KIT ’s IWSLT 2020 SLT Translation System (2020)

Abstract: This paper describes KIT’s submissions to the IWSLT2020 Speech Translation evaluation campaign. We first participate in the simultaneous translation task, in which our simultaneous models are Transformer based and can be efficiently trained to obtain low latency with minimized compromise in quality. On the offline speech translation task, we applied our new Speech Transformer architecture to end-to-end speech translation. The obtained model can provide translation quality which is competitive to a complicated cascade. The latter still has the upper hand, thanks to the ability to transparently access to the transcription, and resegment the inputs to avoid fragmentation.

## introduction the karlsruhe institute of technology (kit) participated in the iwslt 2020 evaluation campaign @xcite in two main tracks: offline speech translation task (slt) and simultaneous text translation. our highlight s the proposal of a novel method for training simultaneous translation models, with the adaptive computation time technique @xcite incorporated to the transformer models @xcite . on the other hand, the end-to-end speech translation models have observed a single deep speech transformer @xcite approaching the performance of a heavily powered cascade. the latter, however, is more transparent because of visible inputs and outputs to each components. it is still the dominant approach, thanks to the segmentation module that adds punctuations and sentence boundaries, so the mt models do not suffer from fragmentation. ## data the overall data that the project employed can be divided into two main sections: speech and text corpora. speech corpora. we gathered the allowed training data included must-c and speech-translation ted talks containing both parallel data for audio to english and german. the tedlium3 and the mozilla common voice data are speech recognition-specific. furthermore we also considered the how2 dataset (the portuguese translation is ignored). the data is further cleaned with asr models (the details are unveiled in section 4.4) to obtain the training time as shown in text corpora. we collected the text parallel training data as presented in for simultaneous speech translation, we deploy a novel model on the text-to-text task based on adaptive computation time (act, @xcite ). at each decoder step, the model makes a decision on whether to read another input token or to write an output token (c. f. @xcite ). in our case, these decisions are made by a mechanism based on act: at each decoder step, we calculate a probability distribution over the encoder timesteps, representing the prediction where the decoder should halt and write an output. specifically, for decoder step i, we calculate: it follows from the definition that α i is a probability distribution. we use this distribution along with the attention mechanism from @xcite to calculate the encoder-decoder attention. in order to incentivise the model to keep the delays short, we employ the ponder loss in addition to the usual cross-entropy: for more information on the ponder loss, see @xcite . by varying the parameter λ, we can produce systems with different latency regimes. however, each model produces many different latency-quality tradeoffs during training. we use sentencepiece @xcite to create a shared 37000 word bpe dictionary for source and target. we then train an offline transformer @xcite model with relative self-attention @xcite . based on this, we train several act models with λ varying from 0.15 to 0.7. for all models, we use the adam optimizer @xcite . we train the offline model for 200 000 steps, varying the learning rate from 2.5 • 10 -4 to 0 with a cosine schedule, then train each of the simultaneous models for 1000 steps with initializing parameters from the offline model. all models use the transformer "base" configuration (layer size 512, feed-forward size 2048, 8 attention heads, 6 layers in encoder and decoder). because the evaluation primarily measures delay in terms of tokens, not time, we could have used a larger model, but we decided to choose our model for a more realistic scenario where evaluation time is an important factor. ## offline speech translation we participate to the offline speech translation task using two different approaches: cascade and endto-end. in the cascade, the audio inputs are fed into our speech recognition component (asr -section 4.1), then the outputs will go through a segmentation module (section 4.2) to have wellformed inputs prior to our machine translation module (mt -section 4.3). the outputs of our mt are the final outputs of the cascade system. on the other hand, the end-to-end approach, as its name suggests, performs trainings for a single model from the english audio inputs to produce text outputs in german (section 4.4). ## conclusion at the iwslt2020 evaluation campaign, we first presented a novel simultaneous model that can efficiently learn to wait and translate using act technique. afterwards, we built two systems for offline speech translation, namely a cascade and an end-toend model using deep transformer networks. we showed that the end-to-end model can rival even the best cascade in challenging speech translation tests.