Without Further Ado: Direct and Simultaneous Speech Translation by A pp T ek in 2021 (2021)

Abstract: This paper describes the offline and simultaneous speech translation systems developed at AppTek for IWSLT 2021. Our offline ST submission includes the direct end-to-end system and the so-called posterior tight integrated model, which is akin to the cascade system but is trained in an end-to-end fashion, where all the cascaded modules are end-to-end models themselves. For simultaneous ST, we combine hybrid automatic speech recognition with a machine translation approach whose translation policy decisions are learned from statistical word alignments. Compared to last year, we improve general quality and provide a wider range of quality/latency trade-offs, both due to a data augmentation method making the MT model robust to varying chunk sizes. Finally, we present a method for ASR output segmentation into sentences that introduces a minimal additional delay.

## introduction in this paper, we describe the apptek speech translation systems that participate in the offline and simultaneous tracks of the iwslt 2021 evaluation campaign. this paper is organized as follows: in section 2, we briefly address our data preparation. section 3 describes our offline st models followed by the experimental results in section 3.6. for the offline end-to-end translation task, we train deep transformer models that benefit from pretraining, data augmentation in the form of synthetic data and specaugment, as well as domain adaptation on ted talks. motivated by @xcite , we also collapse the asr and mt components into a posterior model which passes on the asr posteriors as input to the mt model. this system is not considered a direct model since it is closer to * equal contribution the cascade system while being end-to-end trainable. our simultaneous translation systems are covered in section 4 with discussions on experimental results in section 4.5. we resume the work on our streaming mt model developed for iwslt 2020, which is based on splitting the stream of input words into chunks learned from statistical word alignment. most notably, we can implement a flexible quality/latency trade-off by simulating different latencies at training time. we also meet this year's requirement to support unsegmented input by developing a neural sentence segmenter that splits the asr output into suitable translation units, using a varying number of future words as context which minimizes the latency added by this component. the experiments have been done using rasr @xcite , returnn @xcite , and sisyphus @xcite . ## simultaneous speech translation for the iwslt 2021 simultaneous speech translation @xmath0, we continue exploring our last year's alignment-based approach @xcite , which uses a cascade of a streaming asr system and an mt model. ## final results in comparison to last year's submission @xcite , the result of offline speech translation models have improved. the official results on the tst2020 and tst2021 test sets are shown in our end-to-end direct (an ensemble of 4 models), cascade (a single model) and posterior (a single model) systems correspond to the lines 15, 6 and 7 of ## conclusion this work summarizes the results of apptek's participation in the iwslt 2021 evaluation campaign for the offline and simultaneous speech translation tasks. compared to apptek's systems at iwslt 2020, the cascade and direct systems present an improvement of 0.9% and 2.6% in bleu and ter, respectively, averaging over 3 test sets. this shows that we further decreased the gap in mt quality between the cascade and direct models. we have also explored the posterior model, which enables generating translations along with transcripts. this is particularly important for applications when both sequences have to be displayed to users. for the simultaneous translation systems, this year we are able to provide configurations in a wide latency range, starting at al values of 2 words and 1.8s for text and speech input, respectively. for speech input, a maximal translation quality of 25.8 bleu is achieved on tst-he, 3% bleu improvement compared to the previous system at a similar latency. by using future context of variable length we are able to do reliable sentence segmentation of asr output designed to introduce minimal additional delay to the system.