Towards Efficient Simultaneous Speech Translation: CUNI - KIT System for Simultaneous Track at IWSLT 2023 (2023)

Abstract: In this paper, we describe our submission to the Simultaneous Track at IWSLT 2023. This year, we continue with the successful setup from the last year, however, we adopt the latest methods that further improve the translation quality. Additionally, we propose a novel online policy for attentional encoder-decoder models. The policy prevents the model to generate translation beyond the current speech input by using an auxiliary CTC output layer. We show that the proposed simultaneous policy can be applied to both streaming blockwise models and offline encoder-decoder models. We observe significant improvements in quality (up to 1.1 BLEU) and the computational footprint (up to 45% relative RTF).

## introduction simultaneous speech translation (sst) is the task of translating speech into text in a different language before the utterance is finished. the goal of sst is to produce a high-quality translation in real-time while maintaining low latency. however, these two objectives are conflicting. if we decrease the latency, the translation quality also drops. last year's iwslt evaluation campaign @xcite showed that current methods for simultaneous speech translation can approach the translation quality of human interpreters @xcite . the disadvantage is a higher computation footprint that might make a widespread application prohibitive. this paper describes the cuni-kit submission to the simultaneous translation track at iwslt 2023 @xcite . following our last year's submission @xcite , we continue in our effort to onlinize the robust offline speech translation models. however, the main goal of this submission is to improve the computational footprint. to this end, we propose a novel online policy based on ctc. as we experimentally document, the online ctc policy can be used to onlinize the offline models achieving a 45 % improvement in real time factor (rtf) as well as to improve the quality of the streaming blockwise models @xcite . aside from improving the online policy, we also adopt the novel improved streaming beam search @xcite that further improves the translation quality. our contributions are as follows: ## methods in our submission, we use two different model architectures -a traditional offline st architecture and a blockwise simultaneous st architecture @xcite . in this section, we describe the methods applied to achieve simultaneous st using these architectures. ## submission in this section, we summarize our submission to the simultaneous track at iwslt 2023. in total, we submit 10 systems for all three language pairs. ## conclusion and future work in this paper, we present the cuni-kit submission to the simultaneous track at iwslt 2023. we experimented with the latest decoding methods and proposed a novel ctc online policy. we experimentally showed that the proposed ctc online policy significantly improves the translation quality of the blockwise streaming models. additionally, the proposed ctc policy significantly lowers the computational footprint of the onlinized large offline models. unaware of a data overlap issue in 2022, we eventually chose to use our last years' models in the official evaluation also this year.