System Description on Automatic Simultaneous Translation Workshop (2022)

Abstract: This paper describes our system submitted on the third automatic simultaneous translation workshop at NAACL2022. We participate in the Chinese audio->English text direction of Chinese-to-English translation. Our speech-to-text system is a pipeline system, in which we resort to rhymological features for audio split, ASRT model for speech recoginition, STACL model for streaming text translation. To translate streaming text, we use wait-k policy trained to generate the target sentence concurrently with the source sentence, but always k words behind. We propose a competitive simultaneous translation system and rank 3rd in the audio input track. The code will release soon.

## introduction simultaneous translation refers to translating the message from the speaker to the audience in realtime without interrupting the speakers, which is a challenging task and has become an increasingly popular research field in recent years. in this paper, we describe our system submitted at the 3rd automatic simultaneous translation workshop, which consists of a rhymeological features based audio split model, an end to end speech recognition model and a wait-k @xcite based streaming text translation model. the system input is chinese audio file and the output is english translation text. a temporary streaming transcription is obtained by audio split and speech recognition model, then transmitted into machine translation model to get the target system output. for automatic audio split model, we calculate the rhythmological features @xcite of the audio input, resort to adaptive policy to set short-term energy threshold and zero crossing rate threhold for speech split. for automatic speech recognition model, we use asrt model foot_0 , which is based dcnn model and ctc decoder @xcite . whilst, we expand the training data set by adding aishell-1 @xcite and thchs-30 @xcite datasets. for streaming text translation, our model is based on stacl @xcite . we use some human rules and the pre-trained language model to filter the parallel corpus. at the step of inference, we apply the waitk words policy. both the pre-processing and postprecessing are applied to improve the terminology translation and deal with the word error produced by the asr system. since our submission is a pipeline system, the rest of this paper describes separately regards to audio split, automatic speech recognition and matchine translation sub-modules. we firstly describe the training and development datasets we used, then the data precessing methods is introduces. secondly, we describe our system architecture and experiment results. lastly, we draw a conclusion of our system by analyzing the experiments. ## dataset for audio data of asr, we use qianyan audio datasets provided by naacl workshop @xcite , aishell-1 @xcite , thchs-30 @xcite . for text data of mt, we use cwmt19 foot_1 and the simultaneous translation corpus provided by the organizer of the workshop. ## system description our system consist of a rhymeological features based audio split model, an end to end speech recognition model, and a wait-k based streaming text translation model. the model training process for speech recognition and machine translation model are implemented on a device with four gpus of nvidia 1080ti. ## conclusion this paper describe our submission to the 3rd automatic simultaneous workshop at naacl2022. we detail our process of data filtering and model training. the consecutive wait(cw) of the best point reached to 14.06, while we get the bleu value of 6.17 in the audio input track. in future work, we will continue to research on end-to-end speech translation model.