BIT ’s System for Multilingual Track (2023)

Abstract: This paper describes the system we submitted to the IWSLT 2023 multilingual speech translation track, with input being English speech and output being text in 10 target languages. Our system consists of CNN and Transformer, convolutional neural networks downsample speech features and extract local information, while transformer extract global features and output the final results. In our system, we use speech recognition tasks to pre-train encoder parameters, and then use speech translation corpus to train the multilingual speech translation model. We have also adopted other methods to optimize the model, such as data augmentation, model ensemble, etc. Our system can obtain satisfactory results on test sets of 10 languages in the MUST-C corpus.

## introduction speech translation refers to the technology of translating source language speech into target language text (or speech). this task has a very broad application space in real life, such as in international conferences, lectures, and overseas tourism; adding speech translation to short videos or real-time subtitles in some foreign language videos can provide users with a better experience. early speech translation is the combination of speech recognition and machine translation. firstly, the speech recognition model recognizes source language speech as source language transcribed text, and then the machine translation model translates the recognized source language text into the target language text, which is also called cascade method. the advantage of cascade model is that it can use a large amount of data in speech recognition and machine translation to train the model, and it is relatively simple to implement. however, the disadvantages of cascade model are also obvious: errors in speech recognition results will be transferred to the next machine * *corresponding author translation task. so researchers have focused on end-to-end speech translation. at present, bilingual end-to-end speech translation has achieved very good results, but using a single model to complete multiple language translations has always been a goal pursued by researchers, that is multilingual speech translation. compared to bilingual speech translation, the advantages of multilingual speech translation include: (1) completing multilingual translation with fewer parameters; (2) low resource languages can learn knowledge from high resource languages. in this paper, we conducted one-tomany multilingual speech translation, and submitted our system to the iwslt 2023 @xcite multilingual speech translation track. here is an introduction to our submitted system: we first use convolutional neural networks to downsample the input features, then input them into the transformer model for further processing, and finally output the translation results at the output layer. the encoder for speech translation needs to complete both acoustic feature extraction and semantic feature extraction tasks. in order to reduce the encoding pressure of the model, we use speech recognition task to pre-train the parameters of the encoder. before inputting the data into the model, we applied the specaugment @xcite ) method for data augmentation, which increased data diversity and resulted in better results for the model. after training the multilingual speech translation model, we calculated the average value of the model parameters obtained for the last 10 epochs to generate the model we used during testing, the model with the obtained average parameters can have better results. the target language includes arabic, chinese, dutch, french, german, japanese, farsi, portuguese, russian, and turkish. the training data for these languages can be found in the commonly used corpus for speech translation -must-c @xcite . we downloaded the data for the relevant languages from must-c v1.0, must-c v1.2, and must-c v2.0, merged them, and preprocessed them to obtain our training dataset. we used the fairseq @xcite toolkit to conduct our experiment, and after the training was completed, we scored the translation quality using the sacrebleu metric. our model achieved our expected results on 10 target languages. ## data preparation as shown in the ## conclusion this paper introduces our system submitted on the iwslt 2023 multilingual speech translation track. we used convolutional neural networks combined with transformer models to complete the task of english speech to 10 target language texts. our system is characterized by its simplicity and efficiency, effectively modeling local and global features in speech, and completing modal and language transformations within the model. our system has achieved satisfactory results on the test set of 10 languages in must-c corpus.