HW - TSC at IWSLT 2023: Break the Quality Ceiling of Offline Track via Pre-Training and Domain Adaptation (2023)

Abstract: This paper presents HW-TSC’s submissions to the IWSLT 2023 Offline Speech Translation task, including speech translation of talks from English to German, Chinese, and Japanese, respectively. We participate in all three conditions (constrained training, constrained with large language models training, and unconstrained training) with models of cascaded architectures. We use data enhancement, pre-training models and other means to improve the ASR quality, and R-Drop, deep model, domain data selection, etc. to improve the translation quality. Compared with last year’s best results, we achieve 2.1 BLEU improvement on the MuST-C English-German test set.

## introduction the goal of the offline speech translation task is to examine automatic methods for translating audio speech in one language into text in the target language. in recent years, end-to-end system and cascade system are fundamental pipelines for speech translation tasks. traditional cascade system is comprised of continuing parts, automatic speech recognition (asr) is responsible for generating transcripts from audios and machine translation (mt) model aims at translating asr outputs from source language into target language. asr model like conformer @xcite and @xcite are commonly used. mt models like transformer @xcite can be considered as a standard configuration. the end-to-end systems use a model to directly recognize speech into target text in another language. the cascade system will cause some "missing information" due to the two encoding and decoding processes of asr and mt. at the same time, the disadvantage of the end-to-end system is the lack of sufficient training data. however, with a fully trained cascade system, the accuracy of asr and mt will reach a higher level. so from the results, the bleu of the cascaded system will be higher than that of the end-to-end system. currently in the industry, the mainstream speech translation system is still based on the cascade system. we use the cascade system for this task, mainly to further improve the performance of speech translation. in this work, we carefully filter and preprocess the data, and adopt various enhancement techniques, such as pre-training model, data enhancement, domain adaptation, etc., to optimize the performance of asr. we build machine translation systems with techniques like back translation @xcite , domain adaptation and rdrop @xcite , which have been proved to be effective practices. the main contribution of this paper can be summarized as follows: section 2 focuses on our data processing strategies while section 3 describes the training techniques of asr, including model architecture and training strategy, etc. section 4 describes the training techniques of mt, and section 5 presents our experiment results. 2 datasets and preprocessing ## experiments and results we use the open-source fairseq @xcite for training, word error rate (wer) to evaluate the asr models and report case-sensitive sacrebleu @xcite scores for machine translation. we evaluated our system on the test sets of must-c tst-common (tst-com). we analyze the main reasons for the similar results of the three systems: 1. the three systems use the same mt, and our mt system has the ability to correct wrong input after the robustness is en-hanced. 2. using the same data to finetuning the three asr systems, the wer are relatively close. ## conclusion this paper presents our offline speech translation systems in the iwslt 2023 evaluation. we explored different strategies in the pipeline of building the cascade system. in the data preprocessing, we adopt efficient cleansing approaches to build the training set collected from different data sources. we tried various asr training strategies and achieved good performance. for the mt system, we have used various methods such as multilingual machine translation, r-drop, domain adaptation, and enhanced robustness. finally, compared with last year's best results, we have improved by 2.1 bleu in the must-c english-german test set.