The HW - TSC ’s Offline Speech Translation System for IWSLT 2022 Evaluation (2022)

Abstract: This paper describes the HW-TSC’s designation of the Offline Speech Translation System submitted for IWSLT 2022 Evaluation. We explored both cascade and end-to-end system on three language tracks (en-de, en-zh and en-ja), and we chose the cascade one as our primary submission. For the automatic speech recognition (ASR) model of cascade system, there are three ASR models including Conformer, S2T-Transformer and U2 trained on the mixture of five datasets. During inference, transcripts are generated with the help of domain controlled generation strategy. Context-aware reranking and ensemble based anti-interference strategy are proposed to produce better ASR outputs. For machine translation part, we pretrained three translation models on WMT21 dataset and fine-tuned them on in-domain corpora. Our cascade system shows competitive performance than the known offline systems in the industry and academia.

## introduction in recent years, end-to-end system and cascade system are fundamental pipelines for speech translation tasks. traditional cascade system is comprised of continuing parts, automatic speech recognition (asr) is responsible for generating transcripts from audios and machine translation model aims at translating asr outputs from source language into target language. obviously, the asr part and mt part of this system are independent to some extent. therefore, this paradigm enables people to utilise state-ofthe-art asr models and mt models and conduct experiments by different permutations and combinations. and those experiments can help us find the best combination of choice of asr and mt model. asr model like conformer @xcite and @xcite are commonly used. mt models like transformer @xcite can be considered as a standard configuration. on the contrary, there is also a disadvantage when applying cascade systems. the main aspect is that some important information such as the intonation and emphasis of speakers could not be explicitly expressed in the transcripts. this "missing information" might be the key to distinguish the gender of speaker, or the sarcasm and symbolism behind the texts. it means, there is a risk of losing important information under the condition of cascade system. correspondingly, end-to-end system preserves the competitive edge to learn the "missing information", because it is directly trained on the speech-to-text dataset without any transit process. due to this property, end-to-end system has been paid attention in research and there is encouraging progress. for instance, conformer @xcite can also be used in this task. however, there are some disadvantages for the end-to-end system. firstly, due to the lack of large scale high quality bilingual speech translation datasets, training a productive end-to-end st model can be nontrivial. next, the mapping from speech space to the target language space is far more difficult than the mapping to the source language space, leading to greater demand on the scale of the training set. this paper presents our work in iwslt 2022 @xcite offline speech translation track. the main contribution of this paper can be summarized as follows: ## conclusion this paper presents our offline speech translation systems in the iwslt 2022 evaluation. we explored different strategies in the pipeline of building the cascade and end-to-end system. in the data preprocessing, we adopt efficient cleansing approaches to build the training set collected from different data sources. domain controlled generation was used in the training and decoding of asr models to fit the requirement of the evaluation test set. we also investigated the positive effect of context-aware lm reranking aiming at improving the quality and consistency of asr outputs. fi-nally, we demonstrated that the cascade system consisted of reranking asr system and mt model has the best performance than end-to-end system. in our future works, we would like to investigate more strategies on improving the consistency of asr outputs beyond reranking, as well as better training and data augmentation strategies for endto-end models.