HW - TSC ’s Speech to Text Translation System for IWSLT 2024 in I ndic track (2024)

Abstract: This article introduces the process of HW-TSC and the results of IWSLT 2024 Indic Track Speech to Text Translation. We designed a cascade system consisting of an ASR model and a machine translation model to translate speech from one language to another. For the ASR part, we directly use whisper large v3 as our ASR model. Our main task is to optimize the machine translation model (en2ta, en2hi, en2bn). In the process of optimizing the translation model, we first use bilingual corpus to train the baseline model. Then we use monolingual data to construct pseudo-corpus data to further enhance the baseline model. Finally, we filter the parallel corpus data through the labse filtering method and finetune the model again, which can further improve the bleu value. We also selected domain data from bilingual corpus to finetune previous model to achieve the best results.

## introduction this article describes the indic track speech-to-text translation task submitted by hw-tsc at iwslt 2024. from a system architecture perspective, current research on speech-to-text translation can be divided into two forms: end-to-end and cascade systems. cascade systems usually consist of a speech recognition (asr) module and a text-to-text machine translation (mt) module. although integrating these modules may be complex, the results are still very satisfactory as long as there are sufficient data resources to train each module. additionally, the end-to-end approach can generate translation results directly from the unified model with speech input. however, what we need to know is that the parallel data required to train an end-to-end speech translation model is extremely scarce. ## methods our approach ultimately adopts a cascade approach. ## conclusion in this paper, we report on our work on iwslt2024 speech-to-text translation evaluation in indic track. we mainly introduce our cascade system and the main optimization processes and methods of the mt model. we improve the final results by focusing on optimizing the mt model. for cascade systems, the impact of the mt model on the results is crucial. for the future we plan to further explore the direction of end-to-end systems.