HW - TSC ’s submission to the IWSLT 2024 Subtitling track (2024)

Abstract: This paper introduces HW-TSC’s submission to the IWSLT 2024 Subtitling track. For the automatic subtitling track, we use an unconstrained cascaded strategy, with the main steps being: ASR with word-level timestamps, sentence segmentation based on punctuation restoration, further alignment using CTC or using machine translation with length penalty. For the subtitle compression track, we employ a subtitle compression strategy that integrates machine translation models and extensive rewriting models. We acquire the subtitle text requiring revision through the CPS index, then utilize a translation model to obtain the English version of this text. Following this, we extract the compressed-length subtitle text through controlled decoding. If this method fails to compress the text successfully, we resort to the Llama2 few-shot model for further compression.

## introduction in recent years, the demand for subtitles across various media platforms has surged, driving the need for efficient and high-quality subtitling solutions. two main approaches have emerged for automatic subtitle generation: cascaded strategies and end-toend models. cascaded strategies traditional cascaded strategies involve a multi-step pipeline @xcite , where each component handles a specific subtask. this typically begins with an automatic speech recognition (asr) system that transcribes the audio into text. the transcribed text is then segmented into subtitles, accounting for timing constraints and reading speeds. finally, the segmented subtitles may undergo text compression to ensure they fit within spatial limitations while retaining critical information. end-to-end strategies in contrast, end-to-end models @xcite aim to directly generate subtitles from audio or audio-visual inputs using a single unified framework, typically leveraging recent advances in deep learning and sequenceto-sequence modeling. such models can jointly learn and optimize all subtitling tasks, mitigating error propagation issues. in this paper, we employ a cascaded strategy. due to whisper @xcite 's remarkable achievements across multiple domains, the cascaded strategy is expected to perform well. at the same time, it allows us to leverage our existing text-to-text machine translation capabilities. in the process of automatic subtitle generation discussed above, regardless of the method employed, subtitle compression emerges as a pivotal element. this is due to the restricted display space for subtitles, and the necessity of adapting subtitles to the playback speed of the video, as well as the reading speed of the audience. consequently, once the automatic generation of the subtitle file is finalized, it becomes essential to compress content for overly long subtitles. by retaining the basic information and meaning, this compression significantly enhances the quality of the subtitles. traditional text compression strategies encompass deletion-oriented approach @xcite and @xcite . in addition to the aforementioned methods, training sequence-to-sequence models with parallel data of both the original and compressed text can enhance efficiency in text compression while more effectively preserving semantic integrity @xcite . in this paper, we leverage a model generation approach to accomplish the task of subtitle compression. uniquely, in the absence of extensive parallel data of original and compressed text for model training, we deviate from traditional model compression methods. instead, we employ a machine translation model to execute the task. this requires the compression and reformation of text, and the deployment of large language models to manage the compression task on certain texts that pose challenges for rewriting. ## automatic subtitling we propose a whisper-based cascaded automatic subtitling strategy, with the details as follows: ## subtitle compression in the task of subtitle compression, our explicit objective is to rewrite the original subtitle text, leveraging its content to fulfill the parameters of characters per second (cps @xcite bleurt @xcite indicators to the highest degree possible. ## conclusion although our performance in the experiment did not achieve best results, the comparison between our own systems can also illustrate some issues: