The PASSAGE project : S tandard G erman Subtitling of S wiss G erman TV content (2022)

Abstract: We present the PASSAGE project, which aims at automatic Standard German subtitling of Swiss German TV content. This is achieved in a two step process, beginning with ASR to produce a normalised transcription, followed by translation into Standard German. We focus on the second step, for which we explore different approaches and contribute aligned corpora for future research.

## introduction swiss german, a primarily spoken language with many regional dialects and no standardised written form @xcite , is spoken by two thirds of the population of switzerland. it is widely used on swiss tv, e.g. in news reports or interviews, which are subtitled in standard german to make them accessible to people who cannot understand spoken swiss german. producing these subtitles automatically would be advantageous in terms of time and cost. this task is the focus of the passage project (nov. 2020-dec. 2022) "sous-titrage automatique du suisse allemand en allemand standard", which is a collaboration between geneva university, srf (schweizer radio und fernsehen) and recapp. 1 in this project a first automatic speech recognition (asr) step is used to produce a normalised transcription of spoken swiss german, keeping the original syntax and expressions but using standard © 2022 the authors. this article is licensed under a creative commons 3.0 licence, no derivative works, attribution, cc-by-nd. 1 https://www.media-initiative.ch/project/subtitling-of-swissgerman-into-standard-german-automatic-post-editing/ german words. in a second step, different approaches are explored to transform this normalised transcription into correct written standard german (see figure 1 ). to achieve this, multiple issues must be dealt with: asr errors, incorrect detection of sentence boundaries, features related to spontaneous spoken language, such as dysfluencies or informal language, and finally the syntactic divergences between swiss german and standard german @xcite . the three goals of the project are 1) to create data sets for swiss german, 2) to build systems for the translation of asr output into standard german, and 3) to evaluate the usability of the system output. ## data the following data were provided by srf: based on these data, we have so far created several aligned corpora, which were used to train and specialise the first systems: finally, our project partner recapp foot_1 provided real asr output, which was used for evaluation, and is currently being aligned with the original subtitles to create another parallel resource. ## systems the project aims at investigating different approaches suitable for tasks where only few changes are needed and applicable to low-resource languages. we will also explore different settings, such as the impact of different training data, e.g. transcriptions vs asr output, automatic vs manual alignment. ## first results the first two approaches tested for this task are 1) a neural machine translation (nmt) transformer architecture with copy attention @xcite and 2) an edit-based model (ed) with a taskspecific attention mechanism that predicts types of edits @xcite . our first system evaluations were carried out on normalised transcriptions, which simulate a perfect asr result. an automatic evaluation using the post-edited version as reference showed that overall the nmt system was slightly better than the ed system (bleu 64.91 vs 61.49), and that nmt makes more edits than ed (hter 22.59 vs. 12.69). another round of evaluations was performed on real asr output, with a focus on the systems' ability to transform swiss german syntactic phenomena into their standard german counterparts. here the nmt system outperforms the ed system. for nmt, the addition of targeted synthetic training data improves the results, in terms of transformed phenomena and precision. next steps include an evaluation with end-users to assess the impact on satisfaction.