STT 4 SG -350: A Speech Corpus for All S wiss G erman Dialect Regions (2023)

Abstract: We present STT4SG-350, a corpus of Swiss German speech, annotated with Standard German text at the sentence level. The data is collected using a web app in which the speakers are shown Standard German sentences, which they translate to Swiss German and record. We make the corpus publicly available. It contains 343 hours of speech from all dialect regions and is the largest public speech corpus for Swiss German to date. Application areas include automatic speech recognition (ASR), text-to-speech, dialect identification, and speaker recognition. Dialect information, age group, and gender of the 316 speakers are provided. Genders are equally represented and the corpus includes speakers of all ages. Roughly the same amount of speech is provided per dialect region, which makes the corpus ideally suited for experiments with speech technology for different dialects. We provide training, validation, and test splits of the data. The test set consists of the same spoken sentences for each dialect region and allows a fair evaluation of the quality of speech technologies in different dialects. We train an ASR model on the training set and achieve an average BLEU score of 74.7 on the test set. The model beats the best published BLEU scores on 2 other Swiss German ASR test sets, demonstrating the quality of the corpus.

## introduction we present stt4sg-350, a corpus of swiss german speech, annotated with standard german text at the sentence level. the corpus represents all swiss german dialect regions and contains 343 hours of speech. swiss german is a family of german dialects spoken by around 5 million people in switzerland. it differs from standard german regarding phonology, vocabulary, morphology, and syntax. there are significant differences among the swiss german dialects as well, particularly regarding phonology and vocabulary. swiss german is primarily a spoken language. it is also used in writing, but mainly in informal text messages. in most other contexts, including formal letters, laws, and newspapers, standard german is used instead. one important reason for this is swiss german's lack of a standardized orthography. the diversity among dialects, exacerbated by the lack of a standardized orthography, leads to a large number of written variants for each word. this, together with the small amount of text resources compared to standard german, makes automated processing of swiss german text challenging. stt4sg-350 is, to the best of our knowledge, the largest public speech corpus for swiss german. while the primary use case is automatic speech recognition (asr), it is also a useful resource for text-to-speech (tts), dialect identification, and speaker recognition. by providing roughly the same amount of data per dialect region, irrespective of its population size, the corpus contributes to improving speech technology for underrepresented dialects. in addition, the test set, which contains the same spoken sentences in each dialect, allows a fair evaluation of the quality of speech technologies in different dialects. furthermore, it contributes to more inclusive speech technology by keeping a balanced gender ratio and featuring speakers of all ages. ## related work the sds-200 corpus @xcite ) contains 200 hours of speech by around 4,000 speakers with standard german transcripts. the recordings cover a large part of the swiss german dialect landscape. the number of recordings per speaker follows a long-tail distribution. for example, the top 3 speak-ers account for 23% of recordings. the swiss parliaments corpus or spc @xcite ) contains 299 hours of speech in the bernese dialect. the text is standard german, taken from parliament minutes, and is not a fully accurate transcription. text and audio are automatically aligned. the swissdial corpus @xcite contains 26 hours of studio-quality recordings by 8 speakers, each speaking a different dialect, with both standard german and swiss german transcripts. the radio rottu oberwallis corpus @xcite ) contains 8 hours of speech transcribed in swiss german, of which 2 are also transcribed in standard german. the archimob corpus @xcite ) contains 69 hours of speech with swiss german transcripts. for swiss german asr, the desired output text language is standard german for the vast majority of use cases. tackling speech-to-text translation with an end-to-end approach is feasible as shown by @xcite . applying a similar approach to swiss german asr and therefore avoiding swiss german text and its challenges altogether lead to promising results in recent years, see @xcite @xcite . @xcite experiment with tts for swiss german. their models achieve a 5-scale mean opinion score of 2.9 to 4.1. importantly, their approach requires swiss german input text. ## data collection data for stt4sg-350 was collected in two phases: 1) the test set with 76 participants from december 2021 until march 2022, and 2) the train and validation sets with 240 participants from may until november 2022. ## corpus the corpus is publicly available foot_5 under the meta-share noncommercial noredistribution license 7 . the distribution format and the included metadata is described in appendix b. potential ## automatic speech recognition baseline we train a baseline model to demonstrate the use of the stt4sg-350 corpus for swiss german asr. we fine-tune xls-r (1b) foot_7 @xcite on the train_balanced split. xls-r is a model based on wav2vec 2.0 @xcite with 965 million parameters pretrained on 436k hours of unlabeled speech data covering more than 128 languages. swiss german was not part of the training data. we provide the fine-tuning details and experimental setup in appendix c. we report the results of our fine-tuned model on three publicly available swiss german datasets and the stt4sg-350 validation and test sets in ## conclusion we have described stt4sg-350, which is, to the best of our knowledge, the largest public speech corpus for swiss german with 343 hours of speech. our asr baseline model trained on the corpus achieves a bleu score of 74.7 on the test set. in addition, it beats the best published bleu scores on 2 other test sets, demonstrating the quality of the corpus. stt4sg-350 is balanced across the 7 dialect regions, and the test set allows a fair comparison of asr performance on different dialects. we intend to take advantage of these properties in future work and conduct in-depth experiments to explore differences in asr quality between dialects. subsequently, we want to find ways to improve performance for underrepresented dialects.