Dialect Transfer for S wiss G erman Speech Translation (2023)

Abstract: This paper investigates the challenges in building Swiss German speech translation systems, specifically focusing on the impact of dialect diversity and differences between Swiss German and Standard German. Swiss German is a spoken language with no formal writing system, it comprises many diverse dialects and is a low-resource language with only around 5 million speakers. The study is guided by two key research questions: how does the inclusion and exclusion of dialects during the training of speech translation models for Swiss German impact the performance on specific dialects, and how do the differences between Swiss German and Standard German impact the performance of the systems? We show that dialect diversity and linguistic differences pose significant challenges to Swiss German speech translation, which is in line with linguistic hypotheses derived from empirical investigations.

## introduction there are three main challenges when building swiss german speech-to-text systems. first, swiss german is a spoken language with no formal writing system. thus, the task is formulated as the translation of swiss german audio into standard german text @xcite without access to an intermediate textual representation of the source language. this leads to difficulties where swiss german and standard german differ, e.g. in the usage of tenses (where swiss german does not use preterite) or lexical items that are distinct between the two languages. the second challenge is that swiss german consists of many dialects that tend to differ. this issue makes training a speech translation system from swiss german to standard german a task of translating many dialects into a single language. the third challenge is that swiss german is a low-resource language with only around 5 million speakers. furthermore, the fact that some dialects differ significantly yields even fewer speakers for each dialect (e.g., valais only has about 80k speakers). these three challenges motivate the investigation of the following two questions: the first question investigates the issue of the diversity of dialects in a low-resource setting, while the second question investigates the differences between swiss german and standard german. contributions. to answer these questions, we first review the swiss german dialect landscape and the differences to standard german. in particular, we devise a set of hypotheses from the literature stating which dialects are expected to differ from each other and which swiss german phenomena are expected to impact the performance. we then investigate empirically whether the hypotheses match the results of training st models in various settings. our findings show that the empirical results follow the linguistic investigations. that is, there are dialects that we expect to differ significantly from others, which is confirmed by our empirical results. furthermore, the differences between swiss german and standard german impact the performance, where the past tense has the highest impact. the sociolinguistic situation in germanspeaking switzerland is particular: unlike in other linguistic areas, high prestige is associated with dialects @xcite , which are considered important markers of regional identity. this means that swiss german dialects are spoken in most everyday situations, including formal ones. they are, however, only written in informal contexts, such as social media/text messages, and sometimes also for advertisement purposes. on the other hand, formal correspondence, laws, and newspapers are always written in standard german. the standard german of switzerland differs from the varieties in germany and austria and is therefore often referred to as "swiss standard german." 2.2 differences between swiss german and standard german alemannic dialects (al), including swiss german, differ from standard german (de) in several ways such as: phonology: middle german monophthongs and diphthongs are preserved in swiss german (al "huus" vs de "haus" (house) and al "[li@b]" vs de "[li:b]" (dear)) and most dialects have completed the high german consonant shift (al "['xast@]" vs de "['kastn]" (box)). stress is placed on the first syllable of a word in swiss german more often than in de. grammar: al nouns have no genitive case but rather use possessive constructions ("s huus vom buur" -"the house of the farmer" and "im buur sis huus" -"the farmer his house") and the accusative and nominative cases are conflated (except for personal pronouns). the verbal system has no preterite tense, perfect constructs are used instead. relative clauses always use the particle "wo", and some verbs reduplicate in present infinite form when forming a complex predicate with another verb (e.g. "du lohsch mi lo ässe" -"you let me [let] eat"). lexis: there are a rather large number of swiss german / alemannic vocabulary items which are not intelligible to speakers of standard german foot_2 . in swiss german, some of these originate from french (e.g., "trottoir" for pavement). further, a highly typical swiss german word formation process is "-li" suffixation (e.g., "hündli" -little dog). in this work, we will investigate the impact of two differences between swiss german and standard german: first, preterite tense, which does not exist in swiss german (section 6.1), and second, swiss german vocabulary items that are expected to notably differ from standard german (section 6.2). ## transfer experiments we are interested in the interplay of the dialects: which dialects benefit from other dialects being in the training data, and are there dialects where specialized fine-tuning is more adequate? for this, we run two experiments: for both experiments, we compare the results to the all-dialects setting, which consists in training the model on the full stt4sg-350 corpus, i.e., on all dialects. for all experiments, we report the bleu score @xcite , which is calculated using the script provided by @xcite . to analyse the difference between alldialects and the different settings, we compute the ratio between the bleu score achieved by the alldialects setting and the bleu score achieved by the specific experiment (retainment ratio). we visualise this performance retainment using heatmaps, which show the percentage of performance retained on the j th (column) when leaving out the i th (row) dialect. thus, a value of 0.9 is to be read as the model achieving 90% of the bleu score that alldialects achieved. the average at the end of each row indicates the average retainment of the other dialects (i.e., excluding the value of the diagonal), which summarizes the influence of one dialect on other dialects. the absolute values are provided in ## swiss german differences to standard german we investigate two main differences between swiss german and standard german (cp. section 2.2). first, the usage of the past tense, i.e., samples where the standard german text includes the preterite (simple past), which is not used in swiss german. second, the usage of words which are different in swiss german compared to standard german. ## conclusion our findings show that the empirical results are consistent with the linguistic distances between the dialects and the differences between swiss german and standard german. for example, dialects similar to other dialects (such as zh) positively affect others when included in the training data. in contrast, dialects that differ from others (such as vs) need in-dialect data to perform well. the usage of preterite and vocabulary specific to swiss german impact the bleu score negatively. thus, future work is concerned with investigating methods dealing with more efficient dialect transfer and handling the differences between swiss german and standard german.