ASR for Non-standardised Languages with Dialectal Variation: the case of S wiss G erman (2020)

Abstract: Strong regional variation, together with the lack of standard orthography, makes Swiss German automatic speech recognition (ASR) particularly difficult in a multi-dialectal setting. This paper focuses on one of the many challenges, namely, the choice of the output text to represent non-standardised Swiss German. We investigate two potential options: a) dialectal writing – approximate phonemic transcriptions that provide close correspondence between grapheme labels and the acoustic signal but are highly inconsistent and b) normalised writing – transcriptions resembling standard German that are relatively consistent but distant from the acoustic signal. To find out which writing facilitates Swiss German ASR, we build several systems using the Kaldi toolkit and a dataset covering 14 regional varieties. A formal comparison shows that the system trained on the normalised transcriptions achieves better results in word error rate (WER) (29.39%) but underperforms at the character level, suggesting dialectal transcriptions offer a viable solution for downstream applications where dialectal differences are important. To better assess word-level performance for dialectal transcriptions, we use a flexible WER measure (FlexWER). When evaluated with this metric, the system trained on dialectal transcriptions outperforms that trained on the normalised writing. Besides establishing a benchmark for Swiss German multi-dialectal ASR, our findings can be helpful in designing ASR systems for other languages without standard orthography.

## introduction over the last few years, advancements in speech technology (word error rates (wer) less than 5%, @xcite @xcite ) have lead to an increased demand for automatic speech recognition (asr) systems outside of a small set of standardised, high resource languages. however, developing asr systems for non-standard languages and dialects is difficult since, in addition to the lack of a writing standard, one has to deal with a high degree of regional variation coupled with limited training resources. thus, in such settings, wers of more than 40% are common @xcite . swiss german is a typical case of a non-standard language for which there is a growing interest in asr technology. the term swiss german describes a family of mutually intelligible allemanic dialects spoken in the northern two-thirds of switzerland. despite carrying the official status of a dialect, swiss german is spoken in all spheres of oral communication and has long enjoyed a high degree of cultural value and prestige, unlike many other dialects @xcite . in the past, the linguistic situation in the region has been described as a case of 'medial diglossia' @xcite , in which standard german is used for written communication and spoken in formal settings, while swiss german is primarily used for everyday spoken communication. being the primary language for approximately five million people in the region, swiss german has a growing demand for automatic processing and in particular asr. however, without a single orthography it is difficult to determine not only how to convert speech to text, but also which text to convert it to. in this paper, we propose the first multi-dialect speech-to-text (stt) framework for swiss german dialects foot_0 and explore the efficacy of two potential target textual representations. the first uses dialectal transcriptions (option 1 in figure 1 ), a writing system proposed by @xcite intended to accommodate all dialects of swiss german in a manner which closely reflects the sound of the spoken language, while still being easily readable. this is a loosely phonemic spelling system that draws on familiar spelling norms from standard german, restricting the space of possible writings for a given word. it is, however, often subject to transcribers' interpretation leading to considerable intra-and inter-dialectal inconsistency. the second option is to use normalised transcriptions (option 2 in figure 1 ), a wordlevel many-to-one mapping between swiss variants and a single canonical writing resembling standard german whenever possible. using a normalised transcription should aid asr by reducing the effect of lexical variability on the target side. on the other hand, it might be harder to establish alignments between the dialectal sound and the standard writing which does not represent the original pronunciation. there is currently no conclusive evidence as to which of the two options yields better results. the experiments presented in this paper are intended to identify an optimal approach to dealing with swiss german and non-standard languages in general. the resulting asr systems, featuring a single unified acoustic model (am) trained with the time-delay neural network (tdnn) architecture @xcite ) following kaldi's foot_1 wsj chain recipe @xcite , are the first multi-dialectal solutions for swiss german and constitute a baseline for future improvements. ## related work in addressing the topic of multi-dialectal asr for non-standard languages, our work draws on previous studies carried out in the context of arabic dialects (e.g. @xcite ). in particular, @xcite successfully trained a multi-dialect asr system with a single acoustic model (am) using approximately 1,200 hours of dialectal arabic speech data. leveraging a combination of tdnns and (bidirectional) long-short term memory networks (bi-lstms), trained on speed and volume perturbed data, they reported a wer of 14.7% on held-out test data sampled from different dialects. in the context of swiss german, previous studies on asr have typically focused on one particular regional variety. for example, @xcite trained an asr system on the mediaparl corpus, containing swiss german speech data from the canton of valais with text transcriptions in standard german. the authors used a triphone-based am trained with gaussian mixture models and hidden markoc models (gmm-hmm) and a bigram language model (lm), reporting wer scores of 68.4%. following this, more advanced ams have been applied to the same dataset, namely a three-layer ann-hmm @xcite , a three-layer cnn-hmm @xcite , and a three-layer smbr discriminative ann-hmm optimised towards the state-level error rate (dubagunta and doss, 2019), with wers of 25.5%, 23.5% and 18.7%, respectively. in another study, garner et al. ( 2014 ) trained an asr system on a corpus of radio news broadcasts in valais german annotated with approximate dialectal transcriptions, achieving 19.4% wer. note though that the language model used (lm) was trained on the training and evaluation data together and thus failed to provide an accurate estimation of performance on unseen data. finally, @xcite adapted a standard german asr system to better handle swiss german varieties given a small corpus of weather reports, reporting wers of 23.8% on a challenge corpus of dialectal and standard german speech developed by @xcite . an important aspect of non-standardised-language speech recognition is its evaluation. usual metrics such as wer and character error rate (cer) assume that only one word or character is correct for each particular position in a sequence. for languages without a standardised orthography, there can be multiple permissible spelling variants for the same word. therefore, neither wer nor cer are flexible enough to provide an accurate picture of system performance. @xcite proposed werd (word error rate for dialects) for evaluating dialectal arabic asr. they gathered a large collection of potential spelling variants from social media, using a small context window for a given target word or phrase. valid variants were then found according to occurrence frequencies and normalised with edit distance scores. then, when comparing the system hypothesis to the reference transcription, a word was considered correct if deemed a valid spelling variant of the corresponding word in the reference. in line with this approach, we exploit the many-to-one mappings provided by the normalised transcriptions in our corpus to better evaluate performance on the dialectal transcriptions. ## data for our experiments, we used the second release of the archimob corpus of swiss german @xcite . the corpus consists of 43 interview recordings with native speakers in 14 different swiss german dialects, totaling approximately 70 hours of raw speech data. 3 these interviews have been manually transcribed by five native-speaker annotators, with the audio signals aligned to utterance segments typically of around 4-8 seconds in length. ## methods in this section, we describe our approach to building a multi-dialectal asr system for swiss german which follows the classical pipeline approach depicted in figure 2 . in particular, we discuss the steps taken to establish a suitable pronunciation lexicon, our choices in training ams and lms and the evaluation set up. ## results & discussion the evaluation results presented in if we consider the flexwer evaluation measure, which accounts for permissible spelling variants, the best scores are achieved by the dialectal system tdnn-ivector-dial90k (21.27%). the major disadvantage of systems trained on dialectal transcriptions is that the higher degree of lexical variability hinders the contribution of the lm and, for getting more reliable performance estimation, additional normalisation of the results, or even manual evaluation, should be performed. at the same time, the normalised models trained on transcriptions that poorly reflect the pronunciation and thus rely on a simple g2p-based pronunciation lexicon generally miss more subword-level information. according to cer, which is more sensitive to more precise correspondence between the acoustic signal and the annotation than wer, the systems trained on the dialectal transcriptions perform slightly better than their normalised counterparts. this is most noticeable on the development set, where tdnn-ivector-dial90k and tdnn-ivector-norm80k systems score 14.97% and 16.48%, respectively. therefore, these models may be preferable when dialectal variability is desirable in the output transcriptions. for the normalised transcriptions, leveraging a slightly larger lm results in only minor improvements in wer over the baseline. we hypothesise that this is a consequence of the restricted domain of the archimob dataset, which limits the effectiveness of any additional information provided by extending lm training data. further evaluation on ood data is still needed to answer the question of how well the system is able to adapt to new domains given improved lms. the evaluation results can differ between dialects depending on how well a dialect is represented in training data and the degree to which certain dialects differ from the majority of the training data. to attain an impression of these effects, we report wer scores as a distribution over individual dialects (see figure 4 ). firstly, the effect of the proportion of a corresponding dialect in the training data does not seem to be borne out. the graubünden dialect has the lowest wer but is represented in the archimob corpus with only a single interview. similarly, basel-landschaft and schaffhausen varieties, which also have only one interview each in the corpus, show lower error rates than regional varieties which are wellrepresented in the corpus (e.g. aargau, bern and lucerne). secondly, the degree of similarity between different dialects demands further investigation and is out of the scope of the current paper. some tendencies, however, can be noticed from the plot: the system performs considerably worse when evaluated on the valais dialect compared to other dialects. this observation reflects the fact that swiss german from the canton of valais is further removed from the other varieties. most notably, evaluation on the graubünden dialect is consistently good, outperforming the results on the zurich dialect which comprises the largest part of the training data (12 interviews). this peculiarity may be due to the fact that recognition of a dialect, in our case, is very speaker specific. in the archimob corpus, many dialects have only one interview. this means that they are also represented by only a single speaker. therefore, in order to attain a more reliable picture of dialect-specific speech recognition with a unified asr system, more speakers are needed for under-represented dialects. a general limitation of our work is that it uses a relatively closed domain covered by the archimob corpus (personal narratives on a similar topic). to get an idea of how our models perform outside of this domain, we perform an additional ood evaluation for our dialectal systems using a small data set with appropriate dieth orthography transcriptions. the results of this evaluation are predictably worse: 59.81% wer and 37.83% flexwer for the tdnn-ivector-baselm system. we suspect that potential improvements can be gained by (a) applying advanced training architectures including end-to-end systems; and, of course, (b) increasing the amount of training data to cover more domains and speakers. since this paper focuses on the efficacy of the target textual representation and the corresponding pronunciation lexicons, we leave further investigations of these factors for future work. ## conclusion in this paper, we have introduced the first multi-dialect asr framework for swiss german that exploits a single unified acoustic model. we have tested currently popular techniques for dealing with noisy data and established a solid baseline for future studies in asr for swiss german. we have shown a clear benefit of working with normalised transcriptions (provided a suitable pronunciation lexicon is at hand), but also that loosely phonemic dialectal writing yields output potentially interesting for downstream applications specifically targeting regional variation in non-standard languages.