Low S axon dialect distances at the orthographic and syntactic level (2022)

Abstract: We compare five Low Saxon dialects from the 19th and 21st century from Germany and the Netherlands with each other as well as with modern Standard Dutch and Standard German. Our comparison is based on character n-grams on the one hand and PoS n-grams on the other and we show that these two lead to different distances. Particularly in the PoS-based distances, one can observe all of the 21st century Low Saxon dialects shifting towards the modern majority languages.

## introduction we are investigating dialect similarity in 19 th and 21 st century low saxon based on data from germany and the netherlands. traditionally, low saxon dialect classification has mostly been based on phonological and morphological traits, such as the ones presented by @xcite . in this study, however, we focus on the orthographic and the syntactic side and compare how these relate to each other. we compare two levels as we expect the intensity and nature of the majority language influence to differ here. the choice of these two particular levels was motivated by the fact that orthography can be inspected without annotation and for syntax, we could train sufficiently reliable pos taggers 1 , which at this point is not possible for morphology and phonology. furthermore, we investigate how the dialect closeness on both levels has changed over time. an interesting area to pay attention to with respect to dialect distance is the dutch-german border. like goossens (2019) observed, the low saxon dialects along the border have started to diverge under the influence of the majority languages. according to him, this divergence is most pronounced at the lexical level, but convergence towards the majority language has also been attested 1 around 85% accuracy based on a manually annotated test set. in phonology, morphology and syntax. while studies on the divergence of dialects along the border often focus on the occurrence and frequency of particular traits based on interviews, cf. smits (2011), we address the overall (dis)similarity in prose texts. since in the 19 th century school education and majority language media played a smaller role in everyday life compared with today, we assume the effect of language contact with dutch and german to be less visible in the morphology and syntax of 19 th century low saxon, as such changes to the language system itself take time and gradually add up. on the other hand, the border is probably already clearly discernable at the orthographic level due to reading and writing education in the majority language, which we assume to have had a more immediate influence, particularly in areas where the low saxon literary production had ceased (nearly) completely after middle low saxon times. therefore, from the 19 th to the 21 st century, we expect a greater change in distance towards the majority languages at the pos level than at the character level. we thus hypothesize that the low saxon dialects will appear closer to each other on the syntactic side with distance to the majority languages decreasing over time, while 19 th century dialects might already group together with the respective majority language at the orthographic level. ## background the west germanic language low saxon (also called "low german") today is primarily spoken in northern germany and the north-eastern netherlands by around 5 million people and enjoys official recognition in both countries @xcite . as a result of the lack of an interregional standard language, low saxon speakers tend to use their own dialects in all language use cases. as there is no official common orthography either, one needs to take into consideration two layers of variation: on the one hand spelling variation and on the other ac-tual dialect variation. people may for instance stick to their own dialect but switch writing systems depending on whom they address. 2 this multilayered variation poses challenges to the development of nlp for low saxon but at the same time presents an interesting case for historical dialectology of written language. figure 1 shows the major dialect groups of modern low saxon. the eastern dialects east pomeranian (pom) and lower prussian (npr) were spoken in these areas prior to wwii. ## data the majority of our dataset is taken from the lsdc dataset @xcite since, as far as we are aware, this is the only dataset for modern low saxon annotated for dialect and century. especially in regard to the 19 th century data, we supplemented it with relevant prose texts from leopold and leopold (1882) 3 and the twentse taalbank (van der vliet, 2021). the overall size of the dataset is 120,720 sentences and 2,410,261 tokens and it covers eight dialect regions: dutch north saxon, german north saxon, dutch westphalian, german westphalian, eastphalian, mecklenburgish-west-pomeranian, brandenburgish-south-marchian and low prussian. in this rough division, dutch westphalian includes all dutch low saxon dialects except for gronings, which consequently is identical with dutch north saxon here. the first five of these dialects are included in our current experiments. as we currently lack anno-2 personal observation from conversations on social media. 3 digitised by dbnl: https://dbnl.nl/tekst/ leop008sche00_01/ tated data from mecklenburgish-west-pomeranian (mvp), brandenburgish-south-marchian (bra) and lower prussian (npr) for the 20 th and 21 st century, we cannot yet perform diachronic comparisons and thus exclude these dialects from our experiments as well. furthermore, we do not use the 20 th century data in our comparisons as it still consists mostly of data from only two dialects. in our experiments, we thus used data from the five dialects presented in the low saxon data was converted to conll-u format and automatically pos tagged with the help of the stanza tagger (qi et al., 2020) 4 trained on ud data in danish @xcite , dutch @xcite , german @xcite , and swedish @xcite in addition to manually annotated low saxon data. in connection with the publication of the paper, our dataset, as well as the n-gram counts that form the basis for our experiments, will be added to lsdc-morph repository foot_1 on the helsinki-nlp github page. ## methods dialect similarity at the orthographic level based on character n-grams 6 will be compared to dialect similarity based on pos tag sequences to investigate if these lead to different dialect groupings. @xcite observed in their experiments for identifying swiss german dialects that approaches based on character n-grams outperform word-based ones and, in their study on british dialects, @xcite have employed part-of-speech n-grams for corpus-based dialectometry, concluding that this approach can achieve results comparable to manually selected features. ## results as expected, the pca-based closeness and the clustering at the character-based level differ clearly from the pos-based results, but not all of the divergences correspond to our expectations. ## discussion based on our knowledge of and about low saxon dialects, the overall results appear meaningful despite the comparatively low tagging accuracy of 85%. in the pos-based experiments, the fact that a noticeable distance between neighbouring dialect regions divided by a country border can already be observed in the 19 th century data raises the question of how representative the written dialect is of the actual low saxon spoken by the average population. given that written low saxon is commonly produced by people who have received their education in the majority language, this may have an influence on the kind of written language produced. on the other hand, one needs to keep in mind the size of the dialect regions. both the german westphalian group (dwf) and particularly the german north saxon (dns) group stretching from the dutch border to schleswig-holstein are on their own larger than the whole dutch low saxon area and not all of the texts included are written in varieties particularly close to the border. a more fine-grained dialect subdivision, where e.g., the groningen dialect could be compared with east frisian, would therefore be desirable for the future as well. however, this does not seem feasible in our research project at this point due to the lack of sufficient data sources for many of these dialects. the noticeable distance between german low saxon and german in the character-based experiments compared with the closeness of dutch and dutch low saxon might partly be explained by the greater phonological differences between german and low saxon, but in addition to that, one might also consider that local writing systems for german low saxon tend to adhere to certain orthographic principles not found in the german orthography. one of these is that even the umlauted vowels ä, ö and ü may occur as digraphs, especially in closed syllables, e.g., in the words däänsch/däänsk 'danish', sööt 'sweet' and düüster 'dark', according to both the sass foot_4 spelling @xcite and the münsterland spelling @xcite . the overall pos-based distance of dutch low saxon and standard dutch appears to be comparable to the overall distance between german low saxon and standard german. this is interesting as, due to the greater phonological similarity (e.g. no high german consonant shift) on the one hand and the character n-gram results on the other, one might expect the distance between dutch low saxon and dutch to be relatively smaller on the syntactic level as well. the relatively greater distance of 21 st century german westphalian to the other two german low saxon dialects deserves some attention, too. one possible explanation could be the westphalian dialects' more conservative morphology. whereas several dialects of german westphalian still inflect nouns in three cases and have preserved subjunctive forms of verbs (lindow et al., 1998) 9 , it might be the case that dutch low saxon, german north saxon and eastphalian more commonly resort to prepositions and auxiliary verbs. the relative closeness of german and dutch in the pos-based results came as a surprise as well, but the genre might play a role here: whereas dutch and german data largely represents more formal language from non-fiction texts such as news texts, much of the low saxon data sources belong to various forms of literature. while the possibility of an influence of genre differences on the distance between 19 th and 21 st century low saxon dialects cannot be completely ruled out either, it seems less likely as the majority of the data from both centuries consists of fiction texts and stories. due to the relatively modern data in dutch and german, the conclusions to be drawn from our comparison are restricted. for a more meaningful comparison, one should include 19 th century dutch and german as -even though gradual assimilation to the majority language is what one would expect -it might still be the case that the distance between 19 th century low saxon and the dutch and german of that time was not as significant as the distance presented here would suggest. ## future research in our future research, we will include more low saxon dialects, especially mecklenburgish-west-pomeranian, and add the 20 th century as well as dutch and german data from relevant time periods. the eastern dialects like mecklenburgish-west-pomeranian would constitute a meaningful addition since we could then examine the extent to which the common division into west low saxon and east low saxon / east low german is apparent at the levels of language under scrutiny. morphological tagging would be a valuable addition as well, which we plan to include in the future. at this point, the accuracy is still too low, at around 60-70%, which is why more annotation work is required. in the future, we will create more training data for both pos and morphological tagging through manual correction of the automaticcally tagged data. regarding dimensionality reduction, we intend to more closely inspect which features are considered most central by the model to investigate whether the dialect distances are based on actual dialect characteristics or if the results have been influenced by artifacts of the dataset. we hope that the datasets gathered and annotated by us will facilitate the development of nlp tools for and research into low saxon.