Interactive maps for corpus-based dialectology (2025)

Abstract: Traditional data collection methods in dialectology rely on structured surveys, whose results can be easily presented on printed or digital maps. But in recent years, corpora of transcribed dialect speech have become a precious alternative data source for data-driven linguistic analysis. For example, topic models can be advantageously used to discover both general dialectal variation patterns and specific linguistic features that are most characteristic for certain dialects. Multilingual (or rather, multilectal) language modeling tasks can also be used to learn speaker-specific embeddings. In connection with this paper, we introduce a website that presents the results of two recent studies in the form of interactive maps, allowing visitors to explore the effects of various parameter settings. The website covers two tasks (topic models and speaker embeddings) and three language areas (Finland, Norway, and German-speaking Switzerland). It is available at https://www.corcodial.net/ .

## introduction the traditional data collection method in dialectology has relied on structured surveys conducted in a particular language area. the results of such surveys can be presented in maps, typically one map per linguistic feature. these collections of maps, known as dialect atlases, are an important source of information about dialect divisions of different languages. for instance, the dialect atlas of lauri kettunen @xcite still forms the basis of the division of finnish dialects, even though it was collected almost 100 years ago. as this example shows, dialect atlases were typically conceived in the first half of the 20th cen-tury and presented as paper maps. this poses problems of accessibility for modern dialectology, where computational models are often applied on dialect data, e.g. in the subfield of dialectometry @xcite . some atlases have already been digitized and can thus be used in computational analyses @xcite @xcite . when digitized, the atlases are typically presented as two-dimensional data tables where the columns present linguistic features and the rows locations. digitized atlases also make interactive visualizations possible @xcite . in our recent research, we have experimented with topic modeling @xcite and representation learning @xcite to explore the dialectal divisions arising from corpora instead of atlases. dialect corpora typically consist of spoken data (mostly interviews) which have been phonetically transcribed. compared to the straightforward twodimensional tabular data presented in dialect atlases, corpus data is more difficult to analyze computationally, because individual characteristics of speakers (addressed topics, length of interview, richness of vocabulary, etc.) are mixed with dialect features. in the following sections, we briefly present the data and experiments, while focusing on the interactive website visualizing the results. ## data we work with three datasets consisting of dialect interviews or conversations, which have been both phonetically transcribed and normalized to a standard variety. the datasets cover the finnish, norwegian and swiss german language areas. while the topic modeling experiments (section 3.1) only make use of the phonetic transcriptions, the representation learning study (section 3.2) is based on a dialect-to-standard normal-ization task and uses both transcription layers. ## visualization the website https://www.corcodial. net/ provides interactive visualizations of the two experiments described in the previous section. the maps are drawn with the leaflet foot_4 mapping toolkit. the map backgrounds use the stamen terrain style from stadia maps, foot_5 which are based on openstreetmap data. foot_6 the server-side backend is implemented in flask. foot_7 all these libraries and sources are licensed under creative commons or other open source licenses. the current setup does not require any database, since all the data is available in precomputed csv or json files. figure 1 shows a screenshot of a topic modeling experiment. the map itself takes up most of the screen, whereas the rightmost part is reserved for user interaction (e.g. to select a different parameter) and metadata display (e.g. the legend associating colors to topics). each point on the map corresponds to one interview and each color corresponds to one inferred topic. the main benefit of topic models is that an interview can "belong" to several topics to varying degrees. the pie charts on the maps show the degree of membership to the different topics. a simpler visualization that only shows the dominant topic for each interview, is available by selecting dominant topic. further information about the composition of the topics (i.e., the tokens most strongly associated with each topic) can be shown in a popup window (not shown in figure 1 ). technically, and quite similarly to geographic information systems in general, such a visualization relies on two data files: a corpus-specific geo-json file that describes the points (with their coordinates and ids), and a task-specific json file that contains the distribution of topics for each point. leaflet makes it easy to add the geojson file as an additional layer on top of the map background, and to define the style (e.g. the colors) of each point based on the json file. a particularity of corpus-based analyses is that there can be several interviewed persons from the same place, and the corresponding points on the map would be superimposed. the current implementation detects superimposed points and moves them away from their original locations to ensure their visibility. we plan to further improve this functionality. the representation learning experiment is illustrated by figures 2 and 3 . figure 2 shows a pca of the speaker embeddings of the norwegian ndc dataset. as is commonly the case with dimensionality reduction techniques, the map clearly shows the major dialect areas (southwestern dialects in dark brown, eastern dialects in red, central dialects in light blue and northern dialects in light green), without showing clear-cut borders between the areas. figure 3 , on the other hand, visualizes the speaker embeddings of the finnish skn dataset. in this case, a hierarchical clustering algorithm has been selected. the result shows clearly identifiable dialect areas corresponding relatively well with atlas-based divisions. 8 an exception is the cluster represented in blue on the map, which includes points in the greater helsinki area, in a transition area in the southwest, as well as in northern finland. at the moment, the visualization website supports two clustering algorithms (ward and k-means) and any number of clusters between 2 and 10. ## conclusion following up on our recent research where we propose to use topic modeling and representation learning to explore the dialectal divisions arising from corpora of transcribed dialect speech, we present an interactive website where it is possible to view the experimental results in the form of maps. different parameter settings and modes of visualization can be easily chosen. at the moment, the website covers two tasks (topic modeling and representation learning) and three linguistic areas (finland with the skn corpus, norway with the ndc corpus, and germanspeaking switzerland with the archimob corpus). the design of the website is modular and permits the easy inclusion of additional tasks, language areas and corpora.