dialect R : Doing Dialectometry in R (2022)

Abstract: We present dialectR, an open-source R package for performing quantitative analyses of dialects based on categorical measures of difference and on variants of edit distance. dialectR stands as one of the first programmable toolkits that may freely be combined and extended by users with further statistical procedures. We describe implementational details of the package, and provide two examples of its use: one performing analyses based on multidimensional scaling and hierarchical clustering on a dataset of Dutch dialects, and another showing how an approximation of the acoustic vowel space may be achieved by performing an MFCC (Mel-Frequency Cepstral Coefficients)-based acoustic distance on audio recordings of vowels.

## introduction the quantitative analysis of dialect relatedness has yielded respectable results in the field of dialectometry, where sophisticated methods of measuring linguistic distance have been developed which correlate to a large degree with perceptual measurements of intelligibility @xcite . the use of such methods offers an objective basis to the determination of dialect distributions, including boundaries at times, and which overcomes some of the subjective biases inherent in earlier approaches that utilized the notion of isogloss for dialect classification. however, despite the success of these methods, access to their use has generally relied on gui-based software such as visual dialectometry (vdm) @xcite , diatech @xcite , and gabmap @xcite , which are easy to use, but which accept the trade-off of impeding easy modification for those who wish to extend existing methods. users who wish to perform statistical analyses outside of what is provided or make changes to the existing pipeline do not have easy access to the internals of such software, and consequently have to start from a higher technical threshold. in fact, these packages have not been modified by others. a notable exception is the l04 software, foot_1 which operates in the unix ecosystem and would allow for some degree of user modification, but few, if any users have taken advantage of this. in addition to providing for more flexibility that exists in current packages, the present effort also facilitates the work of those who like the provisions of the older packages, but who wish to try out contemporary approaches, something the existing packages likewise do not readily support. in view of this situation, we present dialectr, an open-source software package that allows the construction of dialectometric pipelines in the statistical programming language r (r core team, 2020). it is largely inspired by gabmap, but attempts to overcome some shortcomings of its monolithic presentation. our vision is to facilitate more wideranging dialectological experimentation with the data analysis possibilities in r. for example, dialectologists should be able to experiment more directly with geostatistical analyses, which, with honorable exceptions @xcite , have largely been ignored in dialectometry. for a second example, we note that, although dialectometry makes extensive use of multi-dimensional scaling (see below), other dimension-reducing techniques (for non-distance matrices), such as factor analysis or principal component analysis, have received less attention, again with some honorable exceptions @xcite . the present paper offers a foundation from which much more extensive experimentation may be launched. we offer further examples below. ## software design the component of dialectr which most interests users is probably the edit distance computation of pronunciation differences based on transcriptions, which is written in c++11 and interfaced to r through the r package rcpp @xcite . once a distance matrix between data-collection sites has been produced, dialectr additionally offers a number of ready-made functions for common analyses, such as an rgb-based multidimensionsal scaling for visualizing dialect continuua @xcite , or a function for visualizing discrete dialect groupings based on hierarchical clustering @xcite . moreover, in the case where the input data is acoustic, we show how an additional acoustic distance proposed in @xcite can be leveraged, something missing in all alternative packages and web applications. we describe specifics of these components in the following subsections. ## example session we show in this section an example session, by analyzing dutch dialect data in the goeman-taeldeman-van reenen-project with dialectr. the ipa transcription dataset comes installed with the package, along with a sample keyhole markup language (kml) file that is required in order to sites concepts aarde adem appels aalsmeern h p6rde p6d@m pap@ls aalstbelb e@t os@m ap@ls aalstbeov eerd@ os@m ap@l@n the keyhole markup language is an xml-based markup language for geographic data, and is principally associated with google earth, foot_4 which users may utilize to create kml files for their own data. an excerpt of the phonetic transcriptions is shown in the transcription data can be called with the data function built natively in r, and the geographic data can be loaded with get_points and get_polygons, which respectively extract the points and polygon data from the kml file into dataframes: library(dialectr) data(dutch) @xmath2.file("extdata", "dutchkml.kml", @xmath3 with the transcription data and geographic information ready, we can call distance_matrix and set the option of alignment_normalization to true, which computes the edit distance between the pronunciations of all corresponding words in all pairs and normalizes the score by length; we also set funname to leven, which uses the plain edit distance for its computation, as opposed to vc_leven, which implements the vowelconsonant constraint. the details of both of these options are discussed in section 2.1. we call mds_map upon the resulting distance matrix along with the required geographic information, which results in figure 4 : @xmath4, @xmath5, @xmath6, dutchpoints, dutchpolygons) we briefly remark that friesland (the area in blue) clearly stands out as a variety most distinctly separate from its surroundings, which is consistent with its status as an independent language. the low saxon area (the green area on the top right) and the west of flanders (lower left) also show a notable similarity, which @xcite also noted. for purposes of illustration, we also show here how the edit distance and its variants as implemented in distance_matrix can be called independently of the function: where the alignment_normalization parameter normalizes the distance by dividing the length of the alignment between two strings, and the delim parameter allows for comparing multiple responses in one or both of the sites with bilbao distance. to gain more specific insights into how one might classify significant similarities in a given area, we are now in a place to complement the multidimensional scaling analysis as performed above with hierarchical clustering. in dialectr this can be called via cluster_map, which results in figure 5 : we observe that the projection of our hierarchical clusters onto the geographic locations of the collection sites results in sensible aggregate isoglosses that largely correspond with the classification of dialectologists. ## conclusion and future work we presented dialectr, an open-source package that attempts to facilitate community-based extensions to dialectometric methods by situating itself in the statistical environment of r. in doing so, we echo the sentiment in @xcite regarding the future of gabmap, a web application for dialectometry that served as the primary reference for the present package: "[t]here are also opportunities for further development. probably the most important of these would involve making it easier for others to contribute modules, i.e. adopting an open-source development mode. once it becomes easier for others to contribute, then scientific imagination is the limiting factor". we suggested several lines of research above which dialectr might be used to support, includ-ing the use of geostatistical analysis or a wider range of dimension-reducing techniques. we further demonstrated how dialectr could be used to incorporate acoustics-based aggregate analyses in sec. 2.3 above. so it is fitting that we close with yet another suggestion for work that dialectr might be used to support. edit distance measures for phonetic transcriptions have been shown to improve in sensitivity when used with sensitive segment weights @xcite . work in this direction has sought to take into account that frequent sound substitutions should be taken as more similar than infrequent ones (e.g., a substitution of [e] should count as more similar to [e] than to @xcite ). such a procedure has been used for the measurement of foreign accent strength @xcite and for the rectification of "field worker isoglosses", which refers to a systematic difference in transcription that occurs due to the field workers preferences, as opposed to any real linguistic differences between the dialect sites @xcite . these applications together point towards its usefulness as a future module, either to be incorporated into the current package, or, alternatively, to be made available alongside it. as increasingly sophisticated statistical methods come to be used to examine dialect data @xcite , the possibility of interfacing with dedicated packages in r facilitates the community-based effort to keep the latest methods within the reach of the general user.