Rediscovering the S lavic Continuum in Representations Emerging from Neural Models of Spoken Language Identification (2020)

Abstract: Deep neural networks have been employed for various spoken language recognition tasks, including tasks that are multilingual by definition such as spoken language identification (LID). In this paper, we present a neural model for Slavic language identification in speech signals and analyze its emergent representations to investigate whether they reflect objective measures of language relatedness or non-linguists’ perception of language similarity. While our analysis shows that the language representation space indeed captures language relatedness to a great extent, we find perceptual confusability to be the best predictor of the language representation similarity.

## introduction the relationship between a group of human languages can be characterized across several dimensions of variation @xcite , including (1) the temporal dimension, wherein languages have diverged from a common historical ancestor as in the case of romance languages; (2) the spatial dimension, wherein the speaker communities are geographically adjacent as in the case of the indo-aryan and dravidian languages of india; and (3) the socio-political dimension, wherein languages have evolved under shared political and/or religious forces as in the case of arabic and swahili. languages, or language varieties, can be related across all these dimensions, which often results in a dialect continuum. speakers of languages that constitute a dialect continuum can usually communicate with each other efficiently using their own mother tongue. the degree of intercomprehensibility between speakers of different language varieties within a continuum is mainly determined by linguistic similarities. a notable case of this phenomenon is the mutual intelligibility among the slavic languages, which we study in this paper. one of the goals of linguistics is to study and categorize languages based on objective measures of linguistic distance. the degrees of similarity at different levels of the linguistic structural organization can be seen as preconditions for, as well as predictors of, successful oral intercomprehension. for closelyrelated languages, similarities at the pre-lexical, that is the acoustic-phonetic and phonological, level have been found to be better predictors of cross-lingual speech intelligibility than lexical similarities @xcite . in a different, yet relevant research direction, @xcite have investigated non-linguists' perception of language variation using data from the popular spoken language guessing game, the great language game @xcite . by analyzing the confusion patterns of the glg's human participants, the authors have shown that factors predicting players' confusion in the game correspond to objective measures of similarity established by linguists. for example, both phylogenetic relatedness and overlap in phoneme inventories have been identified as factors of perceptual confusability (and by implication, similarity) of languages in glg. the development of automatic systems that determine the identity of the language in a speech segment has received attention in the speech recognition community (see @xcite for an overview). state-of-the-art approaches for automatic spoken language identification, henceforth lid, are based on multilayer deep neural networks (dnns). dnn-based lid systems are parametric models that learn a mapping from spectral acoustic features of (untranscribed) speech to high-level feature representations in geometric space where languages are linearly separable. these models have shown tremendous success not only in discriminating between distant languages but also closely-related language varieties @xcite @xcite . nevertheless, none of the previous works in spoken language recognition has analyzed the emerging representations from neural lid models for related languages. thus, it is still unknown whether the distances in these representation spaces correspond to objective measurements of linguistic similarity and/or to non-linguists' perception of language variation. in this paper, we aim to fill this gap and consider the family of slavic languages as a case study. our key contribution is two-fold: in this paper, we attempt to bridge different lines of research that have so far remained unconnected. on the one hand, we employ neural architectures from the field of spoken language recognition and build a robust model to identify languages in contemporary acoustic realizations of slavic speech. on the other hand, we analyze the emerging language representations using techniques established by previous research in multilingual natural language processing (nlp). we consequently shed light on the speech modality and show how (untranscribed) speech signals can complement research done in computational studies of linguistic typology and language variation. ## representation similarity analysis in this section, we present our representation similarity analysis. from the robust lid model presented in the previous section, we obtain 512-dimensional representations for the 11 languages from the rbs evaluation set (500 5-second speech segments for each language). these representations are the output of the last fully-connected layer of the language classifier (i.e., layer fc5) before the softmax output layer. foot_0 it is worth pointing out that our lid model has been trained on a subset of only six languages out of the 11 languages that we analyze in this section. ## discussion many recent works have shown that deep neural networks are good models of human perception. for example, @xcite and @xcite have shown that emerging representations from neural models trained on visual recognition tasks are predictive of human similarity judgments. for auditory recognition, neural speech recognition models have been shown to capture human-like behavior in cross-lingual phonetic perception @xcite . following the same spirit, our objective is to the investigate the extent to which neural models of spoken language identification capture language similarity. nevertheless, and because of the complex space in which language variation can be realized, the similarity between two languages is a multidimensional phenomenon that cannot be expressed in a single number @xcite . we therefore do not consider a single reference as a "ground truth" in our analysis, but consider several reference criteria of distance including genetic, geographic, and perceptual distance. our representation similarity analysis shows that emerging representations in our spoken lid model capture language similarity. the representation visualization illustrated in fig. 2 demonstrates the gen-eralization ability of our model to project speech segments of non-observed (held-out) languages into subspaces of their respective subgroups. given that the data in our study constitute contemporary realizations of slavic speech that do not explicitly encode diachronic sound changes, we first hypothesized that the geographic distances between the linguistic communities would be a good predictor of the distances in the representation space of the lid model. this turns out to indeed be the case as we observe a high positive correlation between geographic distances and cosine distances within prototype language representations. on the other hand, we were less optimistic about our lid model capturing the genetic signal between slavic languages (that is, whether the language representations encode the historical relationships between languages). earlier works that have investigated computational approaches to generating genetic language trees have either employed historical etymological data capturing phonological sound changes @xcite , sequences reflecting syntactic patterns in different languages @xcite , or word lists reflecting lexical similarity @xcite . arguably, these sources of language data are more likely to preserve the relationship between languages across the temporal dimension than the contemporary slavic speech we use in this study. therefore, our initial intuition was that the resulting tree would reflect variation across the spatial dimension and not the temporal dimension. nevertheless, the tree generated by our analysis is an adequate approximation of the slavic genetic tree given the contemporary nature of the data sources. finally, it is striking to observe the similarity between our generated tree and the slavic subtree constructed from player confusion patterns of glg participants @xcite . there are two main factors that contribute to this similarity of our analysis to that of @xcite : (1) west-slavic and east-slavic branches are clustered together before joining the south-slavic branch to form a single slavic cluster, and (2) the deviations from the widely accepted slavic grouping are present in both analyses at approximately the same node locations since bulgarian is grouped with east-slavic and slovak is grouped with south-slavic. these findings provide evidence that the perceived language similarity is the factor that can best predict the geometric distance between languages in the emerging representation space from neural models of spoken language identification. ## conclusion we have presented a convolutional neural model for slavic language identification in speech signals and analyzed the extent to which its emerging representations reflect language similarity. our analysis has shown that the distances in the emergent language representations reflect language variation across the temporal (phylogenetic) and spatial (geographic) dimensions. moreover, by comparing our clustering analysis to the confusion patterns of the great language game participants, we have shown that perceptual confusability is a better predictor of language representation similarities than phylogenetic and geographic distances.