Investigating variation in written forms of N ahuatl using character-based language models (2021)

Abstract: We describe experiments with character-based language modeling for written variants of Nahuatl. Using a standard LSTM model and publicly available Bible translations, we explore how character language models can be applied to the tasks of estimating mutual intelligibility, identifying genetic similarity, and distinguishing written variants. We demonstrate that these simple language models are able to capture similarities and differences that have been described in the linguistic literature.

## introduction the diversity of language variants foot_1 in a linguistic continuum presents an interesting challenge to the development of language technology. for marginalized and endangered languages, the general lack of resources in the language as a whole exacerbates this challenge. character-level features have been shown to be effective for a wide range of textual nlp tasks, including language identification @xcite , native language detection @xcite , and machine translation @xcite . furthermore, they offer the advantage of requiring little-to-no preprocessing or linguistic engineering (e.g. word tokenization, morphological segmentation, etc.) other than possibly orthographic normalization 3 . in this paper we investigate the usefulness of character language models in addressing questions about variation within a linguistic continuum. specifically, we examine the extent to which these sim-ple surface-level features of written language correspond to structural phonological and grammatical differences between different variants of nahuatl. we examine three tasks: variant identification, linguistic sub-classification/genetic similarity, and the prediction of mutual intelligibility. ## background in this section we give some background on the language, language modeling, and some relevant related work. ## data the most widely available corpus of text in the variants of nahuatl is the bible. we used translations into 10 different nahuatl variants available from scriptureearth.org foot_4 . the complete list of variants employed in this study is: azz highland puebla, ngu guerrero, nch central huasteca, nhe eastern huasteca, nhy northern oaxaca, ncj northern puebla, nhi western sierra, nsu sierra negra, ncl michoacán, nhw western huasteca. as translators merge verses differently in different languages, to maintain data parity for all of the variants being investigated we only included verses which were present in all variants (7,363 verses). ## methods in order to analyze the three case studies described below, we evaluated the cross-variant perplexity of character language models for each nahuatl variant in our corpus. specifically, we split the data by verse into train (6,258 verses), dev (552 verses), and test (552 verses) partitions. for each variant, we trained a character language model on the training data for 50 epochs (this was manually verified to be sufficient for convergence). the epoch with the lowest perplexity on the dev set was selected, and the perplexity of the model at that epoch on the test set was calculated for all variants. we used pytorch @xcite to train a unidirectional lstm language model with 100-dimension character embeddings (with dropout) and a single recurrent layer with 1024 hidden units. ## case studies in this section we present three case studies using character-based language models. ## concluding remarks our three case studies suggest that a simple character language model can capture a non-trivial amount of information about some of the linguistic properties, relationships, and similarities of written nahuatl variants. the experiments also support existing literature on the utility of character features in the computational modeling of similar languages. we note the limitations of our data set, i.e. that each variant is represented by a parallel text published by the same organization (and likely by a single author per variant), and that our approach may not yield similar results on non-parallel or comparable text. we are also interested in exploring language models under various tokenization schemes, such as unsupervised subword tokenization and morphological segmentation. 8 measurements reported with less than 5 speakers were excluded. two of the measurements, nhi-nsu and nhi-ncj, were reported as "50-60%" in ethnologue. for these data points we used 55%.