From Field Linguistics to NLP : Creating a curated dataset in Amuzgo language (2024)

Abstract: This article presents an ongoing research on one of the several native languages of the Americas: Amuzgo or jny’on3 nda3 . This language is spoken in Southern Mexico and belongs to the Otomanguean family. Although Amuzgo vitality is stable and there are some available resources, such as grammars, dictionaries, or literature, its digital inclusion is emerging (cf. Eberhard et al. (2024)). In this respect, here is described the creation of a curated dataset in Amuzgo. This resource is intended to contribute the development of tools for scarce resources languages by providing fine-grained linguistic information in different layers: From data collection with native speakers to data annotation. The dataset was built according to the following method: i) data collection in Amuzgo by means of linguistic fieldwork; ii) acoustic data processing; iii) data transcription; iv) glossing and translating data into Spanish; v) semiautomatic alignment of translations; and vi) data systematization. This resource is released as an open access dataset to foster the academic community to explore the richness of this language.

## introduction according to the facts reported in the survey analysis of the language technologies in mexico (astlm, 2018), latin america is a linguistic region with a minimum development in the creation of digital resources, specifically, regarding native languages. one of the main causes, pointed out by the authors, is the scarcity of data. the researchers frequently face a lack of materials to study these languages. for instance, several native languages have not even been described, either because they have not been considered academically, or because their grammar is difficult, or data collection is complicated or, even, highly risky. this stresses a gradual loss of worldviews, as well as an augment of the digital divide, which will di-rectly impact on the native communities by making inequalities and marginalization larger. with respect to the mexican context, specialized organisms such as the national institute of statistics and geography or the national institute of indigenous languages report an important linguistic wealth. according to the numbers registered in @xcite , apart from spanish, more than 60 native languages coexist in the country. this diversity is classified in 11 linguistic families, 68 languages, and 364 dialectal varieties. in terms of their vitality, the languages with more speakers are nahuatl (more than one million), mayan (around 800,000), mixtec, and zapotec (over 400,000 speakers each). on the contrary, there exist an important number of languages with fewer than 1,000 speakers @xcite . sadly, despite this enormous linguistic diversity, there is still a lack of resources, tools, and even linguistic materials for the majority of these languages (surprisingly, some of them well studied and described). amuzgo or jny ′ on 3 nda 3 is one of the languages cited in the previous reports. currently, the language has a couple of grammars and several studies about its varieties (see @xcite @xcite ). in accordance with these specialized works, amuzgo is quite different from the major language in the country (spanish), which makes it more complex to directly apply techniques or tools from other languages. in this respect, this work describes an ongoing research about the creation of a dataset in amuzgo. the dataset contains curated linguistic data collected from colloquial speech in amuzgo. these data are presented considering the following phases or levels: acoustic signal processing, transcription, glossed and human translation into spanish, semiautomatic alignment of human translations, and annotation. this dataset is intended to contribute the development of tools for scarce resources languages. the rest of the article is organized as follows: section 2 describes the main linguistic characteristics of amuzgo, focusing on their complexity. section 3 presents the method to create the dataset. section 4 points out some of the results and applications so far. finally, section 5 summarizes our findings and highlights the future work. ## language description amuzgo or jny ′ on 3 nda 3 is spoken in southern mexico. it has over 60,000 speakers, according to the data reported by @xcite . grammatically, the language belongs to the otomanguean family, alongside languages such as chatino, zapotec, mazatec, or popolaca. it is characterized by a wide set of personal pronouns @xcite , and lexical classes, which impacts on its verbal complexity @xcite . in addition, this is a language with five tones to mark change of meaning in different levels @xcite . for instance, examples (1) and ( 2 ) show how the different tones impact, not only on the meaning, but in the linguistic level as well. while in (1), the difference between the low level tone and the high level tone (superscripts 3 and 5, respectively) impacts on a change in the lexical level; i.e. the tonal contrast produces two different types of words, in (2), the change moves to the morphological level; i.e. the tonal change (rising tone, superscript 1, and mid level tone, superscript 4) generates a clear distinction with respect to who possesses the house. ## method in this section, we describe the method to build the dataset. specifically, we highlight how the dataset is curated by means of adding fine-grained linguistic information to the gathered data. the method consist of the following phases: i) collecting acoustic data from fieldwork, ii) acoustic data processing with praat, iii) data transcription, iv) data glossed and human translation, v) semiautomatic alignment of human translations, and vi) data systematization. ## result and projection the outcome of all the processes described so far is a dataset with fine-grained linguistic information in amuzgo. beyond the traditional linguistic analysis, this information could be used to foster and increase the efforts of the nlp community regarding scarce resources languages, such as jny ′ on 3 nda 3 . in addition, a small parallel corpus amuzgo-spanish has been generated to freely explore some characteristics of the language by using common corpus analyses, such as concordances, collocations, or keywords. as an optimistic projection, the creation of resources like this dataset would allow the experimentation in different areas, the generation of new knowledge, the preservation of endangered languages, and the minimization of the digital divide and its negative consequences for the native communities. ## conclusions in this article we have described the creation of a linguistic dataset in amuzgo. the dataset contains curated information provided by specialists in that language. such information covers different linguistic levels, such phonology, morphology, and syntax, as well as a human translation into spanish. in addition, the dataset was used to generate a small parallel corpus amuzgo-spanish by applying a known algorithm in nlp. both resources are freely available for academic purposes. although the materials exceed the 8 hours of recordings, the dataset here described contains only the data of one hour. therefore, as further work, it is planned to process the remaining data to enhance the linguistic description and, accordingly, to improve the dataset.