Developing finite-state language technology for M aya (2023)

Abstract: We describe a suite of finite-state language technologies for Maya, a Mayan language spoken in Mexico. At the core is a computational model of Maya morphology and phonology using a finite-state transducer. This model results in a morphological analyzer and a morphologically-informed spell-checker. All of these technologies are designed for use as both a pedagogical reading/writing aid for L2 learners and as a general language processing tool capable of supporting much of the natural variation in written Maya. We discuss the relevant features of Maya morphosyntax and orthography, and then outline the implementation details of the analyzer. To conclude, we present a longer-term vision for these tools and their use by both native speakers and learners.

## introduction maya 2 is a member of the yucatecan branch of the mayan language family (figure 2 3 ). it is the second most widely-spoken indigenous language of mexico, with around 800,000 speakers primarily in the states of yucatan, quintana roo, and campeche in southern mexico (collin, 2010) (see figure 1 4 ), including a substantial speaker population in california @xcite and a modest population in belize. ## related work the use of finite-state transducers (fsts) for modeling human language has a long tradition spanning multiple decades @xcite and proving effective in areas such as morphological analysis (beesley and @xcite , spell-checking @xcite , among others. it is particularly attractive in the low-resource case since it requires significantly less data than popular statistical approaches. furthermore, finite-state systems can also be leveraged in order to generate data to train better statistical machine-learning models @xcite . the application of finite-state language technology to indigenous languages of mesoamerica also has some precedent, with morphological analyzers developed for nahuatl @xcite @xcite , zapotec @xcite , huave @xcite , and k'iche' @xcite . @xcite present the large-scale development of morphological analyzers and generators for over one thousand languages using the johns hopkins university bible corpus @xcite , including some mayan languages. @xcite is perhaps one of the earliest published works focused on the development and application of language technology to assist in documenting a mayan language, q'anjob'al (spoken in guatemala), training a maximum-entropy part-of-speech tagger. @xcite and @xcite also apply techniques from machine learning and computational linguistics to the documentation of a mayan language (uspanteko, also spoken in guatemala). more recently, @xcite and @xcite developed an annotated linguistic corpus of k'iche' and explored approaches to automated tagging and parsing. maya is also included as one of six mexican languages aligned with spanish in the parallel corpus for mexican languages @xcite . there has also been interest and some work leveraging computational technology to annotate and analyze classic maya heiroglyphic writing @xcite . particularly relevant to motivation and aims of the present project, @xcite , outlines useful applications of computational morphological analyzers for learners of morphologically-rich indigenous languages of the americas. ## orthography the latin alphabet has been used to write yucatec maya since the 16 th century, but the first organized efforts to standardize the orthography took place in the mid-20 th century @xcite . the colonial-era writing practices are described thoroughly in @xcite , and a variant of this orthographic approach is also used in @xcite . many linguistic resources for maya also use an orthography inspired by the americanist phonetic alphabet @xcite , (e.g. using p for the glottal stop). today, the commonly (though by no means unanimously) adopted "contemporary orthography" is laid out in the publication normas de escritura para la lengua maya @xcite . in the classroom, osea teaches a writing system similar to the contemporary one, with a few pedagogically-motivated changes, like the explicit marking of low tone on long low vowels. additionally, there are some differences related to the spelling of specific words. in order to offer students a consistent source for spelling questions (primarily with respect to vowel quantity and tone), osea uses bricker et al. (1998) as an authoritative reference. this is not to say that alternative spellings are incorrect from osea's perspective, but rather that it is valuable for students to have a thorough and consistent guide to reference when making spelling decisions foot_0 . since the project presented here is intended to be used by students and teachers in the osea maya language program, we follow these orthographic norms while still supporting both the colonial and contemporary orthographies. details about this are provided in section 6.5. ## a brief overview of maya morphosyntax an important linguistic property of maya worth mentioning at the outset is that it does not have tenses, per se. instead, it inflects verbs for aspect to reflect whether a given action has been completed, or how long ago it began @xcite . details about this system are explored in greater depth in section 4.2. maya is a split-ergative language, i.e. it follows ergative-absolutive alignment in all but the imperfective aspect, where it follows nominativeaccusative alignment. as will become obvious in the discussion below, maya has a complex derivational system. most word classes can be derived from other word classes, and the transitivity and voice of a verb is derived morphologically as well. ## data for development, we use a small corpus consisting primarily of pedagogical texts used in the classroom by osea. they include lists of sentences and a number of tsikbalo'ob (dialogues). we also include four short stories from @xcite , for which we changed the orthography to reflect the writing norms of osea-cite (with permission from the author). sentence and token counts are listed in ## implementation the morphological analyzer is developed within the apertium project @xcite , and is made up of three principle components: a model of maya morphotactics, a model of phonological processes, and an analysis disambiguation step. a sample of the type of analysis that is produced can be seen in one major advantage of using the apertium platform is that a single morphological model can trivially be extended to additional applications, such as spell-checking and machine translation. here, we describe the development of the morphological analyzer, and briefly discuss a spell-checking application prototype. aspect/mood trans. intr. positional aps derived trans pss imperfective -ik -vl/ø -t-al ø -a'al perfective -ah ø -l-ah -nah -a'ab present perfect -m-ah -a'an -l-aha'an -naha'an -a'an subjunctive -ø/-eh -vk -l-ak -nak -a'ak trans. and intr. refer to transitive and intransitive root verbs, "aps" = antipassive, and "derived trans pss" refers to intransitive roots that are transitivized and then passivized (e.g. hóoken "i went out" →a hóoksahen "you took me out" →hóoksa'aben "i was taken out.") ## coverage on our modest-sized corpus, the morphological analyzer's coverage is about 96% on tokens and 85% on types. of the forms currently not covered by the analyzer, many are interjections that may be author-specific (e.g. "kikiriki", the sound of a rooster crowing), and foreign loans (e.g. "cinco", "greedy"). currently, all of the missed words by our analyzer are hapax legomena. ## concluding remarks and future work we have described in detail a finite-state morphological analyzer for maya, and demonstrated its utility outside of merely performing morphologi-n coverage (%) tokens 3,033 96 types 734 85 for the near future, our first priority is growing the corpus. we are in the process of normalizing the orthography for a number of additional texts which we will then add and use to update the analyzer lexicon. outside of simply improving the vocabulary and coverage of the analyzer, we plan to explore the numerous ways this tool can be of use to students by incorporating it into a browser-extension that aids the user's understanding of maya texts read in the browser. we also hope to improve the spell-checker by adding a better-informed error model that takes into consideration common spelling mistakes. adding support for the spell-checker in other popular word processors is a longer-term goal, as this would greatly improve accessibility of the tool for teachers and students.