Generalized Glossing Guidelines: An Explicit, Human- and Machine-Readable, Item-and-Process Convention for Morphological Annotation (2023)

Abstract: Interlinear glossing provides a vital type of morphosyntactic annotation, both for linguists and language revitalists, and numerous conventions exist for representing it formally and computationally. Some of these formats are human readable; others are machine readable. Some are easy to edit with general-purpose tools. Few represent non-concatentative processes like infixation, reduplication, mutation, truncation, and tonal overwriting in a consistent and formally rigorous way (on par with affixation). We propose an annotation conventionâ€”Generalized Glossing Guidelines (GGG) that combines all of these positive properties using an Item-and-Process (IP) framework. We describe the format, demonstrate its linguistic adequacy, and compare it with two other interlinear glossed text annotation schemes.

## introduction as part of the ongoing wav2gloss project, we are generating interlinear glossed text (igt) from speech using an end-to-end system. in producing igt for various languages of the americas, we encountered a challenge: traditional interlinear glossing schemes are well-suited for the representation of concatenative morphology @xcite where morphological properties are realized by spans of phonological material @xcite . however, the languages that we are working with-mixtec, nahuatl, and totonac-are permeated by morphological operations such as trun-cation, tonal overwriting, reduplication, apophony, and segmental overwriting, that cannot be adequately expressed as the concatenation (or even interleaving) of strings. the shortcoming of most igt notations is that they represent the alignment of affixes with glosses, but they do not explicitly show how non-concatenative processes align to glosses. the contrast between concatenative and nonconcatenative "models of grammatical description" goes back at least to a seminal article by charles hockett @xcite in which he observed that morphology can be viewed as the concatenation of morphemes (item-and-arrangement or ia) or as the application of processes to morphemes (item-andprocess or ip). whatever their ontological nature may be, some morphological operations-for example, apophony and truncation-are more easily expressed as processes than morphemes. in order to gloss these operations (and give them the same status as affixation), we needed to develop an annotation scheme more general than those currently available. therefore, we propose generalized glossing guidelines (or ggg), that build upon existing conventions such as the leipzig glossing rules @xcite but make the framework formally explicit and add consistent and comprehensive support for non-concatenative morphological alternations such as infixation, reduplication, transfixation, apophony, tonal overwriting, and truncation. figure 1 gives an example of ggg from yoloxóchitl mixtec. it shows metadata as well as the four fields, sr (surface representation), lx (lexical representation), gl (gloss), and tr (translation). it shows tonal overwriting in curly brackets, with cliticization shown by =. ## background a large number of glossing conventions, from the very formal (e.g., xigt; @xcite to the relatively informal (e.g., the leipzig glossing rules or @xcite have been proposed and employed in computational applications. for example, a recent sigmorphon shared task on glossing used representations based on lgr. 1 these conventions play two roles: (1) they allow linguists and language workers to communicate with one another with clarity and minimal am-biguity; (2) they allow humans and computers to communicate with one another with respect to the morphosyntax of human languages. in our usecase, they allow neural models to communicate the details of their morphosyntactic analyses to language workers. as such, these annotation conventions need to be both human readable (whether directly or through some kind of user interface) and expressive, without sacrificing explicitness. although lgr largely satisfies these critera when only concatenative morphology occurs, nonconcatenative operations are only supported in a limited and sometimes inexplicit way in this convention. the following example shows the lgr notation for apophony (umlaut) in german: the sequence "\pl" indicates that plural is marked by a non-concatenative process (in this case, apophony), but it does not index the morphological property to a specific formal change. in the generalized glossing guidelines described here, the same example would be the following: lgr also has conventions for annotating reduplication and infixation, but each of these notations is different. compare these examples from motu: compare the following well-known example of infixation in tagalog: in lgr (4a), infixes are surrounded by angle brackets. in the ggg version (4b), infixes are indicated with the same notation as reduplication and all other processes. maximal empirical coverage is achieved with minimal formal equipment. another important framework for representing igt (and morphosyntactic annotations, generally) is xigt @xcite , an xml-based format that associates annotations with spans. it, too, is highly general, machine-readable, and formally rigorous, but its opaque structure makes it difficult to read and write without special software tools. we propose ggg to take the best of the both frameworks. it has the following properties: ## the guidelines ggg attempts to represent igt examples like those in the preceding section in a yaml format, foot_3 preserving to the degree possible the conventions that are present when linguists typeset linguistic data for the consumption of other linguists. this allies it with the sil shoebox format and differentiates it from xigt @xcite and other highly explicit igt formats. this also makes it relatively easy to edit ggg text using off-the-shelf tools (e.g., text editors and transcription tools). ## linguistic adequacy the adequacy of ggg for annotating concatenative morphology is identical to that of lgr, since the mechanism is borrowed from lgr directly. the only modification is that morphemes within a word are divided by spaces in addition to hyphens and equal signs. this means that the headedness of compounds must be stated explicitly (with dependents treated like affixes). the ggg approach, however, has a distinct advantage in the treatment of non-concatenative morphology, as it is able to achieve complete adequacy (though not theoretical correctness or depth of generalization) through the use of a single annotation mechanism: {a?>b?(;c)?}. we show that the convention works well for infixation, reduplication, truncation, apophony, tonal overwriting, segmental overwriting, transfixation, and other similar processes. ## conclusions as should be clear from one cost, because of its explicitness, is that ggg annotation cannot be completed until a linguist has a thorough (though fundamental) analysis of a language's morphology. our goal is to develop tools to facilitate this analysis: to go from basic recordings to interlinear annotations with reduced human intervention. we hope that ggg will be an important part of this ongoing work. but the benefits are great. we are currently using ggg with great success in our ongoing research and hope that other investigators will find it similarly useful.