Expanding U niversal D ependencies for Polysynthetic Languages: A Case of S t. L awrence I sland Y upik (2021)

Abstract: This paper describes the development of the first Universal Dependencies (UD) treebank for St. Lawrence Island Yupik, an endangered language spoken in the Bering Strait region. While the UD guidelines provided a general framework for our annotations, language-specific decisions were made necessary by the rich morphology of the polysynthetic language. Most notably, we annotated a corpus at the morpheme level as well as the word level. The morpheme level annotation was conducted using an existing morphological analyzer and manual disambiguation. By comparing the two resulting annotation schemes, we argue that morpheme-level annotation is essential for polysynthetic languages like St. Lawrence Island Yupik. Word-level annotation results in degenerate trees for some Yupik sentences and often fails to capture syntactic relations that can be manifested at the morpheme level. Dependency parsing experiments provide further support for morpheme-level annotation. Implications for UD annotation of other polysynthetic languages are discussed.

## introduction the universal dependencies (ud) project @xcite provides a cross-lingual syntactic dependency annotation scheme for many languages. the most recent release of the ud treebanks (version 2.7) contains 183 treebanks in 104 languages. however, polysynthetic languages, known for words synthesizing multiple morphemes, are still much under-represented in the ud treebanks. to our knowledge, abaza foot_0 and chukchi @xcite , are the only polysynthetic languages included in ud version 2.7. in this paper, we describe how we annotated a corpus of st. lawrence island yupik (also known as central siberian yupik), a polysynthetic language spoken in parts of alaska and chukotka, russia, within the framework of the ud guidelines. while ud is a framework for word-level annotations, we argue that morpheme-level annotations are more meaningful for polysynthetic languages. we provide morpheme-level annotations for yupik in addition to word-level annotations. 2 we believe that subword-level annotations can help better capture morphosyntactic relations for polysynthetic languages and assist further dependency annotations and morphosyntactic research for polysynthetic languages. previously tyers and mishchenkova (2020) called for the need to annotate parts of words in regard to noun incorporation in chukchi. they proposed annotating a noun incorporated into a verb via morphology as a separate token available in the enhanced dependency structure. while our approach is motivated by a similar need to annotate subword units for another polysynthetic language, our paper focuses on morpheme-level annotations, which may be applied to other types of multi-morphemic words than just noun incorporation. in what follows, we describe the characteristics of the yupik language ( §2) and show how we annotated a corpus at the morpheme level as well as the word level ( §3 and §4). then we present some language-specific decisions we made for morpheme-level annotations and illustrate yupik constructs captured by the new annotation scheme ( §5 and §6). we also compare the performance of the two annotation schemes in automatic parsing experiments ( §7). based on our findings, we conclude that the morpheme-level annotation is essential and effective for polysynthetic languages and discuss implications of the study for other polysyn-thetic languages and the ud framework ( §8 and §9). ## st. lawrence island yupik st. lawrence island yupik (iso 639-3 ess; yupik hereafter) is a polysynthetic language in the inuit-yupik language family, spoken in parts of alaska and chukotka, russia. like other polysynthetic languages, yupik is characterized by its rich morphology. @xcite provides the most thorough descriptions of the yupik grammar with an emphasis on the morphology. yupik is strictly suffixing with the exception of one prefix. yupik words typically have the following form: root (+ derivational morphemes) * + inflectional morpheme (+ enclitic) that is, a typical yupik word has a root, followed by zero or more derivational morphemes (thus forming a stem), followed by obligatory inflectional morpheme(s), finally followed by an optional enclitic. most roots are nominal or verbal, such as mangteghagh-'house' and negh-'to eat' respectively. the language also includes a set of non-inflecting particles, such as quunpeng 'always' or unaami 'tomorrow'. yupik derivational morphology is highly productive; words with up to seven derivational morphemes have been attested (de reuse, 1994, p.53), and words with 1-3 derivational morphemes are very common. the badten et al. (2008) yupik-english dictionary and the chen et al. (2020) yupik finite-state morphological analyzer document about 400 derivational suffixes: • 81 noun-elaborating suffixes (@xmath0. ## morpheme-level dependency relations the ud annotation guidelines are lexicalist @xcite in nature, specifying that syntax dependencies should be annotated at the word level, such that both the head and the child of each dependency relation are words @xcite . in (3), we see the yupik sentence from (1) with dependency relations annotated at the word level, following the ud guidelines. the resulting dependency tree successfully depicts the core syntactic information in the yupik sentence, with the intransitive verb aanut at the root of the dependency tree, with a nominal subject and an oblique argument as children. however, when we annotate the singleword yupik sentence from (2) according to the ud annotation guidelines, the result is a degenerate tree that completely fails to capture any syntactic information about the yupik sentence. in order to adequately represent the syntactic relations in (2), it is necessary to discard the lexicalist hypothesis and annotate relations between morphemes rather than between words. when we contrast (4) with (5), we observe that annotating relations at the morpheme level results in a meaningful linguistic analysis for this yupik sentence. it is clear from these two dependency trees that treating morphemes as the basic unit of syntactic dependency relations is necessary in order to adequately encode the syntax of the yupik sentence in (2). by doing so, we move from a degenerate tree devoid of syntactic information to a tree that successfully encodes a main verb -yug-('to want to') with a complement -ngllagh-('to make'), and an object mangtegha-('house') with a nominal modifier -ghlla-('big'); the inflectional suffixes encode the number and person of the subject (1pl, 'we') and the main verb's mood and valency (ind.intr). in (6) we observe a more complex yupik sentence; we see the sentence taaghtam aghnamun qayunghitesqaa kufi ('the doctor prevented the woman from drinking the coffee') annotated in (7) with dependency relations between words. the resulting dependency tree fails to illustrate the complex verbal structure of the multi-morphemic third word qayunghitesqaa ('he told one not to drink it'); it is only in (8) when we annotate (6) with syntactic relations between morphemes that we are able to observe that aghnamun ('the woman') is the subject of the embedded verb qayu-('to drink') while taaghtam ('the doctor') is the subject of the main verb -sq-('to tell'). that is, parts of the yupik word, the main verb -sq-('to tell') and the embedded verb qayu-('to drink'), participate in different syntactic relations, which cannot be annotated at the word level. the necessity for this type of sub-word annotation is not unique to yupik; see @xcite for a discussion of subword syntactic units in turkish. if sentences that required morpheme-level dependency relations were rare, it might be reasonable to accept the inclusion of a few degenerate and under-annotated trees such as (4) and (7) in a yupik dependency treebank. however, yupik is polysynthetic, and multi-morphemic words involving complex derivation are very common; the same is true of all of the languages in the inuit-yupik language family. for the polysynthetic languages in this language family, there are simply too many sentences that require morpheme-level dependency annotations to annotate only dependency relations between words. in particular, essentially all words formed with derivational suffixes require morpheme-level dependency relations in order to satisfactorily encode the syntax of the sentence. in annotating yupik sentences with dependency relations, we therefore treat each yupik morpheme as a token rather than treating each yupik word as a token. this necessarily requires that yupik words be analyzed and segmented into morphemes prior to dependency annotation; this task was performed using the existing yupik finite-state morphological analyzer @xcite . in cases of ambiguity when the analyzer provided multiple possible analyses for a given word, we selected the gold analysis via manual disambiguation. we chose to represent all yupik morphemes as independent syntactic tokens, including inflectional morphemes. an alternative approach would be to instead not tokenize inflectional morphemes, but rather annotate inflectional information using feature values. a major benefit of our choice is greater compatibility with the existing yupik morphological analyzer @xcite , which treats inflectional morphemes as independent tokens in the underlying lexical form. because the ud annotation guidelines were not designed for morpheme-level annotation, some minor adaptations were required; we discuss these adaptations in §5 and §6 as we discuss the pos tags and dependency relations used in our corpus along with sample sentences. in order to enable the use of morphemes as tokens, we adapted the existing "multiword expressions" annotation mechanism. the ud annotation guidelines recognize that syntactic words do not always align perfectly with orthographic word boundaries; this can occur even in analytic languages such as english, for example, in words involving a clitic or a contraction. for example, in spanish, the word dámelo ('give it to me') may be broken down into dá me lo ('give me it') for the purpose of ud annotations; the annotation scheme records that the single orthographic token (dámelo) is annotated as multiple syntactic words, and that information can be used to collapse the annotations to the single orthographic token when needed. in our case, we treat each multi-morphemic yupik word as a ud "multiword expression," with yupik morphemes serving as the tokens within the "multiword expression." recognizing the ud project's lexicalist view of syntax, we provide a script to convert our morpheme-level annotations into word-level annotations. this script deterministically merges each multi-morphemic word into a single word token using udapi @xcite . because our morpheme-level annotation does not strictly follow the entirety of the ud guidelines, a small number of sentences had to be manually corrected after the conversion. we plan to release our morpheme-level annotation in ud version 2.8 along with descriptions of the conversion process from the morphemelevel annotations to the word-level annotations. ## corpus the annotated corpus is comprised of exercise sentences from the yupik reference grammar @xcite . the grammar book, designed to teach yupik at the college level, provides end-of-chapter exercises with sample yupik sentences. morphological segmentation and analyses were performed using the chen et al. (2020) yupik morphological analyzer and manually verified when needed. ## pos tags we annotated our yupik corpus using the tags shown in qikmi--lgu--yug--tu--nga noun verb verb x x dog to.have to.want.to ind.intr 1sg object xcomp root dep:infl dep:infl we tagged nominals and nominal bases as noun and verbals and verbal bases as verb. we tagged derivational suffixes that yield nominal stems (@xmath1, @xmath2, @xmath3. for example, (9) shows the morpheme-level annotation for the word qikmilguyugtunga 'i want to have a dog'. in the annotation, the nominal root qikmi-'dog' combines with a verbalizing derivational suffix (-lgu-'to have', @xmath4. then this extended base combines with the verb-elaborating suffix (-yug-'to want to', @xmath5, which is followed by inflection. the two verb-yielding derivational suffixes are tagged as verb. uninflected words or particles were given the particle tag (part). many yupik particles are borrowed from chukchi, a geographically neighboring language, and are mostly adverbial or connective in meaning (de @xcite . examples include ighivgaq 'yesterday' and qayughllak 'because'. the two additional pos tags available only at the morpheme level were x and cconj . the pos tag x is reserved for words that are outside of pos tags defined within the ud framework. we used the x tag for inflectional suffixes such as -tuand -nga as in ( 9 ). coordinating conjunctions (cconj) were only found at the morpheme level because they are only expressed as an enclitic in the language: =llu 'and' as in ( 10 ). our morpheme annotation scheme makes use of 25 types of dependency relations while our word annotation scheme makes use of 14 dependency relations. in general, we followed the ud annotation guidelines, except in cases where polysynthetic nature of yupik made divergence from the guidelines necessary. the full documentation on pos tags, morphological features, and dependency relations used in the treebank is available at the language's ud documentation page. 4 the most notable difference between the two annotation schemes is the dep relation. within the ud framework, the dep relation is reserved for unspecified relations. because morpheme-level annotations require multiple dependency relations specified for subword units, we created a few dependency relations under the dep relation for the morpheme-level annotation only. note that some relations that are commonly annotated at the word level for other languages (e.g. auxiliary, copula) are only available at the morpheme level in yupik. when we can, we expanded existing relations, defined at the word level, to morphemes (e.g. nmod for nominal modifier). whenever that was not possible, we created a version of the corresponding dependency relation in our morpheme annotation scheme. for example, we used dep:aux for verbelaborating (@xmath6. for example, the @xmath7. similarly, we included the dep:mark relation to represent the marker (mark) relation at the morpheme level. in (12) we observe a word that acts as a subordinate clause in a sentence and is roughly translated as 'in order to see them'. the second morpheme of the word -namarks the word as a subordinate clause to the main verb, a mark relation in the word level ud annotation. 5 again, because of some limitations of using this relation at the morpheme level, we created the dep:mark relation for morpheme-level anntoations. on a similar note, the dep:cop relation was added to represent the copula (cop) relation at the morpheme level. in (13), the verbalizing (@xmath8, turning the nominal base as a verbal stem, which combines with the inflection to form a verbal word meaning 'it is a land' in the sentence meaning 'chaplino is a land'. the dep:infl was used for the relation between the stem and its inflectional suffix as shown in ( 13 ). because all yupik words other than particles require one or more inflectional morphemes, the dep:infl relation was the most frequently used in the morpheme-level annotation. in general, morpheme-level annotation was needed to capture some of important morphosyntactic relations present in yupik words. the aux and cop relations are only available at the morpheme level in yupik. while a small number of particles act as marker, the mark relation was also primarily attributed to derivational suffixes. when annotating yupik sentences at the word level, such dependency relations are lost. only when we annotate at the morpheme level can we find such constructions, which may be invaluable in subsequent linguistic inquiries or computational applications alike. ## parsing experiments in order to investigate the practical usage of the annotations, we conducted automatic parsing experiments using udpipe 1.2 @xcite and udpipe 2.0 @xcite . the udpipe project foot_5 provides a trainable pipeline for any ud treebanks in the conll-u format. ## discussion the universal dependencies project is intended as a de-facto standard for consistent dependency syntax annotations across all of the world's languages @xcite . our attempt to construct a ud corpus of yupik can be viewed as a kind of stress test for the ud annotation project. if the ud guidelines truly are universal in nature, then it should be possible to construct dependency trees for yupik while fully following the ud guidelines; to the extent that this is not possible, any such disconnect may serve to illuminate ways in which the ud guidelines might be improved upon in order to be more language universal. one of the core assumptions of the ud guidelines is lexicalism, the assumption that the fundamental token of syntax should be the word. this assumption has been widely adopted in many syntactic formalisms, including the lexical-functional grammar theory of syntax that ud in part draws upon. it has, however, been widely debated (for a thorough recent critique of lexicalism, see @xcite , and other theories such as distributed morphology @xcite explicitly reject the lexicalist hypothesis, asserting that large parts of morphology and syntax operate using a common hierarchical mechanism. the ud guidelines already explicitly recognize that phonological and orthographic boundaries do not always coincide with syntactic words. @xcite recognize that clitics act as words from the viewpoint of syntax, even though phonologically (and orthographically) they must attach to a host word; as such in ud annotations clitics are treated as independent syntactic tokens. similarly, the ud annotation guidelines recognize that contractions should be treated as the combination of two independent syntactic tokens. finally, the ud guidelines recognize that some larger units such the english expression in spite of act syntactically as a single token. however, the existing ud guidelines indicate that derivational morphemes should not be treated as syntactic words for the purposes of dependency annotation. for example, in an english dependency tree, the word dancer would be treated as a single syntactic token, rather than as two (verbal root dance-+ nominalizing suffix -er). in this paper, we have observed that this approach to derivational morphology fails when applied to yupik. the languages in the inuit-yupik language family are polysynthetic and rely heavily on productive derivational morphology. st. lawrence island yupik has around 400 derivational suffixes, around half of which are verb-elaborating (@xmath9. it is essentially impossible to adequately annotate the syntax of yupik sentences without recognizing that significant parts of yupik grammar are handled by yupik derivational morphology. in this paper, we have chosen to treat every yupik morpheme (both derivational and inflectional) as a syntactic token. in future work, it may be beneficial to build upon work by @xcite and treat only some derivational morphemes as syntactic tokens, while not tokenizing other derivational morphemes and perhaps all inflectional morphemes. at a minimum, this work shows that in order to be universal, the ud project must acknowledge that at least some derivational morphemes must be treated as syntactic tokens. ## conclusion this paper presents the first ud treebank for st. lawrence island yupik, the first ud treebank to be annotated at the morpheme level as well as the word level to our knowledge. the polysynthetic language has rich morphology, characterized by a theoretically unlimited number of possible derivations and multimorphemic words. in order to capture the morphosyntactic relations among morphemes, we annotated a corpus @xcite at the morpheme level and converted the morpheme-level annotations into word-level annotations. while the morpheme-level annotation may require more linguistic resources (e.g. morphological analyzer, morphological segmentation), it provides a deeper insight into the language and better automatic parsing performance. morpheme-level syntactic dependency annotation may be a better way to represent polysynthetic languages within the framework of ud. milan straka and jana straková. 2017. tokenizing, pos tagging, lemmatizing and parsing ud 2.0 with udpipe. in proceedings of the conll 2017 shared task: multilingual parsing from raw text to universal dependencies, pages 88-99, vancouver, canada. association for computational linguistics. francis m. tyers and karina mishchenkova. 2020. dependency annotation of noun incorporation in polysynthetic languages. in proceedings of the fourth workshop on universal dependencies (udw 2020), pages 195-204. daniel zeman, jan hajič, martin popel, martin potthast, milan straka, filip ginter, joakim nivre, and slav petrov. 2018. conll 2018 shared task: multilingual parsing from raw text to universal dependencies. in proceedings of the conll 2018 shared task: multilingual parsing from raw text to universal dependencies, pages 1-21, brussels, belgium. association for computational linguistics.