Cross-lingual annotation: a road map for low- and no-resource languages (2020)

Abstract: This paper presents a “road map” for the annotation of semantic categories in typologically diverse languages, with potentially few linguistic resources, and often no existing computational resources. Past semantic annotation efforts have focused largely on high-resource languages, or relatively low-resource languages with a large number of native speakers. However, there are certain typological traits, namely the synthesis of multiple concepts into a single word, that are more common in languages with a smaller speech community. For example, what is expressed as a sentence in a more analytic language like English, may be expressed as a single word in a more synthetic language like Arapaho. This paper proposes solutions for annotating analytic and synthetic languages in a comparable way based on existing typological research, and introduces a road map for the annotation of languages with a dearth of resources.

## introduction: cross-linguistically informed semantic annotation in recent years, there has been a surge of interest in annotation schemes that allow natural language texts to be parsed into semantic representations usable for information extraction, machine translation, and other downstream purposes. good results have been achieved in automatically parsing natural language texts into abstract meaning representations @xcite and discourse representation structures @xcite , among others, as demonstrated in various shared annotation tasks @xcite . however, most efforts in developing and testing such annotation schemes have focused on a restricted set of (typically indo-european) languages with large native speaker populations. for example, the shared annotation tasks reported on in @xcite and @xcite were all based on english. large amr corpora exist, to our knowledge, only for english and mandarin -both morphologically isolating languages, with comparatively little inflectional and derivational morphology. propbank @xcite has been extended to large-scale languages with derivational morphology, such as hindi and arabic. but the annotation of derivational morphology relies on a thorough documentation of its role in the language, which often isn't available for low-resource languages. for morphosyntactic annotation, this bias is less apparent: the universal dependencies project @xcite has annotated treebanks from 96 languages, representing 20 linguistic families. however, indo-european languages are disproportionately represented (53/96 languages). in terms of native speaker populations, 62 out of the 96 ud languages have more than 1 million native speakers, belonging to the largest 6% of languages in the world @xcite . only 24 ud languages have relatively small native speaker populations (10 ud languages are ancient languages). this apparent bias in languages represented in computational linguistic work likely has consequences for the structure of annotation schemes. the world atlas of language structures chapter on the morphological structure of verbs @xcite looks at a sample of 145 languages and finds that, on average, languages express 5.52 inflectional categories within the verb. the 19 ud languages with more than 10 million native speakers that are also included in @xcite sample express only an average of 4 inflectional categories within the verb -exemplifying a known correlation between morphological complexity and demographic factors @xcite . the annotation schemes developed in the context of high-population, typically indo-european languages, may therefore not carry over well to smaller-scale, often more morphologically complex languages. many smaller-scale languages do not have a long history of linguistic analysis, and therefore understanding of their structure may be progressing in tandem with annotation efforts. expanding annotation efforts to such morphosyntactically diverse languages may, apart from simply expanding typologically sound coverage of annotation efforts, improve the overall utility of annotation schemes. cross-linguistic annotation schemes must incorporate a certain amount of flexibility in order to deal with differences in conventionalized semantic distinctions, and the morphosyntactic expression of these distinctions. this flexibility in design can also benefit monolingual annotation, by allowing for flexibility with annotators who have different levels of linguistic training. for that reason, this paper proposes solutions for extending amr to as many languages as possible, including a "road map" for languages with few existing resources. this provides a starting point for flexible but consistent annotation of a number of semantic categories. it also describes how the cross-linguistic diversity in morphosyntax, paired with pre-existing linguistic analyses and resources, can inform the design of a flexible annotation process. this road map provides steps towards more detailed semantic annotation, as the linguistic analysis of the language progresses and computational resources are created, in order to eventually arrive at the same level of specificity in annotations as in high-resource languages. the creation of a comparable cross-linguistic semantic annotation scheme is of course a larger topic than can be covered in a single paper; this paper sets forth a general approach for dealing with differences in linguistic properties and resource availability (the road map), and specific annotation solutions for certain semantic categories and morphosyntactic phenomena (e.g., synthesis). in this paper, examples are drawn from three no-or low-resource languages: sanapaná, kukama, and arapaho. sanapaná (enlhet-enenlhet) has about 1000 native speakers living in paraguay. aside from an ongoing documentation project (van gysel, 2020), there are only exploratory analyses of the morphology @xcite . kukama (tupian) has about 1000 native speakers living in peru. existing linguistic resources include a descriptive grammar @xcite , a kukama/spanish bilingual dictionary @xcite , translated and morphologically analyzed texts @xcite , and some pedagogical materials. arapaho (algonquian) is spoken by two communities in the united states, the northern arapaho in wyoming and the southern arapaho in oklahoma. among the northern arapaho, there are around a hundred native speakers, and several hundred with passive knowledge of the language. linguistic resources include a grammar @xcite , an online lexical database with detailed part-of-speech labelling and argument structure information, and an annotated text database of nearly 100,000 sentences with accompanying audio and/or video. ## cross-linguistic annotation: typological issues certain typological issues arise when constructing a semantic annotation scheme that can, in theory, be applied to any language. three general types of issues are described here. first, some types of morphosyntactic differences do not hinder the annotation of semantic information, and can therefore largely be ignored in a semantic annotation scheme. for example, languages may indicate grammatical roles via constituent order or case affixation of argument phrases, but argument phrases in both types of languages can be annotated for their semantic roles in the same way. next, there are major typological differences in the conventionalized semantic distinctions that languages make in their grammar, i.e. how languages 'carve up' conceptual space. for example, some languages distinguish only singular from non-singular nominal number, other languages distinguish singular, dual, trial, and plural (more than three), still other languages have a few (including singular) vs. many nominal number system (corbett, 2000, chapter 2). for these types of semantic differences, the use of lattices of category values has been proposed to allow flexible but consistent annotation @xcite ; we adopt this approach and incorporate it into the road map in §3. finally, languages differ in terms of how concepts are packaged into words and sentences. as discussed in §1, languages that are more synthetic, packaging many morphemes/concepts into a single 'multiconcept' word, have not been well-represented in past annotation efforts. this also presents a practical issue: for languages at an earlier stage of documentation, it may not be possible for annotators to morphologically decompose multiconcept words. therefore, the issue of how to maintain consistent annotation across both more analytic and more synthetic languages will be the main focus of this section. even 'word' does not have a consistent definition across languages. most languages have a languageinternal concept of 'word', at least as a cognitively salient unit of the language @xcite . but these units do not share consistent linguistic traits across languages, nor is there a widely-accepted definition of what should constitute a word across languages @xcite , but see zingler (2020). ## the road map section 2 covered solutions to typological issues that are raised by the inclusion of low-and no-resource languages in semantic annotation efforts. this section puts forth a "road map" approach to annotation, which synthesizes the typological solutions with practical solutions for the inclusion of languages with few existing computational or documentary resources. stage 0 stage 1 annotation targets treat derivational morphology as single word, separate inflectional morphology indicate derivational morphological relations in lexicon aspect coarse-grained categories on lattice fine-grained categories on lattice modal strength annotate with only modstr and placeholder values; no conceivers use modal lexicon with modal strengths, fill in remaining unspecified modal strengths participant roles general semantic roles lexicalized roles, annotate implicit roles the road map approach both ensures comparability across diverse languages, and allows for flexibility in the annotation of any one language. the road map specifies a starting point for languages with few resources (stage 0), the end point for fully specified annotation (stage 1), and a process for moving between these, defined for each annotation category. these are not discrete annotation stages, and languages will move gradually from the stage 0 to stage 1 annotation. where a language begins on the road map for each annotation category is determined by the typological features of its grammar, its state of documentation, and the computational resources developed thus far. the road map allows for flexibility across languages and annotation categories. languages with a paucity of linguistic or computational resources can still begin annotation efforts. languages with typological features that complicate the annotation of certain semantic categories can still be annotated for those categories, albeit at a less detailed level. within a language, different annotation categories may be annotated at different stages, depending on the language's typological features and existing resources. the road map approach also ensures comparability across languages, even when languages are at different stages, because annotation values retain their meaning across the road map stages. this also ensures that different-stage annotations for the same language are compatible. as annotation and documentation efforts continue, the annotation of a language may progress along the road map. but, the annotations done at the beginning stages are still accessible and comparable to the later stage annotations. for languages that have limited resources in terms of time investment by speakers and/or field linguists, having this type of compatibility built into the annotation scheme is critical. the remainder of this section will demonstrate how the road map approach functions with regard to a number of annotation categories: annotation targets, participant roles, aspect, and modal strength and polarity. the road map for these categories is summarized in ## conclusion this paper recognizes issues not previously dealt with in the annotation of cross-linguistic semantic information: multiconcept words and no-resource languages. as multiconcept words are more common in languages with a smaller speech community, they have not been dealt with in past annotation schemes. we present solutions for extending amr across languages, including the annotation of multiconcept words; these depend on the semantic category of the concept. we have also outlined a road map approach to beginning annotation on very low or no-resource languages, ensuring that the annotation is truly crosslinguistic in terms not only of typological diversity but of resource availability as well.