Automatic Extraction of Rules Governing Morphological Agreement (2020)

Abstract: Creating a descriptive grammar of a language is an indispensable step for language documentation and preservation. However, at the same time it is a tedious, time-consuming task. In this paper, we take steps towards automating this process by devising an automated framework for extracting a first-pass grammatical specification from raw text in a concise, human- and machine-readable format. We focus on extracting rules describing agreement, a morphosyntactic phenomenon at the core of the grammars of many of the world’s languages. We apply our framework to all languages included in the Universal Dependencies project, with promising results. Using cross-lingual transfer, even with no expert annotations in the language of interest, our framework extracts a grammatical specification which is nearly equivalent to those created with large amounts of gold-standard annotated data. We confirm this finding with human expert evaluations of the rules that our framework produces, which have an average accuracy of 78%. We release an interface demonstrating the extracted rules at https://neulab.github.io/lase/

## introduction while the languages of the world are amazingly diverse, one thing they share in common is their adherence to grammars -sets of morpho-syntactic rules specifying how to create sentences in the language. hence, an important step in the understanding and documentation of languages is the creation of a grammar sketch, a concise and human-readable description of the unique characteristics of that particular language (e.g. @xcite for en-glish, or @xcite for the world's languages). one aspect of morphosyntax that is widely described in such grammatical specifications is agreement, the process wherein a word or morpheme selects morphemes in correspondence with another word or phrase in the sentence @xcite . languages have varying degrees of agreement ranging from none (e.g. japanese, malay) to a large amount (e.g. hindi, russian, chichewa). patterns of agreement also vary across syntactic subcategories. for instance, regular verbs in english agree with their subject in number and person but modal verbs such as "will" show no agreement. having a concise description of these rules is of obvious use not only to linguists but also language teachers and learners. furthermore, having such descriptions in machine-readable format will further enable applications in natural language processing (nlp) such as identifying and mitigating gender stereotypes in morphologically rich languages @xcite . the notion of describing a language "in its own terms" based solely on raw data has an established tradition in descriptive linguistics (e.g. harris @xcite ). in this work we present a framework (outlined in figure 1 ) that automatically creates a first-pass specification of morphological agreement rules for various morphological features (gender, number, person, etc.) from a raw text corpus for the language in question. first, we perform syntactic analysis, predicting part-of-speech (pos) tags, morphological features, and dependency trees. using this analyzed data, we then learn an agreement prediction model that contains the desired rules. specifically, we devise a binary classification problem of identifying whether agreement will be observed between a head and its dependent token on a given morphological property. we use decision trees as our classification model because raw text λιµάνι της ηγουµενίτσας συνδέεται µε πολλά λιµάνια της ιταλίας και της αλβανίας port.sg det igoumenítsa.gen connect.sg with many port.pl det italy.gen and det albania.gen dependency parsed data λιµάνι της ηγουµενίτσας συνδέεται µε πολλά λιµάνια της ιταλίας και της αλβανίας noun;neut det;fem propn;fem verb adp adj;neut noun;neut det;fem propn;fem cconj det;fem propn;fem mod det udep comp:obj mod det mod mod det conj training data extraction training sample agree? noun det det yes propn det det yes noun mod adj yes propn mod noun no propn mod propn yes they are easy to interpret and we can easily extract the classification rules from the tree leaves to get an initial set of potential agreement rules. finally, we perform rule labeling of the extracted rules, identifying which tree leaves correspond to probable agreement. this is required because not all agreeing head/dependent token pairs are necessarily due to some underlying rule. for instance, in figure 1 's example of greek gender agreement, both the head and its dependent token ιταλίας→αλβανίας have feminine gender, but this agreement is purely bychance, as correctly identified by our framework. ## problem formulation for a head h and a dependent d that are in a dependency relation r, we will say that they agree on a morphological property f if they share the same value for that particular property i.e. f @xmath0. some agreements that we observe in parsed data can be attributed to an underlying grammatical rule. for example, in figure 2 the spanish a.1 shows an example of where subject (enigmas) and verb (son) need to agree on number. we will refer to such rules as required-agreement. such a required agreement rule dictates that an example like a.2 is ungrammatical and would not appear in wellformed spanish sentences, since the subject and a.1 los enigmas son fáciles det.pl riddle.pl be.pl easy.pl _ req __ 'the riddles are easy.' a.2 *los enigmas es fácil det.pl riddle.pl be.sg easy.sg wrong b.1 mi hermano tiene un perro my brother.sg has.sg art.sg dog.sg _ req __ _____ chance _____ 'my brother has a dog.' b.2 mi hermano tiene muchos perros my brother.sg has.sg many.pl dog.pl _ req __ _____ correct _____ 'my brother has many dogs.' the verb do not have the same number marking. however, not all word pairs that agree do so because of some underlying rule, and we will refer to such cases as chance-agreement. for example, in figure 2 the object (perro) and verb (tiene) in b.1 only agree in number by chance, and example b.2 (where the object of a singular verb is plural) is perfectly acceptable. our goal is to extract, from textual examples, the set of rules r f l that concisely describe the agreement process for language l. concretely, this will indicate for which head-dependent pairs the language displays required-agreement and for which we will observe at most chance-agreement. canonically, agreement rules are defined over syntactic features of a language as seen in figure 2 where we have the following rule for spanish: "subjects agree with their verbs on number". 3 to formalize this notion, we define a rule to be a set of features which are defined over the dependency relation, head and dependent token types. in this paper, we make the simplifying assumption that head and dependent tokens are represented by only part-ofspeech features, as we would like our extracted rules to be concise and easily interpretable downstream, although this assumption could be relaxed in future work. the rule discovery process consists of two major steps: a rule extraction step followed by a rule labeling and merging step (also see figure 1 ). 3 sometimes semantic features are used for agreement for eg. united nations is, despite united nations being plural, it is treated as singular for purposes of agreement. ## experimental settings and evaluation our experiments aim to answer the following research questions: (1) can our framework extract linguistically plausible agreement rules across diverse languages? and (2) can it do so even if gold-standard syntactic analyses are not available? to answer the first question we evaluate rules extracted from gold-standard syntactic analysis (sec. §4). for the second question we experiment in low-resource and zero-shot scenarios using crosslingual transfer to obtain parsers on the languages of interest, and evaluate the effect of noisy parsing results on the quality of rules (sec. §5). ## future work while we have demonstrated that our approach is effective in extracting a first-pass set of agreement rules directly from raw text, it focuses only on agreement between a pair of words and hence might fail to capture more complex phenomena that require broader context or operate at the phrase level. consider this simple english example: "john and mary love their dog". under both ud and sud formalisms, the coordinating conjunction "and" is a dependent, hence the verb will not agree with either of the (singular) nouns ("john" or "mary"). also, deciding agreement based on only pos tags is insufficient to capture all phenomena that may influence agreement for e.g. mass nouns such as 'rice' do not follow the standard number agreement rules in english. we leave a more expressive model and evaluation on more languages as future work. we also plan to expand our methodology for extracting grammar rules from raw text to other aspects of morphosyntax, such as argument structure and word order phenomena.