Predicting Declension Class from Form and Meaning (2020)

Abstract: The noun lexica of many natural languages are divided into several declension classes with characteristic morphological properties. Class membership is far from deterministic, but the phonological form of a noun and/or its meaning can often provide imperfect clues. Here, we investigate the strength of those clues. More specifically, we operationalize this by measuring how much information, in bits, we can glean about declension class from knowing the form and/or meaning of nouns. We know that form and meaning are often also indicative of grammatical gender—which, as we quantitatively verify, can itself share information with declension class—so we also control for gender. We find for two Indo-European languages (Czech and German) that form and meaning respectively share significant amounts of information with class (and contribute additional information above and beyond gender). The three-way interaction between class, form, and meaning (given gender) is also significant. Our study is important for two reasons: First, we introduce a new method that provides additional quantitative support for a classic linguistic finding that form and meaning are relevant for the classification of nouns into declensions. Secondly, we show not only that individual declensions classes vary in the strength of their clues within a language, but also that these variations themselves vary across languages.

## introduction to an english speaker learning german, it may come as a surprise that one cannot necessarily predict the plural form of a noun from its singular. this is because pluralizing nouns in english is relatively simple: usually we merely add an -s to the end (e.g., @xmath0. of course, not all english nouns follow such a simple rule (e.g., @xmath1, @xmath2, etc.), but those that do not are few in number. compared to english, german has comparatively many common morphological rules for inflecting nouns. for example, some plurals are formed by adding a suffix to the singular: insekt 'insect' → insekt-en, hund 'dog' → hund-e, radio 'radio' → radio-s. for others, the plural is formed by changing a stem vowel: foot_0 mutter 'mother' → m ütter, or nagel 'nail' → nägel. some others form plurals with both suffixation and vowel change: haus 'house' → häus-er and koch 'chef' → köch-e. still others, like esel 'donkey', have the same form in plural and singular. the problem only worsens when we consider other inflectional morphology, such as case. disparate plural formation and case rules of the kind described above split nouns into declension classes. to know a noun's declension class is to know which morphological form it takes in which context (e.g., @xcite @xcite @xcite @xcite . but, this begs the question: what clues can we use to predict the class for a noun? in some languages, predicting declension class is argued to be easier if we know the noun's phonological form @xcite or lexical semantics @xcite . however, semantic and phonological clues are, at best, only very imperfect hints as to class @xcite @xcite @xcite @xcite . given this, we quantify how much information a noun's form and meaning share with its class, and determine whether that amount of information is uniform across classes. to do this, we measure the mutual information @xcite both between declension class and meaning (i.e., distributional semantic vector) and between declension class and form (i.e., orthographic form), as in figure 1 . we select two indo-european languages (czech and german) that have declension classes. we find that form and meaning both share significant amounts of information, in bits, with declension class in both languages. we further find that form clues are stronger than meaning clues; for form, we uncover a relatively large effect of 0.5-0.8 bits, while, for lexical semantics, a moderate one of 0.3-0.5 bits. we also measure the three-way interaction between form, meaning, and class, finding that phonology and semantics contribute overlapping information about class. finally, we analyze individual inflection classes and uncover that the amount of information they share with form and meaning is not uniform across classes or languages. ## declension classes in language the morphological behavior of declension classes is quite complex. although various factors are undoubtedly relevant, we focus on phonological and lexical semantic ones here. we have ample reason to suspect that phonological factors might affect class predictability. in the most basic sense, the form of inflectional suffixes are often altered based on the identity of the final segment of the stem. for example, the english plural suffix is spelled as -s after most consonants, like in cats, but as -es if it appears after an s, sh, z, ch etc., like in 'mosses', 'rushes', 'quizzes', 'beaches' etc. often differences such as these in the spelling of plural affixes or declension class affixes are due to phonological rules that are noisily realized in orthography; there could also be regularities between form and class that do not correspond to phonological rules but still have an effect. for example, statistical regularities over phonological segments in continuous speech guide first-language acquisition @xcite , even over non-adjacent segments @xcite . statistical relationships have also been uncovered between the sounds in a word and the word's syntactic category @xcite @xcite and between the orthographic form of a word and its argument structure valence @xcite . thus, we expect the form of a noun to provide clues to declension class. semantic factors too are often relevant for determining certain types of morphologically relevant classes, such as grammatical gender, which is known to be related to declension class. it has been claimed that there are only two types of gender systems: semantic systems (where only semantic information is required) and formal systems (where semantic information as well as morphological and phonological factors are relevant) @xcite . moreover, a large typological survey, @xcite , finds that meaningsensitive grammatical properties, such as gender and animacy, can be decoded well from distributional word representations for some languages, but less well for others. these examples suggest that it is worth investigating whether noun semantics provides clues about declension class. lastly, form and meaning might interact with one another, as in the case of phonaesthemes where the sounds of words provide nonarbitrary clues about their meanings @xcite @xcite @xcite @xcite @xcite . therefore, we check whether form and meaning together share information with declension class. ## data for our study, we need orthographic forms of nouns, their associated word vectors, and their declension classes. orthographic forms can be found in any large text corpus or dictionary. we isolate noun lexemes (i.e., or syntactic category-specific representations of words) by language. we select czech nouns from unimorph @xcite and german nouns from celex2 @xcite . for lexical semantics, we trained 300-dimensional word2vec vectors on languagespecific wikipedia. 2 we select the nominative singular form as the donor for both orthographic and lexical semantic representations because it is the lemma in czech and german. it is also usually the stem for the rest of the morphological paradigm. we restrict our investigation to monomorphemic lexemes because: (i) one stem can take several affixes which would multiply its contribution to the results, and (ii) certain affixes come with their own class. 3 compared to form and meaning, declension class is a bit harder to come by, because it requires linguistic annotation. we associated lexemes with their classes on a by-language basis by relying on annotations from fluent speakerlinguists, either for class determination (for czech) or for verifying existing dictionary information (for german). for czech, declension classes were derived by an edit distance heuristic over affix forms, which grouped lemmata into subclasses if they received the same inflectional affixes (i.e., they constituted a morphological paradigm). if orthographic differences between two sets of suffixes in the lemma form could be accounted for by positing a phonological rule, then the two sets were collapsed into a single set; for example, in the "feminine -a" declension class, we collapsed forms for which the dative singular suffix surfaces as -e following a coronal continuant consonant (figurka:figurce 'figurine.dat.sg'), -i following a palatal nasal (pirana:pira ni 'piranha.dat.sg'), and as -ȇ following all other consonants (kráva:krávȇ 'cow.dat.sg'). as for meaning, descriptively, gender is roughly a superset of declension classes in czech; among the masculine classes, animacy is a critical semantic feature, whereas form seems to matter more for feminine and neuter classes. for german, nouns came morphologically parsed and lemmatized, as well as coded for class in celex2. we also use celex2 to isolate monomorphemic noun lexemes and bin them into classes; however, celex2 declension classes are more fine-grained than traditional descriptions of declension class-mappings between celex2 classes and traditional linguistic descriptions of declension class @xcite are provided in after associating nouns with forms, meanings, and classes, we perform exclusions: because frequency affects class entropy @xcite , we removed all classes with fewer than 20 lexemes. 4 we subsequently removed all lexemes which did not appear in our word2vec models trained on wikipedia dumps. the final tally of czech yields 2672 nouns in 13 declension classes, and the final tally of german yields 3684 nouns in 16 declension classes, which can be broken into 3 types of singular and 7 types of plural. the remaining lexemes were split into 10 folds: one for testing, another for validation, and the remaining eight for training. ## results our main experimental results are presented in as a final sanity check, we measure mutual information between class and gender mi(c; g) (see ## discussion and analysis next, we ask whether individual declension classes differ in how idiosyncratic they are, e.g., does any one german declension class share less information with form than the others? to address this, we qualitatively inspect per-class half-pointwise mutual information in figure 2a-2b . see czech. in general, declension classes associated with masculine nouns (@xmath6; w | g) than classes associated with feminine (@xmath7, masculine, plural -ata'. this class ends exclusively in -e or -ȇ, which might contribute to that class' higher mi(@xmath8; w | g). that mi(@xmath9; w | g) is high for feminine and neuter classes suggests that the overall mi(c; w | g) results might be largely driven by these classes, which predominantly end in vowels. we also note that the high mi(@xmath10; w | g) for feminine 'plural -e', might be driven by the many latin or greek loanwords present in this class. with respect to meaning, masculine declension classes can reflect degrees of animacy: 'animate1' contains nouns referring mostly to humans and a few animals (kocour 'tomcat', colek 'newt'), 'an-imate2' contains nouns referring mostly to animals and a few humans (syn 'son', křest'an 'christian'), 'inanimate1' contains many plants, staple foods (chléb 'bread', ocet 'vinegar') and meaningful places (domov 'home', kostel 'church'), and 'inanimate2' contains many basic inanimate nouns (kámen 'stone'). of these masculine classes, 'inan-imate1' has a lower mi(@xmath11; v | g) than its class size alone might lead us to predict. feminine and neuter classes show no clear pattern, although neuter classes '-eni' and '-o' have comparatively high mi(@xmath12; v | g). for mi(@xmath13; v ; w | g), we observe that 'masculine, inanimate1' is the smallest quantity, followed by most other masculine classes (e.g., masculine animate classes with -ové or -i plurals) for which mi(@xmath14; w | g) was also low. among non-masculine classes, we observe that feminine german. mi(@xmath15; w | g) for classes containing words with umlautable vowels (i.e., s3/p1u, s1/p1u) or loan words (i.e., s3/loan) tends to be high; in the prior case, our models seem able to separate umlautable from non-umlautable vowels, and in the latter case, loan word orthography from native orthography. mi(@xmath16; v | g) quantities are roughly equivalent across classes of different size, with the exception of three classes: s1/p4, s3/p1, and s1/p3. s1/p4 consists of highly semantically variable nouns, ranging from relational noun lexemes (e.g., glied 'member', weib 'wife', bild 'picture') to masses (e.g., reis 'rice'), which perhaps explains its relatively high mi(@xmath17; v | g). for s1/p3 and s3/p1, mi(@xmath18; v | g) is low, and we observe that both declension classes idiosyncratically group clusters of semantically similar nouns: s1/p3 contains "exotic" birds (papagei 'parrot', pfau 'peacock'), but also nouns ending in -or, (traktor 'tractor', pastor 'pastor'), whereas s3/p1 contains very few nouns, such as names of months (märz, 'march', mai 'may') and names of mythological beasts (e.g., sphinx, alp). tripartite mi is fairly idiosyncratic in german: the lowest quantity comes from the smallest class, s1/p2u. s1/p3, a class with low mi(@xmath19; v | g) from above, also has low tripartite mi. we speculate that s1/p3 could be a sort of "catch-all" class with no clear regularities. the highest tripartite mi comes from s1/p4, which also had high mi(@xmath20; v | g). the existence of significant tripartite mi results suggests that submorphemic meaning bearing units, or phonaesthemes, might be present. taking inspiration from pimentel et al. 2019, which aims to automatically discover such units, we observe that many words in s1/p4 contain letters {d, e, g, i, l}, often in identically ordered orthographic sequences, such as bild, biest, feld, geld, glied, kind, leib, lied, schild, viech, weib, etc. while these letters are common in german orthography, their noticeable presence suggests that further elucidation of declension classes in the context of phonaesthemes could be warranted. ## conclusion we adduce new evidence that declension class membership is not wholly idiosyncratic nor fully deterministic based on form or meaning in czech and german. we quantify mutual information and find estimates which range from 0.2 bits to nearly one bit. despite their relatively small magnitudes, our estimates of mutual information between class and form accounted for between 25% and 60% of the class' entropy, even after relevant controls, and mi between class and meaning accounted for between 13% and nearly 40%. we analyze results per-class, and find that classes vary in how much information they share with meaning and form. we also observe that classes that have high mi(@xmath21; v | g) often have high mi(@xmath22; w | g), with a few noted exceptions that have specific orthographic (e.g., german umlauted plurals), or semantic (e.g., czech masculine animacy) properties. in sum, this paper has proposed a new information-theoretic method for quantifying the strength of morphological relationships, and applied it to declension class. we verify and build on existing linguistic findings, by showing that the mutual information quantities between declension class, orthographic form, and lexical semantics are statistically significant.