Conditioning, but on Which Distribution? Grammatical Gender in G erman Plural Inflection (2020)

Abstract: Grammatical gender is a consistent and informative cue to the plural class of German nouns. We find that neural encoder-decoder models learn to rely on this cue to predict plural class, but adult speakers are relatively insensitive to it. This suggests that the neural models are not an effective cognitive model of German plural formation.

## introduction in recent years, neural models of natural language have proven to be powerful statistical learners, capable of representing linguistic patterns and the conditions under which they generalize to new forms (e.g. @xcite . artificial language learning experiments show that humans are also statistical learners: when patterns appear consistently with certain cues in the input, speakers consistently rely on those cues to generalize patterns to new forms @xcite . our research examines how two different statistical learners -neural encoder-decoder (ed) models and adult german speakers -use the cue of grammatical gender in plural inflection of novel words. gender has a high statistical association with plural suffix: the feminine noun wahl ("vote") is wahlen in the plural, but the rhyming neuter noun mal ("time") has the plural form male. we expect that both speakers and the ed model will produce distributions over plural forms which are heavily conditioned on the gender of the input word. we find that the neural model is highly sensitive to grammatical gender; however, speaker productions, while slightly influenced by gender, appear more consistent with a distribution over plural suffixes which is unconditioned on gender. this surprising result suggests that, even though gender is a very informative cue to plural class, speakers may preferentially attend to different cues. ## background german plural inflection is realized by five major suffixes, none of which commands a majority in either type or token frequency. 1 the two most frequent suffixes, -e and -(e)n, each apply to roughly 35-40% of german nouns (figure 1 , upper). with no majority class, how can a learner determine which pattern to generalize to new words? one major cue to plural class comes from a noun's grammatical gender. the strong statistical association between grammatical gender and plural inflection class is widely recognized in the literature @xcite @xcite , to cite some recent examples), and readily apparent in figure 1 (lower), which shows the distribution of plural suffixes by noun gender in the unimorph corpus @xcite . of the two most frequent plural suffixes, -(e)n is highly associated with feminine nouns, and -e with nonfeminine (masculine and neuter); the tendency is so strong that some researchers have analyzed these suffixes as gender-conditioned "defaults" @xcite . from this perspective, grammatical gender provides a highly consistent cue to plural class membership, which ought to inform a statistical learner's generalizations. fur-thermore, grammatical gender is expressed on the article preceding a noun, and this initial position is perceptually salient to speakers @xcite . these properties suggest that grammatical gender should influence how speakers inflect novel nouns, and indeed, several wug tests foot_1 on german-speaking adults have found significant effects of gender @xcite @xcite . based on the artificial language learning literature (e.g. newport, 2016), we might expect speakers to display conditional probability matching on novel german nouns, such that the probability of a noun taking a certain plural inflection -in particular, the two highly frequent classes -e and -(e)n -depends upon its grammatical gender. neural encoder-decoder (ed) models have recently been proposed for consideration as models of speaker cognition @xcite . this has prompted investigation into the extent to which these models capture speaker behavior @xcite @xcite . earlier work suggests that neural models of german plural inflection are sensitive to grammatical gender: @xcite found that a simple recurrent network learned to favor -e plurals for masculine nouns, and -(e)n when the same nouns were presented as feminine gender. we hypothesize that neural models and adult speakers are equally capable of using the information available from grammatical gender to predict number inflection. we expect both to demonstrate similar gender-conditioned probability matching to the distribution shown in figure 1 (lower), resulting in a majority use of -(e)n for feminine nouns, and -e for masculine and neuter nouns. ## method to compare how grammatical gender influences plural inflection for german speakers and neural models, we use a parallel production task on nonce words (a wug test) for both speakers and model. our study largely follows the data collection and modeling procedures of @xcite . stimuli we use the 24 made-up nouns developed by @xcite , listed in appendix a. by design, these nouns lack strong phonological cues to plural class. 3 in their original study, marcus et al. did not find a significant effect of grammatical gender; however, @xcite used the same stimuli and reported gender effects in the expected direction -participants used -(e)n more on feminine nouns, and -e more for nonfeminine nouns. zaretsky and lange speculate that these discrepant findings stem from differences in the two study designs: scale (the earlier study had 48 participants, the later one 585) and task (acceptability ratings vs. elicited productions). a third differentiating factor is the presence of semantic cues in the marcus et al. study, which provided sentence contexts around the nonce words; for example, a sentence like die grünen brals sind billiger ("the green brals are cheaper") would imply that the nonce word bral referred to an object, whereas die brals sind ein bißchen komisch ("the brals are a bit weird") would imply that bral was a family name. as adult learners can attend to formal and semantic cues under different conditions @xcite , it's possible that this manipulation directed participant focus toward semantic cues rather than grammatical gender. zaretsky and lange provided no semantic context in their experiment, only presenting the indefinite article and word form to participants (e.g. ein bral, "a [masculine/neuter] bral"). our experimental design for both speakers and the neural model is closer to that of @xcite : we elicit plural form productions and provide no semantic cues. this suggests we might also expect to find a robust effect of grammatical gender for these stimuli. human data collection we collected production data from 92 native german speakers foot_3 through an online survey. participants saw each noun in the singular with a definite article indicating grammatical gender (e.g. der bral for masculine, das bral neuter, die bral feminine), and typed a pluralinflected form. participants were randomly assigned to one of three lists. grammatical gender was counterbalanced within lists (each participant saw 8 feminine, 8 masculine, and 8 neuter nouns) and across lists (each noun appeared with a different gender in each list). encoder-decoder model a neural encoderdecoder (ed) model encodes an input sequence into a fixed vector representation and then incrementally decodes it into a corresponding output sequence @xcite . we follow other recent work in using the architecture of @xcite , which has been proposed for cognitive modeling @xcite . for the task of german number inflection, the ed takes as input a character sequence representing the singular nominative form of a noun, preceded by a special character for grammatical gender (e.g. f w a h l; f indicates feminine, m masculine, and n neuter). the model is trained to produce the noun's corresponding nominative plural form as output (e.g. w a h l e n). we used the 11,243 german nouns in unimorph @xcite as our corpus, and added noun gender by merging the dataset with another wiktionary scrape. 5 we follow the modeling procedure of @xcite , who found that neural ed models correctly learned the most frequent plural suffix for neuter stimuli, but did not evaluate sensitivity to grammatical gender; please see their paper for further implementation details. following @xcite , we trained 25 separate random initializations of the same model architecture. this allows separate model instances to be treated as simulated "speakers", letting us aggregate productions and compare more directly to human speaker data. for evaluation, we combined each of the 24 noun stimuli with each of the three grammatical genders, and provided the resulting 72 items as input to each model instance. ## results our results (figure 2 ) show that both speakers and the ed model are sensitive to grammatical gen- overall-tf gender-tf ed speakers .67 .49 .49 (.60, .72) (.40, .56) (.35, .61) ed .41 .62 (.27, .54) (.50, .71) ## discussion we hypothesized that adult speakers and neural encoder-decoder models would make similar use of grammatical gender when inflecting novel words in the plural, as gender is a salient and consistent cue to plural inflection class, especially in an experimental setup where semantic cues are absent. contrary to expectations, our results indicate that both learners attend to grammatical gender, but to different degrees -the neural model is much more sensitive to grammatical gender than adult speakers, whose productions are closer to the overall type frequency of plural suffixes in german. the neural model's use of grammatical gender is not surprising, as it aligns with earlier findings (c.f. @xcite ; however, the speakers' lack of attention to gender is unexpected. in their large-scale production study with the same noun stimuli, @xcite found reliable effects of grammatical gender: their participants used -(e)n for 33% of feminine nouns, versus 19% of non-feminine nouns (compare to our study: 33% vs. 26%). -e also appeared more with nonfeminine nouns (49% vs. 41%), although the effect was not statistically significant. nonetheless, they note that -e was most frequently produced for feminine nouns as well as nonfeminine nouns, consistent with our results, and their data shows a similarly broad distribution over types. despite other differences between our study design and theirs (e.g. online vs. in-person data collection, typed vs. written modality, german speakers from various backgrounds vs. one region), we consider our results fundamentally aligned: speakers show a slight but statistically reliable effect of gender on -(e)n and -e production, in both cases much less than the effect shown by the ed model. one possibility is that the phonological forms of our noun stimuli provide their own statistical conditioning, to a stronger degree than anticipated. this overall-tf gender-tf speakers .78 (.73, .82) .70 (.64, .75) ed .47 (.33, .59) .71 (.61, .78) in conclusion, our comparison of neural encoderdecoder models and adult german speakers found a significant difference in their use of grammatical gender as a cue to plural inflection. although this cue is highly informative, speakers -unlike neural models -appear relatively insensitive to gender in our task. this finding suggests that speakers may attend more readily to other cues such as phonology, and therefore match productions to a different distribution which shows less gender conditioning.