# Where is "regular morphology" in the lexicon?
## Revisiting the role of type vs. token frequency in inflection

Jules Bouton, Olivier Bonami, Sacha Beniamine

► To cite this version:

Jules Bouton, Olivier Bonami, Sacha Beniamine. Where is "regular morphology" in the lexicon? Revisiting the role of type vs. token frequency in inflection. Fifth International Symposium of Morphology (ISMO 2025), Oct 2025, Lille, France. hal-05520841

HAL Id: hal-05520841

https://u-paris.hal.science/hal-05520841v1

Submitted on 20 Feb 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY-SA 4.0 - Attribution - ShareAlike - International
License
# Where is “regular morphology” in the lexicon ?
## Revisiting the role of type vs. token frequency in inflection

Jules Bouton

Olivier Bonami

Sacha Beniamine

Université Paris Cité

Université Paris Cité

University of Surrey

## 1 Motivation

Bybee (1995) argues that the productivity of a morphological pattern strongly correlates with its type frequency in the lexicon. This claim has had a strong influence on further research and is at the foundations of contemporary approaches to paradigmatic predictability (Albright, 2002; Bonami & Boyé, 2014; Beniamine, 2018). We re-examine Bybee’s arguments in the light of new quantitative evidence and show that the debate must be reopened.

A main piece of empirical evidence discussed by Bybee comes from data from a 1927 study by the French psychologist Paul Guillaume. Guillaume (1927/1973) counted the frequency of verbs used by nursery school children during play. He noticed that among the four verb classes recognized by the grammars of that time, the first one was the most likely to be generalized. Since this class also has the highest type frequency but not the highest token frequency, Bybee argues that type frequency predicts the direction of analogy in inflection. Table 1 confirms that using the traditional grammar’s classification, type and token frequencies are not correlated. To supplement Guillaume’s experimental observations, we computed the same statistics using the frequencies available in the Vlexique dataset (Beniamine et al., 2024). Despite being based on the usage of adult speakers a century later, results are in line with Guillaume’s.

<table><caption>Table 1: Frequency properties of the traditional French conjugation classes. Left-hand side: Guillaume’s (1927) numbers (as cited by Bybee 1995, with the 3rd and 4th classes merged); right-hand side: numbers derived from Vlexique 2.0 (Beniamine et al., 2024).</caption><thead><tr><th rowspan="2"></th><th colspan="2">Guillaume (1927)</th><th colspan="3">VLexique 2.0</th></tr><tr><th>Type</th><th>Token (cum.)</th><th>Type</th><th>Token (mean)</th><th>Token (cum.)</th></tr></thead><tbody><tr><td>1st class</td><td>76.0%</td><td>36.2%</td><td>86.6%</td><td>4.1%</td><td>34.8%</td></tr><tr><td>2nd class</td><td>6.1%</td><td>6.0%</td><td>6.4%</td><td>2.0%</td><td>1.2%</td></tr><tr><td>3d class</td><td>17.9%</td><td>57.8%</td><td>7.0%</td><td>93.9%</td><td>64.0%</td></tr></tbody></table>

It is however unclear whether the data at hand allows to conclude that token frequency is irrelevant for pattern prominence. First, the so-called third class in French conjugation is really an aggregate of a large variety of presumably “irregular” morphological classes with a low type frequency. Second, although it is striking that the most token-frequent lexemes tend to be alone in their inflectional behavior, thus forming a class on their own, this should be compared to the *cumulative* token frequency of larger classes rather than their *average* token frequency. Given these two observations, the data in Table 1 does not suffice to conclude that type and token frequency are not correlated across inflection patterns.

To improve the coarse-grained classification of traditional grammars, we use the *Qumin* toolkit (Beniamine & Bouton, 2025) to automatically infer microclasses of lexemes that inflect in exactly the same way. Figure 1 shows a non-trivial relation between the type and token frequencies of these microclasses. As a case in point, a higher type frequency entails a higher token frequency, but a high token frequency can be achieved with a low or a high type frequency.
<figure><img src="image_2.png" /><figcaption>Figure 1: Frequency properties of French conjugation microclasses inferred on Vlexique 2.0 following Beniamine's (2018) approach. Each datapoint represents a microclass.</figcaption></figure>

Although this figure is already telling, it is not as informative as it looks: inflectional macro-classes aggregate information on which pattern relates each pair of forms in the paradigm. In the remainder of this abstract we dig deeper by examining the relationship between type and token frequency in a more local situation of inflectional prediction, specific to the alternation between pairs of paradigm cells.

## 2 Method

We investigate the relation between type and token frequency of inflectional patterns. We take the alternation patterns in inflection as the basic unit of comparison, because they are lower-level than inflectional classes and can be automatically inferred.

In practice, we use the algorithm described by Beniamine (2018) and its implementation in *Qumin* to infer all possible alternation patterns between all pairs of cells in French. For each oriented (predictor → target) pair of cells, we construct classes of predictor forms that are similar in that they match the left hand side of the same set of patterns. Table 2 illustrates how the predictors cluster.

<table><caption>Table 2: French lexemes clustered in predictor classes for the INF → IND.PRS.1PL pair.</caption><thead><tr><th>Class</th><th>Lexeme</th><th>Predictor</th><th>Applicable patterns</th><th>Pattern</th><th>Target</th></tr></thead><tbody><tr><td>1</td><td>FINIR</td><td>fini<sup>k</sup></td><td>k ⇌ s̃<sup>õ</sup>; k ⇌ z̃<sup>õ</sup></td><td>k ⇌ s̃<sup>õ</sup></td><td>fini<sup>s̃</sup></td></tr><tr><td>1</td><td>LIRE</td><td>li<sup>k</sup></td><td>k ⇌ s̃<sup>õ</sup>; k ⇌ z̃<sup>õ</sup></td><td>k ⇌ z̃<sup>õ</sup></td><td>li<sup>s̃</sup></td></tr><tr><td>2</td><td>CROIRE</td><td>k<sup>w</sup>wa<sup>k</sup></td><td>k ⇌ ỹ</td><td>k ⇌ ỹ</td><td>k<sup>w</sup>wa<sup>j̃</sup></td></tr></tbody></table>

We use frequency data from Vlexique to compute the cumulated type and token frequencies of each predictor class. We also compute the conditional probability of an inflectional pattern being used given a predictor class, based on the type or token frequency of the pattern in the lexicon. Type frequency is the proportion of predictors instantiating that pattern. Token frequencies is the probability of (predictor, target) form pairs, weighted by the token frequency of the forms. Table 3 shows an excerpt from the four resulting measures: predictor class
frequencies and pattern probabilities, from type and token frequencies.

<table><caption>Table 3: Sample datapoints used in this study to compare type and token frequencies.</caption><thead><tr><th colspan="2">Cells</th><th rowspan="2">ID</th><th colspan="2">Class of predictors</th><th rowspan="2">Pattern</th><th colspan="2">Pattern</th></tr><tr><th>Predictor</th><th>Target</th><th>Type freq.</th><th>Token freq.</th><th>Type prob.</th><th>Token prob.</th></tr></thead><tbody><tr><td>INF</td><td>IND.PRS.1PL</td><td>1</td><td>4562</td><td>6963929</td><td>ð &harr; e</td><td>1</td><td>1</td></tr><tr><td>INF</td><td>IND.PRS.1PL</td><td>2</td><td>227</td><td>442014</td><td>sð &harr; k</td><td>0.88</td><td>0.02</td></tr><tr><td>INF</td><td>IND.PRS.1PL</td><td>2</td><td>227</td><td>442014</td><td>zð &harr; k</td><td>0.04</td><td>0</td></tr><tr><td>INF</td><td>IND.PRS.1PL</td><td>2</td><td>227</td><td>442014</td><td>ð &harr; k</td><td>0.07</td><td>0.98</td></tr><tr><td>INF</td><td>IND.PRS.1PL</td><td>3</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td></tr></tbody></table>

## 3 Preliminary results

We now turn to a preliminary comparison of the distribution of type and token frequencies within predictor classes. There are 56,130 such classes for 2,250 pairs of cells under consideration in our dataset. Figure 2 plots the size of these classes: the vast majority have a single pattern and are thus irrelevant to our concerns. Nearly 12,000 have 2 patterns, and class sizes larger than 3 are rare.

<figure><img src="image_3.png" /><figcaption>Figure 2: Number of patterns by class</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Figure 3: Histogram of rank correlations</figcaption></figure>

Figure 3 provides one view of the relationship between the type and token frequency of patterns. We plot a histogram of Spearman's rank correlation coefficient $\rho$ between the two pattern probability distributions, within each pair of cell and each class of predictors. $\rho$ varies between -1 for perfect inverse correlation and 1 for perfect correlation. Note that only values of -1 or 1 are possible when comparing only two datapoints, which is what happens here for 73% of classes. We observe that the correlation is perfect in 48% of cases and strictly positive in 69%. Hence the relationship between type and token frequency is far both from a perfect positive or negative correlation.

Rank correlation captures only one aspect of the differences between type and token frequency. As an illustration, we briefly discuss two cases presented in Table 4 where there is perfect rank correlation but important numerical differences. The top part of the table presents
the situation when predicting an imperfect 3PL from the corresponding 2PL ending in -je. There are two patterns here, which differ in type frequency by 1 order of magnitude and in token frequency by 3: clearly, using token frequencies leads to a much more skewed distribution, predicting analogy to the majority pattern to be much more prevalent. The bottom part of the table illustrates the opposite situation. Here, token frequencies smoothen the skewed probability distribution, because patterns with a rarer type frequency have a higher cumulated token frequency.

<table><caption>Table 4: Two examples from the data presented in Table 3 where type and token frequency of the patterns yield a significant difference in the prominence of the patterns.</caption><thead><tr><th>Cells</th><th>Lexeme</th><th>Predictor</th><th>Target</th><th>Pattern</th><th>Type freq.</th><th>Token freq.</th></tr></thead><tbody><tr><td rowspan="2">IND.IPFV.2PL→3PL</td><td>ABOYER</td><td>abwaje</td><td>abwaje</td><td>Ø ⇌ Ø</td><td>0.09</td><td>0.0004</td></tr><tr><td>ABAISSER</td><td>abesje</td><td>abesje</td><td>j ⇌ Ø</td><td>0.91</td><td>0.9995</td></tr><tr><td rowspan="3">IMP.PRS.2PL→INF</td><td>ABDIQUER</td><td>abdike</td><td>abdike</td><td>Ø ⇌ Ø</td><td>0.95</td><td>0.66</td></tr><tr><td>ACOURRIR</td><td>akuke</td><td>akuke</td><td>e ⇌ i̯k</td><td>0.02</td><td>0.11</td></tr><tr><td>ABATTRE</td><td>abate</td><td>abatk</td><td>e ⇌ k</td><td>0.03</td><td>0.22</td></tr></tbody></table>

The evidence provided in this abstract suggests that the situation is more complex than what has long been assumed: type and cumulated token frequencies of patterns relate in non-trivial ways. In the talk we will present a more detailed statistical analysis and draw lessons for the modelling of analogy in inflection.

## References

Albright, Adam C. 2002. *The Identification of Bases in Morphological Paradigms*. Los Angeles: University of California dissertation.

Beniamine, Sacha. 2018. *Classifications flexionnelles. Étude quantitative des structures de paradigmes*: Université Sorbonne Paris Cité - Université Paris Diderot (Paris 7) dissertation.

Beniamine, Sacha & Jules Bouton. 2025. Qumin (v3.0.0) [Python package]. Zenodo. doi: 10.5281/zenodo.15008374.

Beniamine, Sacha, Maximin Coavoux & Olivier Bonami. 2024. Vlexique2.0: A rich lexicon of French verbal inflection with form-level frequencies. In *21st International Morphology Meeting*, Vienne.

Bonami, Olivier & Gilles Boyé. 2014. De formes en thèmes. In Florence Villoing, Sarah Leroy & Sophie David (eds.), *Foisonnements morphologiques. Études en hommage à Françoise Kerleroux*, 17–45. Paris: Presses Universitaires de Paris Ouest.

Bybee, Joan. 1995. Regular morphology and the lexicon. *Language and Cognitive Processes* 10(5). 425–455. doi:10.1080/01690969508407111.

Guillaume, Paul. 1927. Le développement des éléments formels dans le langage de l'enfant. *Journal de psychologie normale et pathologique* 24. 203–229.

Guillaume, Paul. 1973. The Development of Formal Elements in the Child's Speech. In Charles A. Ferguson & Dan I. Slobin (eds.), *Studies of child language development*, 240–251. New York.