Mandarin classifier systems optimize to accommodate communicative pressures
Authors: Yamei Wang
Géraldine Walther
Venue: EMNLP 2023 Findings (2023)
Keywords: Mandarin Chinese
classifiers
noun classes
noun class processing
word embeddings
mutual information
GAM
TL;DR: Noun-classifier combinations are sensitive to the nouns’ frequencies, contextual similarity, and the NP context nouns appear in; differential behaviors reflect opposing communicative pressures of processing vs. learning.

Abstract: Previous work on noun classification implies that gender systems are inherently optimized to accommodate communicative pressures on human language learning and processing (Dye. et al 2017, 2018). They state that languages make use of either grammatical (e.g., gender) or probabilistic (pre-nominal modifiers) to smoothe the entropy of nouns in context. We show that even languages that are considered genderless, like Mandarin Chinese, possess a noun classification device that plays the same functional role as gender markers. Based on close to 1M Mandarin noun phrases extracted from the Leipzig Corpora Collection (Goldhahn et al. 2012) and their corresponding fastText embeddings (Bojanowski et al. 2016), we show that noun-classifier combinations are sensitive to same frequency, similarity, and co-occurrence interactions that structure gender systems. We also present the first study of the effects of the interaction between grammatical and probabilisitic noun classification.

Mandarin classifier systems optimize to accommodate communicative pressures
Yamei Wang and Géraldine Walther
George Mason University (ywang78, gwalthe) @gmu.edu
Abstract
'uyHa' luná Yet, gender systems do not appear present any particular challenges first language acquisition (Arnon and Ramscar, 2012), indicating that gender systems are fact, learnable.
Previous work noun classification implies that gender systems are inherently optimized accommodate communicative pressures human language learning and processing (Dye al., 2018). The authors state that languages make of either grammatical (e.g.. gen- der) or probabilistic (pre-nominal modifiers) to smoothe the entropy noums in context. We show that even languages that are considered genderless, like Mandarin Chinese, possess noun classification device that plays the same functional role as gender markers. Based on close to MM Mandarin noun phrases extracted from the Leipzig Corpora Collection (Gold- hahn et al., 2012) and their corresponding fast- Text embeddings (Bojanowski et al., 2017), we show that noun-classffier combinations sen- sitive to same frequency, similarity, and co- occurrence interactions that structure gender systems. We also present the first study of the interaction between grammatical and prob- abilistic classification.
As speakers, humans are confronted with two main challenges, which reflect what Blevins al. (2017) refer to as 'competing communicative pres- sures': the first is learning their language, for which regularities across similar words (or struc- tures) are the most optimal principle of organi- zation; the second is using their language to ef- ficiently communicate with others, for which en- hanced discriminability of words and structures provides the most optimal principle. Irregulari- ties make words and structures more item specific and thus increase their discriminabilityy which allows for faster and more efficient communica- tion. A fully regular system would not be opti- mized for communication, whereas fully irreg- ular would make language extremely diffi- cult if not impossible learn. However, not all parts of the system present the same challenge for lcarning: Highly frequent words and structures are typically encountered early by learners and con- stantly reinforced. Once they are acquired, learn- ing is no longer necessary. Irregularities in early acquired words only present minor challenge to- wards the beginning of human language develop- ment, but do not pose any problems later in life. Their discriminative properties do however pro- vide significant benefits for language processing, making communication faster and more efficient. Fast discrimination especially useful for words and structures that speakers use most frequently. Infrequent words and structures, on the other hand, typically only encountered late in life, if at all. Even proficient adult speakers need lever- age their knowledge about related words and struc- tures when they first encounter them. Regularity across similar structures is essential for low fre- quency words and structures. At the same time,
1 Introduction
The world's languages make use of a variety of strategies to classify nouns into sub-classes. While gendered languages like Spanish German typi- cally split nouns across two (masculine M vs. fem- inine F) three (M, and neuter N) classes, other languages, c.g., the Bantu family, may split nouns across more than 10 different classes (Cor- bett et al., 1991; Creissels, 2006). The recurring fascination with gender systems that can be found in the linguistic literature lies in the perceived ar- bitrariness of gender classes, which, among other things, makes them notoriously difficult to acquire for second language learners: for example, in French or Spanish the word for sun (fr: "soleil', es: 'sol') will be and the word for moon (fr: 'lune', es: 'luna') will be F, while in German the genders are reversed ('Sonne' F, 'Mond' M) and in Russian 'comune' sólnce sun would n and moon

---

faster processing of rarer structures not crucial increasing overall communicative efficiency.
example, classifiers, such as K zhäng and * tiáo, are typically associated with different types of ref- erents; H zhāng with wide and flat items like beds, nets, and maps, and th riáo with long and nar- row objects like rivers, scarfs, and legs. Yet, just like gendered determiners, classifiers can also be seen grammatical means for classifying nouns into different noun classes: Mandarin classifiers are part of the noun phrase. They typically ap- pear before nouns. According to Li and Thomp- son ( (1989), they are obligatorily present after nu- merals or demonstratives: for instance, in phrases like jij liāng *(Sk zhāng) 7 zhuōzi 'two tables' and i zhè *( liáo) Kij J yizi 'this bench', the classifiers (highlighted in bold) cannot be omitted. Nouns will only combine with certain subsets of classifiers, but not others. Just like gender markers, combining noun with the wrong classifier would be perceived as ungrammatical.
Recent work by Williams et al. (2019) showed that gender assignment tends not to be as arbi- trary as would appear, but instead correlates with nouns' semantics. Earlier work by Dye (2017) and Dye et al. (2017b, 2018) had also already shown that gender systems are inherently optimized to accommodate communicative pressures on human language learning and processing via interactions between semantic similarity, frequency, and co- occurrence likelihood: since across all languages, nouns tend to be the syntactic category with the most diverse number of items (Petrov et al., 2011), 1), syntactic processing of nouns tends to correspond to locus of heightened cognitive effort. This is also reflected in the fact that nouns are the most fre- quent sites for disfluencies, incorrect retrieval, and mishearings (Clark and Wasow, 1998; Vigliocco al., 1997). Gender classes that make use of gen- dered determiners reduce the uncertainty about up- coming nouns in context by restricting the subset of possible nouns to the ones that are consistent with the gender indicated by the determiner. Dye et al. (2017b) show that the existence of interactions between nouns' semantics, their frequencies, and their assignment to gender classes closer reflects cognitively optimized system than an arbitrary one. Dye et al.'s 2017a paper "Cute Little Puppies and Nice Cold Beers: An Information Theoretic Analysis of Prenominal Adjectives' further illus- trates how languages without gender markers, such as English, make use of probabilistic counterpart to grammatical noun classification, where a larger number of pre-nominal modifiers progressively re- duces the uncertainty of nouns in context. Accord- ing to the authors, languages can make use of either grammatical (e.g., gender) or probabilisitic (pre- nominal modifier sequences) to smoothe the uncer- tainty (entropy (Shannon, 1948)) of nouns in con- text.
Since knowing a noun's 's classifier appears just as crucial to correctly speaking Mandarin as knowing a noun's s gender would be required for German or Spanish, previous studies have investigated the re- lation between a noun's 's meaning and the classifiers it combines with. Liu et al. (2019) show that classi- fiers cannot be fully predicted from the semantics of their nouns. Diverse nouns such as benches, smoke, passwords, and even abstract entities like regulations can all be quantified with the same clas- sifier 1/4 tiáo. However, semantically more similar nouns like bench and table do not share classifiers (* tiáo for bench, but ik zhāng for table). The challenges they present for second language learn- ers are thus very similar to those encountered by learners of gender systems (Liang, 2008). On a functional level, Wang and Walther (2023) show that classifiers also substantially reduce the con- ditional entropy of upcoming nouns Mandarin processing. This indicates that knowledge of a pre- nominal classifier can help speakers predict up- coming nouns. Lau and Grüter (2015) and Grüter et al. (2020) also report that speakers use classifier- referent associations to predict upcoming nouns in Mandarin, thus facilitating noun processing.
Our study focuses on another, different, noun classification system which is based on NOMINAL CLASSIFIERS. It relates its structure to its cogni- tive processability and learnability. As grammat- ical category, classifiers constitute one of the most prominent features of East and South East Asian languages, such as Mandarin (Li, 2013). Classi- fiers categorize nouns based on distinctive seman- tic properties such as shape, animacy, humanness, among others (Aikhenvald and Mihas, 2019). For
While classifiers appear to share a great number of properties with gendered determiners, they also differ from gender markers in number of signifi- cant aspects: (i) they do not operate over one-to- one correspondence with specific nouns: a given noun will typically combine with multiple classi- fiers, of which subsets are turn also combined

---

with other subsets of nouns (examples of those as- sociations are given in Table 1); (ii) classifiers are significantly more numerous: Mandarin, for exam- ple, has hundreds of different classifiers; (iii) clas- sifiers can sometimes appear grammatically more ambiguous since the same token will sometimes be used as a noun and sometimes as a classifier; partly consequence of ii (i.e., the greater diversity of classifiers) and ii (which includes the fact that classifiers can sometimes retain part of the mean- ing of the corresponding noun), (iv) classifiers are inherently more semantically transparent than their counterparts in gender systems.
across different classifiers when they are frequent. Additionally, we find interesting differential pat- terns between nouns immediately following classi- fiers and nouns also preceded by one or more nomi- nal modifiers, an interaction that had never been di- reetly investigated and reported before: classifiers will display the expected behavior of grammat- ical noun class marker in the sense of Dye et al. (2017a,b; 2018) in the scenario where they imme- diately precede the nouns and are the only contrib- utor to smoothing noun entropy. In cases where additional modifiers (i.c., elements providing addi- tional probabilistic classification) are present in the noun phrase, classifier-noun pairs do not show the structure expected of grammatical noun classifica- tion. In such cases and noun classification appears to be most correlated with noun frequencies. Our study is the first to quantitatively show the effects of the co-existence of grammatical and probabilis- tic noun classification. Mandarin, they manifest through differential patterns of class membership depending on both (a) noun pair similarity and (b) noun phrase structure.
The goal of our study is to investigate the struc- ture and cognitive function of classifier systems compared to better known gender systems. In par- ticular, we seek to understand how system that is neither fully categorical (like gender systems where noun is only ever assigned one gender) nor fully probabilistic (adjective-noun combina- tions of which none would be strictly ungrammat- ical) handles noun class membership and makes use of classifier-noum combinatorics to accommo- date competing communicative pressures of learn- ing and processing language (Blevins et al., 2017). A fully regular classifier system would translate into semantically similar nouns always sharing the same sets of classifiers. A fully irregular system would translate into semantically similar nouns never sharing the same classifiers. The hypothesis that we are testing here is whether Mandarin classi- fiers do indeed organize Mandarin nouns along the two competing communicative pressures related to learning and processing, which have been found to structure gender systems, and strike a balance be- tween regularity and discriminability.
2 Data
Our study is based on three of the 1M sentence cor- pora of Mandarin Chinese in the Leipzig Corpora Collection (Goldhahn et al., 2012).' We normal- ized the data by first transforming all Chinese char- acters into simplified Chinese utilizing the Open Chinese Convert software.? 2 We then applied the CoreNI_2 Chinese dependency parser (Chen and Manning, 2014) to our dataset. Leveraging the de- pendency structure, we extracted all complete nom- inal phrases ( (1,079,190 instances and 767,721 dis- tinct phrases). Since the results of the parsing was not clean enough conduct a detailed analysis of classifier-noun interactions, we manually filtered the extracted noun phrases as described below. First, we removed nominal phrases with un- sual lengths, specifically those exceeding 35 char- acters a total of 91 noun phrases.* Second, any noun phrases that contained elements improp- erly tagged as classifiers, which included sym- bols, non-Chinese characters, or invalid characters.
Using a large set of close to IM Mandarin noun phrases extracted from the Leipzig Corpora Collec- tion (Goldhahn al., 2012) and their correspond- ing fast embeddings (Bojanowski et al., 2017), we show that grammatical noun classification in the sense of Dye et al. (2017,, 2018) extends beyond one-to-one relations like German noun- gender pairs, to cases where the nominal classifiers are in a many-to-many relationship with their as- sociated nouns. Just like gender markers, noun- classifier combinations are sensitive to noun fre- quencies, semantic similarity, and possible noun co-occurrence patterns. We find that semanti- cally similar nouns indeed tend to share classifiers if they are infrequent and tend to be distributed
"2007-2009 news, 2011 newscrawl, and 2015 China web: htlps:/wwortscihe uni- leipzig. http:///ild////////ooo//ool /dwnlood// Chinese.
Phulps: /gtpggttuubbooo www.//om///////ee Voidoid/C /Oen/O VVersion 4.4.0, released in January 2022.
"Extracted phrases of more than 35 characters were judged to be abnormal by one of the authors who is a native speaker.

---

For each noun in data set, extracted the sets of classifiers it could combine with. We also computed all noun frequencies and log frequen- cies. We kept all nouns that occurred a minimum NPs including the same nouns
NPs including the same classifiers
Bi f liāng zhāng zhuōzi 'two tables'
i 4 loj f thé tiáo yizi 'this bench'
T I liāng kuăn zhuōzi 'two types of tables'
1 t ( zhè tiáo yän 'this smoke'
+ liäng pāi zhuōzi 'two rows of tables'
ix MLAE zhè tiáo guiding 'this regulation'
M 4 f liāng chê zhuōzi 'two trucks of tables'
i zhé tião mimă 'this passwords
Table
Nominal phrases illustrate the multifaceted relationship between classifiers and referent nouns in Mandarin Chinese. The characters in bold are classifiers.
were excluded from our data set.² Lastly, nomi- nal phrases featuring post-nominal classifiers were also o After data cleaning, we were left with 760,575 distinct nominal phrases that all in- cluded a pre-nominal classifier. The number of distinct nouns in our data set is 44,620.7 Out of those, we kept 39,011 nouns occurring a minimum of 25 times to ensure that they would be repre- sented in the pre-trained vectors used our exper- iments (see section 3.1).
of 25 times in our original 3M sentence data and extracted the nouns' word vectors from the pre- trained Chinese fastText model (Bojanowski et al., 2017). 10 The vectors were included as quantifi- able measure of noun's meaning based on the dis- tributional hypothesis trbutioonal laid out by Harris (1954) and Firth (1957). We set a minimum of 25 occurrences as a threshold ensure that all words would be rep- resented in the set of pre-trained fastText vectors. For each noun pair, we computed the cosine similarity between the noun vectors. As in pre- vious studies (Dye, 2017; Dye et al., 201 7b,a, 2018; Williams et al., 2019), cosine similarity was used as measure of semantic similarity between nouns."!
Since our goal included evaluating the effect of frequency on noun class membership, we di- vided our noun data into frequency bands of 30 and sampled 1% from each. We generated non- ordered noun pairs for those 1% and computed their summed frequency. From the pairs, we gen- erated a data frame in which the higher and lower frequency elements of pairs were balunced* across first and second nouns.
We further extracted the co-occurrences of nouns in our corpus within bidirectional two word window and computed their pointwise mutual information (PMI) comparing their co- occurence probability to the product of the nouns' independent probabilities, as defined by Church and Hanks (1990):12
Our resulting data is set of 390 nouns, sam- pled from 39,011 nouns across all frequency bands to ensure that the sample reflects distributions in the full data set. From those nouns, we generated 37,831 noun pairs.?
n(11,n2) PMI(n1,n2) - log( P(n1)P(m2)
(1)
3 Method
Cosine similarity is used measure of gen-
3.1 Computing the relevant measures
1"The same vectors were also used in the study by Williams et al. (2019). The High Dimensional Explorer (Shaoul and Westbury, 2010) was run to generate vectors for Dye et al. (2017h,a).
IIn response one reviewer's concern that (i) the pres- ence of classifiers in the data that had been used for train- ing the pre-trained fastText vectors might impact effects classifier-noun combinations and (ii) the reliance of fastText on sub-word information might bias results towards form properties rather than distributional ones, also trained cus- tomized Text and Word2Vec (Rehūřck and Sojka, 2010) vectors version of our corpus from which had pre- viously removed all instances of classificrs. Contrarily fastText, Verd2Vec does not integrate sub-word information. Both models were trained using default settings. Overall, there were no significant differences in results involving cus- tomized vectors that would warrant change the experimen- tal setup.
"Some examples of these improperly tagged classifiers are 1. K. $. '''', and 'Iue997'.
"This group represents approximately 0.75% of the total data and commonly occurs during item enumeration, as in the phrase -- yōnhángkā qi zhâng 'seven bank cards'.
GGiven that Mandarin nouns do not inflect, we do not dis- tinguish between lemmas and word forms.
"The mean frequencies and standard deviations between the first and second nouns are similar.
'The total of 37,831 pairs was generated under two sepa- rate conditions: 19.110 pairs were derived from 196 distinct nouns that were immediately precoded by a classifier, while 18,721 pairs were generated from 194 distinct nouns in noun phrases that also included pre-nominal modifiers.
The window size is the same as the one used by Dye et al. to evaluate corresponding effects on German noun classifica- tion based on gender.

---

eral semantic similarity. PMI relates to whether nouns are likely to be used together. The PMI measure specifically targets whether speakers will be likely to encounter two nouns closely together and need to differentiate between them in context. Two words could independently have fairly simi- lar distributions without ever actually co-occurring with one another. For example, despite their very similar meaning, 'gentleman' and "guy' are less likely to be used together, than 'knife' and 'fork'. Words that, in addition to being semantically simi- lar, would also be likely to co-occur are the ones for which additional disambiguation would be func- tionally most relevant. Introducing PMI as an addi- tional measure allows to test whether language systems are sensitive to co-occurrence likelihoods. Only in calculations involving PMIs, did we dis- card 25,073 noun pairs with negative PMIs, since they would have been computed on co-occurences too small to yield reliable information. For all other calculations the full set was used.
3.3 Evaluating the interactions
We used Generalized Additive Model (GAM) (Wood, 2012; Team, 2020) to evaluate the interac- tions between noun log frequencies, cosine simi- larities, and non negative PMI in predicting SCM scores as defined above.
In order to be able to evaluate two separate conditions in which (a) classifiers were the only element in noun phrase contributing to lower- ing an upcoming noun's entropy and (b) classi- fiers and pre-nominal adjectives were jointly con- tributing lowering noun's entropy, had further split our data into two separate scenarios: nouns directly following classifiers, and classifer- modifier(s) -noun sequences. We generated GAM models for each one of those scenarios and for the combined data set that did not distinguish between the two.
3.4 Probing the effects of noun phrase structure
To further evaluate the relevance of noun phrase structure noun-classifier combinatorics, we also computed the following measures for classifiers im- mediately preceding their nouns vs. classifiers sep- arated from their nouns by additional pre-nominal modifiers:
3.2 Class membership
The goal of study is to evaluate whether Mandarin classifiers show the distribution of grammatical noun class markers defined by Dye et al. (2017b.a, 2018). The expected profile would present a correlation between same noun class membership and interactions between noun frequencies, similarities, and likelihood of co- occurrence. When evaluating gender systems, studies typically consider two nouns as belonging the same class when they have the same gen- der. Since classifier systems do not display simple one-to-one correspondences, we defined a numeric same class membership score based on nouns' shared classifiers. Our metric is based on the cumu- lative proportion of shared classifiers across two nouns' classifier sets. For each noun pair (n1, n2), we used the classifier sets extracted from the full of nouns to compute their
The mean frequency of nouns in the two sce- narios: According to Dye et al. (2018), if a noun class system makes use of probabilisitic noun classification, it should allow for lower frequency nouns to appear in noun phrases that contain addi- tional mntropy-smoothing pre-nominal modifiers. This should be reflected in the mean frequencies of nouns directly following classifiers vs. nouns in classifier-modiffer(() noun sequences.
The overall mutual information between clas- sifiers and nouns: Mutual information (MI) (Cover and Thomas, 2012) indicates how much in- formation (in bits) is shared between a classifier and its corresponding head noun. classifiers that immediately precede their nouns play a greater role in noun classification than those combined with probabilistic entropy smoothing, the mutual infor- mation between those classifiers and their nouns should be significantly higher. If C and N rep- resent the sets of all classifiers and nouns respec- tively, and and n their corresponding elements, then MI between each type of classifier and its cor- responding noun is defined as:
data for each shared class membership (SCM) score. SCM is defined as the sum of the number of shared clas- sifiers across both sets (fclfl, clf2)) normalized on set ((clfl)) and the number of shared classifiers ((clfl, clf2)) normalized on set 2 ((cl/2)):
SCM(n1, n2) = (clf1, (df11
(df1,
(cl

---

T(N:C) H(N) - H(NIC)
E
pla, c)log p(n, c)
(3)
p(n)p((c
DCN.cCC
nound 100
4 Results
4.1 Frequency, similarity, and co-occurrence effects
Figures and show the types of interactions re- ported for German by Dyc et al. (2018) on our full Mandarin noun data set. As can be seen from Fig- ure 1, the likelihood of shared classifiers increases when nouns are more similar. This effect is how- ever reduced in cases where nouns are more likely to co-occur, as indicated by higher PMI score (F(9.96) 20.39, 0.0001). This result is consis- tent with the results reported for German and also congruent with the study on gender classes and noun similarity by Williams et al. (2019). The fre- quency effects shown in Figure however are the opposite of the ones found by Dye et al. for Ger- man. In our Mandarin data, likelihood of shared classifiers increases with frequency (F(16.12) 90.19. 0.0001).
nount
Figure 2: 2-way interaction and frequency effects full dataset: the more frequent two nouns are, the more likely they are to share classifiers.
are, the likely they share classifiers, but co-occurrence likelihood dampens that effect; frequency however increases the likelihood of clas- sifier sharing across the board (F(15.96) 27.54, < 0.0001).
tagaa www.
Figure 3:
3-way interaction between noun similarity, PMI, and summed noun log frequencies.
0.0
0.2
0.4
0.6
am
The interactions between individual log frequen- cies and increasing similarities shown Figure 4 are more interesting. When taking into account 3- way interactions, we observe a pattern that could not have been seen when considering 2-way in- teractions only: in low similarity contexts, shared classifiers correlate with frequencies much like the patterns in the 2-way interaction illustrated in Fig- ure in high similarity contexts however, this tendency reverses: low frequency nouns are more likely to share classifiers, while high frequency nouns are more likely to appear with different sets
Figure 2-way interaction between noun similarity and co-occurrence in full dataset: the more similar two nouns are, the more likely they are to share classifiers. This effect however weakened for nouns occurring in similar contexts (high PMI).
4.2
3-way interactions
Figures 3 and 4 show these results 3-way inter- actions. Results shown in Figure simply con- firm the general pattern found in Figure mod- ulated by frequency: the more similar two nouns

---

of classifiers. The high similarity context shows patterns that correspond the frequency patterns reported for German by Dye et al..
0.853
6th
Figure 5: 3-way interactions in immediate classifier- noun sequences are significant (F(41.44) 16.67, p < 0,0001) and their effects are the same as across all nouns with reversed patters for high vs. low similarity con- texts.
Figure 4: 3-way interaction between individual frequen- cies and noun similarity: In high similarity contexts, low frequency nouns more likely to share classifiers, while high frequency nouns are more likely to appear with different sets of classifiers. texts the frequency effect is inversed. This three-way interaction is highly sigmificant (F(43.81) = 33.76, p 0.0001.)
low similarity con-
4.3 Effects with different noun phrase structures
Interestingly, this effect is even stronger when only considering nouns immediately preceded by clas- sifiers, as shown in Figure 5, but it disappears for nouns additionally combined with pre- pre-nominal modifiers. As shown in Figure 6, in that scenario, frequency is the main effect significantly contribut- ing to predicting shared classifiers (with more fre- quent nouns being more likely to share classifiers).
Figure 6: 3-way interactions in classiferrmodiffer(s)) noun sequences are significant (F(32.77) 26.93, p 0.0001) but higher frequency consistently correlates with shared classifiers, independently of similarity.
The effect of similarity in interaction with co- occurrence likelihood also differs across our two scenarios. Lower frequency nouns appear share classifiers when they are more similar in the im- mediate classifier-noun sequence (Figure 7), but the pattern is reversed in noun phrases additionally containing pre-nominal modifiers (Figure 8). With increasing frequency, both pattern reverse and are still almost mirror images of each other in high fre- quency contexts. In both cases PMI contributes to mitigating similarity effects, but this interac- tion seems stronger in immediate classifier-noun sequences.
pending on noun phrase structure.
4.4 Additional effects
Interactions between overall noun frequencies and noun phrase structure further confirm what has been shown by the GAM models. Nouns immedi- ately preceded by classifiers are significantly more frequent on average than nouns additionally mod- ified (F(1) 3.69, p 0.027). Similarly, the mu- tual information between nouns and their immedi- ately preceding classifiers about 7.90 bits, while that between further modified nouns and their clas- sifiers lies at 6.79. A one-way ANOVA confirmed that the difference in mutual information between
Overall, the GAM models reveal interesting 3- way interactions with significant differences de-

---

tive pressures in section 1). Frequency relates speakers' familiarity with the data, the likelihood of whether they know the data or need to infer meaning from other data, and how much they rely on the data in communication. Cosine similarity as measure of general semantic similarity cor- responds to related meaning speakers can lever- age when encountering aare, previously unknown noun. PMI measure of co-occurrence likeli- hood relates to the likelihood that speaker would have to differentiate between two nouns in context. We find that for words that tend to co-occur, there is an increase of likelihood of being combined with different classifiers even if two nouns are seman- tically similar according to their broader distribu- tions. This effect is part of the communicative optimization observe in the data. While two nouns that co-occur would likely be more seman- tically similar than two other random nouns, the fact that two nouns never closely co-occur does not mean that they do not otherwise share similarities in their distributions. In a later experiment, we increased the window size for the PMI measures to respectively 3, 5 and 10 word bidirectional win- dows. The effect was strongest with 2 or 3-word bidirectional window and completely disappeared with 5 and 10 word windows. This further confirms that if the window is too large, co-occurrence is no longer a factor and cosine similarity and PMI no longer capture two different aspects.
Figure 7: 3-way interactions between similarity, PMI and summed log frequencies for noums immediately pre- ceded by classifiers are significant (F(27.49) = 28.28., 0.0001).
What our results illustrate is that languages ap- pear optimize across frequencies to (i) enhance discriminability to facilitate efficient communica- tion across the most widely used (i.c., most fre- quent) parts of the system, especially when words are likely to co-occur, and (ii) leverage regularity in the less frequent part of the system to main- tain learnability of rarer words and structures po- tentially encountered late in life.
Figure 8: 3-way interactions between similarity, PMT and summed log frequencies for nouns preceded by ad- ditional pre-nominal modifiers are significant (F(22.06) = 49.83, < 0.0001).
nouns and their immediately preceding classifiers and other nouns and their classifiers is significant (F(1) 29.41, 0.0001).
5 Discussion and conclusion
5.1 Relation to previous work
Overall, our results show that noun-classifier com- binations are sensitive to the nouns' frequencies and similarities. Nouns are overall significantly more likely to share the same classifiers if they are more frequent and/or more semantically similar. Co-occurrence likelihood (PMI) mitigates some of those effects.
Our results on noun similarity effects are congru- with the cross-linguistic similarity effects ported by Williams et al. (2019). Interestingly, however, while their study used Mandarin as an ex- ample of a genderless language, our results prove that Mandarin does in fact make use of grammat- ical noun classification which is functionally very close to that of gender systems.
Frequency, cosine similarity, and PMT measures capture different communicative scenarios (see de- scription of the interactions between communica-
Our results on similarity and co-oceurrence in- teractions further confirm results reported by Dye

---

al. (2017a.b, 2018). When considering fre- quency however, our models show the opposite of the results reported by Dye et al. (2017a,b, 2018) for German. Interestingly, though, we do find the same results as Dye et al. when taking into account 3-way interactions- - but only in high similarity con- texts. There are several reasons why this might be the case:
countered late by native speakers, if all. High fre- quency nouns thus mostly benefit from association with different classes that facilitate their discrimi- nation in context, while low frequency nouns ben- efit from being associated with noun classes that can easily be extrapolated from the class semanti- cally similar nouns belong to. These are exactly the pattern see emerge for Mandarin classifiers when looking at 3-way interactions. Our study is the first full investigation of the this kind of 3-way interaction between frequency, similarity, and co- occurrence between nouns and their class member- ship.
(i) Mandarin and German noun classification could make use of opposing strategies when inter- acting with frequency. However, this interpreta- tion does not seem likely given the more detailed understanding of the data obtained when consider- ing 3-way interactions.
Our second major finding confirms the idea that noun class systems can make of grammati- cal and probabilistic noun classification simulta- neously. our results, we see that noun phrases that also contain additional pre-nominal modifiers, such as adjectives, do not display the inverting fre- quency effects depending on noun similarity de- scribed above.
(ii) The sampling performed by the authors that only takes into account nouns appearing in all num- ber and case combinations introduces a bias that goes against natural form distributions often de- scribed using Heap's ss law (Ross, 1960) and the way it applies to word form lemma relations. Accord- ing to Heap's law only a small subset of lemmas will ever be encountered in all their inflected forms in any given corpus, and increasing the corpus size to capture more complete paradigms comes with a diminishing return. Artificially restricting the sample of nouns to only nouns appearing full paradigms in their corpus might have introduced bias that the model could not control for.
Yet, when looking correlations between noun frequency and noun phrase structure and mutual information and noun phrase structure, we also find significant differences between nouns directly introduced by classifiers and nouns addionally smoothed by pre-nominal modifiers. We relate these differences to the existence of dual strategies in the noun classification system. High frequency nouns tend appear in noun phrases that do not contain multiple additional pre-nominal modifiers that would probabilistically smoothe their entropy in context. They rely on grammatical noun classi- fication operated by the classifiers directly. This is also reflected in the higher mutual information between such nouns and their classifiers. On the other hand, lower frequency nouns tend to be ad- ditionally smoothed by pre-nominally modifiers, share less mutual information with their classifiers and appear to rely on more probabilistic noun clas- sification in the sense of Dye et al. (2018). For the first time, our study on Mandarin shows how differential behaviors displayed by class member- ship based on (a) noun pair similarity and (b) noun phrase structure reflects interactions between grammatical and probabilistic classification of nouns.
(ii) While Dye et al. discuss possible 3- 3-way in- teractions between frequency, similarity, and co- occurrence, they do not in fact compute any 3- way interactions. If they had taken into account these more complex interactions, they might have found patterns that mitigate their conclusion that frequency correlates with distribution over differ- ent gender classes. Our study is the first to use 3-way interactions between frequency, similarity, and co-occurrence between nouns to provide complete understanding of their contextual intrica- cies related to patterns of noun class membership.
5.2 Novel findings
Interestingly, our first major finding is that, despite partial differences in reported results for 2-way interactions, the 3-way interactions we find con- firm the conclusion presented by Dye al. (2017b, 2018): noun class systems optimize accommo- date opposing communicative pressures that ap- ply to high vs. low frequency nouns: while high frequency nouns are typically learned carly and known by all, low frequency nouns may only be en-
Limitations
A first limitation of this paper is that it does not initially replicate the studies its results are being

---

compared to. We attempted to reproduce the re- sults in (Dye, 2017; Dye al,, 2018), but could not extract sufficienlly precise information from those publications to re-run a version their study. Our conclusions and discussion of their relation to Dye et al.'s previous findings are therefore based on an understanding of their general conclusions rather than concrete results.
reduction of upcoming nouns: Sortal classifiers reduce entropy more than measure words.
Acknowledgements
This project was supported by resources provided by the Office of Research Computing at George Mason University (URL: https://orc.gmm..com funded in part by grants from the National Sci- ence Foundation (Awards Number 1625039 and 20 2018631).
and
Compared to many contemporary computa- tional linguistic studies, the work presented in this paper conducted set. A larger study could be conducted by employ- ing more powerful computational resources. In this paper, we instead decided to develop a con- trolled sampling technique which we applied to an originally large data set (3M sentences from The Leipzig Corpora Collection). Although reducing the amount of data fed to the models, this tech- nique still preserves the frequency distributions of the full data set and has the advantage of being ap- plicable with modest computational resources.
relatively small data
References
Alexandra Y Aikhenvald and Elena Mihas. 2019. Gen- ders and classifters: a cross-linguistic typology. Ox- ford University Press.
Inbal Armon and Michael Ramscar. 2012. Granular- ity and the acquisition of grammatical gender: How order-of-acquisiine affects what gets learned. Cog- nition, 122(3):922-305
James P Blevins, Petar Milin, and Michael Ramscar. 2017. The Zipfian paradigm cell filling problem. Perspectives morphological organization, pages 139-158. Brill.
Finally. classifiers are commonly sub- categorized into measure words and sortal classifiers. However, a previous study (Wang and Walther, 2023), we showed that these two subcategories are functionally and distributionally sufficiently distinct to warrant treating them as separate syntactic (sub-Jeategories. In this study, we were most interested in sortal classifiers, but given their pre-nominal distribution, measure words were not irrelevant to the entropy reduction and noun classification questions we explore here. Preliminary experiments we conducted did not show any major differences between the two types of classifiers regarding the similarity, frequency, and co-occurrence measures presented in this paper. Part of the reason for this may be that even though measure words are typically described as more flexible in their combinations with differ- ent nouns, sortal classifiers also have generic counterpart that can be used with most not all nouns, which mitigates this bias. Nevertheless, the question of whether sortal and mensural classifiers play partly different roles remains interesting and would constitute an interesting extension of this study, that would take closer look at the details of the classifier systems and not just at the functional aspects of noun classification. In (Wang and Walther, 2023), we already found that there was a significant difference between sortal and mensural classifiers in terms of entropy
Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. Enriching word vectors with subword information. Transactions of the Associa- tion for Computational Linguisties,
Danqi Chen and Christopher Manning. 2014. A fast and accurate dependency parser using neural net- works. In Proceedings the 2014 conference on empirical methods in natural language processing (EMNLP), pages 740-750.
Kenneth Church and Patrick Hanks. 1990. Word associ- ation norms, mutual information, and lexicography. Computational linguistics, 16(1):22-29
Herbert H Clark and Thomas Wasow, 1998. Repeating words in spontaneous speech. Cognitive psychology. 3713):201-242.
Greville G Corbett et al. 1991. Gender. Cambridge University Press.
Thomas M Cover and Joy A Thomas. 2012. Elements of information theory. 2012. Google Scholar Google Scholar Digital Library Digital Library.
Denis Creissels. 2006. Syntaxe générale: une introduc- tion typologique. (No Title).
Melody Dye. 2017. Bridging levels of analysis: Learn- ing, information theory, and the lexicon. ProQuest LLC.
Melody Dye, Petar Milin, Richard Futrell, and Michael Ramscar. 2017a. Cute little puppies and nice cold beers: An information theoretic analysis of prenom- inal adjectives. In SogSci.

---

Melody Dye, Petar Milin, Richard Futrell, and Michael Ramscar. 2017b. A functional theory of gender paradigms. In Perspectives morphological orga- nization, pages 212-239. Brill.
Claude E Shannon. 1948. A mathematical theory of communication. The Bell system technical journal. 27(3):379-423.
Cyrus Shaoul and Chris Westbury. 2010. Exploring lex- ical co-occurrence space using hidex. Behavior Re- search Methods, 42(2):393-413.
Melody Dye, Petar Milin, Richard Futrell, and Michael Ramscar. 2018. Alternative solutions to language design problem: The role of adjectives and gender marking in efficient communication. Topics in cog- nitive science, 10(1):20-2224
Core Team. 2020. R Core Team R: a language and enviromment for statistical computing. Foundation for Statistical Computing.
John Firth. 1957.
A synopsis of linguistic theory, 1930- 1955. Studies in linguistic analysis, pages 10-32.
Gabriella Vigliocco, Tiziana Antonini, and Merrill F Garrett. 1997. Grammatical gender is on the tip of italian tongues. Psychological science, 8(4):314- 317.
Dirk Goldhahn, Thomas Eckart, and Uwe Quasthoff. 2012. Building large monolingual dictionaries at the leipzig corpora collection: From 100 to 200 lan- guages. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC' 12), pages 759-765.
Yamei Wang and Géraldine Walther. 2023. Measure words are measurably different from sortal classi- fiers. In Proceedings the Seventh International Conference on Dependency Linguistics (Depling. GURT/SyntaXess 2023), pages 81-90.
Theres Grüter, Elaine Lau, and Wenyi Ling. 2020. How classifiers facilitate predictive processing and chinese: The role of semantic and grammatical cues. Language, Cognition and Neuroscience, 35(2):221- 234.
Adina Williams, Ryan Cotterell, Lawrence Wolf- Sonkin, Damián Blasi, and Hanna Wallach. 2019. Quantifying the semantic core of gender systems. arXiv preprint 1910.11910.1300
Zellig S Harris. 1954, Distributional structure. Word, 10(2-3):166-62.
Simon Wood. 2012. mgev: Mixed GAM Computation Vehicle with GCV/AIC/EMIL smoothness estima- tion.
Elaine Lau and Theres Grüter. 2015. Real-time process- ing of classifier information by 12 speakers of chi- In Proceedings of the 39th annual Boston Uni- versity conference on language development, pages 31 3-1-323.
Charles N Li and Sandra A Thompson. 1989. Mandarin Chinese: functional reference grammar. Univ of California Press.
XuPing Li. 2013. Numeral classifiers in chinese. In Numeral Classifiers in Chinese. De Gruyter Mouton.
Neal Szu-Yen Liang. 2008. The acquisition of chinese shape classifiers by 12 adult learners. Proceedings of the 20th North American Conference on Chinese Linguistics (NACCL-200), volume 1, pages 309-326.
Shijia Liu, Hongyuan Mei, Adina Williams, and Ryan Cotterell. 2019. On the idiosyncrasies of the man- darin chinese classifier system. arXiv preprint arXiv: 1902. 10793.
Slav Petrov, Dipanjan Das, and Ryan McDonald. 201 A universal part-of-speech tagset. arXiv preprint arXiv: 104.2086.
Radim Rehüfek and Petr Sojka. 2010. Software Framework for Topic Modelling with Large Corpora. Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, pages 45-50, Val- letta, Malta. ELRA.
Alan SC Ross. 1960. Type-token mathematics: text- book of mathematical linguistics.


Reviews:
[{"review_id": "AqfzydZEDw", "reviewer": "Reviewer_DR8p", "summary": "", "questions": "A. In section 3.1 the authors state \"We generated non-ordered noun pairs for those 1% and computed their summed frequency.\"; was this the measure reported in Figures 3 and 4? In other places only frequency is mentioned which is a bit confusing. \n\nB. Isn't cosine similarity and PMI correlated?\n\nC. What are these measures, i.e., summed frequency, cosine similarity and PMI, capturing about the data? They all seem to be capturing certain kind of similarity? e.g., co-occurences can also be deemed to capture semantic similarity (as in HAL, LSA); similarly if two words are highly frequent then they might also be similar?\n\nD. \"As can be seen from Figure 2, the likelihood of shared classifiers increases when nouns are more similar.\"; i suppose the authors meant Figure 1?\n\nE. \"the more similar two nouns are, the more likely they are to share classifiers, but co-occurrence likelihood dampens that effect\" I wonder if this has something to do with the 2 window threshold for computing PMIs?\n\nF. \"The effect of similarity in interaction with co-occurrence likelihood ...\" does co-occurrence mean PMI? please be consistent with terminology.\n\nG. \"The vectors were included as a quantifiable measure of a noun’s meaning based on the distributional hypothesis laid out by Harris (1954) and Firth (1957).\" where was this used?\n\nF. Taking together the findings of the 3-way interaction and the \"correlations between noun frequency and noun phrase structure and mutual information and noun phrase structure\", will it be reasonable to say that the low frequency effect (in high similarity situations) is being driven by noun phrases with pre-nominal modifiers? and the high frequency pattern (in the low similarity situations) is driven by noun phrases with classifiers? and if that is the case then the replication of Dye et al. is only applicable for the noun phrases with pre-nominal modifiers?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "*Thank you for taking the time to review our paper and for your thoughtful comments and questions!*\n\n### Response to reasons to reject\n\n**_1. I thought that the rationale to include various metrics was not clearly mentioned. For example, while it is clear that the correspondence between nouns and classifiers should be influenced by nominal semantics. It is unclear why noun pair frequency and PMI should be relevant?_**\n\nThank you for highlighting where our explanations were not entirely transparent, or came too late to be helpful.\n\nThe reason why frequency and co-occurrences are expected to play a significant role in the organization of noun classes derives from their relation to competing communicative pressures as defined by Blevins et al. 2017. The explanations we give in the discussion are probably introduced too late and we should have added some more details as early as in the introduction. We will certainly do so in the final version of this paper should it be accepted. We are summarizing the interactions between communicative pressures below:\n\n**Regarding the role of frequency:**\nAs speakers, humans are confronted with two main challenges: the first is *learning their language*, for which regularities across similar words (or structures) are the most optimal principle of organization; the second is *using their language* to efficiently communicate with others, for which enhanced discriminability of words and structures provides the most optimal principle. Irregularities make words and structures more item specific and thus increase their discriminability —  which allows for faster and more efficient communication.\n\nA fully regular system would not be optimized for communication, whereas a fully irregular one would make a language extremely difficult if not impossible to learn. However, not all parts of the system present the same challenge for learning: highly frequent words and structures are typically encountered early by learners and constantly reinforced. Once they are acquired, learning is no longer necessary. Irregularities only present a minor challenge towards the beginning of human language development, but they are of no problem at all later in life. They do however make communication faster and more efficient. This is especially useful for those words and structures that speakers use the most frequently.\nInfrequent words and structures, on the other hand, are typically only encountered late in life, if at all. Even proficient adult speakers will need to leverage their knowledge about related words and structures when they first encounter them. Regularity across similar structures is especially crucial in such cases. Since those words and structures are not as frequent, processing them faster is not as crucial to increasing overall communicative efficiency.\n\nThe theoretical assumption behind those measures is therefore that languages optimize across frequencies to (i) enhance discriminability to facilitate efficient communication across the most widely used (= most frequent) parts of the system and (ii) leverage regularity in the less frequent part of the system to maintain learnability of rarer words and structures potentially encountered late in life. \n\nFor noun classification, regularity comes from grouping together semantically similar nouns. In our case, a fully regular system would translate into semantically similar nouns always sharing the same classifiers. Irregularity on the other hand corresponds to distributing similar items over different classes. In our case, a fully irregular system would translate into semantically similar nouns never sharing the same classifiers.\nThe hypothesis that we are testing here is whether Mandarin classifiers do indeed organize Mandarin nouns along those two competing principles. We find that semantically similar nouns indeed tend to share classifiers if they are infrequent and tend to be distributed across different classifiers when they are frequent. Speakers will overall expect highly frequent nouns to occur. In a given context, having a discriminable classifier will therefore be more efficient for reducing the entropy of the upcoming noun.\n\n**Regarding the role of co-occurrences:**\nNot all semantically similar words co-occur. For example, despite their very similar meaning, *gentleman* and *guy* are less likely to be used together, than *knife* and *fork*. Words that, in addition to being semantically similar, would also be likely to co-occur are the ones for which additional disambiguation would be functionally the most relevant. Introducing PMI as an additional measure allows us to test whether language systems are sensitive to co-occurrence likelihoods. In the case of Mandarin classifier systems, we find that this is indeed the case.\n\n\n**_2. Relatedly, the 3 metrics, namely, frequency, cosine similarity, co-occurrence/PMI can be thought as capturing some kind of similarity. In particular, co-occurrence has been extensively used to determine semantic similarity. It is therefore unclear if cosine similarity and co-occurrence could be correlated._**\n\nCosine similarity evaluates semantic similarity across contexts that are broader than the ones we use for our PMI measures. The PMI measure looks at co-occurrences in a small bi-directional 2-word window and tests whether, in addition to general similarity, actual co-occurrences are reflected in system organization. \n\n### Response to questions\n\n**_A. In section 3.1 the authors state \"We generated non-ordered noun pairs for those 1% and computed their summed frequency.\"; was this the measure reported in Figures 3 and 4? In other places only frequency is mentioned which is a bit confusing._**\n\nThank you for finding this imprecision in the text! The text only mentions frequency, but Figure 3 indeed reports results for interactions between cosine similarity and PMI modulated by the *summed* frequency of individual noun pairs (log), as indicated in the caption. Figure 4 shows the interactions based on *individual* noun frequencies (log) modulated by different degrees of cosine similarity. We will clarify this in the text.\n\n**_B. Isn't cosine similarity and PMI correlated?_**\n\nand **_C. What are these measures, i.e., summed frequency, cosine similarity and PMI, capturing about the data? They all seem to be capturing certain kind of similarity? e.g., co-occurences can also be deemed to capture semantic similarity (as in HAL, LSA); similarly if two words are highly frequent then they might also be similar?_**\n\nThe measures are meant to capture different communicative scenarios (see description if the interactions between communicative pressures above). Frequency relates to speakers’ familiarity with the data, the likelihood of whether they know the data or need to infer meaning from other data, and how much they rely on the data in communication. Cosine similarity is used a measure of general semantic similarity. PMI relates to whether two nouns are likely to be used together.\n\nTwo words could independently have fairly similar distributions without ever actually co-occurring with one another. The PMI measure specifically targets whether speakers will be likely to encounter two nouns closely together. We find that for words that tend to co-occur, there is an increase of likelihood of being combined with different classifiers even if two nouns are semantically similar according to their broader distributions. While two nouns that co-occur would likely be more semantically similar than two other random nouns, the fact that two nouns don’t ever closely co-occur doesn’t mean that they don’t otherwise share similarities in their distributions.\n\n**_D. \"As can be seen from Figure 2, the likelihood of shared classifiers increases when nouns are more similar.\"; i suppose the authors meant Figure 1?_**\n\nYes! Thank you for catching this!\n\n**_E. \"the more similar two nouns are, the more likely they are to share classifiers, but co-occurrence likelihood dampens that effect\" I wonder if this has something to do with the 2 window threshold for computing PMIs?_**\n\nYes, since if they co-occur more, speakers will need to differentiate between them more often. This effect is part of the communicative optimization we observe in the data. \n\nIn response to this question we also ran an additional experiment increasing the window size for the PMI measures to respectively 3, 5 and 10 word bidirectional windows. The effect is strongest with a 2 or 3-word bidirectional window and completely disappears with 5 and 10 word windows. This shows that if the window is too large, co-occurrence is no longer a factor and cosine similarity and PMI no longer capture two different aspects. We will include these additional results and corresponding figures in the paper.\n\n**_F. \"The effect of similarity in interaction with co-occurrence likelihood ...\" does co-occurrence mean PMI? please be consistent with terminology._**\n\nYes, thank you! We will try to be more consistent in the final version of the paper to avoid lack of clarity on this matter.\n\n**_G. \"The vectors were included as a quantifiable measure of a noun’s meaning based on the distributional hypothesis laid out by Harris (1954) and Firth (1957).\" where was this used?_**\n\nThe vectors refer to the fastText vectors we used to capture the nouns’ semantics. This is based on the distributional semantic hypothesis that distributions capture meaning, which goes back to those two references. (We hope we understood the question correctly and our answer clarifies it. We would be happy to specify further if there are any additional questions.)\n\n**_F. Taking together the findings of the 3-way interaction and the \"correlations between noun frequency and noun phrase structure and mutual information and noun phrase structure\", will it be reasonable to say that the low frequency effect (in high similarity situations) is being driven by noun phrases with pre-nominal modifiers? and the high frequency pattern (in the low similarity situations) is driven by noun phrases with classifiers? and if that is the case then the replication of Dye et al. is only applicable for the noun phrases with pre-nominal modifiers?_**\n\nWe would think that it would be the other way around: the replication of Dye et al.'s claims on German only applies to nouns directly preceded by their classifiers. (This is being shown by the pattern inversion in Figure 5, which illustrates how semantically similar nouns will tend to combine with different classifiers the more frequent they are.)\n\nBut in addition to that, our findings also provide quantitative evidence for another claim made by Dye et al., for which they however do not provide any measures — namely that probabilistic classification serves as an alternate classification device. Dye et al. compare grammatical and probabilistic noun classification across two languages with different gender profiles (German = 3 genders, English = no gender). What we show here is that this dual classification co-exists within one language and that there are interactions between the two, with measurable effects. In cases where nouns are only introduced by their classifiers, we observe the expected grammatical classification pattern inversion from Figure 5. In cases where nouns are also introduced by pre-nominal modifiers, there is no inversion. This shows that pre-nominal modifiers are an integral part of the system that helps with noun disambiguation, to the point that they 'disrupt’ the classifier-noun combination patterns found in cases without pre-nominal modifiers. When present, pre-nominal modifiers appear to serve as the primary classification device.  In such cases, the classifiers’ function is less crucial to the way information is managed and the whole system `adapts’ by avoiding the irregularity of the high-frequency/high-similarity scenario found in nouns occurring simple classifier-noun sequences.\n\n*Thank you again for giving us an opportunity to clarify the background, the role of the different measures, and the implications of this study. If this paper is accepted, we will dedicate part of the introduction to ensure all the connections are much more evident from the start. We will also include the additional experiments on different window sizes for PMI calculations and explain implications in greater detail in the Discussion section.*", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper discusses an important property in many languages, namely, nominal classifiers. This is perhaps the first attempt to systematically study the relation between classifiers and nominals from the perspective of communicative efficiency. \n\nThe paper attempts to generalise the various mechanisms that grammars employ to classify nouns. It relates the key results with previous findings on nominal classification based on gender. The paper nicely demonstrates that seemingly arbitrary grammatical features can in fact have functional underpinnings.", "reasons_to_reject": "I thought that the rationale to include various metrics was not clearly mentioned. For example, while it is clear that the correspondence between nouns and classifiers should be influenced by nominal semantics. It is unclear why noun pair frequency and PMI should be relevant?\n\nRelatedly, the 3 metrics, namely, frequency, cosine similarity, co-ocurrence/PMI can be thought as capturing some kind of similarity. In particular, co-ocurrence has been extensively used to determine semantic similarity. It is therefore unclear if cosine similarity and co-ocurrence could be correlated. \n\nThe above issues will have significant implication on the interpretation of the results.", "questions_to_authors": "A. In section 3.1 the authors state \"We generated non-ordered noun pairs for those 1% and computed their summed frequency.\"; was this the measure reported in Figures 3 and 4? In other places only frequency is mentioned which is a bit confusing. \n\nB. Isn't cosine similarity and PMI correlated?\n\nC. What are these measures, i.e., summed frequency, cosine similarity and PMI, capturing about the data? They all seem to be capturing certain kind of similarity? e.g., co-occurences can also be deemed to capture semantic similarity (as in HAL, LSA); similarly if two words are highly frequent then they might also be similar?\n\nD. \"As can be seen from Figure 2, the likelihood of shared classifiers increases when nouns are more similar.\"; i suppose the authors meant Figure 1?\n\nE. \"the more similar two nouns are, the more likely they are to share classifiers, but co-occurrence likelihood dampens that effect\" I wonder if this has something to do with the 2 window threshold for computing PMIs?\n\nF. \"The effect of similarity in interaction with co-occurrence likelihood ...\" does co-occurrence mean PMI? please be consistent with terminology.\n\nG. \"The vectors were included as a quantifiable measure of a noun’s meaning based on the distributional hypothesis laid out by Harris (1954) and Firth (1957).\" where was this used?\n\nF. Taking together the findings of the 3-way interaction and the \"correlations between noun frequency and noun phrase structure and mutual information and noun phrase structure\", will it be reasonable to say that the low frequency effect (in high similarity situations) is being driven by noun phrases with pre-nominal modifiers? and the high frequency pattern (in the low similarity situations) is driven by noun phrases with classifiers? and if that is the case then the replication of Dye et al. is only applicable for the noun phrases with pre-nominal modifiers?", "paper_topic_and_main_contributions": "The paper investigates nominal classification in Mandarin Chinese wrt use of classifiers. The work is contextualised from the perspective of the role of communicative efficiency as a guiding principle in language. \n\nThe key takeaway is that classifiers do in fact behave like gender as far as reducing the uncertainty of the upcoming noun is concerned. This is consistent with findings in German and English where the language relies on gender and pre-nominal modifiers to achieve the same. \n\nThe work also argues for a novel finding illustrated through the interaction between frequency and noun similarity. In particular, when the nouns are less similar, high frequency corresponds to more sharing of classifiers. However, when the nouns are similar, low frequency nouns are more likely to share classifiers compared to high frequency nouns. \n\nAdditionally, the above pattern was found to be contingent on the type of noun phrase. The above pattern was only present in nouns that were preceded by classifier and not in nouns which also had an adjectival modification. This is interpreted as the use of both grammatical as well as probabilistic ways of classifying nouns in a language.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "- \"Since knowing a noun’s classifier appears just as \" --> appears to be\n\n- \"Yet, when looking correlations between ...\" --> looking at"}, "extra_text": {}}, {"review_id": "cGL58XbYDy", "reviewer": "Reviewer_igeR", "summary": "", "questions": "Question A) Why do you only sample 390 nouns from the total set of 39,011 distinct nouns? Furthermore why do you have two different numbers for the number of distinct nouns on line 247 and line 249 of page 3? \n\nQuestion B) What would have constituted a negative result for this theory about classifiers? You get different results than Dye et al. do for German, but it is not clear whether this then means that the theory about classifiers is incorrect or not.\n\nQuestion C) If classifiers, gendered articles and prenomial adjectives serve to reduce entropy about upcoming nouns, how do they work in head-initial languages such as Khmer where classifiers follow the noun? Would you expect a different pattern than in Mandarin?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "*Thank you for taking the time to write a detailed review of our paper, for giving us the opportunity to clarify some of its aspects and for raising interesting questions!*\n\n### Preliminary clarifications\n**_1. You write: “Classifiers are somewhat more complex than gender as different classifiers can be used with the same noun, (although this is typically used for measure words, e.g. for \"five rows of tables\" in English, in Mandarin the phrase would be \"five row_CL tables\" where row_CL is a classifier meaning row.”_**\n\nThis observation is absolutely true. However, when we mention that classifiers are more complex than gender systems, we don’t only mean that a noun can be introduced by more than one classifier at the same time, but most importantly that a given noun can combine with multiple different classifiers in different contexts. A gendered noun will always have the same gender regardless of the context it appears in.\n\n**_2. You write: “They also find that the previous results on German gender systems by Dye et al. 2018, do not seem to occur with Mandarin classifiers (because frequency has an intermediating factor).”_**\n\nOur results in the classifier-noun sequence scenario (Figure 5) are actually in line with what Dye et al. describe for German.\n\n### Response to reasons to reject\n\n**_A. I wonder if the similarity measure is ideal as it certainly encodes information about the words' classifiers. Perhaps a better method would be to train novel embeddings on the corpus with classifiers erased._**\n\nThis is true, but it would be more of a problem if all nouns always occurred with classifiers. We looked at the proportions of nouns occurring with and without classifiers in our data. This data is already heavily biased towards nouns with classifiers. (The data in our models only comprises nouns that occur with classifiers at least once. Those which never occur with classifiers were irrelevant to our experiments.)\n\nAmong the nouns that are part of our data, only about 22% of their instances occur with a classifier (mean 22%; median as low as 7%). If we looked at all nouns, even those never occurring with classifiers, these numbers would be even lower. This means that noun embeddings will have been influenced by the classifiers, but only minimally. It is unlikely that another model would significantly change our overall findings.\n\n**_B. Much of the discussion is prose descriptions of the charts plotting the results. The work could be strengthened by elaborating more on why these results would or would not have been theoretically expected and what kind of predictions could be expected on the basis of these results for other languages. As it stands, the interpretation of the results remains rather difficult for the reader._**\n\nThank you for pointing this out. We will do our best to streamline and emphasize the narrative more if the paper is accepted. In particular, we will dedicate part of the extra space to expanding the introduction to make the background and implications more understandable.\n\n**_C. Chinese classifiers come in two basic flavours, \"count\" and \"mass\". The former has no clear English analogue, e.g. we say five tables not five 张 tables. Mass classifiers do have a clear English equivalent, e.g. sān bàng ròu, \"three pounds of meat\" or yì hé dēngpào, \"one box of lightbulbs\". Mass classifiers do not really have the kind of arbitrary property the way gender systems do. I think it would be important to look at how the results look if one only looks at count-classifiers vs only mass-classifiers._**\n\nThis is a very interesting observation, which we did not have the space to discuss in detail in the submitted version of this paper. Sortal and mensural classifiers are indeed different and there is abundance of linguistic literature trying to decide whether or not they should be considered one or two separate syntactic categories. We are most interested in sortal classifiers in this paper, but given their pre-nominal distribution, measure words are not irrelevant to the entropy reduction and noun classification questions we explore here.\n \nPreliminary experiments we conducted but didn’t include in this study for lack of space did not show any major differences between the two types of classifiers regarding the similarity, frequency, and co-occurrence measures presented here. Part of the reason for this may be that even though measure words are typically described as more flexible in their combinations with different nouns, sortal classifiers also have a generic counterpart that can be used with most if not all nouns, which mitigates this bias. This question however remains interesting and would be a great extension of this study, looking more into the details of the classifier systems and not just into the functional aspects of noun classification. For example, we already found that there was a significant difference between sortal and mensural classifiers in terms of entropy reduction for upcoming nouns. (Sortal classifiers reduce entropy more than measure words.) \n\nWe will be happy to add some more information about this in the final version of this paper if it is accepted.\n\n### Response to questions\n\n**_Question A) Why do you only sample 390 nouns from the total set of 39,011 distinct nouns? Furthermore why do you have two different numbers for the number of distinct nouns on line 247 and line 249 of page 3?_**\n\nThe difference between the two numbers is due to the removal of 5,609 nouns (approximately 12.5% of the data) with frequencies below 25. We will make sure to better explain this point in the final version, as we agree that the two numbers are confusing without additional explanations.\n\nWe sampled 390 nouns from our overall data partly for practical and partly for ethical reasons. Our goal was to run a model that does not require huge processing power without compromising on the reliability of the results. We sampled across frequency bands to have all scenarios represented. (We discuss this in the second point of the limitations section.) We will be including the code to allow the replication of the sampling procedure. We would like our findings to be replicable, even by researchers who do not have access to high powered computing.\n\n**_Question B) What would have constituted a negative result for this theory about classifiers? You get different results than Dye et al. do for German, but it is not clear whether this then means that the theory about classifiers is incorrect or not._**\n\nA negative result would have been the absence of any discernible pattern, or a radically different one from the ones described or predicted by Dye et al. \n\nWe do get the same results as Dye et al. for nouns directly preceded by classifiers. But in addition to what they find, we also measure more complex system interactions, which quantitatively confirm their intuitions about probabilistic classification. The novelty here is that we show quantitative evidence of the interaction between grammatical and probabilistic noun classification. Our results illustrate that when probabilistic noun classification is available, noun classification does not require additional irregularities to discriminate high frequency high similarity nouns in context.\n\n**_Question C) If classifiers, gendered articles and prenomial adjectives serve to reduce entropy about upcoming nouns, how do they work in head-initial languages such as Khmer where classifiers follow the noun? Would you expect a different pattern than in Mandarin?_**\n\nThank you for asking this very interesting question! We would definitely expect a different pattern for languages in which classifiers do not precede nouns. But we would need to measure the effects to be certain. \n\nThat said, even in Mandarin, we have already found some promising results when looking at the rarer post-nominal classifiers. Post-nominal classifiers seem to be more likely to occur with nouns whose frequencies are higher. Our hypothesis is that they contribute to further specifying the meaning of more generic (high frequency) nouns in context. A study on other languages with more systematically used post-nominal classifiers would exceed the scope of this paper, but would constitute a great follow-up to this study!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- It provides an interesting account of a seemingly arbitrary part of natural language that might have a straightforward functional purpose. ", "reasons_to_reject": "- I wonder if the similarity measure is ideal as it certainly encodes information about the words' classifiers. Perhaps a better method would be to train novel embeddings on the corpus with classifiers erased. \n\n- Much of the discussion is prose descriptions of the charts plotting the results. The work could be strengthened by elaborating more on why these results would or would not have been theoretically expected and what kind of predictions could be expected on the basis of these results for other languages. As it stands, the interpretation of the results remains rather difficult for the reader.\n\n- Chinese classifiers come in two basic flavours, \"count\" and \"mass\". The former has no clear English analogue, e.g. we say five tables not five 张 tables. Mass classifiers do have a clear English equivalent, e.g. sān **bàng** ròu, \"three **pounds** of meat\" or yì **hé** dēngpào, \"one **box** of lightbulbs\". Mass classifiers do not really have the kind of arbitrary property the way gender systems do. I think it would be important to look at how the results look if one only looks at count-classifiers vs only mass-classifiers.", "questions_to_authors": "Question A) Why do you only sample 390 nouns from the total set of 39,011 distinct nouns? Furthermore why do you have two different numbers for the number of distinct nouns on line 247 and line 249 of page 3? \n\nQuestion B) What would have constituted a negative result for this theory about classifiers? You get different results than Dye et al. do for German, but it is not clear whether this then means that the theory about classifiers is incorrect or not.\n\nQuestion C) If classifiers, gendered articles and prenomial adjectives serve to reduce entropy about upcoming nouns, how do they work in head-initial languages such as Khmer where classifiers follow the noun? Would you expect a different pattern than in Mandarin?", "paper_topic_and_main_contributions": "This paper aims to explain a functional purpose for the the classifier system in Mandarin. In Mandarin, quantifiers require an obligatory classifier which does not have a direct translation in English in most cases. For example, \"two tables\" is said as \"two 张 tables\"  where 张 is a classifier which is typically used for flat objects, e.g. tables, paper, beds, etc. The classifier is obligatory when quantifying unlike in English. \n\nThis paper argues that the classifier system allows for a reduction in entropy for upcoming nouns (i.e. having heard \"two 张\" you know that the upcoming noun must be something which is flat) and that communicative pressures have lead to this system. This is adapting previous work which argues that gendered languages reduce entropy (e.g. in French hearing \"la\" instead of \"le\" reduces the number of possible nouns to follow). Like gender, often the classifiers connection to a noun is seemingly arbitrary. Classifiers are somewhat more complex than gender as different classifiers can be used with the same noun, (although this is typically used for measure words, e.g. for \"five rows of tables\" in English, in Mandarin the phrase would be \"five row_CL tables\" where row_CL is a classifier meaning row. \n\nThe study looks at a corpus of 1M Mandarin sentences and evaluates the different effects of 1) co-occurence 2) similarity and 3) frequency on whether or not two nouns have an overlap in the classifiers they licence. Then, they fit a generalized additive model on a set a of noun-noun pairs to predict how classifier-overlap is predicted by the entaglement of 1), 2) and 3). For similarity, they use pretrained FastText embeddings and take their cosine similarity.\n\nThey find that high-similarity pairs, low frequency nouns will share classifiers more often than high frequency nouns. Conversely, if the similarity is low, it is high-frequency nouns that will share classifiers more often. They also find that the previous results on German gender systems by Dye et al. 2018, do not seem to occur with Mandarin classifiers (because frequency has an intermediating factor).    \n\n", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "page 2, line 164: While \"token\" might make sense in a NLP context, it would probably be better to use lemma or word-form since you mean the a classifier might also function as a noun in different contexts rather than saying in some sentences, a single usage of a classifier has it also functioning as  a noun.\n\nTable 1 page 3: this password**s**\n\nThe charts are a bit hard to read and could probably benefit from having their axes fully labeled. In particular, it took me some time to determine what  `clf sumprop clf_n` referred to. "}}, {"review_id": "PtF8oZf7vs", "reviewer": "Reviewer_pvuj", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "*Thank you for taking the time to write a review on our paper! We hope that the additional information we provide below will help clarify the parts of our paper that weren’t sufficiently explicit.*\n\n\n### Response to reasons to reject\n**_1. Though linguistically provoking, ACL Rolling Review would not be a befitting venue for this paper. More data and/or modeling innovations would be necessary for an EMNLP long paper._**\n\nThis paper was submitted to the *Linguistic Theories, Cognitive Modeling and Psycholinguistics* track. The purpose of this study is to better understand the linguistic patterns of natural language and their relevance to human cognition. (It was not submitted to the *Language Modeling and Analysis of Language Models* track, where this criticism would potentially have been more relevant.)\n\nWhat we present is an empirical quantitative study of Mandarin for which we chose the data and methods necessary for obtaining significant results to our question: *are classifiers in Mandarin structuring nouns along noun classes that are optimized to accommodate cognitive pressures on language learning and processing?* The results we obtain with the amount of data we use are novel and significant. They show clear patterns that allow us to answer the question we ask. More data or more complex models were not necessary in this case.\n\nIn addition, as long as results are not compromised in the process, there are ethical reasons for not running huge models when less processing power is sufficient and studies can be set up in such a way that they would be replicable by a larger community, including researchers who do not have access to expansive computing resources.\n\n**_2. Since classifiers in Mandarin Chinese are more diverse than nominal genders but less than pre-nominal modifiers such as adjectives, the paper could build a more comprehensive analysis of this continuum by additionally comparing to the probabilistic end (pre-nominal modifier sequences) within Chinese and to other genderless languages such as English._**\n\nThis is partly what we are doing: we are showing that pre-nominal modifiers take over the categorizing function when they are present (see the difference between Figures 5 and 6). To the best of our knowledge, this is entirely new compared to previous studies and provides a really interesting insight into how systems self-organize and optimize for communication. \n\nRunning experiments on English — or German for that matter —  would indeed be interesting, but this would be a different study and outside of the scope of this paper. \n\nThe case of Mandarin is interesting in itself. It is particularly interesting because (i) Mandarin is generally considered genderless, and yet it presents noun classification patterns that mirror those of gendered languages when nouns are directly preceded by classifiers. (ii) We show that it combines grammatical and probabilistic noun classification, a type of complexity that would not be observable in a fully genderless language like English. (iii) Because classifiers are in a many-to-many relationship with their nouns, the system is in itself more complex than that of traditional gendered languages like German, which would likely present interactions between gender-based and adjective-based noun classification, but the gender system itself would be much simpler than a classifier system. \n\n\n**_3. Similarly, more background research on probabilistic pre-nominal modifiers would help understand the continuum. Particularly, classifiers, similar to adjectives, form a many-to-many (rather than one-to-many) correspondence to nouns._**\n\nThat is something we explicitly mention lines 154-160, when explaining the problem, and again lines 310-312 when introducing our shared class membership (SCM) measure. \n\n**_4. What are the reasons to downsample unique nouns from 44,620 to 390 (page 3)? Are those the nouns occurring \"a minimum of 25 times\"?_**\n\nThe nouns that were removed because they occurred less than 25 times were the 5,609 nouns that make up the difference between our original data set of 44,620 nouns and our final dataset of 39,011 nouns (approximately 12.5% of the data).\n\nFrom those 39,011 nouns, we selected 390 nouns (1% of the data). The nouns were randomly sampled across all frequency bands. This controlled sampling method was applied in order to preserve variation in frequency, as this frequency variation was part of the effects we wanted to explore in this study.\n\nWe decided to conduct controlled sampling of our data rather than to run experiments over the whole dataset, so we would be able to obtain reliable results without requiring immense computing resources to run the experiments. Our goal was to design the experiments in  such a way that they would be easily replicable in the future, even by researchers who do not have access to high powered computing. (We also explain this in the paper’s limitation section.) \n\nThe sampling code will be made publicly available with this paper if it is accepted. \n\n**_5. There are more advanced Chinese dependency parsers than CoreNLP (Chen and Manning, 2014)._**\n\nWe apologize if we were not precise enough here. We did not use the 2014 version of the Stanford CoreNLP parser, but version 4.4.0 released in January 2022. The 2014 reference (Chen and Manning, 2014) is the reference that is meant to be cited when using any of the versions of CoreNLP, as indicated on the resource’s website.\n\nCoreNLP is a well-maintained resource. The very last version (4.5.4) was released in March 2023. CoreNLP also serves as a basis for multiple other state-of-the art parsers.\n\n**_6. What does \"communicative pressure\" refer to?_**\n\nWe apologize if that concept was not explained enough at the beginning of the paper. We discuss this in lines 533-545, but we agree that we should introduce more details earlier on. We will dedicate more space to this in the introduction to make sure the paper is easier to read.\n\n“Communicative pressures” refer to competing pressures on language learning and language use, as defined by Blevins et al (2017): As speakers, humans face two main challenges: they have to *learn their language*, for which regularities across similar words (or structures) are the most optimal principle of organization; they also need to *use their language* in a way that allows them to efficiently communicate with others. For that, enhanced discriminability of words and structures provides the most optimal principle. Irregularities make words and structures more immediately recognizable/discriminable and allow messages to be processed more efficiently.\n\nAs a result, a fully regular system would not be optimized for communication (one communicative pressure), whereas a fully irregular one would make a language extremely difficult if not impossible to learn (the other communicative pressure).\n\n**_7. It is unclear how ii and iii lead to the conclusion that \"classifiers are inherently more semantically transparent\" in line 167._**\n\nWe mean to say that classifiers are more semantically transparent than gender markers. \n\n(i) A typical gender system like Spanish or German will have at most 2 or 3 genders, but there are hundreds of classifiers in Mandarin. As a consequence (and even though nouns can typically be introduced by several different classifiers), the combination of a noun with its (set of) classifier(s) is bound to be much more specific. The fact that individual classifiers are not combined with as many different nouns as individual gender markers makes their distributions, in other words their meanings, more specific.\n\n(ii) In addition to that, some tokens will be classifiers in some contexts and nouns in others and, as classifiers, they will retain part of the meaning they have as nouns. The fact that classifiers retain some of the meaning they have when used as a noun makes them semantically more transparent than gender markers.\n\n**_8. Why is the shared membership score (SCM) an addition of 2 fractions (line 325)?_**\n\nWe want the SCM score to reflect how many classifiers are shared between two nouns, i.e., what proportion of the classifiers occurring with noun 1 also occur with noun 2 (size of the set of shared classifiers divided by the overall number of classifiers occurring with noun 1) **and** what proportion of the classifiers occurring with noun 2 also occur with noun 1 (size of the set of shared classifiers divided by the overall number of classifiers occurring with noun 2). The SCM score is the sum of those two proportions. Its values will be between 0 and 2.\n\n\n**_9. The distinction between learning nouns earlier versus later is not introduced until page 8_**\n\nWe recognize that this should be explained more at the beginning of the paper and we will dedicate space to this in the introduction should the paper be accepted.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The analogy between nominal genders and classifiers and their interaction with nominal frequency and semantic similarity are theoretically motivating. Particularly, they observe opposite tendencies for frequent versus infrequent nouns.", "reasons_to_reject": "Though linguistically provoking, ACL Rolling Review would not be a befitting venue for this paper. More data and/or modeling innovations would be necessary for an EMNLP long paper. \n\nSome questions regarding the methodology:\n- Since classifiers in Mandarin Chinese are more diverse than nominal genders but less than pre-nominal modifiers such as adjectives, the paper could build a more comprehensive analysis of this continuum by additionally comparing to the probabilistic end (pre-nominal modifier sequences) within Chinese and to other genderless languages such as English.\n- Similarly, more background research on probabilistic pre-nominal modifiers would help understand the continuum. Particularly, classifiers, similar to adjectives, form a many-to-many (rather than one-to-many) correspondence to nouns. \n- What are the reasons to downsample unique nouns from 44,620 to 390 (page 3)? Are those the nouns occurring \"a minimum of 25 times\"?\n- There are more advanced Chinese dependency parsers than CoreNLP (Chen and Manning, 2014).\n\nSeveral concepts or arguments require further explanation:\n- What does \"communicative pressure\" refer to? \n- It is unclear how ii and iii lead to the conclusion that \"classifiers are inherently more semantically transparent\" in line 167.\n- Why is the shared membership score (SCM) an addition of 2 fractions (line 325)?\n- The distinction between learning nouns earlier versus later is not introduced until page 8", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper analyzes 3-way interactions for nouns in Mandarin Chinese between frequency, similarity, and co-occurrence with nominal classifiers. Noun phrases are extracted from Leipzig Corpora Collection, and they use the fastText embeddings for similarity comparisons.\nThe paper looks into classifiers both immediately preceding and intervened by pre-nominal modifiers and shows that interactions of Mandarin nominal classifiers mirror grammatical and probabilistic nominal elements in other languages.", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: > Summary

This paper investigates the classifier system in Mandarin Chinese, studying whether it shows signs of having been optimised for *communicative efficiency*.
Experimentally, they analyse pairs of nouns; studying how a pair’s frequency, semantic similarity, and co-occurrence probability (or PMI) predicts the amount of overlap in the nouns’ classifiers.
They find that, for low frequency nouns, more similar nouns will *more* likely share classifiers.
Similarly, they find that for high frequency nouns, more similar nouns will *less* likely share classifiers.
They take these results as sign of communicative efficiency.

> Meta review

Overall, the reviewers seem to find the paper’s research question interesting, and appreciate the connection this paper makes between gender and classifier systems. Further, this paper’s experiments seem well motivated. While some important parts of the paper were not clear (e.g., *what is the rationale behind the various used metrics?*, and *what exactly is the hypothesis being tested here, and what would constitute a negative result?*), I believe the author’s thorough responses answered those questions. In case this paper is accepted, I expect the authors will try to clear these sources of confusion in their manuscript as well.

The paper has a few drawbacks, however. In terms of scope, one of the reviewers suggested that the paper could be significantly strengthened by exploring both "count" and "mass" classifiers separately, and by comparing how these systems behave. Similarly, another review suggested that expanding these analyses to both adjectives and gender systems could provide a more comprehensive view of their role in communicative efficiency, especially since the authors argue for a similarity in the role of classifier and gender systems in natural languages.

Experimentally, one reviewer suggests that the used similarity metric here will contain information about the words' classifiers. The authors dismiss this concern by saying that only 22%/7% of the studied nouns’ instances occur with a classifier. Personally, I interpret these numbers the other way around, and believe that this is a large percentage of instances which could seriously impact the paper’s results. On the other hand, unless there is a systematic bias in how frequently nouns appear with classifiers (e.g., more frequent nouns appear with classifiers less often), I’d expect results to not change significantly since this bias will be present in most analysed nouns. In any case, training word2vec/fasttext is not particularly expensive, and I agree with reviewer igeR that running an experiment with novel embeddings trained on the corpus with classifiers erased would make these results more reliable.

On a related note, FastText uses subword information (I am not particularly familiar with how it processes Mandarin Chinese characters though). This means that your semantic similarity metric is likely to also encode wordform similarity as well. While this is not in itself an issue (as similar communicative efficiency arguments might be made for an "efficient" natural language to be optimised to distinguish wordform-similar words), it may change how these results should be interpreted. Rerunning experiments with, e.g., word2vec, could be interesting to analyse the extent to which that could have affected results.
Decision: Accept-Findings