Mapping and Generating Classifiers using an Open C hinese Ontology (2016)

Abstract: In languages such as Chinese, classifiers (CLs) play a central role in the quantification of noun-phrases. This can be a problem when generating text from input that does not specify the classifier, as in machine translation (MT) from English to Chinese. Many solutions to this problem rely on dictionaries of noun-CL pairs. However, there is no open large-scale machine-tractable dictionary of noun-CL associations. Many published resources exist, but they tend to focus on how a CL is used (e.g. what kinds of nouns can be used with it, or what features seem to be selected by each CL). In fact, since nouns are open class words, producing an exhaustive definite list of noun-CL associations is not possible, since it would quickly get out of date. Our work tries to address this problem by providing an algorithm for automatic building of a frequency based dictionary of noun-CL pairs, mapped to concepts in the Chinese Open Wordnet (Wang and Bond, 2013), an open machine-tractable dictionary for Chinese. All results will released under an open license.

## introduction classifiers (cls) are an important part of the chinese language. different scholars treat this class of words very differently. @xcite , the traditional and authoritative native chinese grammar, splits cls into nine different classes. @xcite draw a binary distinction between count-classifiers and massifiers. @xcite splits cls into three categories (measure, collective and sortal classifiers). measure classifiers describe quantities (e.g. 'a bottle of', 'a mouthful of'), collective classifiers describe arrangement of objects ('a row of', 'a bunch of'), and sortal classifiers refer to a particular noun category (which can be defined, for example, by shape). @xcite identify four main classes, individual classifiers, mass classifiers, kind classifiers, and event classifiers. @xcite define five major types of cls: sortal (which classify the kind of the noun phrase they quantify); event (which are used to quantify events); mensural (which are used to measure the amount of some property); group (which refer to a collection of members); and taxonomic (which force the noun phrase to be interpreted as a generic kind). this enumeration is far from complete, and @xcite provides a detailed literature review on the most prominent views on chinese classifiers. most languages make use of some of these classes (e.g. most languages have measure cls, as in a kilo of coffee, or group cls, as in a school of fish). what appears to be specific to some languages (e.g. chinese, japanese, thai, etc.) is a class of cls (sortal classifiers: s-cl) that depicts a selective association between quantifying morphemes and specific nouns. this association is licensed by a number of features (e.g. physical, functional, etc.) that are shared between cls and nouns they can quantify, and these morphemes add little (but redundancy) to the semantics of nounphrase they are quantifying. consider the following examples of s-cl usage in mandarin chinese: examples (1) through (4) show how the simple act of counting in mandarin chinese involves pairing up nouns with specific classifiers, if incompatible nouns and classifiers are put together then the noun phrase is infelicitous, see ( 5 ). different s-cls can be used to quantify the same noun, see ( 1 ) and ( 2 ), and the same type of s-cl can be used with many different nouns -so long as the semantic features are compatible between the s-cl and the noun, see ( 2 ) and (3). extensive work on these features is provided by @xcite -where more than 800 classifiers (both sortal and non-sortal) are linked in a database according to the nominal features they select, but providing only a few example nouns that can be quantified by each cl. these many-to-one selective associations are hard to keep track of, especially since they depend greatly on context, which often restricts or coerces the sense in which the noun is being used @xcite . examples (6-8) show how the use of different cls with ambiguous senses can help resolve this ambiguity. in (6), we can see that with the use of 个 ge, the most general s-cl in mandarin chinese, mu4tou is ambiguous because it does not restrict the noun's semantic features. with the use of 位 wèi (7), an honorific s-cl used almost exclusively with people, it can only be interpreted as "blockhead". and the reverse happens when using 根 gēn (8), a s-cl for long, slender, inanimate objects: the sense of log (of wood) of 木头 mùtou is selected. even though written resources concerning cls are abundant, they are not machine tractable, and their usage is limited by copyright. natural language processing (nlp) tasks depend heavily on open, machine tractable resources. wordnets (wn) are a good example on the joint efforts to develop machine tractable dictionaries, linked in rich hierarchies. resources like wns play a central role in many nlp tasks (e.g. word sense disambiguation, question answering, etc.). @xcite argue that the integration between corpora and knowledge rich resources, like dictionaries, can offer good insights and generalizations on linguistic knowledge. in this paper, we follow the same line of thought by integrating both a large collection of chinese corpora and a knowledge rich resource (the chinese open wordnet: cow @xcite ). cow is a large open, machine tractable, chinese semantic ontology, but it lacks information on noun-cl associations. we believe that enriching this resource with concept-cl links will increase the domain of it's applicability. information about cls could be used to generate cls in mt tasks, or even to improve on chinese word sense disambiguation. the remainder of this paper is structured as follows: section 2 presents related work, followed by a description of the resources used in section 3; section 4 describes the algorithms applied, and section 5 presents and discusses our results; section 6 describes ongoing and future work; and section 7 presents our conclusion. ## related work mapping cls to semantic ontologies has been attempted in the past @xcite @xcite . @xcite is the first description of leveraging hierarchical semantic classes to generalize noun-cl pairs (in thai). still, their contribution was mainly theoretical, as it failed to report on the performance of their algorithm. @xcite and @xcite further develop these ideas to develop similar works for japanese and korean. in their work, cls are assigned to semantic classes by hand, and achieve up to 81% of generation accuracy, propagating cls down semantic classes of goi-taikei @xcite . @xcite develop a similar approach using the japanese wordnet @xcite and the chinese bilingual wordnet @xcite , and report a generation score of 78.8% and 89.8% for chinese and japanese, respectively, on a small news corpus. as it is common in dictionary building, all works mentioned made use of corpora to identify and extract cls. nevertheless, extracting noun-cl associations from corpora is not a straightforward task. quantifier phrases are often used without a noun, resorting to anaphoric or deictic references to what is being quantified @xcite . similarly, synecdoches also generate noise when pattern matching @xcite . ## resources our corpus joins data from three sources: the latest dump of the chinese wikipedia, the second version of chinese gigaword @xcite and the um-corpus @xcite . this data was cleaned, sentence delimited and converted to simplified chinese script. it was further preprocessed using the stanford segmentor and pos tagger @xcite @xcite . the final version of this corpus has over 30 million sentences (950 million words). for comparison, the largest reported corpora from previous studies contained 38,000 sentences @xcite . in addition, we also used the latest version (2012) of the google ngram corpus for chinese @xcite . there are some differences between the usage of classifiers in different dialects and variations of chinese in these different corpora, but our current goal focused on collecting generalizations. future work could be done to single out differences across dialects and variants. we used cow @xcite as our lexical ontology, which shares the structure of the princeton wordnet (pwn) @xcite . to minimize coverage issues, we enriched it with data from the bilingual ontological wordnet (bow) @xcite , the southeast university wordnet (sew) @xcite , and automatically collected data from wiktionary and cldr, made available by the extended omw @xcite . the final version of this resource had information for over 261k nominal lemmas, from which over 184k were unambiguous (i.e. have only a single sense). we filtered all cls against a list of 204 s-cls provided by @xcite . following lai (2011), we treated both huang's individual classifiers and event classifiers as s-cls. ## our algorithm our algorithm produces two cl dictionaries with frequency information: a lemma based dictionary, and a concept based dictionary, using cow's extended ontology. we tested both dictionaries with a generation task, automatically validated against a held out portion the corpus. ## discussion and results in in order to allow a fair comparison, since we decided that the concept based dictionary would contain only unambiguous lemmas, we only use unambiguous lemmas to compare the performance across dictionaries. all results can be compared across the different thresholds discussed in 4.3. τ = 1, 3 and 5 present the results obtained in the automatic evaluation, using minimum frequencies of one, three and five, respectively. the first three reported results report exclusively about the lemma dictionary (including both ambiguous and unambiguous lemmas). lem-all reports the results of the prediction task, lem-allmfcl reports the results of the generation task, and lem-all-no-info reports the relative frequency of lemmas for which there was no previous infor-mation in the dictionary, and which could have boosted both task's performance by falling back on the default cl 个 ge. these initial results show that it was easy to perform better than baseline, and that τ = 1 achieved the best results on both predicting noun-cl pairs, and generating cls that matched the data. comparing different τ s shows that, even considering the over-generation reduction that imposing minimum frequencies brings (validated but not presented here), the best generation performance is achieved by not filtering the training data. and this will be consistent across the remainder of the results. when comparing both dictionaries, we look only at unambiguous lemmas. similar to what was explained above, lem-unamb and wn-unamb report the results of the prediction task for the lemma based and concept based dictionary, respectively. the labels lem-unamb-mfcl and wn-unamb-mfcl report the results for the generation task. and the lem-unamb-no-info and wn-unamb-no-info report about the lack of informed coverage (where backing-off to the default cl might have help the performance). between the lemma and the concept based dictionaries, this automatic evaluation shows that while the concept based dictionary is better at predicting if a noun-cl pair was valid, the lemma based dictionary outperforms the former in the generation task. the final results of this automatic evaluation are shown in column test, where we re-evaluated the dictionary produced by τ = 1 on the test-set. test shows slightly better results, perhaps because the random sample was easier than the dev-set, but the same tendencies as reported above. considering that the concept based dictionary should be able to provide cl information to some lemmas that have not been seen in the training data (either by expansion or by leveraging on a single lemma to provide information about synonyms), we expected the concept based dictionary to present the best results. many different reasons could be influencing these results, such as errors in the ontology, the fact that chinese cls relate better to specific senses than to concepts (i.e. different lemmas inside a concept prefer different cls), or noise introduced by the test and dev-set (since we don't have a hand curated golden test-set). for this rea-son, we decided to hand validate a sample of each dictionary. based on a random sample of 100 concepts and 100 lemmas extracted from each dictionary, a chinese native speaker checked if the top ranked cl (i.e. with highest frequency), that would be used to generate a cl for each of the randomly selected entries, was in fact a valid cl for that lemma or concept. this human validation showed the concept based dictionary outperforming the lemma based dictionary by a wide margin: 87% versus 76% valid guesses. this inversion of performance, when compared to the automatic evaluation, was confirmed to be mainly due to noisy data in the test-set caused by the automatic segmentation and pos tagging. we then looked at a bigger sample of 200 lemmas and found roughly 7.5% of invalid lemmas in the lemma based dictionary. conversely, the concept based dictionary assigns cls by 'bags of lemmas' (i.e. synsets). this allows the noise introduced by a few senses to be attenuated by the 'bag' nature of the concept. more importantly, most of the nominal lemmas included in the extended version of cow are human validated, so the quality of the concept based dictionary was confirmed to be better -since most lemmas included in it are attested to be valid. comparing the size of both dictionaries in lastly, coverage was also tested against data from a human curated database of noun-cl associations @xcite , by replicating the automatic evaluation generation task described in 4.3. this dictionary contains information about more than ples for each cl -and hence it is not designed with the same mindset. testing the best performing dictionaries (τ 1) against the data provided for s-cls, we achieved only 43.9% and 28.3% for prediction and generation, respectively, using the lemma based dictionary; compared to 49.8% and 22.4% using the concept based dictionary. the same trends in prediction and generation are observed, where the concept based dictionary is able to predict better than the lemma base, but it is outperformed by the later in the generation task. ultimately, these weak results show that even though we used a very large quantity of data, our restrictive matching patterns in conjunction with infrequent noun-cls pairs still leaves a long tail of difficult predictions. ## ongoing and future work since our method is mostly language independent, we would like to replicate it with other classifier languages for which there are open linked wn resources (such as japanese, indonesian and thai). this would require access to large amounts of text segmented, pos tagged text, and adapting the matching expressions for extracting noun-cl pairs. more training data would not only help improving overall performance on open data, by minimizing unseen data, but would also allow us to make better use of frequency threshold filters for noise reduction. lack of training data as our biggest drawback on performance, we would like to repeat this experiment with more data -including, for example, a very large web-crawled corpus in our experiments. in addition, we would also like to perform wsd on the training set, using ukb @xcite for example. this would allow an informed mapping of ambiguous senses onto the semantic ontology and, arguably, comparable performance on generating cls for ambiguous lemmas. we will also investigate further how to deal with words not in cow: first looking them up in the lemma dictionary, and then associating cls to the head (character / noun) of unseen noun-phrases, as proposed in @xcite . even though this work was mainly focused on producing an external resource linked to cow, we are also working on adding a new set of sortal classifiers concepts to cow @xcite . the absence of this class of words in cow currently prevents us from using the internal ontology structure to link nouns and classifiers. once they are represented, we will make use of this work to link nominal concepts and corresponding valid classifiers. ## conclusions our work shows that it is possible to create a high quality dictionary of noun-cls, with generation capabilities, by extracting frequency information from large corpora. we compared both a lemma based approach and a concept based approach, and our best results report a human validated performance of 87% on generation of classifiers using a concept based dictionary. this is roughly a 9% improvement against the only other known work done on chinese cl generation using wordnet @xcite . finally, we will merge all three data sets and, from them, produce a release of this data. we commit to make both lemma and wn mappings available under an open license, release along with the chinese open wordnet at http:// compling.hss.ntu.edu.sg/cow/ .