Building the C antonese W ordnet (2019)

Abstract: This paper reports on the development of the Cantonese Wordnet, a new wordnet project based on Hong Kong Cantonese. It is built using the expansion approach, leveraging on the existing Chinese Open Wordnet, and the Princeton Wordnet’s semantic hierarchy. The main goal of our project was to produce a high quality, human-curated resource – and this paper reports on the initial efforts and steady progress of our building method. It is our belief that the lexical data made available by this wordnet, including Jyutping romanization, will be useful for a variety of future uses, including many language processing tasks and linguistic research on Cantonese and its interactions with other Chinese dialects.

## cantonese: an overview cantonese is the second most widely known chinese dialect after mandarin @xcite . it is spoken in guangdong province, guangxi province, the special administrative regions of hong kong and macau, as well as diaspora communities in north america, australia, malaysia, singapore, etc. according to ethnologue, 1 there are 73 million cantonese speakers worldwide. but despite the large number of speakers, credible online resources on cantonese, free or otherwise, are limited, especially in comparison with mandarin. there is a considerable lexical overlap between cantonese and mandarin. @xcite mentions that the difference between can-1 https://www.ethnologue.com/language/yue tonese and mandarin vocabulary ranges from 30-50%. @xcite estimates that about 1/3 of the lexical items used in regular cantonese speech is not found in mandarin. to give an example for a very common item,'umbrella' is yǔsǎn ⾬傘 in mandarin but ze1 遮 in cantonese. in cases where they share the same lexical item, the item is always pronounced differently in the two dialects. for example, 'teacher' ⽼師 is pronounced as lǎoshī in mandarin and lou5si1 in cantonese. the vowels of the first syllables in each case are different, and the onsets of the second syllables are also different, not to mention tonal differences. note that the romanization system is different here as well. mandarin uses pīnyīn, which is based exclusively upon the pronunciation of the beijing dialect. in cantonese, jyutping is used, a point we will come back to later. in the existing mandarin chinese wordnet, simplified characters are used. the simplified script, adopted in 1949, aims to alleviate some of the difficulty associated with use of the traditional script, as a measure to eradicate illiteracy. in hong kong, macau and taiwan, the traditional script is used, though in the former two, changes are happening rapidly since their return to china in 1997 and 1999, respectively. cantonese is primarily a spoken variant. a lot of lexical items, excluding those shared with mandarin, do not have fixed agreed upon characters, these are often called 'characterless' words. it is not always easy to determine which character to use as there is no standardization. in some cases, multiple options are available, while in some other cases, no options are available. we will pick up on this issue in section 4.3. ## methodology there are two main methods to build wordnets @xcite . the first method is known as the 'expansion' approach, where the structure of another wordnet is used as 'pivot', and the main work is essentially a translation effortconserving the structure of the pivot wordnet and translating nodes of the hierarchy. the princeton wordnet @xcite is, by far, the most frequently used 'pivot' for projects that employ this approach. the second method is known as the 'merge' approach. this is usually a slower method, since no pivot structure is assumed, but it ensures a higher degree of freedom to more carefully model the structure of the wordnet based on the language in question, without depending on pre-assumed semantic relations. one of the immediate benefits of this approach is the ability to add new concepts that are not part of the 'pivot' language, a problem many wordnet projects that followed the 'expansion' approach have struggled with. and while the 'merge' approach is perhaps more principled in theory, the major drawback from this approach is that it does not benefit from the parallel translations available from all other projects that used the same pivot. the best example of this benefit is the open multilingual wordnet @xcite , a project that links dozens of open wordnets using pwn as the common structure. this language alignment is very useful for many nlp tasks, such as machine translation and word sense disambiguation. a recent addition to this discussion is the conception of the collaborative interlingual index @xcite -an open, language agnostic, flat-structured index that links wordnets across languages without imposing the hierarchy of any single wordnet. and even though pwn was the main contributor to the initial set of concepts present in cili, this set is no longer constrained by itmultiple projects are now able to contribute to cili's set of concepts, and gain the ben-efits of multilingual alignments without the penalty of being frozen within some imposed structure. to the best of our knowledge, the quickest and easiest way to link to cili and to access these language alignments without an imposed structure is, interestingly enough, to use the expansion approach with pwn hierarchy has pivot because all pwn concepts have direct links to cili. and this was what we decided to do. as we had no urgent need for a high coverage wordnet, for which multiple bootstrapping techniques are available to quickly create high coverage lower quality resources, we decided to build a high quality resource fully checked by native speakers. and although we knew from the start that building a wordnet from scratch would be very time-consuming, without going against our commitment to high quality, we decided to ease our task by leveraging on existing resources as much has possible. we used the chinese open wordnet @xcite as pivot. cow is a high quality hand-checked resource for mandarin chinese that was also created through the expansion approach (using pwn as pivot). this means that by linking our wordnet to cow, we would have easy access to pwn's concept ids and, as a result, also to cili. the basic assumption of our method was that while mandarin chinese and cantonese are fairly different languages, and it was clear from the start that resources from one language would not perform well in tasks for the other language, there is still a fair amount of overlap in the lexical usage. this is not without caveats, since cantonese uses traditional characters and mandarin chinese uses simplified characters -and conversion from simplified to traditional characters is inherently lossy. that being said, we decided to automatically convert the lemmas in simplified mandarin chinese to traditional chinese, and use this to jumpstart the manual construction of our cantonese wordnet. since the other mandarin chinese wordnets such as the sinica bilingual ontological wordnet @xcite , the southeast university chinese wordnet @xcite and the chinese wordnet @xcite included lem-mas that were not present in cow, we started by building a small 'chinese wordnet' from the union of all four chinese wordnets: cow, bow, sew and cwn. this was fairly easy since all these wordnets were linked to pwn's hierarchy. next, we used hanziconv foot_1 to convert all lemmas from simplified mandarin chinese to traditional characters. and finally, we generated a list of all candidate senses (with lemmas converted into traditional characters) that satisfied any of these three criteria: • senses that belong to the 4,960 'core' concepts in princeton wordnet @xcite ) -a usual measure for coverage of wordnet resources; • senses from all concepts in two sense tagged sherlock holmes stories, as reported by ntumc @xcite ; and • senses from any concept with sense sumfrequency score of one or higher, as reported by the pwn (i.e. most concepts yield sum-score of 0); ## statistics since our wordnet is currently pivoting on the hierarchy provided by pwn, through cow, we have no information about semantic relations to report. in further stages of our project, however, we might revise this position and consider taking advantage of cili to adapt our wordnet's semantic hierarchy to better fit the assumptions of cantonese native speakers. as mentioned above, in section 3.1, the process of human validation is still ongoing, and we expect to provide an update to these statistics in the camera-ready version of this paper. ## release this cantonese wordnet will be released under a creative commons attribution 4.0 international license (cc by 4.0) foot_2 . keeping up with the recent changes and requirements of the omw, the cantonese wordnet will be primary released and supported for the recent wn-lmf format, foot_3 developed and maintained by the global wordnet association. the use of wn-lmf is not only required by the most recent version of the omw, but is also an essential vehicle to access the new collaborative interlingual index @xcite . once linked to cili, our wordnet will be able to contribute with new concepts, present only in cantonese such as sap1jit6 濕 熱, an adjective with the literal meaning of 'hot wet' (it describes a general negative health condition resulting from an unhealthy lifestyle, e.g. smoking, sleep deprivation, etc.), or gung1zyu2beng6 公主病, a noun that literally means 'princess disease'. it describes girls who are over-confident, over-reliant and demand princess-like treatment. in addition, this release will also include the tab-separated-value format used by the original omw specifications. these files are still very useful for their size, simplicity, and legacy compatibilities with existing systems. one such example is the use of this data through nltk: python natural language toolkit @xcite -which currently still uses this legacy format. however, the simplicity of this format doesn't come without a cost. due to the flatter nature of this format, the jyutping romanization of cantonese lemmas will be added as separate lemmas (i.e. effectively doubling the number of words and senses within this format). the data for this wordnet is available on github foot_4 . ## conclusions and future work this paper presented the ongoing efforts to build a cantonese wordnet. we have motivated this project with the lack of digital resources available for cantonese -a major chinese dialect. we have introduced our methodology, which is to use existing mandarin wordnets to project cantonese candidate senses. so far our wordnet includes over 3,500 concepts and over 12,000 senses. we have discussed some specific challenges encountered while building our wordnet and how we addressed them. we hope that this new open resource will promote a variety of future uses, including language processing tasks and linguistic research. we would like to continue our efforts to improve the coverage and quality of our cantonese wordnet. this would include: • finish validating and revising the list of candidate senses generated through the methods explained in section 3 (so far we have completed 38.25% of this validation); • add example sentences for each sense, which would be the start of an open, sense-tagged cantonese corpus; • given that cantonese is predominantly used in speech, we would also like to add audio recording for each pronunciation of each lemma; once the cantonese wordnet reaches a sufficient coverage, we would like to use it to research a variety of topics, including: • study the amount of mandarin words that have entered common cantonese speech and writing and, conversely, when and why some mandarin words are never used cantonese; • study the morphologically conditioned tone changes in cantonese such as pinjam and other less understood phenomena; and • shed some light on the potential relation between register (formal register is often tied to written chinese, which is based on mandarin) and tone change (a speech phenomenon);