In this paper , we address a unique problem in Chinese language processing and report on our study on extending a Chinese thesaurus with region-specific words , mostly from the financial domain , from various Chinese speech communities . With the larger goal of automatically constructing a Pan-Chinese lexical resource , this work aims at taking an existing semantic classificatory structure as leverage and incorporating new words into it . In particular , it is important to see if the classification could accommodate new words from heterogeneous data sources , and whether simple similarity measures and clustering methods could cope with such variation . We use the cosine function for similarity and test it on automatically classifying 120 target words from four regions , using different datasets for the extraction of feature vectors . The automatic classification results were evaluated against human judgement , and the performance was encouraging , with accuracy reaching over 85 % in some cases . Thus while human judgement is not straightforward and it is difficult to create a Pan-Chinese lexicon manually , it is observed that combining simple clustering methods with the appropriate data sources appears to be a promising approach toward its automatic construction . Large-scale semantic lexicons are important resources for many natural language processing ( NLP ) tasks . For a significant world language such as Chinese , it is especially critical to capture the substantial regional variation as an important part of the lexical knowledge , which will be useful for many NLP applications , including natural language understanding , information retrieval , and machine translation . Existing Chinese lexical resources , however , are often based on language use in one particular region and thus lack the desired comprehensiveness . Toward this end , Tsou and Kwong ( 2006 ) proposed a comprehensive Pan-Chinese lexical resource , based on a large and unique synchronous Chinese corpus as an authentic source for lexical acquisition and analysis across various Chinese speech communities . To allow maximum versatility and portability , it is expected to document the core and universal substances of the language on the one hand , and also the more subtle variations found in different communities on the other . Different Chinese speech communities might share lexical items in the same form but with different meanings . For instance , the word 居屋 refers to general housing in Mainland China but specifically to housing under the Home Ownership Scheme in Hong Kong ; and while the word 住房 is similar to 居屋 to mean general housing in Mainland China , it is rarely seen in the Hong Kong context . Hence , the current study aims at taking an existing Chinese thesaurus , namely the Tongyici Cilin 同義詞詞林 , as leverage and extending it with lexical items specific to individual Chinese speech communities . In particular , the feasibility depends on the following issues : ( 1 ) Can lexical items from various Chinese speech communities , that is , from such heterogeneous sources , be classified as effectively with methods shown to work for clustering closely related words from presumably the same , or homogenous , source ? ( 2 ) Could existing semantic classificatory structures accommodate concepts and expressions specific to individual Chinese speech communities ? Measuring similarity will make sense only if the feature vectors of the two words under comparison are directly comparable . There is usually no problem if both words and their contextual features are from the same data source . Since Tongyici Cilin ( or simply Cilin hereafter ) is based on the vocabulary used in Mainland China , it is not clear how often these words will be found in data from other places , and even if they are found , how well the feature vectors extracted could reflect the expected usage or sense . Our hypothesis is that it will be more effective to classify new words from Mainland China with respect to Cilin categories , than to do the same on new words from regions outside Mainland China . Furthermore , if this hypothesis holds , one would need to consider separate mechanisms to cluster heterogeneous regionspecific words in the Pan-Chinese context . Thus in the current study we sampled 30 target words specific to each of Beijing , Hong Kong , Singapore , and Taipei , from the financial domain ; and used the cosine similarity function to classify them into one or more of the semantic categories in Cilin . The automatic classification results were compared with a simple baseline method , against human judgement as the gold standard . In general , an accuracy of up to 85 % could be reached with the top 15 candidates considered . It turns out that our hypothesis is supported by the Taipei test data , whereas the data heterogeneity effect is less obvious in Hong Kong and Singapore test data , though the effect on individual test items varies . In Section 2 , we will briefly review related work and highlight the innovations of the current study . In Sections 3 and 4 , we will describe the materials used and the experimental setup respectively . Results will be presented and discussed with future directions in Section 5 , followed by a conclusion in Section 6 . In this paper , we have reported our study on a unique problem in Chinese language processing , namely extending a Chinese thesaurus with new words from various Chinese speech communities , including Beijing , Hong Kong , Singapore and Taipei . The critical issues include whether the existing classificatory structure could accommodate concepts and expressions specific to various Chinese speech communities , and whether the difference in textual sources might pose difficulty in using conventional similarity measures for the automatic classification . Our experiments , using the cosine function to measure similarity and testing with various sources for extracting contextual vectors , suggest that the classification performance might depend on the compatibility between the words in the thesaurus and the sources from which the target words are drawn . Evaluated against human judgement , an accuracy of over 85 % was reached in some cases , which were much higher than the baseline and were very encouraging in general . While human judgement is not straightforward and it is difficult to create a Pan-Chinese lexicon manually , combining simple classification methods with the appropriate data sources seems to be a promising approach toward its automatic construction .

Challenge: Although there are varieties in the Chinese language, current lexical resources based on one variant leave questions of compatibility to others when to be extended.
Approach: They sample 30 finance-related words from several Chinese languages and evaluate if a classifier can match them to correct categories in well studied variant.
Outcome: The evaluation with human judgements shows that classification accuracy reaches up to 85% and also finds that the performance depends on word sources.