This article presents our recent work for participation in the Second International Chinese Word Segmentation Bakeoff. Our system performs two procedures: Out-ofvocabulary extraction and word segmentation. We compose three out-of-vocabulary extraction modules: Character-based tagging with different classifiers -maximum entropy, support vector machines, and conditional random fields. We also compose three word segmentation modulescharacter-based tagging by maximum entropy classifier, maximum entropy markov model, and conditional random fields. All modules are based on previously proposed methods. We submitted three systems which are different combination of the modules. We compose three systems: Models a, b and c for the closed test tracks on all four data sets. For Models a and c, three out-of-vocabulary (OOV) word extraction modules are composed: 1. Maximum Entropy (MaxEnt) classifier-based tagging; 2. Maximum Entropy Markov Model (MEMM)-based word segmenter with Conditional Random Fields (CRF)-based chunking; 3. MEMM-based word segmenter with Support Vector Machines (SVM)-based chunking. Two lists of OOV word candidates are constructed either by voting or merging the three OOV word extraction modules. Finally, a CRFs-based word segmenter produces the final results using either of the voted list (Model a) or the merged list (Model c). Most of the classifiers use surrounding words and characters as the contextual features. Since word and character features may cause data sparse problem, we utilize a hard clustering algorithm (K-means) to define word classes and character classes in order to overcome the data sparse problem. The word classes are used as the hidden states in MEMM and CRF-based word segmenters. The character classes are used as the features in character-based tagging, character-based chunking and word segmentation. Model b is our previous method proposed in (Goh et al., 2004b) : First, a MaxEnt classifier is used to perform character-based tagging to identify OOV words in the test data. In-vocabulary (IV) word list together with the extracted OOV word candidates is used in Maximum Matching algorithm. Overlapping ambiguity is denoted by the different outputs from Forward and Backward Maximum Matching algorithm. Finally, character-based tagging by MaxEnt classifier resolves the ambiguity. Section 2 describes Models a and c. Section 3 describes Model b. Section 4 discusses the differences among the three models. Models a and c use several modules. First, a hard clustering algorithm is used to define word classes and character classes. Second, three OOV extraction modules are trained with the training data. These modules, then, extract the OOV words in the test data. Third, the OOV word candidates produced by the three OOV extraction modules are refined by voting (Model a) or merging (Model c) them. The final word list is composed by appending the OOV word candidates to the IV word list. Finally, a CRF-based word segmenter analyzes the sentence based on the new word list. We perform hard clustering for all words and characters in the training data. Kmeans algorithm is utilized. We use R 2.2.1 (http://www.r-project.org/) to perform k-means clustering. 134
This problem has been observed by some participants in the Bakeoff 2005 REF , where they applied the IOB tagging to recognize OOVs, and added the OOVs to the lexicon used in the HMMbased or CRF-based approaches.
Combination of Machine Learning Methods for Optimum Chinese Word Segmentation
Proceedings of the Fourth SIGHAN Workshop on Chinese Language Processing
Computer Science