tt,ttt tit Machine) YamCha Penn Chinese Treebank (base phrase chunking) 2 (Support Vector MOZ 88% MOZ 4% YamCha 93% : YamCha, MOZ
T ATSUMI Y OSHIDA † and K YONORI OHTAKE †† and K AZUHIDE Y MAMAMOTO †† :††† We will report performances of currently and publicly available Chinese analyzers and resources.  We use YamCha, a tool based on Support Vector Machines, and the Penn Chinese Treebank as a language resource. Combining these two, we mea- sure the performances of Chinese analysis, i.e., word segmentation, part-of-speech tagging, and base phrase chunking. In the experiment of word segmentation and part-of-speech tagging, we also report the performance of MOZ, a statistical mor- phological analyzer, which is also available to the public.  We found that the accuracy of morphological analysis using YamCha attains around 88%, which is over 4% higher than that of MOZ, although it is computationally very expensive.  We also found that the accuracy for base phrase chunking is approximately 93%. KeyWords: Chinese analysis, Support Vector Machine, Chunking, YamCha, MOZ i Vol. 10 No. 1 Jan.  2003 (
“到北京去”
“到”
“寒川77
“お気”う”
2002) CTB
YamCha SVM Penn Chinese Treebank ( (Support Vector YamCha 2.2 (base phrase chunking) 2 3 44 2 MOZ CTB CTB 11 110 2 2.1 CTB Penn Chinese Treebank (CTB) Pennsylvania) Chinese Treebank Project B.1
CTB A 17 SVM 2.2 SVM
(
2
f
SVM B
SVM SVM (1) 2 ( ) (
2001) ( ) (2) (chunking) 111 (University of 33 Vol. 10 No. 1 Jan.  2003 ) ( ) SVM YamCha (Yet Another Multipurpose CHunk Annotator)  1 YamCha SVM 2 (tagging)
pairwise classification(Kreßel 1999)
SVM (
, 2002) SVM ( , 2000) 3 YamCha SVM YamCha YamCha (chunk) 11 SVM CTB 3.1
YamCha
YamCha 11 1 YamCha ( ) 1 2 YamCha 1 EOS IOB2 (Sang and ¹ http://cl.aist-nara.ac.jp/~taku-ku/software/yamcha/ 112 Veenstra 1999) B o I o CTB CTB “-NONE-” B/I " _ " CTB YamCha 11 CTB学習データ形港語
1: YamCha YamCha YamCha SVM 2 5 2 YamCha 2 YamCha TinySVM 0.08 ² SVM Jan. 2003 Vol. 10 No. 1
2:
3.2
MOZ
YamCha MOZ³（ 2000) MOZ MOZ MOZ CTB 2 ) bi-gram MOZ tri-gram (data sparseness) bi-gram Wi
POSi Wi (1) w i POSi
(1) POS j
128 (2) (3) 3 http://cl.aist-nara.ac.jp/student/tatuo-y/ma/ 114 p(POS j | POS j ) p(w i | POS i ) = | / log( )) (3) (4) = [| log( ) × n (4) 3.3 CTB s) (4181 (closed test) 1 418 418 (5) (6) (recall) (precision) MOZ YamCha CTB (5) = =
F
P (6) F (F-measure) (7) F
(7) 10 2 1 3.4 CTB 10 (cross validation)
1 9 YamCha MOZ 1 115 / 2 Vol. 10 No. 1
Jan. 2003

10し10 4 null YamCha 1 1 600 MHz :  256 MB   OS: Linux 5 CPU: Pentium III YamCha ( ) CPU: Pentium III 733 MHz 650 MB : 960
116

MB
5
80MB
: 256 MB 5 WWW
7
6 CPU: Pentium III 600MHz YamCha 0.1 MOZ 3700 ( 5 http://www.slt.atr.co.jp/~kohtake/ 6 2002 YamCha 0.2 11
117
Vol. 10
No. 1
し1 ( 400 ) 1 3.5 . 40 
 
 F 1 1111 ( ( ） 30 ) 118 10 11 10 2954 7 11 4 4 6 CTB
Jan. 2003
UK(t i )   w ∈ UK(t i ) w ∈ USeg(t i ) 8 MOZ 1 null MOZ
ti U Seg(t i ) USP(t i ) MOZ
USP(t i ) 1 MOZ
119
Vol. 10 Jan. 2003

3.6
CTB MOZ 2 YamCha 3 3.6.1 YamCha CTB SVM 3.0 B.3 CTB 10 102310 4117 CTB
1, 3, 4, 5 CTB 10 6 JUMAN7 41
YamCha
3 JUMAN 3.6.2 CTB 4181 99720 10 7 http://ww-nagao.kuee.kyoto-u.ac.jp/nl-resource/juman.html 121 Vol. 10 No. 1 Jan. 2003 CTB CTB B.2 PKU ) 8 39 44011 ( 1121447 PKU 7 (Bg: 8, Mg: 7, Rg: 10, Yg: nx ) 1, na:  1, nx:  459, vvn:  1 6 nx 1118794 PKU       43913 PKU     CTB        11 PKU 11 1  ） 10 PKU10 PKU 10 PKU10 CTB 10 PKU 6 CTB = 41.10/1.72 = 23.90   PKU10 CTB PKU 1 CTB 1 = 40.71/1.64 = 24.82 PKU10 10 YamCha CTB MOZ YamCha 1 SVM MOZ MOZ YamCha PKU PKU 3993 101218 MOZ 1024 128 8 http://www.icl.pku.edu.cn/research/corpus/addition.htm 1 1 x 3 1 1 2 2 7 122 100 YamCha 11 3.7 YamCha CTB 87.9% ) MOZ 83.3% YamCha (F YamCha CTB YamCha MOZ SVM YamCha YamCha 40% 45% (51.5% 6.4%) MOZ null 1 YamCha YamCha YamCha moz YamCha MOZ YamCha 123 1 2 11 3 ( ) 2 1 4 Vol. 10 No. 1 Jan. 2003 CTBしCTB 33 6 CTB YamCha MOZ PKU 100 95% F 95% tri-gram tri-gram 2002) ( , YamCha 4 YamCha (base phrase chunking) (Abney 1991) SVM 4.1 CTB CTB 41し3 1 F 1 (      , 2002) 124 4.2 CTB 9) (3572 1 ) (357 9 7746 YamCha 7745 7741 YamCha 7740
4.3 10 IOB YamCha 7734.4 YamCha 10 7691.9
1 5 11 12 1وJan. 2003
Vol. 10
No. 1
5 SVM Penn Chinese Treebank . SVM
4% 88% 4 89% 110
.
2 357 Abney, S. (1991). "Parsing by Chunks." In Berwick, R. C.,Abney, S. P.,and Tenny, C. (Eds.), Principle-Based Parsing: Computation and Psycholinguistics , pp. 257–278. Kluwer Aca- demic Publishers, Boston. . " (2002).  “ (3), 685–695. Kreßel, U. H.-G. (1999). "Pairwise Classification and Support Vector Machines." In Schölkopf, B.,Burges, C. J.,and Smola, A. J. (Eds.), Advances in Kernel Methods , pp. 255–268. The MIT Press. , (2000). “Support Vector Machine 2000-NL-140, pp. 9–16. Chunk . " , (2002).  “ , 43 (6), 1834–1842. (2001). “        ! , 42 (7), 676–683. Sang, E. F. T. K. and Veenstra, J. (1999). "Representing Text Chunks."  In Proceedings of EACL’99, pp. 173–179. (2000). “Support Vector Machine , 41 (4), 1113–1123. Xue, N. and Xia, F. (2000).  "The Bracketing Guidelines for the Penn Chinese Treebank (3.0).". http://www.ldc.upenn.edu/ctb/parseguide.3rd.ch.pdf. ." (2000).  “ , 7 (3), 39–56. , (2002).  “ , , pp. 132–135. ." –." ." , ." 8
127
Vol. 10 No. 1 Jan. 2003 A
CTB 2000)
CTB (Xue and Xia measure word (including classifiers) some particles common nouns proper nouns temporal nouns ordinal numbers onomatopoeia prepositions (excluding地and被) pronouns punctuation被in short bei-construction sentence-final particle predicative adjective copula 差有as the main verb other verbs LB被in long bei-construction LC localizer LCP phrase formed by “XP + LP” list marker noun phrase preposition phrase parenthetical quantifier phrase unidentical coordination phrase verb phrase DNP phrase formed by “XP + DEG” DP determiner phrase DVP phrase formed by “XP + DEV” fragment
simple clause headed by I (INFL) IP
128

UCP VP adjective phrase adverbial phrase headed by AD (adverb) CLP classifier phrase
clause headed by C (complementizer) ADJP ADVP
Penn Chinese Treebank
B.1
(University of Pennsylvania) Chinese Treebank Project 1998 12 (LDC2000T48) 2000 Version 2.0 (LDC2001T11) 2001 1994 1998 325 Agency) 10 CTB新华社通讯325
2 1
Penn Treebank Linguistic Data Consortium (LDC) LDC LDC2000T48 2000 US$100 uri Chinese Treebank Project http://www.ldc.upenn.edu/ctb/ LDC http://www.ldc.upenn.edu/ CTB http://www.ldc.upenn.edu/Catalog/LDC2000T48.html CTB Version 2.0 http://www.ldc.upenn.edu/Catalog/LDC2001T11.html B.2 (北京大学) Penn Chinese Treebank (新华社通讯, Xinhua News English
1,300 730 1
129
Vol. 10 10 WWW北京大学计算语言学研究所11 uri http://pr.fujitsu.com/jp/news/2001/08/28.html北京大学计算语言学研究所http://www.icl.pku.edu.cn/Introduction/corpustagging.htm http://www.icl.pku.edu.cn/research/corpus/addition.htm B.3
2 1
(JUMAN)
CD-ROM CD-ROM http://www-nagao.kuee.kyoto-u.ac.jp/nl-resource/corpus.html 130 : 1979
2002
e-mail:  gaizi@smlab.tutkie.tut.ac.jp : 1973 2001 (ATR) e-mail:  otake@fw.ipsj.or.jp : 1969 1996 (ATR) 1998 2002 , , , ACL e-mail:  yamamoto@fw.ipsj.or.jp )
） ） )