35 n-gram 7869>= k-NN 4<?;:@BDAFEGC 34AE-01/HLN PO0Qy
RJSTST. UTKMI1ZW[Y/ä ÉoÑæ®¼îí=|P = 0.964912
gl|¥ÇéîÅj«¢b3áû×(Å¶ºP = 1 .02>5,Ç8ÿ°gl|Å<¶ÀÏ;²ÄÖÕÀ«P = 0.0Ô£¶¯¶¢ø¯¼Û±µÕÄ«Â«¬gl|ÇçQÇÜºÎ¢OÛ&±ÅÑÔ2>5ø"°3ÚZÅhÓ¢8
\:2É Ñ  ÅÅ.: ¥ ÂÅ(¿²2>5,Uý×±/\É®ì¤í«úÐ=|%­ÔÅÈF½À«Ä«£
P= 0.964912%üÇ.: ¥ ÂÅ(¿²2>5,UýÏ¢P= 0.035087
ÇéÐ÷æÇ.: ¥ &ÁÇuÇÙìÄÃP = 0.526316
ÂÇ6Æ«ÏªÓ¢úTÇÇ¥¥ÏnGP = 0.45614
P= 0.0175439
Á±Ä²Ä½À±À«Ô£ä
1:l8ÿÁxu·Ô-gram¥Ç2P´ÕÅ<¶À¢lÁÈùJ "Ç -gram ¥×¥¯ÒX¶¢¸Ç¥¯Ò -NN ý×u«À
2>5ø"®ÑÉóH9çÇõ+×Ä¬b32>5 nn lm
nom
,Uý× ·Ô£ñ*Ç5,¢ôvÄ¥Ç#Ó nnoprqstÌa×ÄÖÄ«LÁÏ 93% 1²ÇëÂ¢Ww
Ç2>5,¡Ç 1.5 i¢q 20000 ùJ / òÂ
ð 1: EDR ¥4ù¯ÒXµÕº -gram«¬ÚZ8°R{µÕº£3 k -NN dY\^uxwyz
2 a`X^cg n-gram ijhl8ÿÁ ·Ô -NN ýÅÑÔ2>5åÈ!)Ç
ùJ -gram ¥×u«ÔÂ3ÚZÂ¶ÀÇ2>5 UÙÁÄÖÕÔ£ø"×Ä¬ïv°Ä²ÄÔ£´Ç­Å(¿«À¢ [3] 1. h\ùÇ" ×'Í4öøùJ_× -gramÈùJ -gram ¥×  ÅÑÔb32¥¯Ò7B¶¢%ùJæ×1µÂ9Ä¶À>5,Åu¶º£¶¯¶¢´ÇüýÁÈ¢ÚZL&Ï1«¥ (nearest neighbors) ×2A·Ô£° ÇkÞÅí^·ÔºÎRuÁÈÄ«£2AµÕº -gram ¥Á" Å<(·Ô"¸´ÁlUýÁÈÑÓ ØÄ -NN ý×xu·ÔºÎ 2. Á2>5°-ÕÔ
[×(Å¢h\ùÇ" ÁÅ¢ -gram ¥Ï  Áu«ÒÕÔ -gram 2>5°-ÕÔ / -ÕÄ«×4¶¢¸ÕÒÇ;æ¥ÂÈ$ÄÔ2PÇÏÇ×Du·Ô¦ä 1§£´´4ÅÑÓ¢¸Ç"Á2>5°-ÕÔ¯Ã¬¯×ÁfÄÇÈ¢ -gram ¥×è·Ô¤Ç-Õ4·Ô£o"ÂùJÅ [áû×õs¶À«ÔÁªÔ£l8ÿÁÈ¢´Ç -gram ¥ ( = 1  4) × EDR 3. 2. ÂdtÇ;æ4ÅÑÓ¢ùJÅs­ÔóH9
ç×4·Ô£¥4ù 207,802 ù [1] ¦ð 1§Âñ*uÇ 1000 ù×Ü«º 206,802 ù¯Ò¸Õ¹ÕX¶ÀM0ÁYÊÔ
4. -ÕÔÂ4µÕº"Ç&ÇùJ_×2>5Â
2>5å×Ä½À«Ô£¶ÀSÓX¶¢¸ÇÇùJÅs­ÒÕºóH9ç¢? [×K¾9ç×¸Ç2>5ÇóH93çÂ·Ô£nky!ÝÇUÙÈ -NN ýÇ = 1 ÇLÅ7`·Ô£Ëfgº¢¥&Ç1µ×ÝYÇÑ¬ÅÎÔ´ÂÁ¢àÇ -NN ýÁp@ÂÄÔ nearest neighbors Ç7B×OÛ7BÂdtÇ Á7BÁ±ÔÑ¬ÅÄÔ£
1b0b
1b…

28ú]0]1ú

08ú

1Wú

1oWú

1nnk-gram

1-gram2-gram3-gram4-gram-gram

9,448,229

8,799,431

8,172,812

7,724,156

5710

345,108

1,572,361

3,319,892

1654.68

25.4985.19782.3266-gram

nkninnnnkniinnnn

A Stochastic Morphological Analysis for Japanese employ-ing Character-Gram andKenji Nagamatsu-NN Method

Hidehiko Tanaka

knaga, tanaka @mtl.t.u-tokyo.ac.jp

Faculty of Engineering,

University of Tokyo, 7-3-1 Hongo,

Bunkyo-Ku, Tokyo, 113, Japankk

Segmentation into Morphemes

Processing Time

100%30¶¶«­°¢¶µ«´µ¢¶´«µ´¢Accuracy Rate95%¶²«¯®¢¶±«´­¢¶±«°µ¢¶²«®µ¢¶±«²µ¢¶°«µ¶¢¶°«±´¢¶¯«´¯¢¯µ«µ¯°«³¯­«³10®´«³®¯«­en

en

op

en

1=

op

2=

0N

N

3=

op

een

eos

op

cl

4NN

N=os
1

2=20

´°«±­¢

=

os

cle

eos

cl

N=

3

cl
4=NmN))

xtxt(e3en(eanen

op

1en

as

1=chN

´°«±¯¢juen
2=opeenop
3NNN

N=opeos4=os

cl
1=cleeos
cl

2=N=

3oscl
N=4(e(e13anjuNmasenxtxt))

90%ch´¶«­³¢´¶«­µ¢

Processing Time(sec)¶´«°³¢¶´«¯´¢¶³«´¶¢¶³«¯­¢¶²«´±¢

chasen 1

recall

N=4

N=3

N=2

N=1

precisionä 2: 2>5ùJ_Ç%

ä 4: 10000 ù`ÓÇ,L&{e5Pos Tagging to Morphemes-NN ýÇçQ¯Òr6µÕÔÓ¢ close ñ*Ç)ÁÈxu·Ô -gram ¥ÇV] ( ) °?±²ÄÔ¢100%k¶µ«±¶¢¶µ«¯±¢nN,ëÏÝ°Ô´Â°NµÕÀ«Ô£%ü¢ open ñ*Ç)ÁÈ = 3 ÇLÅÏë°
²ÄÔ°¢´ÕÈUÙ 2. Å®«À -gram ¥Ç' Çáû°uµÕºßÅë°)°Ô´ÂÅ#Ô£ 4-gram Å®«À¢¸Õ°ÏyÅ{ÖÕººÎ»ÂEÖÕÔ£ÚZ8Ç×9ÔÂ¢ open ñ*ÁÏ«ë°eÒÕÔ = 3 ÇßÅ¢q 20000 ùJ / òÇ8Á,Á±¢´ÕÈ.: ¥ ÂÅ(¿²,¡ä 3: óH9çËÁ'Îº%
ÁªÔ chasen 1.0 Çq 1.5 iÂÄÔ£-NN ýÇ8ÿÁÈ¢I^Ç#Ó Ì°ÚZ8Âø4 b2]ëÇ}ÝÅ)s·Ô´Â°ÒÕÀ«Ô£l8ÿÇ´Ç ÅÑÔ2>5å×¢ EDR ¥ -@Â¶ÀÏ¢{¢¥¯ÒXµÕºËËDu×zÅ¶Àñ*¶º£Du¶º -gram ¥È30 ¶À«Ô -gram ¥¯ÒôvÄÏÇ×#Ó Í´ÂÅN¶ºÏÇÁ¢ñ*uÅC¶º 1000 ù×,¶À5 Á¢~ÄÔ,ëÂÚZ8Ç}Ý°+=µÕÔ£,° EDR ¥ÇêÂ$4Å%·Ô¯Ã¬¯Á¢V_]Y¨2>5ùJ_Â¶ÀÇ%©Â¨óH9çËÁ'Î 6 lÁÈ¢ùJ-gram ¥¯Ò -NN ýÅÑÓ2º%©×þÎº£>5ø"®ÑÉóH9çÇõ+×Ä¬2>5,UýËº¢xu·Ô -gram ¥×¨4-gram ËÁ©¨3- × ¶º£ñ*Ç5,¢ -NN ýÂ«¬ ØÄüý×gram ËÁ©¨2-gram ËÁ©¨1-gram »³©Ç 4 Óu«Ô»³ÁÏ 93% Â«¬,ëÂ¢ -NN ýÇ
Çâ0 ( = 1 2 3 4) ×/¶¢¸Õ¹Õñ*ù°* ØµÅÑÔÂÄÔ -gram ¥ (close) Â¢m ÂÄÔ -gram µÕº£ 20000 ùJ / òÂ«¬8Ä,8°R{¥ (open) ×u«ºßÇ3 8 ÓÅ<¶Àñ*× f{cvÄ½º£
ñ*5,¯Ò¨2>5ùJ_Â¶ÀÇ%©×ä 2Å¢¨óH9çËÁ'Îº%©×ä 3ÅN·£ juman3.0beta
Â chasen1.0[2] Ç5, (ext) È.í!Á±Ä«ºÎ¢Japanese Morphological Analysis
ãUÁ-¯Ã¬¯Çk ×Ä½ºÏÇÁªÔ£
System ChaSen Manualä 4È close ñ*u¥×u«ºßÅ EDR ¥Ç1c 10,000 ù¦404,994 ùJ§×,·ÔÇÅv
¶ºL& (user time) ×þÎºÏÇÁªÔ£WwÇ¡| | } ~£§¢ÂÇí!ÇºÎ¢ chasen1.0 Å<¶ÀÏdtÇ/¨ ¦  ¥¦¡   ©Ç)Áñ*¶º£ ª¤Accuracy Rate¶³«´­¢¶³«®¶¢

N95%¶°«¯±¢n¶°«¯¯¢¶¯«³­¢¶¯«±µ¢¶®«±²¢¶­«³²¢

recall

en

1op

en

op

=

N

N

=

2op

op

3N

=

N

=

4cl

en

µ¶«²®¢ ³¯«²¶¢eneose1N=2closeN=3close=

N

N

=

4clos

en

op

en

1

N

=

2

op

en=

NN

=

3

op

en

e

4=

N

³¯«³°¢³´«°¶¢

op

ose

cl

1=

NN

=

2closeoseosclcl

3

N=4=N

¶­«²³¢³´«±±¢90%

N

precision

knnnknk

Nn;

;

k;

n

[1] J. Electronic Dictionary Research Institute Ltd. Edrelectronic dictionary technical guide, 1995.

[2] Y. Matsumoto, A. Kitauchi, T. Yamashita, O. Imaichi,and T. Imamura.

.

Nara Institute of Scienceand Technology, 1997. NAIST Technical Report NAISTIS-TR97007.

[3],.

.

, pp. 421{424, mar 1997.

3