Pointwise Prediction for Robust, Adaptable Japanese Morphological Analysis
Graham Neubig,Yosuke Nakata, Shinsuke Mori
Graduate School of Informatics, Kyoto University, JapanOverview

Features for Pointwise MA

Objective: Create a Japanese morphologicalanalyzer (word segmentation + POS tagging) thatis robust and adaptable to new domains●Approach: Use pointwise prediction, which
estimates all tags independently of other tags
●Pointwise prediction:● Robust: does not rely on dictionaries as much asprevious methods● Adaptable: it can be learned from single
annotated words, not full sentences● Works with active learning: Single words toannotate can be chosen effectively●Evaluation on Japanese morphological analysis
shows improvement over traditional methods
1●Specify features using character n-grams, character
type n-grams, length-annotated dictionary presence

Joint: Predict word boundaries+tags simultaneously● Use HMMs, CRFs, or language models

Boundary Pointhon

本WS

宝石を磨くmaxmigaku

宝石NをP磨V

くSuf

2-Step: First predict word boundaries, then POSs
● Can use Logistic Regression, SVM, CRF● LR and SVM are pointwise, CRF notTags

投yo与suすruる

X0 投 X1 与X-1 を投 X0 投与 X1 与す
X-1 を投与 X0 投与すT0K
T1K
T-1HK T0KK T1KH
T-2KHK T-1HKK T0KKH T1KHH
D0L1( 投 ) D0R1( 与 ) D0I2( 投与 )GeneralTarget

Test
87.5k
17.3k

Tested three systems● Joint: Kudo et al.'s CRF-based method, as
implemented by the MeCab toolkit● 2-CRF: The 2 step method using CRFs as a solver● 2-LR: 2 step pointwise method using LR

Char n-gram + Type n-gram
Word Identity W 投与 + Dictionary DN DVPOS●Key point: None of the features require wordboundaries or surrounding tags

Train
782k153k●3Morphological analysis underperforms on out-ofdomain text → we would like to adapt● We have an in-domain unannotated text, andsome annotator time
● Goal is to maximize the effect for annotator time
● Use active learning to choose data to annotate●●Train

Test

Joint

2-CRF

2-LRGeneralGeneral

97.31%

98.08%

98.03%

GeneralTarget94.57%

95.39%

95.13%

Gen+Tar

Target96.45%

96.91%

96.82%

2-LR slightly worse than 2-CRF, better than Joint

Tested 2 annotation strategies for domain adaptation● Sentence: annotate the sentence with lowest
overall posterior probability● Partial: annotate the word with lowest prob. margin●96.40%

Full annotation: Choose sentences with low prob.
● Can train any model on this annotated data

宝石を磨く

95.90%
95.40%
2-lr/part2-lr/sent
2-crf/sent
joint/sent

94.90%
94.40%
0●Partial annotation: Choose words with low prob.
● Only pointwise prediction can be usedho-seki wo miga ku

2000

4000

6000

8000

10000

Words Annotated

本剤 /N を /P 投与 /N する /V ため /N
Annotated (5)0 1 1 1

宝石 を 磨 く
max * 4
POS Tags
N P V Sufをtou●

ho-sekiwomigaku

max*4Boundary剤

woReference
本剤 /N を /P 投与 /N する /V ため /N
Automatic Result
本 /Pre 剤 /N を /P 投 /N 与 /N する /V ため /N
0.8
0.91 1.0 0.98 0.94 0.998 0.997●2-step

Char 1-gram
Char 2-gram
Char 3-gram
Type 1-gram
Type 2-gram
Type 3-gram
Dictionaryzai

F-measure

Word/POSPairs

ho-sekiwomigaku

wo●Annotation Methods●

ho-sekiExperiments performed on the Balanced Corpus ofContemporary Written Japanese (BCCWJ)● General domain: News, white papers, books● Target domain: Web text

●Morphological Analysis Methods

JointExperiments

Partial annotation much more effective●

本剤 /N を 投与 /N
2

Annotated (2)Available Open Source!
http://www.phontron.com/kytea/

するためUnannotated4

Chinese models, Japanese pronunciation
estimation also available5