Using Decision Trees to Construct a Practical Parser

This l)al)er describes novel and practical . lal)anesc parsers that uses decision trees, l"irst, we COl> struct a single, decision tree to estimate modifica-lion probabilities; how one phrase tends t.o modify another. Next, we introduce a boosting algorithm in which several decision t.rees are COllst.ructed and then combined for probalfility estiinat.ion. 'lThe two constructed parsers are evalua.ted I)y using the El)t{ . Japanese annotated corpus. The single-tree method outperforlns the conventional Japanese stochastic reel.hods by 4%. Moreover, the boosting version is shown to h;we significant adwmtages; 1 ) better parsing accuracy than ils single-tree counl.erparl for any alnoullt o[" training data and 2) no over-titling 1o data. for va.rious iterations. 1 I n t r o d u c t i o n Conventional parsers with practical levels of per f o r mance require a number of sophisticated rules that. haw" to be hand-crafted by human linguists. It is time-consuming and cumbersome to mainl.ahl tltese rules for two reasons. * The rules are specific to the application domain. * Specific rules handling collocat.ional expressions create side effects. Such rules often deteriorate the overall performance of the parser. q'he stochastic approach, on the other hand, has the potential to overcome these difficulties. Because il induces stochastic rules to maximize ow~'ra.ll per['onnance against t.raining data, it. llOf Ollly adapts to any application domain but also may avoid ow>rfitting to the data. In the late 80s and early 90s, lhe induction and parameter estimation of l)robabilis tie context, free grammars (PCF(',s) from corpora were intensively studied, la;ecause these grammars comprise only nonterminal and part-of-speech tag symbols, their performances were not enough to be used in practical applications (Charniak, 1993). A broa.der range of information, in lmrticular lexical inforinatiolq was tbund to be essential in disambiguating the syntactic structures of real-world sentences. SI 'ATTEt{ (Magerman, 1995) augmented the pure I'(:I"G by introducing a. mnnl)er of lexical at.tributes. The parser controlled applications of each rule by using the lexical constraints induced by decision tree algorithnl (Quinlan, 1993). rFhe SI)N]'TEt{ parser attained 87% accuracy and first made stochastic parsers a practical choice. The other type of highprecision parser, which is based on dependency ana[5'sis was introduced by Collins (Collins, 1996). l)ependency a.mdysis first, segments a sentence into syntactically meaningful sequences of words and then considers the modificatioll of each segment. Collins" parser computes the likelihood that each segment modifies the other (2 term relation) by using large corpora. These moditication probabilities are conditioned by head words of two S{"glnelltS, distance between the two segments and otlmr syntactic %atures. Although these two parsers have showll silnilar performance, the keys of their success are slight ly diflk~renl. SPA'[. 'TER parser perforlnanee greatly depelldS on the feat.tire sehection ability of the decision tree algorithm rather than its linguistic representation. On t, he other hand, dependency analysis plays an essential role in Collins' parser for elficienlly extracting inK)rmation from corpora. In lifts i)al)er, we (lescribe practical .]apanes(" dependency parsers that uses decisio11, trees, in the .lal)anese language, dependency analysis has I)ecn showll to ])e powerful because seglllellI (bullselsll) order in a sentence is relal:ively free compared to l! ',uropean languages. Japanese dependency parsers generally proceed in three steps. l. ,qeglnent a sentence into a sequence of t)unsetsu. 2. Prel)are a modification matrix, each value of which rel)resents how one ])/illSet.Sll is likely to modify another. 3. Find optimal modifications in a sentence by a dynanfic progranmfing technique. The most diIficult part is the second; how to construct a. sophisticated lnodifieation matrix. With conventiolml ,]apanese parsers, the linguist nmst classify the tmnset.su and select appropriate fealures to compute modificatioll values. The parsers thus suffer from application domain diversity and t.he side effects of specific rules.