HPSG-Style Underspecified Japanese Grammar with Wide Coverage

This paper describes a wide-coverage Japanese grammar based on HPSG. The aim of this work is to see the coverage and accuracy attainable using an underspecified grammar. Underspecification, allowed in a typed feature structure formalism, enables us to write down a wide-coverage grammar concisely. The grammar we have implemented consists of only 6 ID schemata, 68 lexical entries (assigned to functional words), and 63 lexical entry templates (assigned to parts of speech (POSs))_ Furthermore. word-specific constraints such as' subcategorization of verbs are not fixed in the grammar. llowever, this grammar can generate parse trees for 87% of the 10000 sentences in the Japanese EI)I{ corpus. The dependency accuracy is 78% when a parser uses the heuristic that every bunsetsu ~ is attached to the nearest possible one. 1 I n t r o d u c t i o n Our purpose is to design a practical Japanese grammar based on ItPSG (Head-driven Phrase Structure Grammar) (Pollard and Sag, 1994), with wide coverage and reasonable accuracy for syntactic structures of real-world texts. In this paper, "coverage" refers to the percentage of input sentences for which the grammar returns at least one parse tree, and "accuracy" refers to the percentage of bunsetsus which are at tached correctly. To realize wide coverage and reasonable accuracy, the following steps had been taken: A) At first we prepared a linguistically valid but coarse grammar with wide coverage. B) We then refined the grammar in regard to accuracy, using practical heuristics which are not linguistically motivated. As for A), the first grammar we have constructed actually consists of only 68 lexical en* This research is part ial ly founded by the projec t of JSPS ( JSPS-RFTF96P00502) . 1A bunsetsu is a common unit when syntact ic structures in Japanese are discussed. tries (LEs) for some flmctional words 2, 63 lexical entry templates (LETs) for POSs 3, and 6 ID schemata. Nevertheless, the coverage of our grammar was 92% for the Japanese corpus in the EDR Electronic Dictionary (EDR, 1996), mainly due to underspecification, which is allowed in HPSG and does not always require detailed grammar descriptions. As for B), in order to improve accuracy, the grammar should restrict ambiguity as much as possible. ~br this purpose, the grammar needs more constraints in itself. To reduce ambiguity. we added additional feature structures which may not be linguistically valid but be empiricafiv correct, as constraints to i) the original l,Es and LE'Is. and ii) the I1) schemata. The rest of this paper describes the architecture of our ,}apmmse grammar (Section 2). retinelnent of our grammar (Section 3), experimental results (Section 4). and discussion regarding errors (Section 5). 2 A r c h i t e c t u r e o f J a p a n e s e g r a l l l l r l a r In this section we describe the architecture of the ItPSG-style Japanese grammar we have developed. In the HPSG framework, a grammar consists of (i) immediate dominance schemata (ID schemata), (ii) principles, and (iii)lexical entries (LEs). All of them are represented by typed feature structures (TFSs) (Carpenter, 1992), the fundamental data structures of HPSG. ID schemata, corresponding to rewriting rules in CFG, are significant for constructing syntactic structures. The details of our ID schemata are discussed in Section 2.1. Principles are constraints between mother and daughter feature structures. 4 LEs, which compose the lexicon, are detailed constraints on each word. In our grammar, we do not always assign LEs to each word. Instead, we assign lexical entry 2A functional word is assigned one or more LEs. 3A POS is also assigned one or more LETs. 4We omit fur ther explanat ion about principles here due to limited space.