Some Properties of Preposition and Subordinate Conjunction Attachments

Determining the attachments of prepositions and subordinate conjunctions is a key problem in parsing natural language. This paper presents a trainable approach to making these attachments through transformation sequences and error-driven learning. Our approach is broad coverage, and accounts for roughly three times the attachment cases that have previously been handled by corpus-based techniques. In addition, our approach is based on a simplified model of syntax that is more consistent with the practice in current state-of-the-art language processing systems. This paper sketches syntactic and algorithmic details, and presents experimental results on data sets derived from the Penn Treebank. We obtain an attachment accuracy of 75.4% for the general case, the first such corpus-based result to be reported. For the restricted cases previously studied with corpusbased methods, our approach yields an accuracy comparable to current work (83.1%). 1 I n t r o d u c t i o n Determining the attachments of prepositions and subordinate conjunctions is an important problem in parsing natural language. It is also an old problem that continues to elude a complete solution. A classic example of the problem is the sentence "I s a w a m a n w i t h a t e l e s c o p e " , where who had the telescope is ambiguous. Recently, the preposition attachment problem has been addressed using corpus-based methods (Hindle and Rooth, 1993; Ratnaparkhi * This paper reports on work performed at the MITRE Corporation under the support of the MITRE Sponsored Research Program. Useful advice was provided by Lynette Hirschman and David PMmer. The experiments made use of Morgan Pecelli's noun/verb group annotations and some of David Day's programs. et al., 1994; Brill and Resnik, 1994; Collins and Brooks, 1995; Merlo et al., 1997). The present paper follows in the path set by these authors, but extends their work in significant ways. We made these extensions to solve this problem in a way that can be directly applied in running systems in such application areas as information extraction or conversational interfaces. In particular, we have sought to produce an attachment decision procedure with far broader coverage than in earlier approaches. Most research to date has focussed on a subset of the attachment problem that only covers 25% of the problem instances in our training data, the socalled binary VNP subset. Even the broader V[NP]* subset addressed by (Merlo et al., 1997) only accounts for 33% of tile problem instances. In contrast, our approach attempts to form attachments for as much as 89% of the problem instances (modulo some cases that are either pathological or accounted for by other means). Work to date has also been concerned primarily with reproducing the structure of Treebank annotations. In other words, the underlying syntactic paradigm has been the traditional notion of full sentential parsing. This approach differs from the parsing models currently being explored by both theorists and practitioners, which include semi-parsing strategies and finitestate approximations to context-fl'ee grammars. Our approach to syntax uses a cascade of rule sequence processors, each of which can be thought of as approximating some aspect of the underlying grammar by finite-state transduction. We have thus had to extend previous work at the conceptual level as well, by recasting the preposition attachment problem in terms of the vocabulary of finite-state approximations (noun groups, etc.), rather than the traditional syntactic categories (noun phrases, etc.).