A Learning Natural Language Parser

A natural language parser is described that analyzes the syntactic structure of an input sentence in relation to a specified grammar and generates all possible syntax trees of the sentence, along with estimates of the probability of each being the correct parse. The grammar used is based on X-bar theory, and the parsing algorithm is a chart parse – a topdown parser which uses dynamic programming for efficiency in cases where the grammar leads to ambiguities. The parser has a database of the frequency of application of each syntax rule in the grammar as well as a lexicon of known words and their lexical categories and frequency of use in each category. These are used in a probabilistic context-free grammar model to yield the likelihood judgments of the candidate parses and are updated by user feedback, leading to more accurate subsequent estimations. Introduction Approaches to natural language understanding have usually modeled language as having levels of structure corresponding to traditional levels of linguistic analysis. Thus, speech input undergoes acoustic and phonetic analysis, then phonological, morphological, syntactic, and finally semantic analysis. Each level in this schema imposes a layer of structure on the input with the goal of eventually arriving at a logical representation of the information it imparts. Natural language does not bend conveniently to these structural analyses, unfortunately. Ambiguity is prevalent at every level of analysis, and some ambiguity cannot even be resolved by a native speaker. When humans process language, structure and meaning are inferred using contextual information. The first problem this poses to computers is that, given a string of words, there are no hard and fast rules to govern the syntactic structure of that string. Information from the semantic level of analysis is required to disambiguate many grammatical ambiguities. Since, in other respects, semantic analysis takes place after syntactic analysis, this poses a problem for the level-by-level strategy, necessitating backtracking. Many recent trends in linguistics and natural language processing move away from the multi-level, structure-heavy approach in favor of methods based on statistical analysis and positional relationships. These ideas have been very productive, but there is still room for structural approaches. By combining them with the statistical and probabilistic strategies, structural analyses can be made flexible enough to deal with ambiguities. This paper examines an example of how this can be accomplished, applied specifically to the syntactic level of analysis: parsing. Parsing is the logical first step to computer understanding of natural language text. Most approaches to determining the meaning of a natural language statement first analyze the syntactical structure of the statement. The syntactical analysis of a sentence contains the information of what roles the words in the sentence are playing as well as the phrase structure of the sentence – that is, what parts constitute the subject and predicate of the sentence and what the adjectives, adverbs, and prepositional phrases refer to. Unlike programming languages, which have relatively simple formal grammars, natural languages have not been successfully formalized, so modifications must be made to adapt traditional parsing algorithms to them. One major challenge is that natural language grammars frequently generate multiple possible parses for a single sentence. These are ambiguities that humans resolve through the application of a wealth of semantic and contextual knowledge that computers are not yet able to replicate. This project's goal was to develop a system to parse grammatical English sentences into phrase structures in a way that deals gracefully with ambiguity. The resulting parser first generates all possible phrase structures using the chart parser algorithm. It then uses a probabilistic grammar to assign each of these possibilities a likelihood measure, which is the estimated probability of that parse being the correct representation of the sentence. These measures are normalized probabilities of the incidence of each result based on the sentences that the system has seen in the past. Thus, as the parser is exposed to more sentences, it learns what are the most common and the least common grammatical constructions in the language, and therefore improves its judgments of ambiguous cases. Background X-Bar Grammar The output of a parser such as the one described here is a parse tree, a construct describing the phrase structure of the input sentence in relation to a specified grammar. The internal nodes of a parse tree represent non-terminal symbols in the grammar and the leaf nodes are the words of the input sentence. The grammar used in this parser is a context-free grammar based on X-bar theory. Although X-bar grammar grew out of the Chomskyan generative grammar tradition that frowns upon probabilistic methods (Russell & Norvig, 2003), it still has many features that are useful in computational analysis. It lends itself well to abstraction, as its rules are at most binary branching and are categorized in terms of a few abstract patterns. It also serves as the base framework for much work on describing syntactic phenomena such as agreement, pronoun reference, and structural transformations (Haegeman, 1994). Figure 1: X-bar theory’s categorization of non-terminal symbols means that the syntax rules conform to a set of abstract patterns. On the right is a generic XP schema. XP Spec X' X' X | X' Comp | Comp X' | X' Adjunct | Adjunct X'