Trigger-Pair Predictors in Parsing and Tagging

In this article, we apply to natural language parsing and tagging the device of triggerpair predictors, previously employed exclusively within the field of language modelling for speech recognition. Given the task of predicting the correct rule to associate with a parse-tree node, or the correcl lag to associate with a word of text , and assuming a particular class of parsing or tagging model, we quantify the information gain realized by taking account of rule or tag trigger-pair predictors, i.e. pairs consisting of a "triggering' rule or tag which has already occurred in the document being processed, together with a specific "triggered" rule or tag whose probability of occurrence within the current sentence we wish to estimate. This information gain is shown to be subslantial. Further, by utilizing trigger pairs taken from the same general sort of doculnent as is being processed (e.g. same subject mat ter or same discourse type) as opposed to predictors derived from a comprehensive general set of English texts we can significantly increase this information gain. 1 I n t r o d u c t i o n If a person or device wished to predict which words or grammatical constructions were about to occur in some document, intuitively one of the most helpful things to know would seem to be which words and constructions occurred within the last half-dozen or dozen sentences of the document. Other things being equal, a text that has so far been larded with, say, mountaineering terms, is a good bet to continue featuring them. An author with the habit of ending sentences with adverbial clauses of confirmation, e.g. "as we all know", will probably keep up that habit as the discourse progresses. Within the field of language modelling for speech recognition, maintaining a cache of words that have occurred so far within a document, and using this information to alter probabilities of occurrence of particular choices for the word being predicted, has proved a winning strategy (Kuhn et al., 1990). Models using trigger pairs of words, i.e. pairs consisting of a "triggering" word which has already occurred in the document being processed, plus a specific "triggered" word whose probability of occurrence as the next word of the document needs to be estimated, have yielded perplexity 1 reductions of 29 38% over the baseline tr igram model, for a 5 million-word Wall Street Journal training corpus (Rosenfeld, 1996). This paper introduces the idea of using trigger pair techniques to ~ssist in the prediction of rule and tag occurrences, within the context of natural language parsing and tagging. Given the task of predicting the correct rule to associate with a parsetree node, or the correct tag to associate with a word of text, and assuming a particular class of parsing or tagging model, we quantify the intbrmation gain realized by taking account of rule or tag trigger-pair predictors, i.e. pairs consisting of a "triggeriilg" rule or tag which has already occurred in the document being processed, plus a specific "triggered" rule or tag whose probabili ty of occurrence within the current sentence we wish to estimate. In what follows, Section 2 provides a basic overview of tr igger-pair models. Section 3 describes the experiments we have performed, which to a large extent parallel successflfl modelling experiments within the field of language modelling for speech recognition. In the first experiment, we investigate the use of trigger pairs to predict both rules and tags over our full corpus of around a million words. The subsequent experiments investigate the