Lexical Resource Reconciliation in the Xerox Linguistic Environment

This paper motivates and describes those aspects of the Xerox Linguistic Environment (XLE) that facilitate the construction of broad-coverage Lexical Functional grammars by incorporating morphological and lexical material from external resources. Because that material can be incorrect, incomplete, or otherwise incompatible with the grammar, mechanisms are provided to correct and augment the external material to suit the needs of the grammar developer. This can be accomplished without direct modification of the incorporated material, which is often infeasible or undesirable. Externally-developed finite-state morphological analyzers are reconciled with grammar requirements by run-time simulation of finite-state calculus operations for combining transducers. Lexical entries derived by automatic extraction from on-line dictionaries or via corpus-analysis tools are incorporated and reconciled by extending the LFG lexicon formalism to allow fine-tuned integration of information from difference sources. 1 I n t r o d u c t i o n The LISP-based LFG Grammar Writers Workbench (GWB) (Kaplan and Maxwell, 1996) has long served both as a testbed for the development of parsing algorithms (Maxwell and Kaplan, 1991; Maxwell and Kaplan, 1993) and as a self-contained environment for work in syntactic theory. The C/Unix-based Xerox Linguistic Environment (XLE) further develops the GWB parsing algorithms, extends them to generation, and adapts the environment to a different set of requirements. This paper motivates and describes the morphological and lexical adaptations of XLE. They evolved concurrently with PARGRAM, a multi-site XLF_~ based broad-coverage grammar writing effort aimed at creating parallel grammars for English, French, and German (see Butt et. al., forthcoming). The XLE adaptations help to reconcile separately constructed linguistic resources with the needs of the core grammars. The paper is divided into three major sections. The next section sets the stage by providing a short overview of the overall environmental features of the original LFG GWB and its provisions for morphological and lexicon processing. The two following sections describe the XLE extensions in those areas. 2 T h e G W B D a t a B a s e GWB provides a computational environment tailored especially for defining and testing grammars in the LFG formalism. Comprehensive editing facilities internal to the environment are used to construct and modify a data base of grammar elements of various types: morphologicalrules, lexical entries, syntactic rules, and "templates" allowing named abbreviations for combinations of constraints. (See Kaplan and Maxwell, 1996; Kaplan and Bresnan, 1982; and Kaplan, 1995 for descriptions of the LFG formalism.) Separate "configuration" specifications indicate how to select and assemble collections of these elements to make up a complete grammar, and alternative configurations make it easy to experiment with different linguistic analyses. This paper focuses on the lexical mapping process, that is, the overall process of translating between the characters in an input string and the initial edges of the parse-chart. We divide this process into the typical stages of tokenization, morphological analysis, and LFG lexicon lookup. In GWB tokenizing is accomplished with a finite-state transducer compiled from a few simple rules according to the methods