CRYSTAL: Inducing a Conceptual Dictionary

plg /9 50 50 20 9 M ay 1 99 5 In Proceedings of the Fourteenth International Joint Conference on Arti cial Intelligence (IJCAI '95) CRYSTAL: Inducing a Conceptual Dictionary Stephen Soderland, David Fisher, Jonathan Aseltine, Wendy Lehnert Department of Computer Science University of Massachusetts, Amherst, MA 01003-4610 fsoderlan d sher aseltine lehnertg@cs.umass.edu Abstract One of the central knowledge sources of an information extraction (IE) system is a dictionary of linguistic patterns that can be used to identify references to relevant information in a text. Automatic creation of conceptual dictionaries is important for portability and scalability of an IE system. This paper describes CRYSTAL, a system which automatically induces a dictionary of \concept-node de nitions" su cient to identify relevant information from a training corpus. Each of these concept-node de nitions is generalized as far as possible without producing errors, so that a minimum number of dictionary entries cover the positive training instances. Because it tests the accuracy of each proposed de nition, CRYSTAL can often surpass human intuitions in creating reliable extraction rules. 1 Information Extraction An information extraction (IE) system analyzes unrestricted natural language text and produces a representation of the information from the text which is considered relevant to a particular application. Information extracted from the text is represented as case frames, called \concept nodes" (CN's) in the University of Massachusetts BADGER sentence analyzer, which performs selective concept extraction similar to that of the previous CIRCUS system [Lehnert, 1991; Lehnert et al., 1993]. The domain knowledge needed to identify relevant references is stored in a dictionary of \CN de nitions" that describe the local syntactic and semantic context in which relevant information is likely to be found. Before the CN de nition is applied, BADGER segments the input text to identify syntactic constituents such as subject, verb phrase, direct and indirect object, and prepositional phrases, and also looks up the semantic class of each word in a domain-speci c semantic lexicon. A CN This research was supported by NSF Grant no. EEC9209623, State/ Industry/ University Cooperative Research on Intelligent Information Retrieval. Thanks also to David Aronow for help as our medical domain expert. de nition speci es a set of syntactic and semantic constraints that must be satis ed for the de nition to apply to a segment of text. Examples in this paper are from a medical domain, where the task is to analyze hospital discharge reports and identify references to \diagnosis" and to \sign or symptom". These are further classi ed with subtypes. Diagnosis: Sign or Symptom: con rmed present ruled out absent suspected presumed pre-existing unknown past history The example shown in Figure 1 is a CN de nition from this domain that identi es references to absent symptoms. This CN de nition extracts the phrase in the direct object bu er when the subject bu er has the word \patient" (semantic class <Patient or Disabled Group>), the verb is \denies" in the active voice, and the direct object has the semantic class <Sign or Symptom>.