Bootstrapping a Domain-specific Terminological Taxonomy from Scientific Text

We present an approach to automated extraction of a taxonomy of domain-specific terms from scientific discourse. The approach has been developed and evaluated in the domain of computational linguistics. Concept pairs in is-a relation have been extracted from a subset of the ACL Anthology and WeScience. Correctness of the resource has been verified by crowdsourcing: To attract domain experts to identify correct and invalid is-a pairs, we used “games with a purpose”. The popular games of Tetris and Invaders were modified to support concurrent and efficient annotation of domain term pairs during playing. High quality of the resulting annotations was ensured by exploiting redundancy: at least five-way agreement was required for a candidate is-a pair to be considered correctly extracted. Based on the crowdsourced evaluation the extraction method achieved precision around 80%. 1 Motivation and Related Work Information on lexical relations between words or ontological relations between concepts often constitutes indispensable knowledge in language processing applications, in particular in applications involving inference. WordNet (Fellbaum, 1998) is nowadays a de facto standard source of knowledge on lexical relations between (English) words. A “Wordnet” is often one of the first resources developed for a new language.1 Wordnets, however, are typically general-purpose lexicons with little technical terminology from specialized domains. Building domain-specific ontologies manually is a time-consuming task, requiring access to domain experts whose time is an expensive commodity. “Wordnets” exist or are being developed for over 50 languages; see http://www.globalwordnet. org/ The present work focuses on building a subset of specialized “Wordnet”: a taxonomy of domain terms for computational linguistics (CL). Computational approaches to relation learning have used dictionaries (Chodorow et al., 1985), linguistic patterns (Hearst, 1992; Brin, 1999; Pantel et al., 2004; Snow et al., 2005; Yang and Callan, 2009), clustering (Caraballo, 1999; Bisson et al., 2000; Cimiano et al., 2004) and, in particular, bootstrapping as a method of pattern induction (Etzioni et al., 2005; Pantel and Pennacchiotti, 2006; Girju et al., 2006; Kozareva et al., 2008). Building on this work, we find pairs of CL concepts in is-a relation with each other2 based on bootstrapping lexico-syntactic “is-a patterns” from a corpus. We show that bootstrapping-based acquisition of lexico-syntactic patterns achieves satisfactory results on OCR’ed computational linguistics scientific papers, i.e. documents whose text is prone to have been partially corrupted. We also show that it is possible to evaluate the quality of the extracted patterns by crowdsourcing experts. We develop variants of two popular comuputer games which serve as “annotation tools” for evaluating the quality of the extracted patterns in a “game with a purpose” scenario. Outline Section 2 introduces the corpora. Section 3 summarizes the taxonomy creation. Section 4 describes the games and the evalution procedures. Extraction results are outlined in Section 5. Section 6 concludes with an outlook.