Minimally Supervised Learning for Unconstrained Conceptual Property Extraction

Minimally Supervised Learning for Unconstrained Conceptual Property Extraction Colin Kelly (colin.kelly@cl.cam.ac.uk), Anna Korhonen (anna.korhonen@cl.cam.ac.uk) Computer Laboratory, University of Cambridge 15 JJ Thomson Avenue, Cambridge, CB3 0FD, UK Barry Devereux (barry@csl.psychol.cam.ac.uk) Centre for Speech, Language, and the Brain, University of Cambridge Downing Street, Cambridge, CB2 3EB, UK Table 1: Top ten properties from McRae norms with produc- tion frequencies for knife and pig . Abstract We present a highly performant, minimally supervised system for the challenging task of unconstrained conceptual property extraction (e.g., banana is fruit, spoon used for eating). Our technique employs lightly supervised support vector machines to acquire promising features from our corpora (Wikipedia and UKWAC) and uses those features to anchor the search for plau- sible unconstrained relations in our corpus. We introduce a novel backing-off method to find the most likely relation for each concept/feature pair and produce a number of metrics which act as potential indicators of true relations, training our system using a stochastic search algorithm to find the opti- mal reweighting of these metrics. We also introduce a human semantic-similarity dataset; our output shows a strong corre- lation with human similarity judgements. Both our gold stan- dard comparison and direct human evaluation results improve on those of previous approaches, with our human judgements evaluation showing a significant 20 percentage point perfor- mance increase. knife is sharp used for cutting is dangerous has a handle has a blade a weapon a utensil made of steel is serrated found in kitchens pig an animal lives on farms is pink has a tail has a curly tail has a snout eaten as bacon oinks is fat is dirty hope to extract features for a given concept as well as those features’ relationship with that concept; specifically, we aim to extract properties in the form of concept relation feature triples (e.g., knife used for cutting , pig lives on farm ), where both the relation and the feature are unconstrained. Our task is particularly challenging because while we seek a very spe- cific ‘type’ of information (namely, conceptual properties), there is an enormous amount of variation across the features and relations of properties which exhibit such characteristics. Previous approaches to our specific conceptual property extraction task (Baroni, Murphy, Barbu, & Poesio, 2009; De- vereux, Pilkington, Poibeau, & Korhonen, 2009; Kelly, De- vereux, & Korhonen, 2010, 2012) have been successful to varying degrees, however each has suffered from limitations. Baroni et al., for example, did not explicitly offer relations between their extracted concepts and features. The relations extracted by the Devereux et al., system were rather unso- phisticated, with the relation corresponding to the verb found along the grammatical relation path linking concept to fea- ture. The Kelly et al. (2010) system had reasonable perfor- mance but was founded on manually constructed rules and relied heavily on WordNet for its feature selection. The system of Kelly et al. (2012) approached this task as one of relation classification. The relations generated were derived directly from its training set; it was therefore unable to posit new or unseen relationships between its extracted concepts and features. We believe their feature output, how- ever, was promising and we extend and enhance their feature extraction method in the first component of our own system. Our system works by first employing a wealth of lexical, syntactic and semantic machine-learning attributes to train a support vector machine for feature-extraction. Unlike other Introduction Recent theories in cognitive psychology attest a property- based, distributed and componential model of conceptual representation for concrete concepts (e.g., elephant , screw- driver ) in the brain (Farah & McClelland, 1991; Tyler, Moss, Durrant-Peatfield, & Levy, 2000; Randall, Moss, Rodd, Greer, & Tyler, 2004). To explore the validity of these the- ories, researchers employ real-world knowledge taken from property norming studies where human volunteers are asked to list properties for concepts. McRae, Cree, Seidenberg, and McNorgan (2005) performed the largest such study to date, collecting properties for over 500 concrete nouns (we call these the ‘McRae norms’). Some example properties from these norms can be found in Table 1. However, as has been widely discussed (Murphy, 2002; McRae et al., 2005), these studies suffer from a number of weaknesses. For example, human participants often under- report certain properties, even when they are facts presum- ably known by the volunteers: though all participants are likely to have known that animals have hearts, has heart is not reported as a property for any animal concept. Similarly, is animal is listed as a property of all animals in the norms while breathes is only cited as a property for whale . A re- lated issue is inconsistency across similar concepts: has legs is listed as a property of leopard but is absent for tiger . Our task is to automatically extract such conceptual repre- sentations from large text corpora using NLP techniques. We