Meaning Representation in Natural Language Categorization

Meaning Representation in Natural Language Categorization Trevor Fountain (t.fountain@sms.ed.ac.uk) and Mirella Lapata (mlap@inf.ed.ac.uk) School of Informatics, University of Edinburgh 10 Crichton Street, Edinburgh EH8 9AB, UK Abstract A large number of formal models of categorization have been proposed in recent years. Many of these are tested on artificial categories or perceptual stimuli. In this paper we focus on cat- egorization models for natural language concepts and specif- ically address the question of how these may be represented. Many psychological theories of semantic cognition assume that concepts are defined by features which are commonly elicited from humans. Norming studies yield detailed knowl- edge about meaning representations, however they are small- scale (features are obtained for a few hundred words), and ad- mittedly of limited use for a general model of natural language categorization. As an alternative we investigate whether cate- gory meanings may be represented quantitatively in terms of simple co-occurrence statistics extracted from large text col- lections. Experimental comparisons of feature-based catego- rization models against models based on data-driven represen- tations indicate that the latter represent a viable alternative to the feature norms typically used. Introduction Considerable psychological research has shown that people reason about novel objects they encounter by identifying the category to which these objects belong and extrapolating from their past experiences with other members of that cat- egory. This task of categorization, or grouping objects into meaningful categories, is a classic problem in the field of cog- nitive science, one with a history of study dating back to Aris- totle. This is hardly surprising, as the ability to reason about categories is central to a multitude of other tasks, including perception, learning, and the use of language. Numerous theories exist as to how humans categorize ob- jects. These theories themselves tend to belong to one of three schools of thought. In the classical (or Aristotelian) view cat- egories are defined by a list of “necessary and sufficient” features. For example, the defining features for the concept BACHELOR might be male, single, and adult. Unfortunately, this approach is unable to account for most ordinary usage of categories, as many real-world objects have a somewhat fuzzy definition and don’t fit neatly into well-defined cate- gories (Smith and Medin, 1981). Prototype theory (Rosch, 1973) presents an alternative for- mulation of this idea, in which categories are defined by an idealized prototypical member possessing the features which are critical to the category. Objects are deemed to be members of the category if they exhibit enough of these features; for example, the characteristic features of FRUIT might include contains seeds, grows above ground, and is edible. Roughly speaking, prototype theory differs from the classical theory in that members of the category are not required to possess all of the features specified in the prototype. Although prototype theory provides a superior and work- able alternative to the classical theory it has been challenged by the exemplar approach (Medin and Schaffer, 1978). In this view, categories are defined not by a single representation but rather by a list of previously encountered members. Instead of maintaining a single prototype for FRUIT that lists the fea- tures typical of fruits, an exemplar model simply stores those instances of fruit to which it has been exposed (e.g., apples, oranges, pears). A new object is grouped into the category if it is sufficiently similar to one or more of the FRUIT instances stored in memory. In the past much experimental work has tested the predic- tions of prototype- and exemplar-based theories in laboratory studies involving categorization and category learning. These experiments tend to use perceptual stimuli and artificial cat- egories (e.g., strings of digit sequences such as 100000 or 0111111). Analogously, much modeling work has focused on the questions of how categories and stimuli can be rep- resented (Griffiths et al., 2007a; Sanborn et al., 2006) and how best to formalize similarity. The latter plays an impor- tant role in both prototype and exemplar models as correct generalization to new objects depends on identifying previ- ously encountered items correctly. In this paper we focus on the less studied problem of cat- egorization of natural language concepts. In contrast to the numerous studies using perceptual stimuli or artificial cate- gories, there is surprisingly little work on how natural lan- guage categories are learned or used by adult speakers. A few notable exceptions are Heit and Barsalou (1996) who attempt to experimentally test an exemplar model within the context of natural language concepts, Storms et al. (2000) who eval- uate the differences in performance between exemplar and prototype models on a number of natural categorization tasks, and Voorspoels et al. (2008) who model typicality ratings for natural language concepts. A common assumption underly- ing this work is that the meaning of the concepts involved in categorization can be represented by a set of features (also referred to as properties or attributes). Indeed, featural representations have played a central role in psychological theories of semantic cognition and knowl- edge organization and many studies have been conducted to elicit detailed knowledge of features. In a typical procedure, participants are given a series of object names and for each object they are asked to name all the properties they can think of that are characteristic of the object. Although fea- ture norms are often interpreted as a useful proxy of the struc- ture of semantic representations, a number of difficulties arise