The Emergence of Semantic Categories from Distributed Featural Representations

The Emergence of Semantic Categories from Distributed Featural Representations Michael J. Greer *(mgreer@csl.psychol.cam.ac.uk) Maarten van Casteren ^ ( maarten.van-casteren@mrc-cbu.cam.ac.uk) Stuart A. McLellan *(sam26@cam.ac.uk) Helen E. Moss *(hem10@cam.ac.uk) Jennifer Rodd * ( jrodd@csl.psychol.cam.ac.uk) Timothy T. Rogers ^ ( tim.rogers@mrc-cbu.cam.ac.uk) Lorraine K. Tyler * (lktyler@csl.psychol.cam.ac.uk) *Centre for Speech and Language, Department of Experimental Psychology, University of Cambridge ^MRC Cognition and Brain Sciences Unit, Cambridge, UK Abstract This paper presents a computational model of semantic memory, trained with behaviourally inspired vectors. The results are consistent with the conceptual structure account (Tyler, Moss, Durrant-Peatfield & Levy, 2000), which claims that concepts can be understood, and the effects of random damage predicted, based on (i) the number of correlations its features make, and (ii) the distinctiveness of those correlated features; the former indicating category membership, the latter distinguishing between concepts. The model shows a changing direction of domain-specific deficits as damage accumulates (animals concepts lost first, then objects upon severe lesioning). Also, the pattern of error differs between domains; animals tend to be confused for other members of the same category, whilst object errors disperse more widely across categories and domain. Recent neuropsychological evidence demonstrates a similar pattern for semantically impaired patients. For both patients and the model, this can be attributed to the timing of featural loss: distinctive features are lost earlier than shared features. The model demonstrates that the relative timing of feature loss differs between domains, resulting in the emergence of domain-specific effects. Introduction The neuropsychological literature on semantic memory shows patients can develop an impairment in one domain of knowledge, whilst the other is relatively spared. Most commonly, semantically impaired patients show a deficit for living things (e.g. Warrington & Shallice, 1984), with the reverse pattern being rarer (e.g. Hillis and Caramazza, 1991). There are three main types of explanation for the double dissociation. One postulates physically separate and functionally independent stores in the brain for dissociable categories of knowledge (e.g. Goodglass, Klein, Carey and Jones, 1966; Carramazza & Shelton, 1998). Another suggestion is that concepts may vary by domain according to the type of semantic information upon which they depend, with living things depending more on sensory information and artefacts depending more on functional properties (Warrington & Shallice, 1984; Warrington & McCarthy, 1983; 1987). Selective brain damage to one type of semantic information will lead to a category-specific deficit. This account assumes neuro-anatomical specialisation for type of property rather than category per se, to permit their independent disruption by brain damage. Finally, and most recently, attempts to account for category-specific deficits suggest that they can emerge from the internal structure of concepts alone without any type of neural or functional specialisation. Computational models have shown that random damage to a unitary, distributed system can produce category-specific deficits (e.g. Devlin, Gonnerman, Anderson and Seidenberg, 1998; Tyler et al, 2000). These models draw on structural aspects such as property correlation and distinctiveness. Conceptual Structure Account Common to all distributed accounts of semantic memory (see McRae, de Sa, Seidenberg, 1997; Devlin et al, 1998; Tyler et al, 2000) is the observation that similar concepts tend to have overlapping sets of semantic features. Properties that frequently co-occur in concepts will serve to predict each others presence, a fact that a distributed connectionist network will exploit during training, leading to mutual activation of those properties. A consequence of mutual activation is resilience to damage of those properties, and hence their continued ‘availability’ to a stricken network when identifying concepts. A second important factor, is the distinctiveness 1 of features (cf. Devlin et al, 1998). A feature that is present in only one concept can be used Distinctiveness is calculated as 1/number of concepts for which the property is given.