Organization and Emergence of Semantic Knowledge: A Parallel-Distributed Processing Approach

Organization and Emergence of Semantic Knowledge: A Parallel-Distributed Processing Approach James L. McClelland (jlm@cnbc.cmu.edu) Department of Psychology and Center for the Neural Basis of Cognition Carnegie Mellon University, Pittsburgh, PA 15213 How do you know that Socrates is mortal? More generally, how do you know what properties to attribute to an object? How is the relevant knowledge acquired? How is the knowledge organized in the brain, and how is it affected by brain damage? My colleagues and I have been seeking to answer such questions by developing computational models of semantic cognition and its development. Differentiation and Disintegration of Conceptual Knowledge In our approach, cognition begins not so much with booming buzzing confusion but with a bland conceptual uniformity. At first, all things are represented with highly similar, undifferentiated patterns of activation. As the network experiences the various properties different things manifest in different contexts, it gradually comes to differentiate them. This differentiation process is sensitive to patterns of coherent co-variation of properties of objects. That is, it picks up on the fact that there are many objects in the world that have wings and feathers and can fly and many others that have four legs, a tail, a wet nose, and can bark. First general and subsequently more specific differentiations occur. The representations of objects reflect their underlying conceptual similarity even as they become more and more differentiated. After learning, progressive damage to the network results in gradual disintegration of the conceptual knowledge, largely reversing the patterns seen in development. These differentiation and disintegration processes closely follow patterns seen in children’s cognitive development and in the progressive loss of conceptual knowledge in patients with semantic dementia (Rogers et al, 2005). These effects coexist, in both the model and in human performance, with early emergence of the ‘basic level’, and with frequency, typicality, and expertise effects. Parallel-Distributed Processing Our overall framework relies on an approach to semantic cognition first suggested by Hinton (1981). Hinton’s proposal was that our knowledge of the properties of objects as expressed in propositions about them, such as ‘A canary can fly’, is not stored directly in propositional form but in the strengths of connections between simple processing units that allow propositions to be formed mentally by pattern completion – e.g. filling in a pattern of activation representing ‘fly’ when probed with patterns representing ‘canary’ and ‘can’. After the introduction of the back- propagation algorithm, Hinton (1989) and Rumelhart (1990; Rumelhart and Todd, 1993) showed how this learning algorithm could discover useful internal representations that would support generalization. In work building on their efforts, my colleagues and I have proposed an overall architecture for the representation, learning, and processing of semantic information. This architecture provides the context in which we have gone on to address questions like those enumerated above. Complementary Learning Systems Capturing Phenomena Others have Attributed to Innately Constrained Naive Domain Theories A key element of the approach is that semantic cognition takes place within a distributed network of contributing brain areas that work together to allow us to learn, represent, and process semantic and other types of information (McClelland, McNaughton, and O’Reilly, 1995). One part of this network, the neocortical learning system, allows for the developmental elaboration of organized semantic representations through a gradual learning process. The other, located in the medial temporal lobes, provides a mechanism for learning new information rapidly, while avoiding catastrophic interference that would otherwise occur if the new information were quickly incorporated into the neocortical learning system. For present purposes I focus here on recent work (Rogers & McClelland, 2004) addressing processes we think of as taking place primarily within the neocortex. Beyond capturing developmental and neuropsychological progressions, the model can address several findings others have attributed to naive domain theories constrained by innate knowledge. These include: Early signs of sensitivity to domain- and property- specific patterns of generalization of attributions from one object to another (Macario, 1991; Gelman and Markman, 1986) Illusory attribution of properties to objects, e.g. attributing legs to animals that do not have them (Williams and Gelman, 1998). Conceptual reorganization and coalescence of categories with the accumulation of experience across varied contexts (Carey, 1985). The model exhibits these phenomena as a result of gradual learning sensitive to the structure present in its inputs,