Webs, cell assemblies, and chunking in neural nets: introduction.

Abstract This introduction to Wickelgren (1992), describes a theory of idea representation and learning in the cerebral cortex and seven properties of Hebb's (1949) formulation of cell assemblies that have played a major role in all such neural net models. Ideas are represented in the cerebral cortex by webs (innate cell assemblies), using sparse coding with sparse, allor-none, innate linking. Recruiting a web to represent a new idea is called chunking. The innate links that bind the neurons of a web are basal dendritic synapses. Learning modifies the apical dendritic synapses that associate neurons in one web to neurons in another web. Mind: Ideas, Thoughts, and Thinking This section describes some basic psychological concepts and principles that motivate the neural models described in the rest of the paper. Assume that the mind contains a set of N ideas. Ideas are representational atoms, such as: a 45 line segment located 30o to the left of the fovea, the letter "p," pressing the tongue against the upper front teeth, a sound of 680 hz, the word "dog,' the image of a particular dog, the concept of a particular individual dog, the concept of a member of the set of dogs, the concept of the set of all dogs, the proposition that dogs eat meat, etc. I use "idea" very generally to mean any of the fundamental units of representation in any module of the mind, not just those modules concerned with the representation of concepts and propositions in semantic memory. Thus, any sensory or motor feature, segment, image, concept, proposition, action, or mental procedure is an idea. Ideas have at least two states of activation in the mind, active and inactive, and there may be intermediate degrees of activation. Ideas also have various degrees of excitation, which is the potential for future activation of an idea. Ideas with a high degree of excitation may already be active or may become active with a small amount of additional input excitation from associated ideas or the external world. Only active ideas produce associative input that may add or subtract from the excitation of other ideas. Ideas with levels of excitation which are below the activation threshold do not provide associative input to other ideas. This is what it means for an idea to be active, namely, that it provides excitatory or inhibitory input to the excitation of other ideas. In a two-state activation model, a thought is a set of activated ideas. In a continuous activation model, a thought is the N-dimensional activation vector that represents the activation of each of the N ideas in the mind. Attentional set (nonassociative short-term memory) is the N-dimensional excitation vector that represents the excitation of each of the N ideas in the mind. Thinking is a sequence of thoughts. The successor thought is determined by a combination of sensory input, associative input from the ideas in the prior thought, and the persisting (decaying) excitation of each possible idea that results from prior sensory and associative input. For the purposes of Wickelgren (1992), one may regard a set of active ideas as a conscious thought. In a larger context, I would probably not want to identify conscious thought with the entire set of active ideas, but only with the subset of active ideas concerned with semantic memory, language, imagery, and a subset of emotion. The number of active ideas composing a thought is a very tiny portion of all the ideas that the mind contains in its long-term memory. The maximum number of ideas that may be simultaneously active in a thought is the attention span. Sometimes it is alleged that attention span is on the order of four or five ideas, whereas the total number of ideas in an adult human mind is surely in the millions or billions. I suspect that there are separate attention spans for parts of the mind, which I will refer to as modules. Most of Wickelgren (1992) is concerned with modeling a single module of the mind, but I am not prepared to specify the nature of our limited attention capacity in either the mind or a module, beyond the principle that the number of active ideas is a tiny fraction of the number of ideas in the module. …