The Emergence of Orientation Selectivity in Self-Organizing Neural Networks

Introduction The past three decades have witnessed a host of impressive findings concerning the functional architecture of the brain. One trend that much recent research suggests is that there is a high degree of specialization within the brain: different parts of the brain do different, highly specific tasks. Furthermore, most estimates place the number of cortical neurons alone at 1010, with roughly 1014 total synapses connecting them. Given this enormous number of neurons, and considering the extreme structural complexity embodied by a healthy and fully-developed mammalian brain, the project of elucidating this structure in some ways pales in comparison to that of explaining how it came to be. It is clear that the mammalian genome “cannot, in any naive sense, contain the full information necessary to describe the brain” (von der Malsburg, 1990). Yet somehow, in the process of development, the brain acquires its structure. The compelling nature of this quandary makes the notion of self-organization very appealing. The term “self-organization” is generally used to describe the evolution of complex behavior in systems that consist solely of many very simple parts. The brain is an excellent example of such a system, for while neurons are by no means simple, many of their principle functional characteristics are believed to be relatively easy to approximate and model. In this review I will focus on research that suggests that the early stages of the mammalian visual system can be implemented solely through the self-organizing properties of neural networks. Central to all such research is the assumption that synaptic modification occurs via some form of Hebbian learning, whereby the strength of a synapse between two neurons is increased if the activities of the neurons are correlated in time, that is, if one tends to be active at the same time that the other is. This learning rule was first suggested by Hebb in 1949 without an accompanying brain mechanism or body of evidence for its existence. Since then, the discovery of long-term potentiation (LTP) in the hippocampus has provided evidence that a form of Hebbian learning does occur in the brain, although how it is accomplished is still not clear. While skepticism about LTP remains, it is generally agreed that some form of local learning is bound to control connectivity in many cases, if only because the idea of completely central control seems inconceivable. Most efforts to show that self-organization can occur in structures resembling the brain thus make use of Hebbian learning. Vision research has been among the most fruitful areas of neuroscience, and the basic structure of the early levels of the mammalian visual system are for the most part established. The research that I will be discussing concerns the visual system up to and including the primary visual cortex (V1). The visual pathway begins at the retina, where there are several layers of cells which project via the optic nerve to the lateral geniculate nucleus (LGN) in the thalamus, which in turn projects to layer 4C of V1. Cells in the inner layers of the retina and in the LGN are characterized by spatial-opponent receptive fields: Stimuli placed in a central, circular region of the receptive field of a cell in these regions tend to excite the cell, while stimuli placed outside the excitatory region tend to inhibit the cell. In addition, the retina and LGN are topographic: the spatial layout of their cells' receptive fields is topologically similar to the physical layout of the cells themselves. V1 cells differ from those in the LGN or retina in that many of them have orientation selective receptive fields; a cell in V1 will tend to respond strongest to a line of a particular orientation placed in its receptive field. Furthermore, as the Nobel Prize-winning research of David Hubel and Torsten Wiesel revealed, orientation selective cells are laid out in an orderly fashion. In addition to being retinotopically organized, a group of cells whose receptive fields correspond to a certain point in space will be arranged in a line such that their orientation preferences vary continuously along that dimension in cortex. Many have speculated that this layout has the favorable characteristic that each small portion of V1 has all the machinery necessary to perform the first steps in extracting information from the portion of the visual field which it represents. The research reviewed here is exciting because it suggests that many of the characteristics I have just described can be accomplished through usupervised self-organization. A host of experiments have shown the influence of the environment on the development of the nervous system. Although many results in this literature are controversial, it is