The Temporal Correlation Hypothesis of Visual Feature Integration: Still Alive and Well

Having reviewed some of the available evidence for, as well as against, the temporal correlation hypothesis, it is appropriate to ask what conclusions can be drawn with confidence and what questions remain open for future studies. As I see it, these questions fall into four general categories. The first concerns the psychophysical foundations of the theory. As I have outlined here, the correlation hypothesis relies on the notion that perceptual grouping is a rapid, preattentive process that acts in parallel over the visual field to “bind” features with common properties into larger units. Although there is much ongoing debate concerning the details of this process, nearly 80 years of psychophysical research indicate that this fundamental concept is sound.Given this conclusion, the second category of questions concerns the neurophysiological basis of perceptual grouping. Because of the retinotopic organization of visual cortex and its parallel nature, grouping must involve the integration of information from distributed populations of cells. I have briefly reviewed the evidence for two candidate mechanisms, hierarchical convergence and population coding. The evidence indicates that both mechanisms operate together, but concerns raised by the correlation hypothesis suggest that additional mechanisms must be involved. In its simplest form, convergence coding cannot solve the problem, because the constraints on flexibility, anatomical wiring, and numbers of cells are too severe. Population coding appears to solve these problems, but may itself suffer from a superposition problem that is due to the parallel nature of the grouping process. Some mechanism should be available to distinguish one group of bound features, and hence one population of active neurons, from another. This argument appears valid, but it should be pointed out that it is primarily a theoretical construct. We don't know for certain that a “binding” problem exists that requires a neural solution. If we accept that superposition is a problem, then it appears that some form of assembly-based representation is required. The correlation hypothesis posits that cell assemblies are formed when a common pattern of synchronous firing is established among the participating cells.This raises a third set of questions. Does existing evidence support this definition of cell assemblies, and if so, do they exhibit the properties required by the theory to account for perceptual grouping? The answer to the first question appears to be yes. There is an abundance of evidence, from visual cortex and many other neural structures, demonstrating the existence and properties of dynamic ensembles of synchronously firing cells. The answer to the second question is more equivocal. The evidence for synchronous activity clearly satisfies some of the requirements of the correlation hypothesis. But the strongest predictions of the theory are also the ones for which the least evidence is available. In spite of this, and in contrast to the viewpoints expressed by some critics of the theory, none of the existing evidence points to an outright rejection of the theory. The resolution of these issues will have to await the results of future experimental studies.A fourth category of questions concerns the physiological plausibility of the theory. It has been argued that cortical neurons may not possess the requisite biophysical mechanisms to establish synchronous interactions with subsets of their active inputs. I have argued to the contrary that cortical neurons possess an impressive array of mechanisms to generate and respond to patterns of synchronous activity. It remains to be determined how these mechanisms operate together to select certain inputs for processing. In this context, the anatomical connectivity that provides the substrate for perceptual grouping is critical. The appropriate connections must be available to provide a wide repertoire of possible dynamic assemblies. In his early formulations of the theory, von der Malsburg 1981xSee all Referencesvon der Malsburg 1981 proposed the existence of a network of horizontal connections in visual cortex that links populations of neurons with similar receptive field properties. In the years since, much evidence has accumulated to support this prediction (Gilbert and Weisel, 1983, 1989; Blasdel et al., 1992, Soc. Neurosci., abstract; 364xRelationship between intrinsic connections and functional architecture revealed by optical imaging and in vivo targeted biocytin injections in primate striate cortex. Malach, R., Amir, Y., Harel, M., and Grinvald, A. Proc. Natl. Acad. Sci. USA. 1993; 90: 10469–10473Crossref | PubMed | Scopus (297)See all References, 332xAnatomical substrates for early stages in cortical processing of visual information in the macaque monkey. Levitt, J.B., Lund, J.S., and Yoshioka, T. Behav. Brain Res. 1996; 76: 5–19Crossref | PubMed | Scopus (22)See all References, 700xRelation between patterns of intrinsic lateral connectivity, ocular dominance, and cytochrome oxidase-reactive regions in macaque monkey striate cortex. Yoshioka, T., Blasdel, G.G., Levitt, J.B., and Lund, J.S. Cereb. Cortex. 1996; 6: 297–310Crossref | PubMed | Scopus (96)See all References, 56xOrientation selectivity and the arrangement of horizontal connections in tree shrew striate cortex. Bosking, W.H, Zhang, Y., Schofield, B., and Fitzpatrick, D. J. Neurosci. 1997; 17: 2112–2127PubMedSee all References, 283xOrientation-specific relationship between populations of excitatory and inhibitory lateral connections in the visual cortex of the cat. Kisvarday, Z.F., Toth, E., Rausch, M., and Eysel, U.T. Cereb. Cortex. 1997; 7: 605–618Crossref | PubMed | Scopus (122)See all References; Schmidt et al., 1997). When these data are combined with the finding that local (i.e., <500 μm radius) intracortical connections are relatively homogeneous, the anatomical network appears to be well suited for linking activity across a wide array of features (156xStimulus-dependent neuronal oscillations in cat visual cortex (inter-columnar interaction as determined by cross-correlation analysis) . Engel, A.K., Konig, P., Gray, C.M., and Singer, W. Eur. J. Neurosci. 1990; 2: 588–606Crossref | PubMed | Scopus (185)See all References, 119xTopography of contextual modulations mediated by short-range interactions in primary visual cortex. Das, A. and Gilbert, C.D. Nature. 1999; 399: 655–661Crossref | PubMed | Scopus (190)See all References). These findings suggest the interesting possibility that the Gestalt criteria for perceptual grouping reside in the network of intracortical connections (Roelfsema et al. 1996xThe role of neuronal synchronization in response selection (A biologically plausible theory of structured representations in thevisual cortex) . Roelfsema, P.R., Engel, A.K., Konig, P., and Singer, W. J. Cogn. Neurosci. 1996; 8: 603–625Crossref | PubMedSee all ReferencesRoelfsema et al. 1996). If so, this would provide for a large number of possible interactions, with the stimuli and the underlying neuronal dynamics ultimately determining which interactions take place and which features become grouped (563xModeling perceptual grouping and figure ground segregation by means of active reentrant connections. Sporns, O., Tononi, G., and Edelman, G.M. Proc. Natl. Acad. Sci. USA. 1991; 88: 129–133Crossref | PubMed | Scopus (101)See all References, 507xThe role of neuronal synchronization in response selection (A biologically plausible theory of structured representations in thevisual cortex) . Roelfsema, P.R., Engel, A.K., Konig, P., and Singer, W. J. Cogn. Neurosci. 1996; 8: 603–625Crossref | PubMedSee all References, 695xExtraction of perceptually salient contours by striate cortical networks. Yen, S.-C. and Finkel, L.H. Vision Res. 1998; 38: 719–741Crossref | PubMed | Scopus (160)See all References, 334xVisual segmentation by contextual influences via intra-cortical interactions in the primary visual cortex. Li, Z. Network Comput. Neural Syst. 1999; 10: 187–212Crossref | PubMedSee all References, 335xA neural model of contour integration in the primary visual cortex. Li, Z. Neural Comput. 1999; in pressSee all References, 696xPerceptual grouping in striate cortical networks mediated by synchronization and desynchronization. Yen, S.-C., Menschik, E.D., and Finkel, L.H. Neurocomputing. 1999; in pressSee all References).In conclusion, the temporal correlation hypothesis remains an attractive theoretical framework for investigating the neural basis of perceptual grouping. The experiments I have proposed provide one way to test several of the key predictions of this theory, but these studies are still correlative in nature. A true test of the predictions would be one in which a causal link could be established between synchronous activity and perception. Recent experiments in the insect olfactory system demonstrate that tests of this sort can be accomplished in simpler nervous systems (313xDynamical representation of odors by oscillating and evolving neural assemblies. Laurent, G. Trends Neurosci. 1996; 19: 489–496