What simple and complex cells compute.

Visual neuroscientists seek to answer two related questions. First, what does the visual system do? Second, how does it do it? While an answer to the second question is a description based on anatomy and biophysics, an answer to the first question is a description of computations performed on images. Important steps in providing this description were made with the publication of two classical studies on primary visual cortex by Movshon et al. (1978a,b). These studies have not aged in their readability, and do not require an interpretive key to be enjoyed by the contemporary reader. As an introduction, however, this Perspectives article briefly reviews the state of the art at the time of their appearance, their main findings, and their influence on three decades of subsequent investigations. The 1960s brought important advances in the understanding of the computations performed by the retina. An influential view emerged that described retinal ganglion cells as linear filters, i.e. as processes that compute a weighted sum of the intensities in the stimulus, with weights given by the receptive field (Enroth-Cugell & Robson, 1966). This simple ‘linear model’ applied to the majority of ganglion cells in the cat retina (those of the X type, corresponding to P cells in primates). It was quite powerful, as it promised to predict the responses of X ganglion cells to arbitrary stimuli, on the basis of the simple knowledge of the receptive field. This purely linear view was complemented by the discovery that another kind of ganglion cells, those of Y type (which correspond to M cells in primates), perform non-linear operations, suggesting that they sum the distorted output of subunits that in turn have linear receptive fields (Enroth-Cugell & Robson, 1966; Hochstein & Shapley, 1976). This arrangement confers position invariance: for stimuli of high spatial frequency, Y cells respond equally regardless of position within the receptive field. These advances suggested new ways of looking at simple and complex cells in primary visual cortex. Hubel & Wiesel (1962) had defined simple cells as having distinct antagonistic regions in their receptive fields, and had suggested that knowing those regions, one could predict ‘the responses to any shape of stimulus, stationary or moving’. They had defined complex cells as any cell that was not simple, and had reported that complex cells achieved position invariance within their receptive field: they would respond to a stimulus of the appropriate orientation regardless of position within the receptive field. These attributes of simple and complex cells resembled those of X and Y ganglion cells. Some authors proposed that the correspondence could be anatomical, i.e. that it reflected predominance of X inputs to simple cells and of Y inputs to complex cells (Stone, 1972; Movshon, 1975), a suggestion that was not later confirmed. More generally, however, the linear and non-linear models and the stimulation procedures that had been so useful to analyse X and Y cells (Enroth-Cugell & Robson, 1966; Hochstein & Shapley, 1976) constituted promising starting points for a concise and precise characterization of simple and complex cells. In addition to the breakthroughs in retinal physiology, another force was pushing towards the use of quantitative engineering techniques in primary visual cortex: such techniques were proving successful to study human perception. In particular, the University of Cambridge – where Movshon, Thompson and Tolhurst operated – was a hotbed of research into the relations between single neuron responses and perceptual phenomena. These phenomena were investigated with rigorous psychophysical measures, and described with quantitative models based on image filtering (Graham, 1989; Wandell, 1995). Much of this research rested on the concept of ‘channels’, which are linear filters. Simple cells in the cortex seemed to be good candidates for such a role. Did they exhibit linear summation? The time had come to apply the power of linear systems analysis and related techniques to primary visual cortex. In their first article, Movshon et al. (1978a) applied linear systems analysis to the responses of simple cells. They measured responses of simple cells to gratings and bars, and asked if such responses were consistent with the output of a linear receptive field (Fig. 1A). Much as Enroth-Cugell & Robson (1966) had done for X retinal cells, they asked if the responses to drifting sinusoidal stimuli were sinusoidal, as one would expect from a linear filter. The results supported this view, provided that the responses were rectified by the spike threshold, which shows only the part of the responses that lie above threshold (Fig. 1A). The authors then asked how the responses depended on the spatial phase of a standing grating whose contrast oscillated sinusoidally in time. This test had been applied to X and Y retinal cells by Hochstein & Shapley (1976). Here the linear model was put to a quantitative test, and the fit was good, provided again that the output of the receptive field was passed through a rectification stage that thresholded it (Fig. 1A). Movshon, Thompson and Tolhurst were even able to suggest how high the threshold should be relative to rest. They expressed this threshold in the units of firing rate responses, spikes s−1. For example, for the cell in their Fig. 4, the estimated threshold was 8 spikes s−1 (if the receptive field were to output 12 spikes s−1, the neuron would output 4 spikes s−1). Finally, the authors asked a key question: was the selectivity of simple cells predictable on the basis of the receptive field alone, as had been suggested by Hubel and Wiesel, and as would be expected of a linear filter? To test this hypothesis, they turned again to the approach that had demonstrated the linearity of X retinal cells (Enroth-Cugell & Robson, 1966). First, they measured responses to drifting gratings of various spatial frequencies (Fig. 2A). Then, they measured responses to bars flashed at various positions, thus estimating the profile of the receptive field (Fig. 2B, histograms). According to the linear hypothesis, the first data set could be used to predict the second one. This was indeed the case for the cells (Fig. 2B, curve): just as predicted by the linear model, the selectivity of simple cells for spatial frequency could be