A crucial step toward understanding visual processing is to obtain a comprehensive description of the relationship between visual stimuli and neuronal responses. Many neurons in the visual cortex exhibit nonlinear responses, making it difficult to characterize their stimulus-response relationships. Here, we recorded the responses of primary visual cortical neurons of the cat to spatiotemporal random-bar stimuli and trained artificial neural networks to predict the response of each neuron. The random initial connections in the networks consistently converged to regular patterns. Analyses of these connection patterns showed that the response of each complex cell to the random-bar stimuli could be well approximated by the sum of a small number of subunits resembling simple cells. The direction selectivity of each complex cell measured with drifting gratings was also well predicted by the combination of these subunits, indicating the generality of the model. These results are consistent with a simple functional model for complex cells and demonstrate the usefulness of the neural network method for revealing the stimulus-response transformations of nonlinear neurons. n the primary visual cortex, response properties such as the spatial structure of the receptive field (RF) and tuning to orientation of the stimuli have been studied extensively with simple stimuli (e.g., light spots, bars, and sinusoidal gratings) that were specifically designed to measure these properties (1, 2). Although these studies have provided crucial insights into the functions of the primary visual cortex, the information obtained with such a paradigm may be insufficient for understanding the neural responses to complex spatiotemporal inputs, which are frequently encountered in the natural visual environment. An alternative approach is to use large ensembles of complex stimuli to probe the stimulus-response relationship. One example of this approach is the use of white-noise stimuli and the reverse correlation method in characterizing sensory neurons (3-6). The resulting linear RFs can be used to predict the neuronal responses to other arbitrary stimuli (6-9). However, although the linear method has been useful in characterizing neurons in the early visual pathway, including simple cells in the primary visual cortex (4, 10), it is much less applicable to complex cells due to nonlinearities in their responses (1, 11). Thus a method for characterizing the responses of nonlinear neurons to complex inputs is desirable. Multilayer feed-forward neural networks have been successfully applied to a variety of problems to extract the input-output relationship of nonlinear systems (12). Despite the structural simplicity of these artificial networks, they have led to insights into the functions of various biological circuits (13-17). Here, we have trained neural networks with the back-propagation algorithm (18) to predict the responses of V1 neurons to random-bar inputs. For complex cells, the random initial connections in the networks consistently converged to patterns that were highly regular: the RFs of the hidden units had alternating ON and OFF spatial subregions, resembling the RFs of simple cells. The responses to the random-bar stimuli as well as the direction selectivity of the complex cells were well predicted by the combination of these units. Thus, despite the anatomical complexity of the cortical circuitry, responses of complex cells can be approximated by the combination of a small number of computational subunits resembling simple cells (11, 19) . Physiology. Adult cats (2-3 kg) were initially anesthetized with isoflurane (3%, with O 2 ) followed by sodium pentothal (10 mg͞kg, i.v., supplemented as needed). Anesthesia was maintained with sodium pentothal (3 mg͞kg͞hr, i.v.) and paralysis with vecuronium bromide (0.2 mg͞kg͞hr, i.v.). Pupils were dilated with 1% atropine sulfate, and nictitating membranes were retracted with 2.5% phenylephrine hydrochloride. The eyes were mechanically stabilized and optimally refracted. Endexpiratory CO 2 was maintained at 4-4.5% and core body temperature at 38°C. ECG and electroencephalogram were monitored continuously. All procedures were approved by the Animal Care and Use Committee at the University of California, Berkeley. Visual Stimulation. Stimuli were generated on a personal computer and presented on a Barco (Kortrijk, Belgium) CCID 121 monitor (40 ϫ 30 cm, refresh rate 120 Hz). The random-bar stimulus was presented in a rectangular patch covering the RF of each cell, containing 16 bars with their length equal to or slightly longer than the RF. The contrast of each bar was temporally modulated according to a pseudorandom binary m-sequence (5, 20) (luminance: Ϯ 39 cd͞m 2 from the mean of 40 cd͞m 2 ). The full m-sequence was 32,767 frames long, updated every other frame for an effective frame rate of 60 Hz. Neural Recording. Single-unit recordings were made from 38 cells in V1 with tungsten electrodes (A-M Systems, Everett, WA). Each cell was briefly characterized with drifting sinusoidal gratings. Cells were classified as simple if their RFs had clear ON and OFF subregions (1) and if the ratio of the first temporal harmonic to the mean response to an optimally oriented drifting grating was greater than one (21). All other cells were classified as complex. The full m-sequence was repeated at least three times for each cell, and RF stability was checked between presentations. Two cells were excluded because of eye movement. Seven cells were excluded because of low responsiveness (Ͻ1 spike͞sec) to the m-sequence stimulus. For the remaining 29 cells (21 complex, 8 simple), spike trains were smoothed with a Gaussian filter ( ϭ 10 msec) and then binned at the stimulus frame rate and averaged. Network Structure and Training. We trained two-layer neural networks with a tapped-delay line architecture (Fig. 1 ). There were 256 inputs, representing the spatiotemporal luminance signals S 1 (t), S 1 (t Ϫ 1), . . . , S 1 (t Ϫ 15), . . . , S 16 (t Ϫ 15), where the subscripts index the spatial positions of the bars, and t, . . . , Abbreviations: RF, receptive field; STRF, spatiotemporal receptive field.
Lau et al. trained a multi-layer perceptron with inputs from different delayed time points to predict responses of V1 neurons to random bars REF .
Computational subunits of visual cortical neurons revealed by artificial neural networks
Proceedings of the National Academy of Sciences of the United States of America
Proceedings of the National Academy of Sciences of the United States of America
Medicine Computer Science