License to See: for One Eye Only?
Authors: Stefan Treue
Journal: Neuron (1997)

While crossing our eyes results in the disorienting percept of double vision, it also offers a glimpse into a host of issues of visual information processing. The misalignment of the two eyes results in incompatible images projected onto the two retinae. While the slight differences between the left and the right image caused by the different positions from which the eyes view the world (called binocular disparities) are used to recover the distance or 3-D shape of objects in a process called binocular fusion, the two images when squinting are too different to be fused and now compete for access to our conscious perception. Rather than perceiving two images transparently overlayed on top of each other, we see a mosaic of objects and features patched together from the images on the retinae, something aptly called binocular rivalry. This patchwork is in constant change as the balance of power between the two images is shifting, leading to periods when the percept is almost entirely dominated by the image from one eye and periods when each eye only contributes portions of the perceived image. In the laboratory, binocular rivalry can be generated under much more controlled conditions and without the need to squint by presenting independent stimuli to the left and right eye. The phenomenon of rivalry has been known for a long time, yet one might ask what makes the investigation of such an obscure phenomenon worthwhile. First, binocular rivalry has been used in efforts to determine where the signals from the two eyes are combined in the brain to create the unified visual percept that we experience. Therefore, a better understanding of binocular rivalry might teach us something about how other signals are combined, such as those used to recover depth from binocular disparity or differences between the sounds picked up by our two ears to locate sound sources. Second, binocular rivalry shares many aspects with the perception of ambiguous figures (like the famous Necker cube illusion), and therefore its investigation might help us understand how sensory ambiguities in general are resolved. Finally, binocular rivalry is an example of a selection process, and thus could give us an insight into the implementation of other selection processes. An increasing realization that vision is fundamentally a segmentation and selection process is what makes recent studies of binocular rivalry so important for a general understanding of visual information processing, and therefore this aspect will be the emphasis of this review. Selection processes are of critical importance in visual processing as they reduce the staggering amount of information received by our senses to a level that can be adequately processed. A well-studied example of such a selection process is selective attention, the ability to allocate processing resources in a nonhomogeneous manner, enabling the detailed analysis of stimuli of interest while at the same time not wasting neural processing resources on uninteresting information. Selection processes such as binocular rivalry or attention cannot work without segmentation. Only if we can segment the visual input can we effectively discriminate and separate significant from insignificant information. As we shall see later on, some of these selection processes appear to work on the level of surfaces and objects, suggesting an important role for these tokens in visual perception. This list amply demonstrates the possibilities offered by a better understanding of binocular rivalry to almost all of current visual (and by extension other sensory) system neuroscience. It is thus no surprise that the phenomenon has attracted a lot of psychophysical interest. A number of investigations have established the precise conditions that will cause rivalry but, interestingly, have resulted in notably different predictions about where in the visual pathway it is produced. Some experiments suggest that binocular rivalry occurs when the signal from the two eyes are still segregated, i.e., before or at the level of primary visual cortex (V1, the area where visual information from the eyes first reaches cortex). For example3Fox R Check R J. Exp. Psychol. 1968; 78: 388-395Crossref PubMed Scopus (76) Google Scholar have shown that during rivalrous suppression of stationary stimuli, motion stimuli briefly presented to the suppressed eye were also suppressed, as evidenced by much higher detection thresholds than when the motion stimuli were presented to the dominant, nonsuppressed eye (the latter being detected at levels comparable to measurements under nonrivalrous presentations). These studies suggest that the suppression seen during binocular rivalry operates nonselectively across stimuli and depends on the state of the monocular pathway that is the source of the signal. This has led to the hypothesis that rivalry is caused by the reciprocal inhibition of monocular neurons in V1 (e.g.,1Blake R.R Psychol. Rev. 1989; 96: 145-167Crossref PubMed Scopus (498) Google Scholar). Other experiments, however, have shown that suppression can occur at the level of stimulus representations independent of the eye that is the source of the various parts of the stimulus, suggesting a mechanism that is more high level like that used in scene segmentation or that behind phenomena of Gestalt psychology. Last year6Logothetis N.K Leopold D.A Sheinberg D.L Nature. 1996; 380: 621-624Crossref PubMed Scopus (424) Google Scholar added strong support to the latter view with a psychophysical experiment employing an elegant new paradigm. They created rivalry by presenting flickering gratings of orthogonal orientation to the two eyes. As expected, subjects reported normal rivalry in this first experiment with phases of complete dominance of one stimulus of up to several seconds. When plotting the frequency of the various dominance durations of one stimulus (and thereby of one eye) over the other, the histograms showed the characteristic dynamics (a gamma distribution when plotting a frequency histogram of the length of the dominance phases) long known and shared by other bistable percepts. The authors then repeated the experiment, but they exchanged the two gratings between the two eyes several times per second. If rivalry reflects suppression of the input of one eye (or of an entire monocular pathway), the percept should reflect the stimulus as seen by the dominant eye, i.e., that of a regularly switching orientation. This would be akin to the subject closing one eye (simulating the suppression of that eye), and the percept would be unlike the prolonged phases of dominance of one orientation experienced by subjects in the first experiment. If, on the other hand, the suppression is stimulus-based, then moving the dominant stimuli from one eye to the other would not interfere with its dominance. Thus, subjects would perceive a constant dominant stimulus for extended periods, the same percept experienced in the first experiment. The latter prediction is the one that Logothetis et al. found fulfilled in their study. This offers convincing evidence for a neural representation of the two stimuli competing for visual awareness independent of the eye through which they are received and for a stimulus-based mechanism behind rivalry that can operate at cortical levels that do not contain monocular stimulus representations. The few studies that have tried to look for the neural correlate of binocular rivalry with physiological methods had to face a major methodological challenge that has hampered its investigation up to now. The initial task for such investigations is to find the point in the processing of visual information at which the neurons do not code the sensory stimulus, but rather reflect the perceptual dominance of one image over the other. The problem in finding these cells is the need to record their activity and at the same time get the animal to report its momentary state of perceptual dominance reliably and truthfully, since no other direct assessment is available. This problem now seems to have been overcome by the Logothetis group in two recent publications (4Leopold D.A Logothetis N.K Nature. 1996; 379: 549-553Crossref PubMed Scopus (700) Google Scholar, 9Sheinberg D.L Logothetis N.K Proc. Natl. Acad. Sci. USA. 1997; 94: 3408-3413Crossref PubMed Scopus (512) Google Scholar), adding to the findings of the psychophysical study mentioned above and a previous physiological paper by 5Logothetis N.K Schall J.D Science. 1989; 245: 761-763Crossref PubMed Scopus (445) Google Scholar. The Logothetis group has recorded single-cell activity from a range of cortical areas at various levels of the processing hierarchy for visual signals in awake, behaving macaque monkeys when they were presented with stimuli under conditions causing binocular rivalry. Macaque monkeys were presented with independent images to the two eyes and were trained to report which, if any, of the two images was dominant at any given moment. A skillful design of the experimental paradigm combined with knowledge from human psychophysical investigations of rivalry enabled these researchers to be certain about something as elusive as the moment to moment perception of the animal. The most recent paper of 9Sheinberg D.L Logothetis N.K Proc. Natl. Acad. Sci. USA. 1997; 94: 3408-3413Crossref PubMed Scopus (512) Google Scholar shall serve as an example of their approach. In this study, the monkeys were trained to maintain fixation and to perform a peripheral discrimination task in which they had to pull the left of two levers whenever they saw a starburst-like pattern and the right lever whenever they saw another figure, such as pictures of humans, animals, or man-made objects. They were also trained not to pull either lever when they were presented with a blend of different stimuli (mixed objects). Trials would consist of a series of random transitions between the different stimulus types, and the monkey was rewarded only after responding correctly to all transitions. False responses during a trial would lead to the immediate abortion of the trial without reward for the animal. Additionally, trials would contain rivalrous periods, i.e., periods during which the two eyes were presented with conflicting images. Since the correct answer is not defined for these periods, any answer by the monkey was considered valid. In their experiments, Logothetis and his colleagues took great care to minimize the likelihood that the animal was simply responding randomly during rivalrous periods. Not only did they interleave periods of monocular stimulation, but the use of mixed objects also added to the range of possible percepts, making it less likely that the monkey would adopt two different response strategies. Most convincingly, Leopold and Logothetis showed that the responses of the monkeys during the rivalrous periods showed characteristic dynamics, the gamma distribution mentioned above, making it almost certain that the monkey indeed reported his perceptual state during the rivalrous periods. Similarly, Sheinberg and Logothetis exploited the fact that limiting the spatial frequency content of an image decreases a stimulus' predominance and demonstrated that the monkeys' reports showed the same dependency of predominance of a visual pattern on its spatial frequency content as human reports. The interesting trials and trial periods in all of these experiments were presentations of an effective or preferred stimulus (one that would normally excite a given cell) and an ineffective stimulus (to which the cell normally would not respond) to the two eyes. The most likely neural correlate of rivalry would be cells that would respond more actively when the animal reported the dominance of the preferred stimulus with no corresponding changes in the stimuli. Logothetis and his colleagues indeed found such cells, and from this series of papers, a remarkable story emerges. When recording from areas early in the visual cortical heirarchy, neurons sensitive to one of the two stimuli respond to its presence, but typically do not alter their firing pattern under rivalrous conditions. Logothetis and colleagues found a significant increase in activity during reported periods of dominance of the preferred stimulus in only ∼18% of V1 cells. As one ascends the cortical hierarchy to intermediate visual areas, significantly more cells showing modulation with rivalry are encountered (38% of V4 cells and 43% of MT cells), but almost all of this increase is due to the appearance of cells decreasing their response during preferred stimulus dominance (∼13% of V4 cells and 20% of MT cells), encoding a signal that does not seem to reach consciousness. Logothetis' group interprets these cells as reflecting the perturbations of a process normally involved in grouping and segmentation through feedforward and feedback connections between visual areas. In their most recent publication, they report the results of recording from area IT in the temporal lobe and from the superior temporal sulcus, about four areas beyond V1. Here, the dominance of nonmodulating cells found in earlier visual areas and representing the sensory stimulus is replaced by a dominance of cells that reflect the reported perceptual experience of the animal. In ∼90% of the IT and superior temporal sulcus cells, the monkey's response could be predicted from the cells' activity, i.e., the cells would respond more strongly during periods in which the monkey reported the dominance of the more effective stimulus. These findings go well beyond a better understanding of how binocular rivalry is generated. They are another important building block in our increasing realization that vision is about segmentation and selection. Traditionally, vision has been studied as a filtering process. Individual neurons in the visual system will only respond to a small subset of images. This is because they act as filters that will only respond when the input matches their preferred combination of image properties, such as spatial location, spatial and temporal frequency content, orientation, etc. Through this filtering mechanism, the visual system can break the input into various streams of information, dealing with the different aspects of the input, such as the forms, colors, motions, etc. present in the input. Treating the visual system as a series of such linear or quasilinear filters has been a particularly useful and fruitful approach that has resulted in important advances in our understanding of visual information processing. Many successful models of orientation or direction tuning are based on this approach. Understanding the filtering issues involved in visual processing has also allowed us to realize the magnitude of the task faced by the system. The amount of information impinging on our retinae is monumental and consequently has to be reduced to a manageable size. Selection and segmentation of the input have emerged as aspects of visual information processing that are as important as filtering. The visual system has implemented these two features with a host of mechanisms. Some of them are bottom-up hardwired into early stages of visual processing, such as the uneven distribution of resolution across the retina and the cortical magnification of foveal visual information; others are top-down processes, such as the allocation of attentional resources based on the need of the task at the moment. What all of these mechanisms have in common is the selection of certain information for special processing. This selection mechanism is complemented with a segmentation process that changes the processing of visual information based on the surface or object to which a given feature belongs. The role of surfaces as one of the most crucial tokens of visual information processing is what links the findings of Logothetis et al. (especially4Leopold D.A Logothetis N.K Nature. 1996; 379: 549-553Crossref PubMed Scopus (700) Google Scholar) with a host of other studies, suggesting an important role for surfaces and surface interpolation in visual perception. Examples of such studies include the psychophysical work by Nakayama and Shimojo (see their 1990 overview) and physiological studies by 11Zipser K Lamme V.A.F Schiller P.H J. Neurosci. 1996; 16: 7376-7389PubMed Google Scholar. These latter researchers' recordings from macaque visual cortex suggest that surface-based processing occurs as early as area V1. They compared the responses of V1 neurons when a texture patch presented inside the receptive field and extending beyond its boundaries was uniform with the surrounding texture (and thus not visible as a separate surface or object) with the response evoked by the same texture when the background differed in disparity, color, luminance, or texture orientation. Even though the stimulus presented inside the receptive field was unchanged between the two conditions, V1 neurons showed a prolonged response when the texture was part of a surface or object that was larger than the receptive field, but differing from the surround; information presumably sent back to V1 from higher areas able to extract the object from the background because of their larger receptive fields. Receptive fields in V1 are very small, and thus studies of this area are particularly prone to artifacts from small systematic changes in eye position or eye movements. Barring contamination of their results from such effects, Zipser et al. seem to have tapped into a segmentation process that could serve as the basis for further effects like suppression in rivalry or processes active in selective attention (e.g.,2Duncan J Perception. 1993; 22: 1261-1270Crossref PubMed Scopus (56) Google Scholar). The demonstration that the response of cells in higher cortical areas can vary without a corresponding variation in the stimuli and that the response to one of two stimuli in the receptive field can be suppressed also links these studies with other examples of top-down effects such as the modulation of IT, V4, MT, and MST cells by the attentional state of the animal (7Moran J Desimone R Science. 1985; 229: 782-784Crossref PubMed Scopus (1941) Google Scholar, 10Treue S Maunsell J.H.R Nature. 1996; 382: 539-541Crossref PubMed Scopus (786) Google Scholar). These studies show that the response of a neuron to several stimuli in the receptive field is not only determined by the stimuli's effectiveness but also by the behavioral relevance of the individual stimuli. If the effective stimulus is behaviorally irrelevant, the response of the neuron is reduced, a suppression reminiscent of the suppression seen when the effective rivalrous stimulus is not the dominant one. Despite this similarity, binocular rivalry and voluntary attention are probably not the same mechanisms; both represent highly nonlinear, top-down properties of sensory information processing. Like these studies of attentional modulation and surface segmentation, studies by Logothetis et al. open our eyes to how much more there is to the visual system than its initial processing of the retinal image that is so well approximated by a series of linear filters. Neurophysiological recordings from the behaving monkey, combined with carefully designed paradigms, prove to be powerful tools for understanding the complexity of visual perception.