Neuroscience Letters xxx (2005) xxx–xxx

Spatio-temporal dynamics of the visual system revealedin binocular rivalry
Fumihiko Taya a,b , Ken Mogi b,∗a Sony Computer Science Laboratories, Takanawa Muse Bldg. 3-14-13, Higashigotanda, Shinagawa-ku, Tokyo 144-0022, Japanb Graduate School of Medicine, Osaka University, 2-2, Yamadagaoka, Suita, Osaka 565-0871, Japan

Received 28 September 2004; received in revised form 23 January 2005; accepted 31 January 2005

Abstract
From the evolutionary viewpoint, animals need to monitor the surrounding environment and capture salient features, such as motion, forsurvival. The visual system is highly developed for monitoring a wide area of visual field and capturing such salient features. In humans andprimates, there is a wide binocular field, suggesting a necessity of integrating the images from the two eyes. Binocular rivalry [R. Blake, A
neural theory of binocular rivalry, Psychol. Rev. 96 (1989) 145–167; R. Blake, N.K. Logothetis, Visual competition, Nat. Rev. Neurosci. 3(2002) 13–21], where incompatible inputs from the two eyes compete to emerge in the subject’s visual percept, has been shown to exhibit
highly adaptive behavior [I. Kovacs, T.V. Parathomas, M. Yang, A. Feher, When the brain changes its mind: interocular grouping during
binocular rivalry. Proc. Natl. Acad. Sci. U.S.A. 93 (1996) 15508–15511; N.K. Logothetis, Single units and conscious vision, Philos. Trans.
R. Soc. Lond. B. Biol. Sci. 353 (1998) 1801–1818]. Here we investigated the spatio-temporal dynamics of the ocular dominance pattern inbinocular rivalry under conditions where conflicting salient features were presented in a temporally varying manner. We found a striking
example of the detailed structure of the dominance wave propagation, by using a spatio-temporal sampling method. The data show in detailthe ability of the visual system to dynamically adapt to the changing stimuli in the context of the massively parallel visual field. We show bymodel prediction that the globally coherent dominance change in the presence of multiple stimuli can be explained by a mechanism based on
local saliency comparison.
© 2005 Elsevier Ireland Ltd. All rights reserved.
Keywords: Saliency; Binocular rivalry; Spatio-temporal sampling method; Dynamical adaptation

The visual system needs to monitor a wide area of visual field
in a massively parallel manner, in order to enable perceptualgrouping [26,31] and capture salient features embedded in
the visual field. The difficulties inherent in dealing with massively parallel distribution of visual features in the wide visual
field is demonstrated by the failures of the visual system toregister salient features, in seemingly familiar situations suchas those associated with inattentional blindness [25].
The percept in the visual field is constructed as a result of afusion of the inputs from the two eyes. In normal conditions,
there is a high degree of correlation between the binocularsignals, and fusion leads to depth perception. When the correlation between the binocular images is low, however, there∗ Corresponding author. Tel.: +81 3 5448 4380; fax: +81 3 5448 4273.

E-mail address: kenmogi@csl.sony.co.jp (K. Mogi).
0304-3940/$ – see front matter © 2005 Elsevier Ireland Ltd. All rights reserved.
doi:10.1016/j.neulet.2005.01.087

is a competition between the two eyes, resulting in binocular
rivalry [4,5]. Studying the nature of binocular rivalry in thepresence of competing salient stimuli is an effective tool for
uncovering how the visual system captures salient features
distributed in the visual field. Elucidating the neural correlates of binocular rivalry is also instrumental in clarifying theneural correlates of visual awareness [10].
The exact nature of neural mechanisms underlying binocular rivalry is not yet known. Signals from the right and left
eye retinae are integrated at the primary visual cortex (V1),so the neural activities in area V1 are the obvious candidate
for the neural correlates of perceptual alternation in binocular
rivalry [4]. However, there are apparently conflicting reports.
The early psychological studies (e.g., [4]) and recent psychological [32] and imaging studies [16,24,28,30] seem tosuggest that area V1 is crucial in inducing rivalry. On the

2

F. Taya, K. Mogi / Neuroscience Letters xxx (2005) xxx–xxx

other hand, the electrophysiological studies on binocular rivalry have suggested that the neural activities in the higher
visual areas, such as area V5/MT, STS and IT, rather thanarea V1 were correlated with the subject’s percept. Someother studies [15,19,29] also seem to support the latter view.
These conflicting evidences mainly arise from the fact that
there are multi-level factors involved in competition between
perceptual states, i.e., low-level sensory features and highlevel cognitive interpretations [6,21]. In order to clarify theinterplay between the lower and higher visual systems, weneed to conduct a detailed analysis of the ocular dominancepattern in the spatio-temporal domain.
In spite of the knowledge of the existence of piecemeal rivalry [15,18,19], most studies on binocular rivalry have been
conducted in an “all-or-none” paradigm, neglecting the spatial heterogeneity of ocular dominance pattern. Studying the
dynamics in the spatial dimension is especially importantin understanding the mechanism behind dominance change
induced by temporally changing salient features. It is wellknown that the sudden onset of visual stimuli induces a perceptual alternation [11,33]. It is also known that the sudden
onset of stimuli induces a dominance wave spreading fromthe locations of the onset stimuli. For example, Wilson andcolleagues have studied the dominance wave propagation induced by a brief contrast change [32]. In general cases where
there are more than one temporally varying features, the visual system must somehow assign dominance in the presenceof competing sources of saliency, resulting in a highly complex dominance wave pattern.
At present, there are technical difficulties involved in monitoring the massively parallel adaptation of the visual system.
Until a satisfactory model of the neural correlates of dominance change supplemented by a noninvasive measurementof neural activities is attained, we need to devise ways to obtain dominance data through the sequential motor output (i.e.,
key stroke) of the subject. Here, by applying a spatio-temporal

sampling method, we show that it is possible to reveal the detailed spatio-temporal structure of ocular dominance patternin the full broadness of the visual field. We report here a striking case where two moving circles induce globally structured
dominance wave propagation. It is shown by model prediction that computation based on local saliency comparison is
sufficient to reproduce the dominance pattern. This result,taken together with earlier studies, provides insights into theneural mechanisms behind rivalry change, especially when arich dynamics of dominance wave propagation is involved.
The subjects were three healthy adults with normal orcorrected-to-normal vision (F.T., male, 24 years old, K.M,
male, 37 years old, and I.I., male 30 years old). All subjects
gave informed consent.
The stimuli used were one or two circle(s) moving on homogeneous background colors. The background colors were
blue for the left eye and yellow for the right eye (Fig. 1a).
These stimuli were chosen to investigate how the visual system can assign dominance to competing salient features ina dynamical manner. Preliminary observations have foundthat the ocular dominance pattern adapts dynamically to themoving visual stimuli and results in the representation of thesalient features continuously. When the subjects were presented with two circles moving with various phase differences (Fig. 1a), they reported that both of the moving circles
were continuously present in the percept. In the case of aphase difference of 180◦ , the subjects experienced a “flipflop” change of the perceived background color. For phasedifferences other than 180◦ , the spatio-temporal ocular dominance pattern was often too complicated to be described frommemory. In various phase difference conditions, the subjects
reported always seeing both circles. When two stationary circles were presented in rivalry, sometimes either one or bothof them disappeared from the percept (Fig. 1b).
We used a liquid crystal shutter eye wear (Crystal Eyes,
Stereo Graphics Corporation, 2171 E. Francisco Boulevard,

Fig. 1. (a) Experimental condition. The subjects were asked to fixate on the fixation point at the center of the screen. The circles move with a constant velocityof 4.4◦ /s. The subjects were asked to report by key pressing the background color at the position of the indicator that flashed with an interval of 1 s. (b) Average
number of circles seen in the moving and stationary circles condition. Error bars show +1 S.D.

F. Taya, K. Mogi / Neuroscience Letters xxx (2005) xxx–xxx

San Rafael, CA 94901, U.S.A.) with a dynamic range of
1000:1, to present the rivaling stimuli to the two eyes. The
computer-generated graphics was presented on the monitor(Eizo E67T) with a frame rate of 120 Hz. The graphic converter extended the upper half and lower half of the graphics tothe whole screen in alternating frames, with the left and rightliquid crystal shutters being shut in an alternating fashion insynchrony with the screen. An infrared emitter synchronized
the liquid crystal shutters. We presented the visual stimuli
by using customized software running on a PC (Endeavor
Pro-400L, Epson).
The image was projected on a computer screen (22◦ (horizontal) and 16.6◦ (vertical) in visual angle). The subjects
were instructed to fixate on the center (Fig. 1a). The circles
were 1.4◦ in diameter, and moved with a constant velocity of
4.4◦ /s. This angular velocity was chosen so that the subjects
experienced robustly the typical background color perceptchange. When the edge of the circle reached the border of thescreen, the circles reversed their directions and continued tomove with a constant velocity in the opposite direction.
Because of the effect of cortical magnification, the effective area of cortex being stimulated by the circles change
depending on the eccentricity. However, we chose to keep
the physical sizes of the circles constant in order to reproduce commonly occurring situations in natural scenes. We
note that the propagation speed of traveling waves of dominance is unaffected by cortical magnification [32]. There isa well-documented temporal hemifield advantage in rivalry[27], and this could possibly affect the result. However, in
our experiment, there was not a significant tendency for amore pronounced dominance of the ipsilateral color whenthe moving circles were in temporal hemifield than the nasal
hemifield. Contours and textures in the background could
affect the resulting dominance pattern. Preliminary experiments suggest that the existence of grating pattern in the
background, for example, can affect the resulting dominancepattern induced by the moving circles. Our present condition,
therefore, represents the case of the most simple backgroundpattern.
In order to study the detailed spatio-temporal dominancepattern, we conducted an extensive sampling study. During
the presentation of the stimuli, an indicator was presented
at one of the five equally spaced positions in the line of the
motion of the circle center. The indicators were at 3.7, 7.3,
11.0, 14.7 and 18.3◦ from the left edge of the screen, at an interval of 1 s. The subject answered the perceived backgroundcolor at the position of the indicator by pressing one of two
specified keys in a two alternative forced choice between twocolors. The key input was recorded and analyzed on the PC.
The probabilities are resampled using the bicubic interpolation method [13].
The positions and timing were chosen so that the indicatornever overlapped with the circles. The indicator was flashed
50 times in one trial. We conducted five trials for each setof values of phase difference (0, 72 and 180◦ ) and indicator
position. Thus, the subject reported the perceived color at 250

3

points in the spatio-temporal domain in one trial. As depicted
in Fig. 3, where red and blue lines indicate the trajectories
of circles, in the case of phase difference of 180◦ (Fig. 3a),the circles always move in the opposite directions. On theother hand, in the case of 72◦ (Fig. 3b), the circle presented
to the left eye (red line) always follows the circle presented
to the right eye (blue line). In the case of 0◦ (data not shown),
two circles move together, thus they are seen as one circle
in effect. In the stationary circles condition, the position of
the circles were taken to represent the snapshots of presented
image in the 180◦ phase difference condition at the momentof the indicator presentation.
The physical configuration of the crystal shutter eyewear
made it difficult to monitor the subject’s fixation at the timeof the experiment. As an indirect control, we measured thesubject’s point of fixation by using the eye tracking system(EMR-8, NAC Image Technology, 15 McCoy Place Simi Valley, CA 93065 U.S.A.) as the image of two moving circles in a
homogeneous color background (blue) were presented binocularly (i.e., no rivalry involved). The point of gaze stayed
within 1.5◦ of the fixation point for 98.7 and 90.1% of thetime for the x- and y-axis, respectively (mean of F.T. and one
additional subject). The informal post-experiment reports by
the subjects suggested that the resulting dominance patterns
were not greatly affected by small deviations of the point ofgaze.
In order to investigate the common neural mechanism involved in the percept change in the single circle and double
circles condition, we made model predictions for the double
circles condition from the data in the single circle condition.
Predictions for the double circles condition from the single
circle data were made by calculating the average probabilityof dominance assuming that the double circles condition canbe approximated by a linear combination of the single circle
conditions, where one circle is moving in the left eye image,the other moving in the right eye image, with the predetermined phase difference value.
Thus,p(E, {L, R}) =p(E, {L}) + p(E, {R})2Here, p(E, S) indicates the probability of E = {L, R} eye dominance, where L indicates left and R indicates right, during
presentation of circle to S = {{L},{R},{L, R}} eye(s).
Fig. 2a shows the spatio-temporal structure of ocular dominance pattern reported by subjects when presented with a single circle moving in a blue background (presented to the lefteye) against a homogeneous yellow background (presented
to the right eye). The left eye dominant region is distributedin the vicinity of the moving circle presented ipsilaterally,
implying that not only the moving circle itself but also theipsilateral regions in the vicinity of the circle tend to win overthe contralateral regions (“vicinity” effect). The left eye dominance is invoked with a certain amount (∼0.7 s) of temporaldelay, in a traveling wave of dominance lagging behind thecircle. Once the left eye dominance for a particular position

4

F. Taya, K. Mogi / Neuroscience Letters xxx (2005) xxx–xxx

Fig. 2. (a) The spatio-temporal structure of ocular dominance pattern during the presentation of a single circle averaged over three subjects. The gray scale
indicates the ocular dominance probability. Red line represents the trajectory of the moving circle presented to the left eye. Yellow dots represent the 0.5 chancelevel. (b) Time course of the ocular dominance change for the average of three subjects during the presentation of a single circle. Here, we show the probabilityof perceiving the ipsilateral color, as a function of the distance from the center of the moving circle. The blue line indicates the interpolated data by means of
cubic spline interpolation method, while the red line indicates the point (−3.1◦ in visual angle) which divide the area surrounded by blue line into two equally
sized parts, suggesting an average temporal delay of 0.7 s in the ocular dominance change. Error bars show +1 S.D.

in the visual field is over, right eye dominance is more likely(fatigue effect) [7].
Fig. 2b shows the probability of perceiving the ipsilateralcolor as the function of the distance between the indicator andthe center of the single circle. The data set used are the same
as for Fig. 2a. As can be seen from the figure, the ipsilateralcolor dominance follows rather than precedes the moving
circle. Note that the diameter of the circle is 1.4◦ . There is analmost exclusive dominance of the ipsilateral color after thepassage of the circle, with the domain of >0.8 probability of
dominance estimated to extend for 6.2◦ .
Fig. 3a shows the spatio-temporal structure of ocular dominance pattern when the subjects were presented with twomoving circles in homogeneous color backgrounds with aphase difference of 180◦ . When the moving circles cross overat the center of the visual field, the left eye and the right eye
dominant regions “flip” their positions. Fig. 3b shows thedominance pattern when the phase difference = 72◦ . Here,the spatio-temporal structure is much more complicated, but
the background color percept still changes in a way so thatboth circles are continuously present in the percept.
As the sampling method reported in this study involves
only one spatial dimension, we cannot reproduce the dominance change in the fullness of the two dimensional visual
field. In the case where two circles are presented with 180◦
phase difference, the subjects’ verbal reports suggest that thedominance changed in a “flip-flop” like manner as depicted
in Fig. 1a, with the dominance spreading distally from theline of motion of the circles. When the two circles movewith other phase differences and in the single circle condition, the dominance in general has a tendency to spread distally from the line of motion, so that a “comet-like” narrow
dominance tail lagging behind the moving circle(s) was notobserved.

Fig. 3. Spatio-temporal structure of ocular dominance pattern during the presentation of the two circles moving in a homogeneous color background, with aphase difference of (a) 180◦ and (b) 72◦ . The gray scale indicates the ocular dominance probability. Red lines represent the trajectory of the moving circles
presented to the left eye. The blue lines represent the one for the right eye. Yellow dots represent the 0.5 chance level. Averaged over three subjects.

F. Taya, K. Mogi / Neuroscience Letters xxx (2005) xxx–xxx

5Fig. 4. The ocular dominance pattern in the double circles condition predicted from the single circle condition, with a phase difference of (a) 180◦ and (b) 72◦ .
The data are shown for subject K.M.

In order to examine the relation between the single circle
dominance pattern of Fig. 2a and the double circles dominance pattern of Fig. 3, we tested how one could predict one
from the other. Fig. 4 shows the model predictions for thedominance probability for subject K.M. in the double circle condition predicted from the data of single circle. Whensampled over 33 (spatial) × 73 (temporal) = 2409 (spatiotemporal) points, for subject K.M. the predicted value wasoff the actual value by an average of 8.2 and 18% for thephase difference of 180 and 72◦ , respectively. For the averageof three subjects, the mean difference between the predicted
and actual data were 8.2 and 17% in the case of 180 and
72◦ phase difference, respectively. Thus, we conclude thatthe spatio-temporal structure of ocular dominance pattern inthe single circle and double circles conditions were fairly
consistent, suggesting a common underlying neural process.
Experiments with varied speed of the moving circles
would facilitate further understanding of the cortical mechanisms involved. Preliminary results suggest that the continuous presentation of two circles in the percept is quite robust
over different velocity of the circles.
Binocular rivalry is a competition not only between rightand left eye [4], but also between features [15,19], leading
to alternative interpretations of the external world. Alais andBlake [1] reported that the globally coherent motion stimuli enhanced the predominance of the local motion stimuli.
Ooi and He [22] suggested that the voluntary attention could
influence the visual perception only during dominant phase.
These observations, taken together with our data, point to ahighly complex dominance assignment process in the presence of dynamically changing salient features. The delay ofocular dominance change in single circle condition (Fig. 2)
may reflect the time required for object-based mechanism towork [21].
In double circles conditions, the visual system behaves as
if it can predict the trajectory of moving circles, in spite of
the lack of prediction in single circle conditions. Barlow [3]has pointed out that “prediction is possible if there is spatio-

temporal redundancy in the input data”. The fatigue after
passing of circles moving in a periodic manner is a possible
neural mechanism for exploring redundancy. This temporal
alternation of the percept might also reflect the “inhibition of
return” (IOR), the process by which the currently attended location is prevented from being attended again [12,14]. Recent
studies have suggested that perceptual alternation in binocular rivalry and ambiguous figures share a common mechanism[2,9,17,18]. The “flip-flop” effect observed in the double circles condition might reflect such a temporal integration [23].
Although the spatio-temporal ocular dominance pattern inthe presence of two moving circles is quite complex, the fairly
good model prediction of ocular dominance pattern in double
circles conditions from that in single circle conditions (Fig. 4)
indicates that the dynamics of ocular dominance change, suchas “flip-flop” effect might be explained by the fatigue after
passing of the salient features, suggesting that the relatively
rapid change may be caused by the interaction within neurons
in the lower visual areas, such as V1. Two factors, namely
the induction of dominance by a moving stimulus, and thefatigue after the passing of the dominance, seems to explain
the spatio-temporal dominance pattern in the single circle
and double circles conditions. Within the set of competingsalient features studied here, local nonlinear dynamics seems
sufficient to explain the dominance wave propagation.
Carlson and He [8] have revealed that conflicting global
stimuli that consist of compatible local elements failed to induce binocular rivalry, implying that the initiation of rivalryoccurred at an early stage of visual processing. Our present
analysis of dominance wave also suggests that the globally
structured dominance change reported here can be explainedby local nonlinear dynamics, suggesting the involvement ofearly visual system. Although the involvement of higher visual areas in general cannot be excluded, we suggest that a
local computation supported by early visual areas can resultin a highly rich dynamics of dominance wave propagation.
We have shown here that a conventional psychophysicalmethod, implemented with an appropriate spatio-temporal

6

F. Taya, K. Mogi / Neuroscience Letters xxx (2005) xxx–xxx

sampling paradigm, can reveal the detailed spatio-temporal
structure of the ocular dominance in binocular rivalry. Such amethod would provide a window to the functionally embedded dynamics of the visual system, especially when combinedwith the brain imaging techniques such as fMRI and MEG[20,24,28–30].

Acknowledgement
We thank Dr. Mario Tokoro for helpful discussions. We
are indebted to the two reviewers for valuable comments.

References[1] D. Alais, R. Blake, Interactions between global motion and local
binocular rivalry, Vision Res. 38 (1998) 637–644.
[2] T.J. Andrews, D. Purves, Similarities in normal and binocularly rivalrous viewing, Proc. Natl. Acad. Sci. U.S.A. 94 (1997) 9905–9908.
[3] H. Barlow, Redundancy reduction revisited, Network 12 (2001)
241–253.
[4] R. Blake, A neural theory of binocular rivalry, Psychol. Rev. 96(1989) 145–167.
[5] R. Blake, N.K. Logothetis, Visual competition, Nat. Rev. Neurosci.
3 (2002) 13–21.
[6] R. Blake, K.V. Sobel, L.A. Gilroy, Visual motion retards alternations
between conflicting perceptual interpretations, Neuron 39 (2003)
869–878.
[7] R. Blake, D. Westendorf, R. Fox, Temporal perturbations of binocular rivalry, Percept. Psychophys. 48 (1990) 593–602.
[8] T.A. Carlson, S. He, Competing global representations fail to initiate
binocular rivalry, Neuron 43 (2004) 907–914.
[9] O.L. Carter, J.D. Pettigrew, A common oscillator for perceptual rivalries? Perception 32 (2003) 295–305.
[10] F. Crick, C. Koch, Are we aware of neural activity in primary visualcortex? Nature 375 (1995) 121–123.
[11] S. Duensing, B. Miller, The Cheshire Cat effect, Perception 8 (1979)
269–273.
[12] L. Itti, C. Koch, Computational modeling of visual attention, Nat.
Rev. Neurosci. 2 (2001) 194–204.
[13] R.G. Keys, Cubic convolution interpolation for digital image processing, IEEE Trans. Acoust. Speech Signal Process. 29 (1981)
1153–1160.

[14] R.M Klein, Inhibition of return, Trends Cogn. Sci. 4 (2000) 138–
147.
[15] I. Kovacs, T.V. Parathomas, M. Yang, A. Feher, When the brainchanges its mind: interocular grouping during binocular rivalry, Proc.
Natl. Acad. Sci. U.S.A. 93 (1996) 15508–15511.
[16] S.H. Lee, R. Blake, V1 activity is reduced during binocular rivalry,
J. Vis. 2 (2002) 618–626.
[17] D.A. Leopold, N.K. Logothetis, Multistable phenomena: changing
views in perception, Trends Cogn. Sci. 3 (1999) 254–264.
[18] N.K. Logothetis, Single units and conscious vision, Philos. Trans.
R. Soc. Lond. B. Biol. Sci. 353 (1998) 1801–1818.
[19] N.K. Logothetis, D.A. Leopold, D.L. Sheinberg, What is rivalling
during binicular rivalry? Nature 380 (1996) 621–624.
[20] E.D. Lumer, K.J. Friston, G. Rees, Neural correlates of perceptual
rivalry in the human brain, Science 280 (1998) 1930–1934.
[21] J.F. Mitchell, G.R. Stoner, J.H. Reynolds, Object-based attention determines dominance in binocular rivalry, Nature 429 (2004) 410–
413.
[22] T.L. Ooi, Z.J. He, Binocular rivalry and visual awareness: the roleof attention, Perception 28 (1999) 551–574.
[23] E. Poeppel, A hierarchical model of temporal perception, Trends
Cogn. Sci. 1 (1997) 56–61.
[24] A. Polonsky, R. Blake, J. Braun, D.J. Heeger, Neuronal activity
in human primary visual cortex correlates with perception during
binocular rivalry, Nat. Neurosci. 3 (2000) 1153–1159.
[25] D.J. Simons, Attentional capture and inattentional blindness, Trends
Cogn. Sci. 4 (2000) 147–155.
[26] W. Singer, C.M. Gray, Visual feature integration and the temporal correlation hypothesis, Annu. Rev. Neurosci. 18 (1995) 555–
586.
[27] F. Tong, Competing theories of binocular rivalry: a possible resolution, Brain Mind 2 (2001) 55–83.
[28] F. Tong, S. Engel, Interocular rivalry revealed in the cortical blindspot representation, Nature 411 (2001) 195–199.
[29] F. Tong, K. Nakayama, J.T. Vaughan, N. Kanwisher, Binocular rivalry and visual awareness in human extrastriate cortex, Neuron 21(1998) 753–759.
[30] G. Tononi, R. Srinivasan, D.P. Russel, G.M. Edelman, Investigating neural correlates of conscious perception by frequency-tagged
neuromagnetic responses, Proc. Natl. Acad. Sci. U.S.A. 95 (1998)
3198–3201.
[31] A. Treisman, The binding problem, Curr. Opin. Neurobiol. 6 (1996)
171–178.
[32] H.R. Wilson, R. Blake, S.H. Lee, Dynamics of travelling waves invisual perception, Nature 412 (2001) 907–910.
[33] J.M. Wolfe, Reversing ocular dominance and suppression in a single
flash, Vision Res. 24 (1984) 471–478.