ProceedingsoftheInternationalSymposiumonMeasurement,AnalysisandModelingofHumanFunctions(ISHF2001).pp.67-72.

Towards a systematic turn in cognitive neuroscience.
K. Mogi
Sony Computer Science Laboratories
Takanawa Muse Bldg. 3-14-13, Higashigotanda
Shinagawa-ku, Tokyo, 141-0022 Japankenmogi@csl.sony.co.jp
Abstract

how the brain functions as a system, rather than hopingthat somehow brain function would emerge when you put

It sounds like a cliche to point out the importance ofsufficient number of neurons together in a sufficiently

seeing the brain as a system. Even so, today we do notcomplex manner.

understand the essential systems level features that makes

It is important to point out at the outset the limit of theit possible for the brain to realize its remarkableapproach of localizing the functionality of the cortical

functionality, including dynamical adaptability, sensori-areas based on such experimental techniques as single unit

motor

context

recording, fMRI, MEG etc. Even if one cortical area isdependent learning. In this paper, some essential systems

indicated to carry one particular function, its ultimatelevel features of the brain are discussed, based on recent

functionality must be grounded within the context of theneurophysiological evidences. Data from binocular rivalry

whole brain system.

experiments are presented as a specific instance. Finally,

It is even possible that multiple functionality is imposedsome basic constraints that a systems level brain model

on a cortical area. One of the evolutionary constraints onshould satisfy are proposed.

the brain is that the cortical information processing has tocoordination,

conceptualization,andbe done with limited spatial resources. Accordingly, the1. Introduction

brain is likely to have employed the strategy to utilize asingle cortical area for multiple purposes. The primary

There are approximately 100 billion neurons in the brain,visual cortex (V1) has been traditionally considered to beeach connected to other neurons via thousands of synapses.

a typically visual area. Recent evidence has shown that it

The strength of these synapses are modified over a wideplays an essential role in judging the orientation of arange of time as the result of the brain development andgrating by touch (Zangaladze 1999). It is thus likely thatexperience. Brain areas are heterogeneous in the sense of

the area V1 functions not only as a primary visual area,

synaptic connectivity and the kind of neurotransmitters

but also as the "hub" of spatial information processing,involved. In order to understand such a complex system, aregardless of the sensory modalities, when a precise

reductionist approach, which tries to reduce everythinggeometrical information such as orientation, distance,

down to the synaptic interaction, is simply not practical.

angle etc. are involved. The "body image" area in theWe need to start from a clear and explicit hypothesis onparietal cortex, on the other hand, seems to be concernedmore with the integral and real-time processing of sensory

the famous “young woman vs old woman” figure (Fig.1),

and motor information, rather than precise geometricalwhat you perceive in terms of sensory qualia is the spatialinformation.

distribution of color qualia with varying shades from

The multiple functionality of a cortical area, as well as

white though gray to black. On the other hand, theother new insights into the brain function revealed by“interpretation”

modern cognitive neuroscience (the case of mirror

interpretation of the picture, which is phenomenallyneurons is a particularly striking example, can be only beperceived as intentional qualia, switches between the twounderstood within the systems level property of the brain.

alternatives. When looking at a “bistable figure” such as

There is a systematic turn in cognitive in the making.

this, what we experience is a dynamic matching betweenofthefigure

isbistable.

Yoursensory qualia and intentional qualia.
2. Sensory and Intentional Qualia

To study the brain function without paying regard toconsciousness is like studying the stomach without paying
regard to digestion. The subjective, phenomenal properties
invoked by the neural activities in the cortex provide
important clues to the systems level principles of the brain
fucntion. In recent years, qualia (sensory qualities such as
the redness of red) have emerged as the central issue in thescientific pursuit of the problem of consciousness(Chalmers 1996).
A subset of neural activities in the brain results in thesubject having sensory qualia (such as

the redness ofred) and intentional qualia (such as the meaning of aword). A growing set of evidences suggests that sensory
and intentional qualia arise from a spatially distributed
cluster of neural firings, each reflecting different aspectsof the systems level property of the brain. The sensory
qualia reflect a relatively stable representation of thegeneric features in the environment. When the subject ispresented with a visual stimulus multiple times, he islikely to experience the same set of sensory qualia. On theother hand, the intentional qualia reflect a dynamic,
context dependent interpretation of the environment. InFig.1 Young woman vs Old woman

Based on neurophysiological evidences, it is possible topin down the neural correlates of these two different kindsof qualia. Sensory qualia, e.g. visual qualia (such as theredness of red) are likely to arise from the cluster of neuralacticities starting from the early sensory areas. In vision,neural activity in area V1 is essential in giving rise tosensory qualia (Weiskrantz 1990). Intentional qualia, on
the other hand, are likely to arise from the cluster ofneural activities within the higher cortical areas. Thehigher visual areas such as IT, MT give rise to intentional
qualia that contribute to the dynamic, context-dependentinterpretation of the visual stimulus.

Recent

intentionalqualia
qualiaintentionalparietalsensory qualia

neurophysiologicalevidencessuggestthatbinocular rivalry result from the interaction between the“pointers” that correlate with the neural activities inhigher visual areas and prefrontal areas, and the sensory

prefrontal

qualia that correlate with the neural activities starting fromarea V1. Here, the pointers are a special form of visual
intentional qualia. In order to see something, it seems thatinferio-temporal

it is necessary that there is a “matching” between
intentional qualia (in this case termed “pointers”) andsensory qualia. What is changing in binocular rivalry is
the matching between sensory and intentional qualia.
Fig.2

Sensory and intentional qualia

Taya and Mogi (1999) studied the dynamical adaptabilityof the matching between pointers and sensory qualia as is

3. Dynamical Adaptability in Binocular Rivalry.

evident in binocular rivalry. In this experiment, the imageof a circle moving horizontally with a constant velocity in

Binocular rivalry is a striking phenomenon where inputs
from the two eyes compete to emerge in visual awareness.
For example, when horizontal and vertical gratings arepresented to the right and left eye, you see either thehorizontal and vertical gratings in the visual field, where
the ocular dominance pattern changes with time in aa homogeneous color background was presented to theright and left (Fig.4). The background color was blue forthe left eye and yellow for the right eye. There was aphase difference between the motion of the two circles.
The resulting color of the background in visual awarenesswas used as the indicator of the ocular dominance pattern.

dynamical manner (Fig.3)

Pointers
What "I" seeatt ention

att ention

pointer s

pointers

qualiaPointers
Images fromthe two eyes
Fig.3

Binocular Rivalry

Fig.4

Stimulus of moving circles.

It was found that the spatio-temporal dominance patterncolor as the function of the distance between the indicator

was strongly influenced by the presence of moving circles.

and the center of the moving circles. The probability ofThe subjects reported that both of the moving circles wereseeing the ipsilateral color is large when the distancealways present in visual awareness. The background color

between the indicator and the circle is small. The tendency

perception changed in a manner consistent with theto see the ipsilateral color in the vicinity of a circle

motion of the circles. The percept of a circle was(vicinity effect) is less evident when the circles are in

surrounded by the percept of the background color on

stationary positions. This tendency is most pronounced

which the moving circle is presented (ipsilateral color).

when the phase difference = 180 degrees (i.e., when theSubjects reported always seeing both circles in “slower”

circles are moving in counter-phase), and disappears whenmotions down to ~0.3 degrees per second, and “faster”

the phase difference = 0 (i.e., when the circles are moving

motions up to ~ 20 degrees per second. When the motion

together, as a single circle in effect). These data indicate

was less than 0.2 degrees per second, sometimes one ofthat the perceived background color (and therefore thethe circles disappeared from visual awareness. When twoocular dominance pattern) changes in such away that both

stationary circles were presented in rivalry, sometimesof the moving circles are present in visual awareness.

either one or both of them disappeared from the percept

Thus, the motions of the circles count as salient features,(Fig.5).

influencing what we see consciously. The ocular2dominance pattern must change in such a way that both ofthe moving circles are always present in visual awareness.

1.5

The visual system behaves as a dynamically adaptive
1system to make such a percept change possible.

0.50Statinary
CirclesMovingCircles

Subject K.M.

Statinary
CirclesMovingCircles

Subject F.T.

Fig.5 Average number of circles seen.
To facilitate a quantitative analysis, the subjects wererequested to report the perceived color at the position of
the indicator which appeared as a flashing point in several
positions during the course of the movement of the circles.
The analysis of the ocular dominance pattern in the spatiotemporal domain shows a strong correlation between thedominance pattern and the position of the moving circles.
Figure 5 shows the probability of perceiving the ipsilateralFig.6 `Probability of dominance of background color

4. The Significance of Noise.

robustly under the presence of a high level of noise,
whereas a digital computer would malfunction by theArea MT is considered to be the center of motion

occurrence of even a single bit of noise. However, from a

information processing in vision. Only about 30% of thesystems level viewpoint, especially keeping the "multiple

variability of neural activities in area MT can beuses of a single cortical area" idea (see Zangaladze et. al.

accounted for by the variability of the motion signal itself

1999) in mind, the large variability of neural activities in(Mogi and Barlow 1999). This can be shown by

area MT could be seen to carry information reflecting

calculating

the efficiency of signal detection based onalternative contexts other than motion signal processing.

the responses of MT neurons (Fig. 7), where efficiency

One possibility is that the variability is closely coupled

is defined as the ratiowith the dynamical adaptability of the visual system, as issuggested by the experiment on binocular rivalry (Taya<external noise> /<internal noise> + <external noise>.

and Mogi 1999).
When we define the noise in the neural system, we onlyHere, the internal noise represents the variability inherent

do so in reference to the most apparent stimulus featurein neural network, and external noise represents thethat the neurons selectively respond to. In this paradigm,

variability in the stimulus itself.

any variability in neural activities that do not directly.

correspond to that feature would be automatically

0.5

Efficiency

0.4

discarded as “noise”. However, it is quite possible that then=2

n=8variability in neural activities actually reflects somen=220.3

contexts other than the stimulus feature. If this is the case,the variability of neural activity is not a meaningless noise.
It rather reflects the multiple contexts within which the0.2

neurons in a particular cortical area become functional. In

0.1
0.0

order to understand the properties of neural activities inthis respect, it is necessary to take a systems point of view,

MonkeyE

MonkeyJMonkeyWwhich has been largely lacking in neuroscience up totoday.

Fig.7 Efficiency values for monkeys E, J, W.
5. Conclusion.
The internal noise that account for about 70% of total
variability is traditionally regarded as a "meaninglessnoise". There is a surprisingly high degree of noise in theneural activities in the cortex. It is sometimes argued thatthe brain is constructed in such a way that it functions

Based on the above arguments and other recent
evidences from neuroscience, the following ideas forunderstanding the systems level properties of the brain aresuggested.

(1) One of the evolutionary pressures on brain function

6. References

has been the limitation on spatial resources. The
brain has therefore likely to have adapted theChalmers, D. (1996) The Conscious Mind. Oxford

strategy of using a single cortical area for multiple

university Press.

purposes.

Iriki, A., Tanaka, M,. and Iwamura, Y. (1996) Coding of(2) The neural activity in single cortical area would

modified body shema during tool use by macaquereflect multiple contexts. This could account for thepostcentral neurons. Neuroreport 7, 2325-2330/

high level of the variability of neural activity as

Mogi, K.

measured by the traditional definition in which any

Mach's Principle. in Riegler, A. & Peschl, M. (eds.)

variability that cannot be explained by the stimulus

Understanding Representation in the Cognitive Sciences.

feature itself is discarded as noise.

New York: Plenum Press. 127-134. (1999)

Response Selectivity, Neuron Doctrine, and(3) In understanding the systems level properties ofMogi, K & Barlow, H B. The source of variability inbrain function, the phenomenal (first person) aspects

neural responses from MT. J.Physiol 515, pp. 101-102

of perception provide important clues. In particular,(1999)

the distinction between sensory and intentional

Taya, F. & Mogi, K.

qualia and their neural correlates provide a goodstructure in binocular rivalry. Perception 28 Supplement

paradigm.

pp.135 (1999).

(4) The sensory qualia provide a relatively stable

Feature induced spatiotemporal

Zangaladze, A.

et al. INVOLVEMENT OF VISUAL

representation of the environment. The intentional

CORTEXTACTILE

qualia represent a dynamic, context-dependent

ORIENTATION. Nature 401, 587 - 590 (1999)

interpretation of the environment.

Weiskrantz, L. (1990) Blindsight.

(5) In cases as binocular rivalry, there is a dynamicalINDISCRIMINATION

OFOxford UniversityPress.

interaction between sensory and intentional qualia.
Vision is an active process, where the external inputs

Acknowledgements. The author would like to thankare matched with internal models.

Yoshi Tamori, Horace Barlow, Fumihiko Taya for helpful(6) In the formation of body image (e.g., Iriki et al.
1996), the interaction between sensory and motor
information is important. The resulting body image isphenomenologicallyperceived

asaformofintentional qualia, reflecting the multiple contexts
necessary for the execution of appropriate motor
activities in interacting with the environment.

discussions.