3B2v8:06a=w ðDec 5 2003Þ:51c
þ model

Prod:Type:FTP
pp:128ðcol:fig::NILÞ

NEUCOM : 10288

ED:BabuB:
PAGN:NMN SCAN:prasanth

ARTICLE IN PRESS
1
3

Neurocomputing ] (]]]]) ]]]–]]]
www.elsevier.com/locate/neucom

5
7

A computational model of anterior intraparietal (AIP) neurons

9Erhan Oztopa,b,, Hiroshi Imamizub, Gordon Chenga,b, Mitsuo Kawatoa,b

11

a

JST-ICORP Computational Brain Project
ATR Computational Neuroscience Laboratories, Department of Cognitive Neurosciences, 2-2-2 Hikaridai, Soraku-gun Seika-cho, Kyoto 619-0288, Japan

b

13
15

Abstract

19

25

The monkey parietal anterior intraparietal area (AIP) is part of the grasp planning and execution circuit which contains neurons that
encode object features relevant for grasping, such as the width and the height. In this study we focus on the formation of AIP neurons
during grasp development. We propose and implement a neural network structure and a learning mechanism that is driven by successful
grasp experiences during early grasp development. The simulations show that learning leads to emergence of units that have similar
response properties as the AIP visual-dominant neurons. The results may have certain implications for the function of AIP neurons andthus should stimulate new experiments that cannot only verify/falsify the model but also advance our understanding of the visuomotor
learning mechanisms employed by the primate brain.
r 2006 Published by Elsevier B.V.

27

Keywords: Anterior intraparietal area; Grasp learning; Affordance; Visuomotor development

O

O

23

PR

21

F

17

29
1. Introduction

33

Humans visually monitor critical kinematic events for
detecting errors in goal-directed movement execution [8]including grasping movements that require cortical integration of visual and somatosensory cues for proper grip
formation [6]. The accumulated neurophysiological data
indicate that the parietal cortex is involved in visuomotor
aspects of manual manipulative movements [19]. In
particular, the neurons in anterior intraparietal (AIP) areaof macaque monkeys discharge in response to viewing and/or grasping of three-dimensional objects representing
object features relevant for grasping [14,16]. Generally,
AIP neurons are classiﬁed as one of motor-dominant(active during grasping, even in the dark), visual-motor,
and visual-dominant (no movement is necessary; sole
object ﬁxation elicits response) types. AIP has strong
recurrent connection with area F5 (ventral premotor

45
47
49
51
53
55
57

TE

EC

R

R

43

O

41

C

39

N

37

U

35

D

31

Corresponding author. ATR Computational Neuroscience Laboratories, Department of Cognitive Neurosciences, 2-2-2 Hikaridai, Sorakugun Seika-cho, Kyoto 619-0288, Japan. Tel.: +81 774 95 1215; fax:
+81 774 95 1236.
E-mail addresses: erhan@atr.co.jp (E. Oztop), imamizu@atr.co.jp (H.
Imamizu), gordon@atr.co.jp (G. Cheng), kawato@atr.jp (M. Kawato).

0925-2312/$ - see front matter r 2006 Published by Elsevier B.V.
doi:10.1016/j.neucom.2005.12.106

cortex) [10] that is involved in grasp planning andexecution [13], and projects to motoneurons that control
ﬁnger muscles [2]. The activity of neurons in the primary
motor cortex (F1) when compared to premotor activity
indicates that the primary motor cortex may be more
involved in dynamic aspects of movement [18], executing‘instructions’ sent by higher motor centers including
premotor regions. Thus, it has been suggested that
AIP–F5–F1 circuit is responsible for grasp planning andexecution [3–5,7]. However the formation/adaptation of
the neural circuitry that extracts object features requiredfor dexterous manipulation (i.e. AIP) is yet to be understood. To this end, it is important to know whether AIP
representation is the ﬁnal step of a series of visual analysis
or the by product of the grasp-related visuo-motor
learning.
In this article, we present a model of AIP visualdominant neurons consistent with monkey grasping circuitthat extends upon our earlier modeling of infant grasp
learning (when we use AIP, we mean AIP visual-dominantfrom now on). Infant motor development studies haveshown that during early grasping period of 4–6 months,
infants do not use vision to guide hand trajectory or toorient the hand toward the object prior to initial contact.

59
61
63
65
67
69
71
73
75
77
79
81
83

NEUCOM : 10288
ARTICLE IN PRESS
E. Oztop et al. / Neurocomputing ] (]]]]) ]]]–]]]

17

2. The model
2.1. Behavioral setting

19

D

2.2. The systems level organization

EC

37

TE

35

U

N

C

O

R

R

We abstract the primate grasping circuit as shown in Fig.
39 1A. The visual input arriving to AIP—although processedat earlier visual areas—is void of geometric information
41 about the object in the visual ﬁeld. The task of AIP–F5
complex is then to transform the visual input into a motor
43 code which when executed yields a successful (stable)
grasping of the object. In the primates, the input to AIP is
45 highly processed as there are multiple relay stations on theway from primary visual cortex to AIP. Nevertheless, these47 areas do not compute information such as width and heightof the object in the visual ﬁeld. To our knowledge, AIP and
49 caudal intraparietal sulcus (cIPS) are the sole areas
reported to encode geometric object properties. cIPS
51 neurons encode orientation or axis of objects and may
provide information for AIP [15]. For simplicity we do not
53 model cIPS explicitly as a separate layer. However, we do
expect to see units similar to cIPS neurons as well in our
55 AIP layer (in spite of the naming). For this article, we focus
only on the properties of unit responses that are compar57 able to AIP-like responses.

61
63
65
67
69
71
73
75
77
79
81
83

2.4. Adaptation mechanism

85

The problem an infant faces during grasp learning iscomputationally stated as to learn the mapping from visual~ that yield successful grasping.
~Þ to motor codes ðGÞðV
Notice that the learning mentioned here is only possible
with the active movements of an infant, although theresulting neural structure may exhibit purely visual
responses not requiring movement. The infant experiences~i g pairs during early grasping. At ﬁrst, this~i ; G
many fV
problem seems to assume a simple function approximation
solution, however, the mapping is not well deﬁned: we canapply different grasps to a given object. Conversely,
different objects can serve as the target for the very samegrasp. The problem can be approached from several
directions. One way is to model the association learning~ of the~; GÞ
as learning the joint probability distribution pðV
visual representation of the object and the motor command. In this study we chose a more biological approachand propose a neural circuit with explicit neural units thatcan solve the problem: we propose that AIP–F5 complex
consists of sub-networks (columns) that compete in the
motor code space for being the one to learn the current~i g (successful grasp association) pair (see Fig. 1B).
~i ; GfV
The competition is implemented using a Kohonen’s
topology preserving self-organizing map (SOM). When agrasping attempt results in successful holding of the target
object, the competition picks a sub-network to learn thecurrent observed vision-motor association. Althoughother alternatives were possible, the learning in the sub-87

PR

The model we present addresses the developmental stageof 4–9 months where a basic grasping skill has been
21
acquired. We present the model in terms of brain areas
belonging to macaque monkeys, however, the develop23
mental data is mainly from humans due to the lack ofinfant studies on other primates. In other words, we
25
implicitly assume that the developmental aspects of thegrasp circuit in humans and other primates follow similar
27
stages.
The grounding assumption of the modeling presented in29
this article is that during early grasp learning infants
associate the vision of the objects with the grasp plan (the31motor code generating the grasp) that provides a stablegrasping.
33

59
To capture the non-speciﬁcity of the visual input (i.e.
lack of geometric information coding) we implemented the
input to AIP as a depth encoding ‘retina’. The visual
processing taking place prior to AIP includes stereopsis, so
this choice is justiﬁed by the monkey neurophysiology [15].
The most notable point of our representation is that it does
not include any high level features extracted by apreprocessing step; what the network sees is just a depthmap (i.e. matrix of real numbers). Notice that instead of an
explicit depth encoding it is also possible to use two
intensity coding retinas corresponding to two eyes, inwhich case we would predict the emergence of binocular
neurons in the hidden layers. Since the depth encoding
retina chosen for computational convenience does notcontain more information than the two retina system(given the simple objects we used) the validity of thearguments we might draw from the model is not weakened
by our choice.
For the motor code (F5) output we used joint angles of
the ﬁngers. Although the motor code in the brain must
address dynamics and intrinsic properties of the muscles
and lower motor control centers we believe the output codeused does not limit the validity of the conclusions we candraw from the model.

F

15

2.3. Input and output encodingO

1 For example, at 4–5 months, reaches are as good withvision available during the reach as when vision is removed
3 after onset of the reach [1]. Only after 9 months of age thevisual features of the objects that are relevant for grasping
5 (orientation, size, etc.) are incorporated into the grasping
actions. This developmental progression suggests that
7 earlier grasp learning may mediate the formation of astronger grasp planning circuit that fully utilizes the visual
9 information available. The model we present addresses thelatter portion of this progression, where the less-visually
11 guided grasp experiences provide (learning) data points for
an infant’s grasp related visuo-motor mapping system.
13

O

2

89
91
93
95
97
99
101
103
105
107
109
111
113

NEUCOM : 10288
ARTICLE IN PRESS
E. Oztop et al. / Neurocomputing ] (]]]]) ]]]–]]]

3

1
59
3

AIP

hidden layer

11
13

F5

W3

W2

AIP column

AIP column

hidden layer

w1

w2

Column 1
outputColumn 2
output…

F5
AIP column F5columncolumnF5F5columncolumnAIP column

hidden layerwN…

F5 column

F5 column

Column N
outputAIP column

AIP column

AIP column

W1

7
9

61

Visual Input

563

AIP column
AIP column

F5columnF5columnF5 column

65
67
69
71

15
73
Lower Motor Areas & Motor Program Selection

17
(A)

75

(B)

19

motor
m2

21

column 1

77
79

25

O

column 2

27

PR

m1

29

v1

(C)

31

35

47
49
51
53
55
57

D

EC

R

R

83

87
89
91
93
95

partition, the vision - motor relation becomes welldeﬁned. This scheme works provided that the number of
sub-networks is sufﬁcient enough to accommodate thenumber of sub-problems required to reduce the association~i g mappings.
~i ! fG
problem into a set of well deﬁned V

97
99
101

O

45

C

43

N

41

networks was implemented using back-propagation learning algorithm for computational convenience. The SOM
clustering groups similar (in the hand conﬁguration space)grasps together and hence creates sub-problems each ofwhich can be solved with a simple function approximatorallowing the sub-networks to learn their sub-problems.
This is conceptually illustrated in Fig. 1C where the curve
represents a hypothetical vision - motor relation. Noticethat at vision ¼ v1, the motor output can be either m1 or
m2. When a function approximator is used to learn thisrelation it will be forced to learn conﬂicting data points,
namely (v1, m1) and (v1, m2). Since the function
approximator can give only single output for the input ofv1, the learning will not be able to reduce the prediction
error to zero (see also [9]). However, if the curve was cut
into horizontal stripes then within each stripe the relation
would be learnable (see the shaded regions in Fig. 1C). Thisis exactly what the self-organization in the motor space
does; it partitions the motor space such that in a givenU

39

visionFig. 1. (A) Overall network representation is shown. The arrows labelled with W and w indicates the adaptive weights. (B) Competition among AIP–F5
columns (sub-networks) for learning the association of the current vision and the executed grasp code is depicted. (C) Conceptual illustration of how an illdeﬁned vision - motor relation (the curve) can be learned using multiple columns. A single neural network cannot learn this mapping because v1 maps toboth m1 and m2. The strips drawn over the curve symbolically indicate the areas where the columns (sub-networks) become winner for the indicated rangeof motor values. In these strips the mapping is well-deﬁned (see the curve segments in the shaded areas).

37

81

85

TE

33

O

F

23

2.5. Learning details

103~i , The SOM 105
Given the successful grasp conﬁguration G2~i Þ where ~update rule used was D~
rk ¼ meLðk ;kÞ=d ð~
rk þ Grk
represents the kth sub-network’s preferred grasp conﬁg- 107
uration, and k* is the winner sub-network. The neighbor2hood function eLðk ;kÞ=d was determined by taking Lðk ; kÞ 109
as the Euclidean distance between the 2D indexes of subnetworks k and k*, and setting the variance as d2 ¼ 2. In all 111
simulations, the update rate of the SOM was m ¼ 0:001,
and the variance was reduced at each time step by 106 of 113
the current value of the variance.

NEUCOM : 10288
ARTICLE IN PRESS
E. Oztop et al. / Neurocomputing ] (]]]]) ]]]–]]]

21

3. Simulations~Þ was modeled as a 32  32 depth
The visual input ðV
map centered on the target object. The motor command~ was modeled as the joint conﬁguration of the hand. WeðGÞ25
used 7 joints: 3 joints for the thumb, 2 joints for the index
and middle ﬁngers. The number of sub-networks was 16,27
and each had 16 or 8 hidden (AIP) units. For the SOM, weused a planar (two dimensional) mesh. The number of29nodes was equal to the number of sub-networks (16), andthe input dimension was equal to the motor output (7). For
31
input, we used three objects with variable dimensions. The
objects were: a rectangular prism (box) which could change
33
size in two dimensions; a vertical cylinder (vertical bar) that
could change in height and diameter; and ﬁnally a35horizontal cylinder (horizontal bar) with variable heightand diameter (Fig. 2, right side).
37

EC

TE

D

PR

23

As mentioned previously, the current model addresses
the stage of motor development where a basic grasping
ability is assumed. Our earlier study of infant grasp
learning provided us the stage required. We used our
implementation of infant grasping learning model (ILGM)[12] to acquire the skill of grasping the aforementioned
objects with different sizes (Fig. 2, left side shows the
ILGM simulation environment). The successful grasps
executed by ILGM then were used to drive the AIP model
presented in this article. In ILGM the objects were modeled
as 3D geometric shapes, which required us to convert theminto depth maps so that they could be fed to AIP. This
conversion is performed by rendering the objects into a
32  32 buffer such that the intensity of a rendered pixel
indicated its depth as shown in Fig. 2, right side.
We used 2000 successful grasping performances fromILGM to test the AIP model. In those grasping movements
the object sizes and types were (uniform) randomly
selected. Note that the grasp plan generation mechanism
in ILGM is stochastic and produces different grasp
conﬁgurations at different instances of an object’s presentation (even when the dimensions are ﬁxed). Technically, we could implement AIP model on top of ILGM
model and run them together, but for practical reasons we
kept the two systems separate. We ﬁrst collected 2000
grasping data points (joint conﬁguration of the hand andthe depth map of the grasped objects) and then used themin adapting our model. For the simulations reported in thispaper we applied a sequential training schedule: ﬁrst the
SOM was adapted, then each sub-network was allowed tolearn for those grasp codes that they became winner for.
The learning was stopped after all the sub-networks had
converged. The norm of the error was 0.45 for the 8 AIP
unit case, and 0.4 for the 16 AIP unit case. This
corresponds approximately to an average of 6 and 5
degrees of error per ﬁnger joint angle, respectively.

F

For a given (visual-input, grasp conﬁguration) data~i Þ, the weights of the winner sub-network was~i ; G
point ðV3 updated using a straightforward gradient descent update~i k2
rule (i.e. back-propagation) on the error E ¼ 0:5k~
BG~i
5 where ~B represents the output generated when the input V
was presented. The output ~B was computed with~i ÞÞ, where f ð:Þ is the sigmoid function given
7 ~z ¼ f ðWf ðwVby f ðxÞ ¼ ð1 þ ex Þ1 . Note also that for the operation of
9 this network the input and outputs were normalized to bewithin the range of [0, 1]. The visual-input-AIP weight@E
11 matrix (W) was updated with DWt ¼ Z@W
þ bDWt1 , and
the AIP-output weight matrix (w) was updated witht1
13 Dwt ¼ Z@E
, where Z and b represents the@w þ bDw
learning rate and the coefﬁcient of momentum term,
15 respectively. The superscripts indicate the update step
number, so t1 means the previous update step. All of the
17 simulations reported in this article used Z ¼ 0:05 andb ¼ 0:6.
19

O

1

O

4

R

39

R

41

O

43

51

N

49

U

47

C

45

53
55
57

Fig. 2. Grasp learning system that provided successful grasps andcorresponding joint angles (left), and the depth map representation usedas the input to the model for various objects are shown (right). The arrows
indicate the dimensions of the objects that were varied during grasp
learning.

59
61
63
65
67
69
71
73
75
77
79
81
83
85
87
89
91
93

4. Results95

After learning has converged, we presented the objects
with varying dimensions recording the elicited response ateach of the AIP units. Since two dimensions of the objects
were altered, a given object provided us with a twodimensional mesh of response values (called the responsemap) for each unit. Before presenting the actual response
maps of the units we brieﬂy report the object encoding
emerged via learning using 16 AIP units per sub-network [8AIP units per sub-network]: for each object type we listed
all the units that have higher activity than in all otherobject presentations by a margin of 0.1 [0.05] (remember that maximal ﬁring is 1.0). Interestingly, number of
selective units for each object was more or less the same foreach object (i.e. the list contained approximately the same
number of units). Approximately, 8 [4] out of 256 [128]units became selective for each object (box, horizontal bar,and vertical bar). And of those only 1 or 2 [1 or none] units
were strictly selective for a speciﬁc size of the object.

97
99
101
103
105
107
109
111
113

NEUCOM : 10288
ARTICLE IN PRESS
E. Oztop et al. / Neurocomputing ] (]]]]) ]]]–]]]

1

width

+1

5

Height Corr.

59
3
61
5

height

63

7
65
9
67
+1

-1

11

69

Width Corr.

13
71
15
73
17
75
19
77
-1

21

79

27

F

O

O

25

Fig. 3. Response of AIP units (after learning) when the box object was presented with varying width and height values (number of AIP units per subnetwork ¼ 16). The thumbnail images (response maps) indicate the AIP units’ response to the varying width and height values of the box. The darker
areas indicate higher response, whereas lighter areas indicate smaller response. The magniﬁcation on the top left shows the axes of the response maps. The
location of a unit’s response map shows the feature encoding by that unit: the ones on the left and right extremes have responses that correlate well withthe width of the presented box. Similarly, the units on the upper and lower extremes have responses that correlate well with the height of the presented box.
Note that although there are 256 AIP response maps, because of the special arrangement of the response maps, some overlapped (because they had close
correlation values for the width and height).

PR

23

29
31

43
45
47
49
51
53
55
57

D

TE

EC

R

R

41

O

39

C

37

N

35

In order to visually inspect the response map of units wehave created image maps (small thumbnail images) inwhich darker regions indicate higher response, while white
regions indicate zero response, as shown in Fig. 3. Avertical gradation in the intensity indicates that the unit
encodes the height of the object, whereas a horizontal
gradation indicates a width encoding. For space limitations
we can only present the response maps obtained from thepresentation of one object (box) using the 16 AIP units per
sub-network (Fig. 3). The location of each response map
was specially chosen: the position of the response maps
within the drawn axes, indicate the level of geometricfeature encoding of units: the horizontal axis indicates thecorrelation of unit response with the width of the box,
where as the vertical axis indicates the correlation of unit
activity with the height of the box. Therefore, the units at
the left and right extremes encode the width of the box,
whereas the units at upper and lower extreme encode theheight. The units around the origin do not have a clear(linear) correlation with the dimensions of the box. Notethat there are units which encode a certain range of widthor height, one example is marked in the ﬁrst quadrant witha circle. In addition, there are a few units that prefer certain
width and height in combination; the unit marked in thethird quadrant, for example, prefers small boxes. Thesetypes of units are rarer than the width and height encoding

U

33

81
83
85
87

units and are located away from the +1 and 1 in the 89
correlation axes as these units do not have a linear
correlation with the object dimensions. The sigmoidal 91
network we used is characterized by the non-local basis
functions implemented by the hidden layers, which are 93
formed via training [17]. It could be speculated that thevariety of neural responses we obtained could be attributed 95
to this fact. If we used local learning networks (i.e. radialbasis function network) the neural responses might have 97
been more stereotypical, having convex shaped high99
response areas when plotted as in Fig. 3.
Because of space limitations we revert to simple scatterplots for presenting the responses of the AIP units for other 101
objects. In Fig. 4, each unit is indicated by a triangle; thehorizontal axis indicates the absolute value of the linear 103
correlation of unit responses with the horizontal extent of
the object presented. Likewise, correlation with the vertical 105
extent is represented on the vertical axis. Row A indicatesthe objects used in computing the correlations. The last 107
column of Fig. 4A indicates that all of the objects wereconsidered in the correlation computation illustrating the 109
emergence of units with size encoding independent of object
identity. Row B shows the results obtained when the 111
number of AIP units was set to 8 per sub-network, whereas
Row C shows the results when the number of AIP units 113
was 16. Comparing Row B and C we can see that the

NEUCOM : 10288
ARTICLE IN PRESS
E. Oztop et al. / Neurocomputing ] (]]]]) ]]]–]]]

6

1

y

z

(A)

0.9

0.8

0.9

0.7

0.8

0.7
0.6
0.5
0.4
0.3
0.2

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

0.7
0.6
0.5
0.4
0.3

0.6
0.5
0.4
0.3

0.4
0.3
0.2
0.1

0

0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

10

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

0

1

1

0.9

0.9

0.9

0.8

0.8

0.8

0.5
0.4
0.3
0.2
0.1

0.6
0.5
0.4
0.3
0.2
0.1
0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

(C)

1

0.6
0.5
0.4
0.3

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

75

0.5
0.4
0.3
0.2

0.1

0.1

77

0
0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

X (thickness) Correlation

10

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

79

X CorrelationF

X (length) Correlation

73

0.6

0.2
0

0

X (width) Correlation

71

0.7

0.7

Y Correlation

0.6

Y (length) Correlation

1

0.7

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

X Correlation

0.9
0.7

69

X (thickness) Correlation

X (length) Correlation

0.8

67

0.5

0.1
0

65

0.6

0.2

1

63

0.7

0.1

0.2

X (width) Correlation

0

23

0.8

Y Correlation

0.8

Y (length) Correlation

Y (thickness) Correlation

0.9

0

Y (height) Correlation

21

ALL OBJECTS
1

15

19

VERTICAL BAR
0.9

(B)

17

HORIZONTAL BAR
1

0.1

13

61

z

1

Y (thickness) Correlation

11

Y (height) Correlation

9

59

x

z

BOX

7

y
x

x

3
5

y

O

Fig. 4. (A) Objects presented for computing the correlations that are shown in the lower rows. (B) Correlation results when the sub-networks were81

25 modelled with 8 AIP units. (C) Results are shown when the number of the AIP units was increased to 16. Each triangle represents an AIP unit, where the

O

PR

27

coordinates of a unit represents its (absolute valued) correlation with the indicated object dimension. So the units that have coordinates close to 1 can besaid to represent geometric features of the presented objects.

29

U

N

C

O

R

R

EC

TE

D

distribution of units have strong similarity even though
31 they were obtained through different simulations withdifferent number of AIP units.
33
5. Conclusion
35
The monkey AIP has neurons with complex visuo-motor
37 properties. Some of them are mainly activated during grasp
execution whereas others can be activated by mere visual
39 ﬁxation of the objects (visual-dominant object-type neurons) [15]. The current model addresses the latter type of
41 neurons that become active even though no subsequent
movement is involved. In literature, a range of object
43 selectivity has been reported for the object-type neurons; ahighly selective neuron shows vigorous activity to a single
45 object and responds weakly to other objects, whereas, abroadly selective neuron may respond to more than one
47 object, often which have similar geometric features [11].
The simulation experiments with our model showed that49 units with a range of object selectivity can emerge via
visuo-motor learning. Most of the units we have found
51 were broadly tuned showing response to more than one
object type. This is consistent with the experimental studies
53 that report that the number of AIP neurons with strict
selectivity is less than the broadly selective neurons [11].
55With the goal of stimulating new experimental studies,
the main focus of the current modeling study was to test
57 whether size selectivity could emerge from visuo-motor

learning. As presented in results, we have found units that
encode object dimensions (some with independent of theobject shape). Indeed, it has been reported that manyneurons found in monkey AIP were selective for the size
and the shape (often in conjunction) of the objects being
viewed [11]. However to our knowledge, there is noexperimental study to show that AIP neurons represent
geometric quantities (i.e. measure of say, thickness rather
than the attributes of thin/thick). The results from oursimulations suggest that if AIP visual-dominant neurons
are part of a visuo-motor grasp learning network, as is
usually accepted [7,16], then (1) shape and size selectivity
emerges naturally via learning, and furthermore (2) thephrase ‘shape and size selectivity’ can be replaced with‘representation of geometric features’, or better with‘representation of affordances for grasping’. It is possible
to test the latter implication of our model with further
neurophysiological experiments that involve systematic size
alterations of the presented objects to the subject monkey,
allowing a reliable correlation analysis on the recordeddata.
An unanswered question by the presented model is how
the two learning systems interact. The model predicts thatduring the early phases of infant grasping, output from thevisuo-motor circuit would not be accurate (due to e.g. lackof sufﬁcient learning data points). Therefore if the infants
have no complete control to suppress this inaccurate
output, the vision of the detailed shape of the object must

83
85
87
89
91
93
95
97
99
101
103
105
107
109
111
113

NEUCOM : 10288
ARTICLE IN PRESS
E. Oztop et al. / Neurocomputing ] (]]]]) ]]]–]]]

7
9
11
13

17

This work was supported by JST-ICORP Computational Brain project. We thank Akira Murata for useful
discussions.

19
References
21

39
41
43
45
47
49
51
53
55
57

Erhan Oztop received his M.S. degree from
Middle East Technical University in 1996; his
Ph.D. degree from the University of Southern
California in 2002. From 2002 to 2003, he was avisiting Researcher at the Advanced Telecommunications Research Institute, Computational
Neuroscience Laboratories, Kyoto, Japan. He is
now a Researcher at Japan Science Technology,
working for the ICORP Computational Brain
Project, and he is still afﬁliated with Advanced
Telecommunications Research Institute. His research interests include
computational modeling of the brain mechanisms of action understanding
and visuomotor transformations, human-robot interaction, robotics,
machine learning and cognitive neuroscience.
Hiroshi Imamizu obtained his Ph.D. (Experimental Psychology, 1995) at the University of Tokyo,
Japan. He worked in ATR Human Information
Processing Research Laboratories in Kyoto from1992 to 1996 as a Research Associate. From 1996
to 2001, he worked as a leader of Computational
Psychology Group in KDB, ERATO, JST. From2001 to 2004, he worked as a senior researcher in
ATR Human Information Science Laboratories.
Since 2004, he is the head of Department of
Cognitive Neuroscience in ATR Computational Neuroscience Laboratories. His research interest is in human sensorimotor learning frompsychological and computational points of view.

PR

D

TE

37

EC

35

R

33

R

31

O

29

C

27

N

25

[1] R.K. Clifton, D.W. Muir, D.H. Ashmead, M.G. Clarkson, Is visually
guided reaching in early infancy a myth?, Child Dev. 64 (1993)
1099–1110.
[2] R.P. Dum, P.L. Strick, The origin of corticospinal projections fromthe premotor areas in the frontal lobe, J. Neurosci. 11 (1991) 667–689.
[3] A.H. Fagg, M.A. Arbib, Modeling parietal–premotor interactions inprimate control of grasping, Neural Networ. 11 (1998) 1277–1303.
[4] L. Fogassi, V. Gallese, G. Buccino, L. Craighero, L. Fadiga, G.
Rizzolatti, Cortical mechanism for the visual guidance of handgrasping movements in the monkey—a reversible inactivation study,
Brain 124 (2001) 571–586.
[5] V. Gallese, A. Murata, M. Kaseda, N. Niki, H. Sakata, Deﬁcit ofhand preshaping after muscimol injection in monkey parietal cortex,
Neuroreport 5 (1994) 1525–1529.
[6] M. Jeannerod, The formation of ﬁnger grip during prehension. A
cortically mediated visuomotor pattern, Behav. Brain Res. 19 (1986)
99–116.
[7] M. Jeannerod, M.A. Arbib, G. Rizzolatti, H. Sakata, Grasping
objects—the cortical mechanisms of visuomotor transformation,
Trends Neurosci. 18 (1995) 314–320.
[8] R.S. Johansson, G. Westling, A. Backstrom, J.R. Flanagan, Eye–hand coordination in object manipulation, J. Neurosci. 21 (2001)
6917–6932.
[9] M.I. Jordan, D.E. Rumelhart, Forward models—supervised learningwith a distal teacher, Cognitive Sci. 16 (1992) 307–354.
[10] M. Matelli, G. Luppino, A. Murata, H. Sakata, Independent
anatomical circuits for reaching and grasping linking the inferiorparietal sulcus and inferior area 6 in macaque monkey, in: Society for
Neuroscience, vol. 20, 1994, p. 404.
[11] A. Murata, V. Gallese, G. Luppino, M. Kaseda, H. Sakata,Selectivity for the shape, size, and orientation of objects for graspingin neurons of monkey parietal area AIP, J. Neurophysiol. 83 (2000)
2580–2601.
[12] E. Oztop, N.S. Bradley, M.A. Arbib, Infant grasp learning: acomputational model, Exp. Brain Res. 158 (2004) 480–503.
[13] G. Rizzolatti, M. Gentilucci, R.M. Camarda, V. Gallese, G.
Luppino, M. Matelli, L. Fogassi, Neurons related to reaching—grasping arm movements in the rostral part of area-6 (area-6a-beta),
Exp. Brain Res. 82 (1990) 337–350.
[14] H. Sakata, M. Taira, M. Kusunoki, A. Murata, Y. Tanaka, K.
Tsutsui, Neural coding of 3D features of objects for hand action in
the parietal cortex of the monkey, Philos. Trans. Roy. Soc. London
Series B-Biol. Sci. 353 (1998) 1363–1373.

U

23

59
61
63
65
67
69
71

Acknowledgments
15

F

5

[15] H. Sakata, M. Taira, M. Kusunoki, A. Murata, K. Tsutsui, Y.
Tanaka, W.N. Shein, Y. Miyashita, Neural representation of threedimensional features of manipulation objects with stereopsis, Exp.
Brain Res. 128 (1999) 160–169.
[16] H. Sakata, M. Taira, A. Murata, S. Mine, Neural mechanisms ofvisual guidance of hand action in the parietal cortex of the monkey,
Cereb. Cortex 5 (1995) 429–438.
[17] S. Schaal, C.G. Atkeson, Constructive incremental learning fromonly local information, Neural Comput. 10 (1998) 2047–2084.
[18] L.M. Shen, G.E. Alexander, Preferential representation of instructed
target location versus limb trajectory in dorsal premotor area, J.
Neurophysiol. 77 (1997) 1195–1212.
[19] S.P. Wise, D. Boussaoud, P.B. Johnson, R. Caminiti, Premotor andparietal cortex: corticocortical connectivity and combinatorial
computations, Annu. Rev. Neurosci. 20 (1997) 25–42.

O3

degrade the grasping performance. The closest experimental ﬁndings to our prediction indicates that 3 months old
infants reach for and contact with glowing and sounding
objects under lighted and dark conditions with similarfrequency, and the onset of successful grasping occurs at
approximately at the age of 15–16 weeks for bothconditions [1]. This means that 3 months old infants do
not take advantage of the visibility of the shape of theobject. Perhaps with more strict experimental conditions it
might be possible to show that at the age of 3–4 months,
the object shape information in fact degrades the grasping
performance.

O

1

7

Gordon Cheng is the head of the Department of
Humanoid Robotics and Computational Neuroscience, ATR Computational Neuroscience
Laboratories, Kyoto, Japan. He is also theGroup Leader for the newly initiated JST
International Cooperative Research Project(ICORP), Computational Brain. Before taking
up these positions, he held fellowships from theCenter of Excellence (COE), Science, and Technology Agency (STA) of Japan. Both fellowships
were taken at the Humanoid Interaction Laboratory, Intelligent Systems
Division at the ElectroTechnical Laboratory (ETL), Japan. At ETL he
played a major role in developing a completely integrated humanoid
robotics system. He received a Ph.D. in Systems Engineering from the
Department of Systems Engineering, The Australian National University.
Bachelor and Master degrees in Computer Science from the University of
Wollongong, Australia. He is a society member of the IEEE Robotics &Automation and Computer Society. He is on the editorial board of theInternational Journal of Humanoid Robotics.

73
75
77
79
81
83
85
87
89
91
93
95
97
99
101
103
105
107
109
111
113

NEUCOM : 10288
ARTICLE IN PRESS
E. Oztop et al. / Neurocomputing ] (]]]]) ]]]–]]]

8

1
3

F
O
O

PR
D

21

TE

19

EC

17

R

15

R

13

O

11

C

9

N

7

Mitsuo Kawato received the B.S. degree in physics
from Tokyo University in 1976, the M.E. and
Ph.D. degrees in biophysical engineering from
Osaka University in 1978 and 1981, respectively.
From 1981 to 1988 he was a faculty member andlecturer of Osaka University. From 1992 hebecame a department head of Department 3,
ATR Human Information Processing ResearchLaboratories. From 2003, Director of ATR
Computational
Neuroscience
Laboratories.
From 2004, he has been jointly appointed as the Director of theComputational Brain Project, ICORP, JST. From 1996 to 2001 he was
jointly appointed as a director of Kawato Dynamic Brain Project,
ERATO, JST. He has been jointly appointed as visiting professor of
Kanazawa Institute of Technology, Nara Institute of Science andTechnology and Osaka University. For the last 15 years he has beenworking in computational neuroscience. He was awarded Yonezawa
founder’s medal memorial special award of the Institute of Electronics,
Information and Communication Engineers in 1991, outstanding researchaward of the International Neural Network Society in 1992, Osaka
Science Prize in 1993, 10th Tsukahara Naka-akira Memorial Award in1996 and Tokizane Toshihiko memorial award. He is a governing board
member of Japanese Society of Neuroscience and Japan Neural Network
Society and Member of Executive Committee of International Association
for the Study of Attention and Performance.

U5