Neurocomputing 44 – 46 (2002) 965 – 972

www.elsevier.com/locate/neucom

On the use of forward kinematic models invisually guided hand position control—analysis
based on ISLES model
Eimei Oyamaa;∗ , Taro Maedab , Susumu Tachib ,Karl F. MacDormanc , Arvin Agahda National Institute of Advanced Industrial Science and Technology, 1-2 Namiki Tsukuba-shi,

305-8564 Ibaraki-ken, Japanb School of Engineering, The University of Tokyo, Japanc Faculty of Systems Engineering, Wakayama University, Japand Department of Electrical Engineering and Computer Science, The University of Kansas, USAAbstract
The human nervous system is equipped with a forward kinematics model, which calculates
the hand positions using proprioceptive information. Since signi2cant evidence suggests that theforward dynamics model is used for general motion control, the forward kinematics model also
seems to be used for the control of visually guided reaching. However, we believe that theforward kinematics model plays no role, or only a supplemental role, in the 2nal stages ofvisually guided hand position control. Instead, we propose that this is mainly handled by theinverse kinematics model. To explain the relatively large errors of the internal models in thehuman nervous system, Maeda et al. (Proceedings of the 1993 International Joint Conference
on Neural Networks (IJCNN’93 Nagoya), 1993, pp. 1317–1320) proposed a neural networkarchitecture called independent scalar learning elements summations (ISLES) model. We will
provide evidence based on experimental results and a mathematical analysis using the ISLESc 2002 Published by Elsevier Science B.V.
model to support our hypothesis. Keywords: Forward kinematics model; Inverse kinematics model; Visually guided reaching; ISLES model∗ Corresponding author. Tel.: +81-298-61-7298; fax: +81-298-61-7201.

E-mail addresses: eimei.oyama@aist.go.jp, eimei0080@aol.com (E. Oyama), maeda@star.t.u-tokyo.ac.jp(T. Maeda), kfm@sys.wakayama-u.ac.jp (K.F. MacDorman), agah@ku.edu (A. Agah).
URL: http://staE.aist.go.jp/eimei.oyama/
c 2002 Published by Elsevier Science B.V.
0925-2312/02/$ - see front matter 
PII: S 0 9 2 5 - 2 3 1 2 ( 0 2 ) 0 0 4 9 8 - 8

966

E. Oyama et al. / Neurocomputing 44 – 46 (2002) 965 – 972

1. Introduction
An important subject matter in neuroscience is the internal model involved in humanmotion control [3]. Ito proposed that the cerebellum provides forward models for avariety of controlled systems [3]. Much evidence supports the assumption that theforward dynamics model is used for motion control.
A person can estimate the position of the hands with a certain precision even when
they are out of sight. It is known that the human nervous system is equipped witha forward kinematics model of the arm that calculates the hand position using proprioceptive information. Without visual feedback, a monkey loses its hand position
when area 2ve of its parietal lobe is cooled [11]. The neuronal activities of the parietal cortex corresponding to the forward kinematics model were discovered [2]. It isoften supposed that the forward kinematics model that calculates the hand position
using proprioceptive information operates in the posterior parietal cortex. Hereafter,
we call this forward kinematics model FKMP (Forward Kinematics Model which uses
Proprioceptive information). It is possible that the other forward kinematics model
uses motor commands to estimate the hand position. Hereafter, we call this forward
kinematics model FKMM (Forward Kinematics Model which uses Motor commands).
Since there is signi2cant evidence supporting the use of the forward dynamics modelin hand position control, it seems plausible that the forward kinematic models are also
used for visually guided reaching. However, we present evidence that supports precisely
the opposite conclusion:(1) The FKMP plays no role, or only a supplemental role, in the 2nal stages ofvisually guided reaching. The human nervous system cannot freely use the forwardkinematics model.
(2) The properties of the FKMP are diEerent from those of the FKMP, if the FKMM
plays a main role in the stages.
(3) It is highly probable that the inverse kinematics model plays the main role.
We will provide experimental evidence and theoretical predictions based on our mathematical model of sensorimotor integration to support these claims.

2. Background2.1. Human forward and inverse kinematics models
This section presents models of the human inverse kinematics solver. Let  ∈ Rm
be the joint angle vector and x ∈ Rn be the hand position=orientation vector givenby the visual system. The relationship between x and  is expressed as x = f (),where f is a C 1 class function. The Jacobian of the hand position vector is expressed
as J () = @f ()=@: Let xd be the desired hand position=orientation vector and e = xd −x=xd −f () be the hand position error vector. We consider the inverse kinematics problem that involves calculating the joint angle vector  satisfying xd =f () from a desired

E. Oyama et al. / Neurocomputing 44 – 46 (2002) 965 – 972

967

hand position vector xd . In this paper, a function f −1 (x) that satis2es x = f (f −1 (x))
is called an inverse kinematics function of f (). The acquired model of f −1 (x) inthe nervous system is called an inverse kinematics model. Hereafter, we call the inverse kinematics model IKM. Let im (x) ∈ Rm be the output of the inverse kinematics
model. Let fm () ∈ Rn be the output of the forward kinematics model, which approximates f ().
A human can conduct a visually guided reaching motion at a certain precision even
without using the visual feedback of the hand position [10]. There are two ways to solve
the inverse kinematics problem without hand position error feedback. The straightforward method is through the use of the IKM. If the IKM im (xd ) is suJciently precise,then an inverse kinematics solution im is calculated as im = im (xd ). The other approach is via the use of the forward kinematics model. A number of researchers haveproposed and used the method that exploits the forward model of the controlled systemand have solved the control problem by using the iterative improvement technique. Themethods obtain an approximate solution for the inverse problems described as xd =f ()by solving the following nonlinear equation:
xd = fm ():(1)

By using the multiple starts of the iterative procedure, Eq. (1) can usually be solved.
For example, Newton’s method can be used for solving Eq. (1). Let (k) be the estimated joint angle vector at step k and J + () be the pseudo-inverse matrix(Moore–Penrose’s generalized inverse matrix) of J (), which is calculated as
J + () = J T ()(J ()J T ())−1 . By using the following iterative computation, a solution that satis2es Eq. (1) can be obtained(k + 1) = (k) + J + ((k))(xd − fm ((k))):(2)

2.2. Independent scalar learning elements summations (ISLES) model
To explain the relatively large errors of the internal models, Maeda et al. proposed aneural network architecture called the independent scalar learning elements summations(ISLES) model [7]. Let q = (q1 ; q2 ; : : : ; qu )T ∈ Ru be the input vector and (q) ∈ Rv
be the output of the ISLES model. The ISLES model consists of a number of scalarlearning elements s{ij} (qj ) (i = 1; 2; : : : ; v; j = 1; 2; : : : ; u). Each element receives one
scalar signal and calculates one scalar output. The ith component of the output i (q)is calculated as follows:
i (q) =us{ij} (qj ):(3)j=1

The element can precisely reproduce any C 1 class function with scalar input. However,the ISLES model cannot precisely reproduce all C 1 class functions whose input is avector. This limitation can explain many psychophysical properties of human beings
such as Helmholtz’s horopter [1], the distortion of vision space [5], the haptic horoptor,
and auditory alleys [9].

968

E. Oyama et al. / Neurocomputing 44 – 46 (2002) 965 – 972

Let  (q) be the desired output signal for i (q) and p(q) be the probability density
function of q. Let Q be all the input space q and Qj be the region, where z = qj issatis2ed. The learning results of the ISLES model can be described as follows:p(q) i (q) dq
Qj:
(4)
s{ij} (z) = E[ i (q)|z = qj ] =p(q) dq
Qj
3. Psychophysical evidence: hand position control without visual feedback does notrely mainly on the FKMP
A number of the human nervous system’s internal models have relatively large errors[6 –8]. By using these errors, we can investigate the use of the internal models andthe con2guration of the control system. Based on JLacksch’s classical psychophysicalexperiment [4], Maeda et al. conducted psychophysical experiments to clarify the properties of the forward kinematics model that handles somatosensory information [6,7].
The experimental setup involved a subject in a darkroom who could not see his or her
hand and could move a spotlight by using a joystick as shown in Fig. 1. First, thesubject was asked to move the right hand to a given position. Then, the subject wasasked to move the spotlight to the right-hand position as estimated by using proprioceptive information. This was achieved by controlling the joystick with the left hand
without the visual feedback of the right-hand position. The spotlight position 2nallycorresponded to the output of the forward kinematics model. Subjects usually 2nish
one task in 15 s. It was found that a human points to a position nearer his=her bodythan the true hand position, as in JLacksch’s experiment, as shown in Fig. 1(b). A whitesquare box indicates the true hand position and a black square indicates the hand position estimated from somatosensory information. An arrow indicates an error vectorof FKMP.
Prablanc, Pelisson and their group conducted a number of experiments to clarify
the properties of visually guided reaching with and without the use of hand position
error feedback [10]. These experiments showed that a human reaches toward a point

Pointer

Z [m]

Z [m]

0.4

0.4

XY-Table

SubjectMirror
To
PC

Frame

0.2

TargetUp

Joy-StickX

0.1

Subject: T.M.

0.1

ZX [m]

X [m]

Down

-0.2

(a) Experimental Setup

0.2

Subject: K.T.

-0.1

00.1

0.2

-0.2

-0.1

00.1

(b) Experimental Results

Fig. 1. Properties of FKMP: (a) experimental setup; and (b) experimental results.

0.2

E. Oyama et al. / Neurocomputing 44 – 46 (2002) 965 – 972
Z [m]

XY-TableTarget

969

Z [m]

Subject: K.Y.

Subject: E.K.

0.4

SubjectTo
PC

Mirror

0.3

Pin
0.2

0.2

0.1

0.1

Frame
Up

X

X [m]

Z
Down

-0.2

-0.1

(a) Experimental Setup

00.1

0.2

X [m]
-0.2

-0.1

00.1

0.2

(b) Experimental Results

Fig. 2. Properties of reaching error without visual feedback: (a) experimental setup; and (b) experimentalresults.

nearer to his or her body than the target position without the hand position error
feedback. Maeda improved those experiments and showed that the reaching error isexpanded by the constraints imposed on the subjects as shown in Fig. 2(a) [6,7].
A subject could measure the target position only by using the rotation angles of theeyes. Fig. 2(b) shows the experimental error of visually guided reaching without hand
position error feedback. A white square box indicates the target position, and a blacksquare box indicates the real hand position as the result of reaching.
If FKMP is used for hand position control, then the real hand position should be
farther from the subjects’ body than the target position. However, the direction of
the real error contradicts the direction of the predicted error of the inverse kinematics
solution, as calculated by using the FKMP. We can conclude that the FKMP is not atall, or is only partially used in the 2nal stage of the reaching motion without visualfeedback. Furthermore, we can conclude that the properties of the FKMM are diEerent
from those of the FKMP if the FKMM plays a main role in the stages.
4. Mathematical analysis based on ISLES model
There are two possible mechanisms for hand position control without using visualfeedback of hand position error:(i) The IKM plays a main role.
(ii) The FKMM plays a main role.
Although both these cases are possible, we suggest that the 2rst case is more probablebecause the ISLES model supports the 2rst case, as shown in this section.
4.1. Forward kinematics learning by ISLES modelFirst, we calculated the output of FKMS, which consisted of an ISLES model. In
these simulations, the visual information was described by using the bipolar latitude 970

E. Oyama et al. / Neurocomputing 44 – 46 (2002) 965 – 972(a) Top View(b) Sidep View

Fig. 3. Visual and motor coordinates: (a) top view; and (b) side view.

and the vergence angle  of the eyes. Fig. 3 shows the visual coordinates x = (; )T◦◦and the joint angle vector coordinates  = (; )T . The range of  was (−100 ; −95 ).
◦◦◦◦The range of  was (−50 ; −10 ). The range of  was (−100 ; −95 ). The range of◦◦ was (−50 ; −10 ).
We calculated the output of the FKMP fm () which consists of an ISLES modelusing Eq. (4). The corresponding hand position on the visual coordinates x = (; )Twas used as the desired output signal for the FKMP as fm () = x. Let pfm () be theprobability density function of . We assumed that the value of pfm () was constant in
the region where ; ; , and  are within the de2ned range and pfm () was 0 in theother region. Fig. 4 shows the simulated properties of the FKMP. If we use multi-layer
neural networks, then there is no signi2cant error of the FKMP. The ISLES model canexplain the properties of the FKMP.
4.2. Inverse kinematics learning by ISLES model
We calculated the output of the IKM im (x) which consists of an ISLES modelusing Eq. (4). im (x) is calculated by solving f −1 (x) analytically. Let pim (x) be theprobability density function of x. We assumed that the value of pim (x) was constantin the region where ; ; , and  are within the de2ned range and pim (x) was 0 inthe other region.
Fig. 5 shows the simulated reaching error by using the IKM, which consists of theISLES model. An arrow indicates a simulated reaching error vector. The IKM, which
consists of the ISLES model, can explain the properties of the errors of visually guided
reaching without hand position error feedback. If the ISLES model is an appropriate
model of human sensorimotor integration, then the inverse kinematics model plays themain role.

E. Oyama et al. / Neurocomputing 44 – 46 (2002) 965 – 972
Z [m]
0.4

0.2

0.1

X [m]
-0.2

-0.1

00.1

0.2

Fig. 4. Simulated error of FKMS.

Z [m]

0.2

0.1

X [m]
-0.2

-0.1

00.1

Fig. 5. Simulated reaching error.

0.2

971

972

E. Oyama et al. / Neurocomputing 44 – 46 (2002) 965 – 972

5. Conclusions
In this paper, we presented experimental evidence suggesting that the forward kinematics model that calculates hand positions from proprioceptive information plays atbest a supplemental role in the 2nal stages of reaching without visual feedback. We alsopresented a theoretical argument to support the conclusion that the inverse kinematics
model plays the main role.
References[1] H.V. Helmholtz, Handbuch der Physiologischen Optik. Vol. 2, 3rd Edition. Voss, Hamburg und Leipzig,
1911.
[2] A. Iriki, et al., Coding of modi2ed body schema during tool use by Macaque postcentral neurons,
Neuroreport 7 (1996) 2325–2330.
[3] M. Ito, The Cerebellum and Neural Control, Raven Press, New York, 1984.
L[4] W. JLacksch, Uberdie Zusammenarbeit von Auge und Hand, Dissertation, Jena, 1937.
[5] R.K. Luneburg, Mathematical Analysis of Binocular Vision, Princeton University Press, Princeton, NJ,1947.
[6] T. Maeda, S. Tachi, Sensory integration of binocular visual space and kinesthetic space in visual reachingexperiment, Trans. Soc. Instrum. Control Engrs. 29 (2) (1993) 201–210.
[7] T. Maeda, E. Oyama, S. Tachi, Sensory integration for space perception based on scalar learning rule,in: Proceedings of 1993 International Joint Conference on Neural Networks, IJCNN’93 Nagoya, 1993,
pp. 1317–1320.
[8] T. Maeda, E. Oyama, S. Tachi, Why is binocular visual space distorted compared to physical space?
Neurocomputing 44 – 46 (2002) 907–914, this issue.
[9] M. Ohkura, Y. Yanagida, T. Maeda, S. Tachi, Measurement of auditory alleys in a virtual environment
and their mathematical models, Systems Comput. Japan 31 (4) (2000) 12–21.
[10] C. Prablanc, D. Pelisson, M.A. Goodale, Visual control of reaching movements without vision of thelimb I, Role of retinal feedback of target position in guiding the hand, Exp. Brain Res. 62 (1986)
293–302.
[11] J. Stein, EEects of parietal lobe cooling manipulative behaviour in the conscious monkey, in: G. Gordon,Ed., Active Touch, The mechanism of recognition of objects by manipulation: a multidisciplinary
approach. Pergamon Press, Oxford, 1978, pp. 79 –90.