Hindawi Publishing Corporation
Journal of Robotics
Volume 2010, Article ID 217867, 15 pages
doi:10.1155/2010/217867

Research Article
Iterative Learning without Reinforcement or
Reward for Multijoint Movements: A Revisit ofBernstein’s DOF Problem on Dexterity
Suguru Arimoto,1, 2 Masahiro Sekimoto,1 and Kenji Tahara3
1 Research Organization of Science and Engineering, Ritsumeikan University, Kusatsu, Shiga 525-8577, Japan
2 RIKEN-TRI Collaboration Center for Human-Interactive Robot Research, Nagoya, Aichi 463-0003, Japan
3 Organization for the Promotion of Advanced Research, Kyushu University, Fukuoka 819-0395, Japan

Correspondence should be addressed to Suguru Arimoto, arimoto@fc.ritsumei.ac.jp
Received 5 November 2009; Accepted 17 May 2010
Academic Editor: Noriyasu Homma
Copyright © 2010 Suguru Arimoto et al. This is an open access article distributed under the Creative Commons Attribution
License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly
cited.
A robot designed to mimic a human becomes kinematically redundant and its total degrees of freedom becomes larger than
the number of physical variables required for describing a given task. Kinematic redundancy may contribute to enhancement of
dexterity and versatility but it incurs a problem of ill-posedness of inverse kinematics from the task space to the joint space. This
ill-posedness was originally found by Bernstein, who tried to unveil the secret of the central nervous system and how nicely it
coordinates a skeletomotor system with many DOFs interacting in complex ways. In the history of robotics research, such illposedness has not yet been resolved directly but circumvented by introducing an artificial performance index and determining
uniquely an inverse kinematics solution by minimization. This paper tackles such Bernstein’s problem and proposes a new method
for resolving the ill-posedness in a natural way without invoking any artificial index. First, given a curve on a horizontal plane fora redundant robot arm whose endpoint is imposed to trace the curve, the existence of a unique ideal joint trajectory is proved.
Second, such a uniquely determined motion can be acquired eventually as a joint control signal through iterative learning without
reinforcement or reward.

1. Introduction
Almost a quarter century ago, “robotics” was defined byProfessor Brady at the first International Conference of
Robotics Research [1] as “the intelligent connection ofperception to action.” After a great deal of researches ondevelopments of industrial robots and their applications, avariety of research projects on “humanoid” have attracted
many roboticists during the past decade and nowadays
robots that can walk with a bipedal mode are not peculiar.
Nevertheless, the present state of the art of humanoid stilllacks dexterity in fulfillment of ordinary tasks that humanencounter in their everyday life. More than a half century
ago preceding the birth of “humanoid,” Bernstein [2, 3]
noted that dexterity of human body movements resides ininvolvement of surplus degrees of freedom of limb joints but
this incurs the ill-posedness of inverse kinematics. This wasintroduced to the robotics community through the famous
textbook [4] in page 303 in such a statement as“The study of human biological motor control mechanisms
led the Russian psychologist Bernstein to question how the brain
could control a system with so many diﬀerent degrees of freedominteracting in such a complex fashion. Many of these same
complexities are also present in robotic systems and limit ourability to use multifingered hands and other robotic systems totheir full advantage.”
Actually, this was originally quoted from Hinton’s article[5] in which he summarized what Bernstein challenged in thefollowing way.
(a) What can we infer about the code that the brain uses
to communicate with the periphery, and what does that tell
us about how the computation is organized?, (b) If the brain
knew just what movements it wanted the body to make,could it figure out what to tell the muscles in order to make2

Journal of Robotics
q3
q3

π/2
q3

q1

q2

Configuration space(local coordinate chart)

q1
q2

O
q2
q1

π/2

q3

A Riemannian manifold
q2

q1

q1

Riemannian distance 1 
 gi j (q)q̇i (t)q̇ j (t) dt

d(q1 , q2 ) = min
q(t)

0

i, j

Figure 1: The set of all possible postures of a planar robot arm
with three joints can be regarded as a Riemannian manifold with
Riemannian metric gi j (q) that constitutes the inertia matrix.

of a double pendulum can be described by an orbit on a twodimensional torus T 2 that is regarded as T 2 = S1 × S1 , where
S1 denotes a unit circle. In line with this notion, an n-DOF
robot arm can be treated on an n-dimensional Rimeannian
manifold like an n-dimensional torus T n , and the stability
problems of PD feedback with damping shaping [11] wereretreated in a Riemannian-geometric manner [12, 13]. Morerecently, the author and his group showed that, given a
robot arm, the set of all possible postures can be regarded
as a Riemannian manifold with the Riemannian metricthat constitutes the inertia matrix [14, 15] (see Figure 1).
Thus, an orbit of motion as a geodesic solution to theEuler equation can be regarded as an inertia-induced motion
without aﬀection of damping and gravity forces [16].
It is well known as in a text book that motion of a robot
manipulator as a serially connected rigid-body system isgoverned by an Euler-Lagrange equation shown in the form
(see [17]) it happen? (c) How is it possible to coordinate a system withso many degrees of freedom that interact in such complex
ways? (d) How does the brain make sensible choices among
the myriad possibilities for movement that the body oﬀers?
This paper discusses what are the fundamentals of
biomimetic control by focusing Bernstein’s DOF problem
and shows one way for solving such diﬃcult problems fromthe standpoint of Riemannian geometry. It is shown in thecase of multijoint reaching with DOF redundancy that, givena starting posture in the robot configuration space (or aRiemannian manifold as a set of all postures) and a target
endpoint in the task space, there exists a unique orbit of joint
motion, provided that the gravity term is compensated ina feedforward manner. Then, it is shown that such an ideal
joint motion can be acquired through the iterative learning
control without introducing any kind of performance index
or reinforcement. In the second illustrative example, a
handwriting motion with DOF redundancy is analyzed fromthe viewpoint of Riemannian geometry under the situationthat writing with a ball pencil is imposed to trace an arbitrarysmooth curve of C ∞ -class on an arbitrary smooth surface
in the three-dimensional Euclidean space. Even in this casethere exists a unique joint motion in the base Riemannian
manifold with DOF redundancy and it can be acquired
through repeated exercises of handwriting motion, that is, an
ILC scheme without introducing any artificial performanceindex. In conclusion, an ideal multijoint motion can beacquired through repeated exercises of motion even underthe existence of redundancy in DOF, irrelevantly to any kindof reinforcement with the aid of some sort of reward [6, 7].

2. Riemannian Manifold and Euler’s Equation
It is widely known among roboticists that kinematics andplanning of multijoint robots are treataed in the configuration space regarded as an n-dimensional numerical spaceRn [8, 9]. On the other hand, Arnold [10] pointed out
the importance of Riemannian geometry in the analysis ofmechanical systems and shown that the dynamics of motion

G q q̈ + 
1    Ġ q + S q, q̇ q̇ + g q = u,2(1)

where q = (q1 , . . . , qn )T denotes the vector of joint angles,
G(q) = (gi j ) does the n × n inertia matrix, u a control toquevector, g(q) = ∂P(q)/∂q with a scalar function P(q) called
the potential, and S(q, q̇) a skew-symmetric matrix S = (Si j )defined as
⎧

⎛

⎞

⎛

⎞⎫

⎬
1 ⎨ ∂ ⎝∂ ⎝Si j = ⎩q̇k gik ⎠ −q̇k g jk ⎠⎭.
2 ∂q j k=1∂qi k=1nn(2)

If we consider a control torque that can exactly compensate
the gravity term, that is, 

u=g q ,

(3)then substitution of (3) into (1) yields
 

G q q̈ +1    Ġ q + S q, q̇ q̇ = 0,2(4)which is considered to be an ideal equation of motion
without aﬀection of gravity and joint damping forces like arobot arm on an artificial satellite in space. It is pointed out
in the recent papers [14, 16] that (4) is equivalently written
in the form
 

 

gik q q̈i + Γik j q q̇ j q̇i = 0,

k = 1, . . . , n,(5)where Γik j denotes the Christoﬀel’s symbol of the second
kind and the symbol of summation with respect to i and jin (5) is omitted by obeying the Einstein’s rule in diﬀerential
geometry [18, 19]. Equation (5) is also expressed equivalently
in the formq̈k + Γkij q̇i q̇ j = 0,

k = 1, . . . , n,(6)

Journal of Robotics

3

Desired point
xd = (xd , yd )End pointx0 = (x0 , y0 )where v should be computed so as to optimize a certain
performance index related to joint position variables (forexample, manipulability index [20], obstacle avoidance [21],etc.). In equation (9), J(q) stands for the Jacobian matrix ofx in q, that is, J(q) = ∂x/∂qT , and J + (q) denotes the pseudoinverse of J(q). The original idea of use of the pseudo-inverse
J + (q) is due to Whitney [22]. Once a desired joint velocityq̇d (t) is planned, it is claimed that the computed torquemethod can be applied for determining the control inputthat must generate the whole joint motion of the robot.
This is called “inverse kinematics approach”. Another idea of
direct generation of a control signal called “inverse dynamics
approach” is based upon a form of control input

xy-plane

q3
q4
q2

y
q1

x

Figure 2: “Reaching” by means of a surplus DOF system of handarm dynamics.

        + I − J + q J q v,
which is called the Euler equation. In (5) and (6), Γik j andΓkij (i.e., called the Christoﬀel’s symbol of the first kind) aredefined as follows:

1 ∂g jk ∂gik ∂gi j
Γik j =
+
−
,
2 ∂qi
∂q j
∂qk
1
Γkij =

n


2 l=1

g


lk

∂g jl ∂gil ∂gi j+−∂qi ∂q j∂ql

1  lkg Γil j ,
=
2 l=1
n

(7)

where (g lk ) denotes the inverse of G(= (gi j )). The equivalence relation of (5) to (4) is shown in the previous paper[15].

3. Multijoint Movements withDOF Redundancy
Let us now consider motion of a redundant planar robot arm
whose endpoint is free to move in the horizontal plane asshown in Figure 2. The Lagrange equation of motion of thewhole arm-hand system depicted in Figure 2 is expressed as 

G q q̈ +1    Ġ q + S q, q̇ q̇ = u,2(8)where q = (q1 , . . . , q4 )T and each rotational axis of the four
joints (shoulder, elbow, wrist, and index finger MP (metacarpophalangeal) joint) is in the direction perpendicular to thexy-plane. Given a robot arm posture q in the configuration
space R4 or in the 4-dim. base manifold {M, gi j }, theendpoint position x can be determined by the forward
kinematics. A vector-valued function x(q) = (x(q), y(q))
of C ∞ -class. However, given an endpoint position xd in
R4 , there arises an infinite number of inverses that realizex(qd ) = xd and thereby the problem for obtaining inverse
kinematics from the 2-dimensional Euclidean space E2 to the
4-dimansional configuration space R4 becomes ill-posed.
A variety of ideas for solving such ill-posedness of inverse
kinematics for redundant robotic systems with excess DOFs
has been proposed in the area of robotics, based upon the useof the formq̇d (t) = J


+





q(t) ẋd + I − J   +q J q v,(9)u = G q J + q ẍd − J˙ q q̇ + g q, q̇



(10)

where v is computed so as to optimize a certain performanceindex related to velocity variables (e.g., kinetic energy [23],torque [24], energy dissipation [25], etc.). In (10), g(q, q̇)means compensation for the remaining nonlinear function
including centrifugal and Coriolis forces and the gravity
eﬀect. In the physiological literature, main concerns arefocussed on the question why human skilled multijoint
reaching movements exhibit typical characteristics that (1)endpoint trajectory becomes a quasistraight line and lessvariable throughout repetitions, (2) velocity profiles of theendpoint velocity becomes bell-shaped, though (3) joint
trajectories are rather variable trials-by-trials [26]. Then, avariety of cost functions for derivation of such propertiesof point-to-point reaching movements has been proposed,
among which a quadratic function of endpoint jerk (rateof acceleration) was the first [27] and successively a costfunction based on joint torques was introduced [28] forplanning not only an endpoint trajectory but also joint
trajectories. However, in the physiological literature, there
is a dearth of papers that attempted to directly deal withreaching movements with redundant joints, though theimportance of Bernstein’s DOF problem [2] has been widely
known among physiologists.
Diﬀerently from the traditional approaches, a simpler
control method for multijoint reaching movements was
proposed very recently in our previous papers [29–31] andshown to be eﬀective in both cases of human and roboticarms with redundant DOFs. In those papers, only planar
motions confined to a horizontal plane are treated andtherefore the control signal is free from gravity with a simple
form (see Figure 3) 

u = −C q̇ − J T q kΔx,

(11)

where C denotes a diagonal positive definite matrix asfollows:
C = diag(c1 , . . . , cn ).

(12)Notwithstanding this simpler form than (10), once damping
factors C and single stiﬀness parameter k are chosencarefully, it is shown that it generates smooth reaching

4

Journal of RoboticsyConfiguration spacex = (x, y)T: endpoint∗q(t)Torque to generate
virtual damper eﬀect

x − xd ∗Pq∗ (t)∗

xd = (xd , yd )T: targetP = q(0)= q0Torque to generate
Virtual spring eﬀectxOx − xd 

Virtualspringx

Figure 4: A set of all postures that have the same endpoint position
xd constitutes a two-dimensional Riemannian submanifold.

Intention
to reach

Figure 3: Virtual spring-damper hypothesis.

movements by realizing a quasistraight endpoint trajectory,bell-shaped velocity profiles, and double-peaked acceleration
profiles typically seen in case of human skilled multijoint
movements [26, 32]. Therefore, the method can get rid of
undesirable fluctuations of the endpoint trajectory tracking
as pointed out in a recent elaborate work [33]. Such fluctuations in task space tracking caused by using the computed
torque under uncertainty in link parameters become more
noticeable in cases of robots with redundant DOFs. In
contrast, in the use of control defined in (11), there is noneed of planning any desired endpoint trajectory. However,
all these treatments are restricted to planar motions as wellas in most of the previous papers on multijoint reaching
movements. In addition, another disadvantage is that choiceof damping factors recommended in [29] is not fit to the scaleof coeﬃcients of viscosity of human muscles [34, 35].
To reduce damping factors in general, another control
method based upon “Virtual Spring-Damper Hypothesis”
was suggested in [31], which in the case of planar motions
without aﬀection of the gravity is expressed by the form  

NP  = {q | x(q) = xd }NP = {q | x(q) = x(0)}VirtualdamperT∗q(t)u = −C0 q̇ − J q ζ kẋ + kΔx

obtained by substituting (13) into (8) and taking the innerproduct of this resulted closed-loop dynamics and q̇ asfollows:   
1
Ġ + S + C0 q̇ + J T q ζ k ẋ + kΔx = 0,
G q q̈ +2(15)

d 1 T  k2
2Tq̇ G q q̇ + Δx = −q̇ C0 q̇ − ζ kẋ .
dt 2
2These results suggest that skilled multijoint movements canbe generated even in the case of robot arms with redundant
DOFs without construction of any inverse dynamics through“error-feedback learning” as claimed in a physiologicaljournal [36] for modifying Equilibrium-Point hypothesis[35], End-Point hypothesis [37], and Virtual Trajectory
hypothesis [37].

4. Existence of Desired Joint-Motion
More recently in the paper [16], an interesting result is foundthat the endpoint trajectory of a solution to the closed-loop
dynamics for a given starting posture resembles considerably
the endpoint trajectory of a geodesic solution to the Euler
equation of (5) or (6) starting from the same given posture
to a certain diﬀerent posture with the prescribed endpoint xd .
The existence of such a geodesic solution to (6) is ascertainedby considering a two-dimensional Riemannian submanifoldthat is defined by the set of all postures q satisfying x(q) = xd ,that is (see Figure 4),    
= −C0 q̇ − ζ kJ T q J q q̇ − J T q kΔx,

(13)



Similarly, define another submanifold
 





NP = q | x q = P (= x(0)) .

where C0 is chosen as follows:
C0 = ζ0 diag(c1 , . . . , cn ), NP = q | x q = P  (= xd ) .

(16)
(17)

For a given endpoint position xd in E2 , NP constitutes a two(14)together with positive constant 0 < ζ0 < 1.0. The eﬀectivenessof this control signal particularly in the case of middle-range
reaching was demonstrated through computer simulationand its performance was compared with that of the controlof (11). Theoretical verification of the eﬀectiveness of thisspring-damper hypothesis was also presented, on the basisof an energy conservation law like a Lyapunov-like relation

dimensional submanifold of the base manifold {M, gi j }, thatis called the equilibrium-point manifold or simply the EPmanifold in this paper. Denote by q(t) any smooth orbit ofmotion of the robot starting at t = a from the same postureq(t = a) = q0 and reaching the submanifold NP at t = b sothat it satisfies x(q(b)) = xd . Then, consider the infimum

d q0 , NP = infq(t) b a 

gi j q q̇i (t)q̇ j (t)dt

i, j

(18)

Journal of Robotics

5

over the set of all such possible orbits of robot motion
connecting q0 and NP . If xd is not so distant from x(0) (=x(q0 )), it is reasonable from the text books of Riemannian
geometry [18, 19] that there exists a unique optimal orbitq∗ (t), t ∈ [a, b], that minimizes the right-hand side of(18). Then, the quantity d(q0 , NP ) is entitled to be called
the Riemannian distance from q0 to the submanifold NP inthe base manifold {M, gi j }. It is well known that the optimal
orbit q∗ (t) must satisfy the Euler equation (6) for t ∈ [a, b]
and, moreover, it must satisfy the following equation:




 



I4 − J + q∗ (t) J q∗ (t) q̇∗ (t) = 0,

 

⎛ ⎞

⎛ ⎞

(20)

provided that the robot dynamics is governed by the
Lagrange equation
 



 
1  
(21)
G q q̇ + S q, q̇ q̇ + C0 q̇ = J T q v,
2
where C0 is a 4 × 4 positive definite damping coeﬃcientmatrix. We assume that the initial posture of the robot at
t = 0 is given by q(0) = q0 and motion of the robot starts
from the still state, that is, q̇(0) = 0. The first problem
is to find an adequate control signal v(t), t ∈ [0, T], sothat the solution to the Lagrange equation of (21) startingfrom q(0) = q0 and q̇(0) = 0 satisfies x(q(t)) = xd (t) fort ∈ [0, T]. In order to find a solution to this problem, we
decompose any solution trajectory of joint velocity q̇(t) insuch a way thatG q q̈ +

⎛ ⎞

v
 ẋ
1
G1 ⎝ ⎠ + Ġ1 + S1 + C1 ⎝ ⎠ = ⎝ ⎠.
2
η̈
η̇
0
ẍ

(25)

Note that S1 is again skew-symmetric. For convenience let us
define1
B1 = Ġ1 + S1 + C1
2

(19)

where J(q) = ∂x(q)/∂qT , the Jacobian matrix of x(q) withrespect to q. In other words, q̇∗ (t) must belong to the image
space of J T (q∗ (t)) at any instant t in [a, b]. That is, q̇∗ (t) does
not have any component in the kernel space of J T .
Consider now an endpoint trajectory tracking problem
for a redundant multijoint arm of Figure 2 in the case thata desired endpoint motion is given as a curve xd (t) : I =[0, T] → E2 . The control task is to maneuver the robot to let
its endpoint trace the given trajectory xd (t) in E2 through atask space control signal
u = JT q v

then, by substituting (22) into (21) and multiplying theresultant equation by the transpose of Q from the left, we
have

(26)

and decompose G1 and B1 in such a way that
⎛

⎞

⎛

⎠,
G1 = ⎝ T
G12 G22

B1 = ⎝

G11 G12B11 B12B21 B22

⎞
⎠,

(27)

where all Gi j and Bi j are of 2 × 2-matrix. Then, it follows from(25) that
G22 η̈ + B22 η̇ = −GT12 ẍ − B21 ẋ.

(28)This equation means that if ẋ(t) is set as ẋ(t) = ẋd (t) andqd (t) is also given then η̇(t) can be determined uniquely fromsolving the diﬀerential equation of (28) as an initial-value
problem by setting η̇(0) = 0. It should be remarked at thisstage that the given curve xd (t) is of C ∞ -class described interms of time parameter t in E2 , that is, it is an E2 -valued
function of t with the initial value xd (0) = x(q0 ), and it has
the continuous time derivative ẋd (t) with the initial value−1ẋd (0) = 0. Now, multiplying (28) by G22
and accompanying
this with (22) by setting ẋ = ẋd and ẍ = ẍd , we have








q̇d = P qd ẋd + W qd η̇,

(29)


−1



















η̈ = −G22 qd B22 qd , q̇d η̇ + G12 qd ẍd−1+ G22
qd B21 qd , q̇d ẋd .



(30)

where x = x(q), ẋ = J(q)q̇, η̇ is a 2 × 1 vector and P is the
4 × 2-matrix defined byThis couple implies a set of six simultaneous diﬀerential
equations of 1st order concerning six variables {qd , η̇}though the right hand side of (30) contains q̇d and hence it
is an implicit function expression. Fortunately, it is possible
to obtain an explicit expression of the six simultaneousdiﬀerential equation by putting     −1

(23)

B22 = B22 qd , P ẋd + W η̇ ,and W is a 4 × 2-matrix whose column vectors w1 and w2 areorthogonal to J T (q) (wi belongs to the kernel space of J T (q))
and satisfy w1  = w2  = 1 and w1T w2 = 0. Then, if we
defineB21 = B21 qd , P ẋd + W η̇ .

⎛ ⎞ẋq̇ = (P, W)⎝ ⎠,η̇P = JT q J q JT q⎛

Q = (P, W),
G1 = QT GQ,

Q

−1

=⎝

(22)

 
= J+ qPT
WT⎞⎠ = QT ,C1 = QT C0 Q,

11
S1 = QT SQ − Q̇T GQ + QT GQ̇,
2
2

(24)









(31)

Hence, the right hand sides of (29) and (30) are nonlinear
in qd and η̇, but they are Lipschitz continuous in qd and η̇
locally. Therefore, for given ẋd and ẍd there exists a uniquesolution {qd (t), η̇(t)} for an interval t ∈ [0, a] with some a >0 satisfying qd (0) = q0 and η̇(0) = 0, where q0 signifies aninitial posture satisfying x(q0 ) = xd (0). This fact was alreadydiscussed in our previous paper [38]. In this paper, we now
prove the unique existence of the solution to the pair of 1storder diﬀerential equations of (29) and (30) over the timeinterval [0, T], provided that C0 is not so small in comprison

6

Journal of Robotics

with the scale of G(q) and both the quantities of ẋd and ẍdare within an adequate physical scale.
First, we analyze (30) or (28) by rewriting them in a moredetailed manner as follows:1
Ġ22 + S22 η̇ + C22 η̇ = −GT12 ẍd − B21 ẋd − C21 ẋd ,G22 η̈ +
2
(32)
where
G22 = W T GW,

C22 = W T C1 W,

1
B21 = Ġ21 + S21 ,2

GT12 = W T GP,

C21 = W T C1 P
(33)

and S22 denote the submatrix (si j ) of S1 for i, j = 3 or
4, and B21 plus C21 constitutes B21 . Note that B21 is linear
and homogeneous in η̇ but G12 are irrelevant to η̇. In thispaper, we restrict our consideration to the case that overall
movements of the robot is confined to a single chart ofthe base manifold in which at any posture in the chart theJacobian matrix is nondegenerate and therefore there exists
a positive constant σ0 such that J(q)J T (q) ≥ σ0 I2 inside thechart. Under this condition, we analyze the following relation
obtained by taking the inner product of (32) and η̈:

d 1 Tη̇ G22 η̇ = −η̇T C22 η̇ − η̇T B21 ẋd
dt 2
− η̇

T



GT12 ẍd + C21 ẋd



(34)

(35)

and β0 depend on the maximum magnitude of ẋd and 1/σ0 .
The third term of the right hand side is bounded from theabove in the following way
!

!
1
! T T
!
!−η̇ G12 ẍd + C21 ẋd ! ≤ η̇T W T GW η̇ + ẍd P T GP ẍd

2
(36)
1 T T
−1
+ ẋd P C0 G C0 P ẋd .
2
As discussed previously in [39], the damping matrix C1 canbe chosen to be of the order of G1/2 and further so as to satisfy
C0 > 2G.

(37)

Then, (34) is reduced to the inequality relationdηd (t) ≤ −γ0 ηd (t) + ξd (t),dtwhere we put with some positive constant γ0(38)1ηd (t) = η̇T (t)G22 (t)η̇T (t),21
ξd (t) = ẍdT (t)P T GP ẍd (t)21+ ẋdT (t)P T C0 G−1 C0 P ẋd (t).
2t0e−γ0 (t−τ) ξd (τ)dτ(40)

which concludes that η̇(t) is uniformly bounded in t ∈[0, T].
Once q̇d (t) and η(t) are obtained for the given ẋd (t) andẍd (t), the desired input signal vd (t) in the image space ofJ(qd ) is obtained by setting q(t) = qd (t), ẋ(t) = ẋd (t),ẍ(t) = ẍd (t) in (25).

5. Iterative Learning Control in the Task Space
Given a desired trajectory of the endpoint in the task space E2
for a redundant robot arm, there exists a unique trajectory ofrobot motion in the joint space for a specified initial posture.
In particular, it is shown in the previous section that there isuniquely a control signal in the task space that maneuvers therobot through the transpose of the Jacobian matrix to realize
the endpoint tracking. However, such a control signal can not
be obtained in any analytical form. Nevertheless, it is possible
for us to acquire such a desired control signal by using asimple iterative learning control scheme, provided that theendpoint trajectory in the task space can be measured byvisual sensing.
At the kth trial of iterative learning, the control signal forthe dynamics of (21) is designed in the form√

vk = −J T qk κΔxk (t) + ζ1 κΔẋk (t)The second term of the right hand side is quadratic in η̇ andtherefore there exists a constant β0 such that

C0 > 2β0 I4 ,ηd (t) ≤

 

.

!
!
! !2
! T
!
!η̇ B 21 ẋd ! ≤ β0 !η̇!

Clearly, since ηd (0) = 0, (38) implies

(39)



+ vk−1 − ΦΔẋk−1 (t),(41)

where J(qk ) means ∂x/∂q at x = x(qk ), and








Δxk (t) = x qk (t) − xd (t),(42)Δẋk (t) = ẋ qk (t) − ẋd (t).

The first term of the right hand side of (41) signifies the innertask-space PD feedback, vk−1 denotes the previous control
signal at the (k − 1)th trial, and Φ is an adequate positive
definite constant matrix. At the first trial, usually we set
v0 (t) = 0 for t ∈ [0, T]. At the second trial v1 (t) must containerroneous terms. Fortunately, without knowing the desired
ideal control vd (t), it is possible to expect that Δxi (t) → 0
and Δẋk → 0 for t ∈ [0, T] as k → ∞. We give an illustrative
example of numerical simulation conducted for the 4-DOF
robot arm shown in Figure 2 with physical parameters given
in Table 1. The values for length, mass, and inertia momentof the first link correspond to those of an upper arm of
average human adult (male), and the values for the second
link do to those of a lower arm. The third link corresponds
to a human palm and the fourth an index finger. The desiredtask is to write a handwritten character “α” on the xyplane. More explicitly, the endpoint trajectory is given by theequation⎡

xd (t) = ⎣

0.00
0.30

⎤

⎡

⎦+⎣

0.075 cos ω(t)0.100 cos 1.5ω(t)

⎤
⎦,

(43)

Journal of Robotics

7
Table 1: Physical parameters of the 4-DOF robot arm.
i
li
lgi
ri
hi
di
mi
Igiz

Link number
Length [m]Center of mass [m]Cylinder radius [m]Cuboid height [m]Cuboid depth [m]Mass [kg]Inertia moment [kg m2 ]10.28000.14000.04000
N/A
N/A
1.407
9.758 × 10−320.28000.14000.03500
N/A
N/A
1.078
7.370 × 10−3

0.2

0

0.2

0
−0.2

0
x-component (m)

−0.2

0.2

(a) 1st trial

0x-component (m)0.2(b) 3rd trial

0.4
y-component (m)0.4
y-component (m)40.09000
0.04500
0.009500
N/A
N/A
0.02552
1.780 × 10−5

0.4
y-component (m)y-component (m)0.4

30.09500
0.04750
N/A
0.08500
0.03000
0.24232.004 × 10−4

0.2

0

0.2

0
−0.2

0
x-component (m)Endpoint trajectory
Desired trajectory(c) 5th trial

0.2

−0.2

0
x-component (m)Endpoint trajectory
Desired trajectory(d) 10th trial

Figure 5: Endpoint trajectories and the initial and final postures of the arm.

0.2

8

Journal of Robotics

Table 2: Initial values and gain settings in the case of the 4-DOF
planar robot arm.
(a)T
q1 (0)
q2 (0)
q3 (0)
q4 (0)

Terminal time
Initial posture

2.0 [s]21.02 [deg]75.47 [deg]44.20 [deg]64.87 [deg]endpoint connecting two given points x(q(0)) = x0 andx(q(T)) = xd by selecting an adequate initial joint velocityq̇(0) that is nonzero. However, the orbit x(t) of the endpoint
can be represented by a curve c(s) on E2 with the aid of lengthparameters. In the case of middle-range reaching, the profileof this endpoint geodesic curve c(s) quite resembles those ofhuman-like multijoint reaching characterized typically by aquasistraight line movement of the endpoint starting from afixed still state.

(b)

k
ζ0
ζ1
Φ
c1
c2
c3
c4

Gains

10 [N/m]
0.5 [−]
2.5 [−]
10.0 [−]
0.862 [Nms]0.569 [Nms]0.129 [Nms]0.0356 [Nms]

6. Extension to the Case of Existence of
Effect of Gravity
Most of the previous results in Sections 4 and 5 can beextended to the case that robot dynamics is subject to theeﬀect of gravity. In such a robot with redundancy in DOF,
robot dynamics is expressed by the Lagrange equation: 

G q q̈ +  
1    Ġ q + S q, q̇ + C0 q̇ + g q = J T q u, (46)2where T = 2.0 [s] and&tω(t) = 2.0π −15
T

4



t
+6
T

5



t
+ 10
T

3

'

.

(44)

The initial posture of the arm is given in Table 2. Based onthese data, the system of diﬀerential equations
 

G qk q̈k +T
1   Ġ qk + S qk , q̇k + C0 q̇k
2

 

√



T

 

where g(q) stands for the gravity term that can be regarded asa gradient vector of a potential function U(q) with respect toq, that is, g(q) = ∂U(q)/∂q. Let us denote again the endpoint
position by x(q) in the m-dimensional Euclidean space Em .
Then, we split g(q) into           

g q = J + q J q g q + In − J + q J q g q(45)+ J qk κΔxk + ζ1 κΔẋk = J qk vk
is numerically solved by using the Runge-Kutta method.
In Figure 5 we show endpoint trajectories at 1st, 3rd, 5th,and 10th trials and their corresponding postures at t = 0
and t = T. On the other side, we are able to obtain thedesired control signal numerically by numerically solving acouple of 1st-order diﬀerential equations of (29) and (30).
Based upon knowing physical data given in Table 1, theinitial posture of q(0) given in Table 2, and the specified
endpoint trajectory of (43), we obtain the desired ideal
control signal shown in Figure 6. It is quite interesting toknow that, through simulations of the iterative learning,calculated control signals vk (t), t ∈ [0, T], approach thedesired one as the trial number k increases as shown in
Figure 6. When k = 10, the trajectory of control signal v10 (t)almost coincides with the ideal one vd (t), that uniquely exists
just in the image space of the Jacobian matrix J(qd ) withx(qd ) = xd for all t ∈ [0, T].
It should be remarked that the desired endpoint trajectory x(t) in the case of multijoint reaching for the robot
arm of Figure 2 is obtained by solving the Euler equation of(4) or (5) as a two-point boundary-value problem when theinitial posture q(0) = q0 at t = 0 is given and the terminal
condition at t = T is partially specified so as to move theendpoint of the arm to meet x(q) = xd at t = T and passit away. This is the problem to find a curved orbit of the  = g 1 q + g2 q .

(47)

That is, g1 (q) is a component of g(q) in the image space ofJ(q) and g2 (q) is that of g(q) in the kernel space of J(q), that is
orthogonally complement to the image space. In accordancewith the split of the term g(q), let us choose an n × (n − m)matrix W(q) = (w1 , . . . , wn−m ) with n-dimensional unit
column vectors wi (i = 1, . . . , n − m) that are mutuallyorthogonal and satisfy W(q)J T (q) = 0 (i.e., W(q)J + (q) = 0).
Then, in a similar way to (22), we define
 

  

 

Q q = J+ q , W q(48)

which leads to

Q

−1

 

⎛

  ⎞

J q

q = ⎝ T  ⎠.
W q

(49)

On the other hand, for a given desired endpoint trajectory
xd (t) for t ∈ [0, T], we consider the control signal with thetask space PD feedback√u = − κΔx + ζ1 κΔẋ + v,

(50)

Journal of Robotics

9

2

4

1

vd2 (N)

vd1 (N)

2
0

0

−1
−2
−2
−4
−3

0

1

2

3

0

1

Time (s)

2

3

2

3

Time (s)(a) vd1(b) vd2

Figure 6: Transient responses of the ILC term vd = (vd1 , vd2 )T .

2

4

1

vk2 (N)vk1 (N)

2
0

0

−1
−2
−2
−4
−3

0

1

2

3

0

1

Time (s)1st3rdTime (s)
5th
10th

1st
3rd

5th
10th

(a) vk1(b) vk2

Figure 7: Transient responses of the ILC term vk = (vk1 , vk2 )T .

where Δx = x − xd . Substituting (50) into (46) yields theclosed-loop dynamics 

G q q̈ +  
1    Ġ q + S q, q̇ + C0 q̇ + g1 q + g2 q
2

 

√



 

+ J T q κΔx + ζ1 κΔẋ = J T q v,

(51)

where v expresses a feedforward task space control signal.
Then, consider the transform of q̇ to (ẋT , η̇T )T with an (n −m)-dimensional vector in such a form that⎛ ⎞⎛ ⎞  ẋ  ẋ
q , W q ⎝ ⎠ = Q q ⎝ ⎠.

 + 

q̇ = J

η̇

η̇

(52)

10

Journal of Roboticsz

In order that the equation of (51) with a desired control
signal vd (t) has a unique solution qd (t) and q̇d (t) so that its
endpoint trajectory x(t) is coincident with xd (t), and ẋ(t) =ẋd (t), the following equations must be satisfied:
⎛

Gd ⎝

ẍd
η̈

⎞

⎛

q1

J3

d



Cd = QdT C0 Qd ,

Qd ,

 
 
1
1
Sd = QdT SQd − Q̇dT G qd Qd + QdT G qd Q̇d
2
2

(54)

with Qd = Q(qd ). The latter (n − m)-dimensional componentof (51) can be expressed byG22 η̈d +1
Ġ22 + S22 + C22 η̇2(55)+ GT12 ẍd + B21 ẋd + WdT gd = 0,
where Gd and Bd = (1/2)Ġd + Sd + Cd are decomposed into⎛

G11 G12⎛⎞B11 B12⎠,
Gd = ⎝ T
G12 G22⎞⎠,
Bd = ⎝
B21 B22(56)q̇d = J





y
q1





qd ẋd + W qd η̇xFigure 8: A handwriting robot with four DOFs whose endpoint
P(= (x, y, z)) is constrained on a hypersurface ϕ(x) = 0, where ϕ(x)is a scalar function of C ∞ -class.

S, it is possible to consider an orbit in the submanifold F(equivalently, φ(F) in the configuration space) starting fromq0 , lying on N(S), and reaching some point lying on NP
that is another single-dimensional submanifold defined by
the set of all q satisfying x(q) = xP and q ∈ F. Thus, it isreasonable to suppose that there exists an optimal orbit q∗ (t)that gives minimization of the Riemannian distance from q0
to NP such that

d q0 , NP = inf

1(

q(t)

and G22 and B22 are of (n − m) × (n − m)-matrix, and G11
and B11 are of m × m-matrix. The simultaneous diﬀerential
equations (53) of 1st order together with+

ϕ(x) = 0

O(J0 )
x



S

P

x

where gd = g(qd ) and
Gd = QdT G qd

q2

J1

0

gd

q4

(53)

⎛  T ⎞ ⎛ ⎞vd
J + q d gd+⎝  ⎠ = ⎝ ⎠,

WT q

q3

⎞

ẋd
1
Ġd + Sd + Cd ⎝ ⎠
2
η̇

⎠+

J2

(57)

determine a unique solution qd (t) and η̇(t) for given xd (t),ẋd (t), and ẍd (t) with the initial conditions qd (0) = q0 ,x(q0 ) = xd (0), and η̇(0) = 0.

=

All the considerations in the previous sections can beextended to the case of a handwriting robot whose last link
is a ball-point pen constrained on a hypersurface ϕ(x) = 0,
where x = (x, y, z)T in E3 and ϕ(x) is a scalar function of C ∞ class (see Figure 8). First, we consider a pure mathematicalproblem of finding a geodesic curve by ignoring the eﬀect ofgravity and any joint damping. In this case, let us consider
an open connected area S on the hypersurface as shown in
Figure 8, on which any point P(= x) satisfies the equality
ϕ(x) = 0. Then, we denote by F a local coordinate chart
defined by F = {q ∈ (M, gi j ) | x(q) ∈ S} and assume that F
is connected and at any q ∈ F the Jacobian matrix J(x(q)) =∂x(q)/∂qT is nondegenerate. Then, given a point P on S withthe cartesian coordinates xP , the set of all q such that x(q) =xP and q ∈ F constitutes a single-dimensional submanifold
NP (see Figure 9). Hence, for another given point P  on

1(
0





gi j q(t) q̇i (t)q̇ j (t)dt(58)

gi j (c(t))ċi (t)ċ j (t)dt,

where the infimum is taken over all the orbits lying on N(S)(see Figure 9), starting from q0 and reaching NP , where q∗ (t)is rewritten by c(t). It is reasonable to conclude that the
optimal orbit c(t) in (58) is a solution to the Euler equation
under the constraint ϕ(x(c)) = 0:
G(c(t))+

7. ILC for Handwriting

0∂ϕ1Ġ(c)+S(c, ċ) ċ(t) = −λJcT (x(c(t))) T ,
2
∂x

(59)

where λ denotes a Lagrange multiplier. In other words, thepath of c(t) from q0 to some point on NP can be called
the geodesic on the Riemannian submanifold N(S) induced
naturally from the constraint ϕ(x(q)) = 0.
Next, consider the full dynamics of the handwriting robot
with 4 DOFs shown in Figure 8 by taking into account theeﬀect of gravity forces and damping torques at joints (see[38]). The dynamics is described by 

G q q̈ + 
1    Ġ q + S q, q̇ q̇ + C0 q̇ + g q
2

= −λJ

T

  ∂ϕ(x)

q

∂xT

(60)

+ u,

where g(q) = ∂U(q)/∂q, U(q) denotes the gravity potential,
C0 a positive definite damping matrix, and J(q) = ∂x(q)/∂qT .
It should be remarked that ∂ϕ/∂x stands for a vector that

Journal of Robotics

11

q4

existence of a control signal in the image space of J(q), we
express it in a decomposed form such that 

u = J T q {vn n(s) + vb b(s) + ve e(s)},(65)

q∗ (t)

q(t)

q(0)

O

where b(s) signifies the unit vector tangent to the surface
in the direction of −ẋ(t) and e(s) = n(s) × b(s). Note that
substituting (65) into (63) yields

q2 (or q3 )

q(t)

= q0

NP 
N(S) = S

NP

)

 

 
1
Ġ + S q̇ + C0 q̇ + (I4 − J + J)g q
2

G q q̈ +

φ(F)

q1

⎧⎛
⎞ ⎛ ⎞⎫⎪gn ⎪⎪⎪⎪⎪ vn − f
⎨
⎜
⎟
⎜
⎟⎬
T
⎜
⎟
⎜
⎟
= −J Π(s) ⎝ vb ⎠ + ⎝gb ⎠ ,
⎪
⎪
⎪
⎪
⎪
⎪
⎭
⎩

S = {q | ϕ(x(q)) = 0}

Figure 9: A geodesic curve starting from q0 lying on NP andreaching some point on NP . Here, φ is a homomorphism mapping
F in M to φ(F) in R4 .

vn

(66)

ge

where
originates at the position x(q) in E3 and is normal to thesurface ϕ(x) = 0 which the ball pen contacts with. Now weintroduce a length parameter s in E3 by the quantitys(t) =t0

ẋ(τ)dτ

 t!
!

 

!
!
= !q̇T (τ)J q(τ) J T q(τ) q̇(τ)!dτ

⎛ ⎞

gn

and define the unit normal at the contact point as follows: 

g T q J + n(s)

ge

⎛ ⎞

ξ˙

⎟
 ⎜
⎜ ⎟
q̇ = Q q ⎜η̇⎟,
⎝ ⎠

(62)

(63)

Note that the inner product of q̇ and the right-hand side of(63) vanish. In other words, ẋ(= J(q)q̇) is orthogonal to n(s)at the contact point in E3 . Under the assumption that boththe endpoint position x(t) and the velocity ẋ(t) are measured
in real time by visual sensing and the Jacobian matrix J(q(t))
is calculated from the measurement data of x(t) and q(t) inreal time, suppose that the control signal must be constructed
through the Jacobian transpose in the formu = J T q v.

(64)

For a given desired endpoint trajectory xd (t), t ∈ [0, T],
together with ẋd (t) and ẍd (t) and a given desired pressing
force fd (t), we are concerned with the problem to finda desired control signal vd (t) of (64) that maneuvers therobot to make the endpoint of the last link (ball pen) tracex(q(t)) = xd (t) on the hypersurface with the pressing
force fd in the direction normal to the surface. To show the

(68)

γ̇
 

 + 

Q g = J  
= − f J T q n(s) + u.

(67)

Note that Π(s) is an orthogonal matrix belonging to SO(3),g = (gn , gb , ge )T is a vector in E3 , and a component of g(q) to

Then, by rewriting f = λ∂ϕ/∂x, (60) can be rewritten into 
1    
G q q̈ +Ġ q + S q, q̇ q̇ + C0 q̇ + g q
2⎞

g q J e(s)

!−1

!
∂ϕ !
! ∂ϕ !
! ! .
n(s) =∂x ! ∂x !

 

⎛

⎟
⎜ ⎟ ⎜
 
T  
⎜ gb ⎟ = ⎜g T q J + b(s)⎟⎟ = ΠT (s)(J + ) g q .
⎝ ⎠ ⎜
⎝  
⎠
T
+

(61)

0

!

Π(s) = (n(s), b(s), e(s)),

 

 



q n(s), W q , J + q (b(s), e(s)) ,(69)

where W(q) is the 4 × 1 unit vector satisfying J(q)W(q) = 0,ξ˙ and η̇ are a scalar and express a 2 × 1 velocity vector suchthat (b(s), e(s))γ̇(s) = ẋ(t) at the contact point between the
tip of the ball pen and the surface. Since ξ˙ = 0 that implies
that the velocity of the tip of the pen in the direction n(s)normal to the surface is zero, substituting (68) into (66) andmultiplying (66) by QT (q) yield








G11 η̈ + 1
Ġ11 + S11 η̇ + W T C0 W η̇ + W T g q2

α η̇, η̈, γ̇, γ̈ = − vn − f + gn ,

(70)

(71)

+ G12 γ̈ + B12 γ̇ = 0,G22 γ̈ +1
Ġ22 + S22 γ̇ + C22 γ̇2⎛ ⎞+ GT12 η̈ + B21 η̇ = −v − ⎝

gb
ge

⎠,

(72)

12

Journal of Robotics
z

q4

q3

q2

Memory

−

σk
n(s)
O

q1

y

b(s)

x
φ (x

- +

ψ

fd

−

s

n(s)

P
e(s)

-

M
-

P
)=

-

J T (q)

E3

q, q̇

f

x, ẋ

−

ẋ

x

Δxk

-

0

PD feedback

b(s)

−
−

Δẋ

xd
ẋd

Figure 10: Handwriting on a hypersurface ϕ(x) = 0.
- −vkwhere G11 is of 1 × 1, G22 is of 2 × 2, and
G = P T GP,⎛

MemoryP = (W, J + (b, e)),
⎞

⎛

⎠,
G=⎝ T
G12 G22B=⎝

G11 G12B11 B12B21 B22

Figure 11: Iterative learning control scheme for robotic handwriting.

⎞
⎠,

(73)

1
B = G˙ + S + P T C0 P,
21
1
S = P T SP − Ṗ T GP + P T GṖ.
2
2
In (70), α is a function of C ∞ -class with respect to, η̇, η̈, γ̇, γ̈,and q. Thus, for a given desired trajectory of the tip of theball pen, γ̇(t) = ẋd (t) and γ̈(t) = ẍd (t), it is possible to provethe existence of qd and η̇(= η̇d ) that satisfies
⎛ ⎞
 η̇d
q̇d = P qd ⎝ ⎠,ẋd

G11 η̈d +(74)

1
Ġ11 + S11 η̇d + C11 η̇d
2







 

√

 
− J T q {vk + σk n(s)},where C11 = W T C0 W. The two equations (74) are the setof five simultaneous diﬀerential equations of the 1st orderin five variables of η̇d and qd . Once the desired trajectoryof joint motion qd (t) with its first and second derivatives isdetermined together with the motion η̇d (t) of excess-DOF,
the desired control signal vd is given by (72) by puttingγ̇ = ẋd , q = qd , and η̇ = η̇d . Moreover, if a desired pressing
force fd is specified, the desired control signal in the direction
normal to the surface can be constructed through (70) suchthatsection that there exists uniquely a control signal v(t) lying
in the image space that makes the endpoint of the robot
trace the curve γ(s(t)) on the hypersurface. Irrespective of
the redundancy of DOF, once a starting posture of the robot
is specified in the Euclidean space E3 with its endpoint
position P = x(0) on the hypersurface ϕ(x) = 0 as shownin Figure 10, the orbit qd (t) that realizes x(qd (t)) = xd (t)itself is determined uniquely correspondingly to the uniquely
existing control signal vd (t) (see Figure 10). Then, there
arises a problem whether it is possible to acquire such adesired control signal without knowing the details of therobot dynamics. In order to discuss the problem, let us
consider a learning control scheme that is expressed by thefollowing form of the control signal (see Figure 11):

uk = −J T q κ(x − xd ) + ζ κ(ẋ − ẋd )+ W T g qd + G12 ẍd + B12 ẋd = 0,



Φ



vn = fd − gn qd − α η̇d , η̈d , ẋd , ẍd .

(75)

8. What Should the Central Processing
Unit Learn?
Given a smooth curve γ(s(t)) on a hypersurface ϕ(x) = 0
over time interval t ∈ [0, T], it is shown in the previous(76)

where κ > 0 stands for a constant expressing the stiﬀness
parameter of position feedback and ζ > 0 a constant lying
in interval [0.5, 2.0] of R1 . The first term of the right-handside expresses the task space (E3 ) PD feedback for a given setof position and velocity (xd , ẋd ) and the second term stands
for the learning control signal at the kth operation that isconstructed as follows:
vk+1 = vk − ΦΔẋk ,σk+1 = σk + ψΔ fk ,(77)

where ψ is an appropriate constant such that 0 < ψ < 1and Φ is a positive-definite 3 × 3 constant matrix. In thecase of ordinary handwriting when the constraint surface isa horizontal plane, it is shown in the previous paper in [38]that the learning update law for the control signal based on(76) makes the motion of the endpoint trace the given curve
xd (t) on the plane and at the same time the joint motions

Journal of Robotics

13

Intention
Motor program(memory)Motor control
Perception

Sensory-motor
coordination

SpinalreflexVisionsensing

Tactile & force
sensing
Motor control
signalFigure 12: Sensory-motor coordination.

qk (t) and q̇k (t) converge to the ideal trajectories qd (t) andq̇d (t) as k → ∞. The proof of convergence of the control
signals vk to the desired one vd as k → ∞ is also given in[39] on the basis of the “passivity” of the input v and theoutput ẋ of the system dynamics of (63) provided that u isgiven through (64) and the gravity eﬀect g(q) is carefullycompensated. Some numerical simulation results are alsopresented in [39].
Returning to Bernstein’s problem discussed in the introductory section, we are fortunately able to quote Latash’scommentary [40] to the book originally written by Bernsteinin Russian cited as [2]. Latash says,
Bernstein’s definition of degrees of freedom relied heavily on
the analysis of kinematic degrees of freedom of various systems.
In particular, when analyzing the human upper limb, Bernsteinconsidered all possible orthogonal axes of rotation in a joint as
independent degrees of freedom, which were later summed over
the joints. The apparent redundancy of the joints of the humanarm in comparison to the three-dimensional space where we
happen to live and where movements take place led Bernstein
to his famous formulation that the essence of motor control
is the elimination of the redundant degrees of freedom. The
beauty and brevity of this formulation is stunning. A skeptic
may want to decide whether control can always be reduced
to the elimination of redundant degrees of freedom. In otherwords, I am suggesting that Bernstein’s famous definition may
not always be correct (blasphemy!!!)Even in the case of simple humanlike multijoint pointto-point reaching, Riemannian geometry suggests a goodreason to think that the central nervous system is functioning
according to a natural law of Newton’s mechanics, that is, thelaw of inertia for multijoint mechanisms. Once a geodesic
curve q∗ (t) connecting the given initial posture q0 andthe equilibrium submanifold NP (see (18)) is determinedwith some initial velocity q̇∗ (0), then the three-dimensional
orbit x(q∗ (t)) for t ∈ [0, T] is obtained. This orbit can be
rewritten into a three-dimensional curve γ(s) with the aid oflength parameter s for a corresponding interval s ∈ [a, b].

Then this curve can be spelled out by a desired trajectory
given as a movement of the tip of the arm in E3 in theform of xd (t) = γ(s(t)) for t ∈ [0, T] with the aid of ascale change of “time” through a monotonously increasing
function s(t). Once a desired orbit xd (t) of motion of a tipof the arm is given in E3 and at the same time an initial
posture of the arm is chosen, the trajectory of motion in
the joint space is uniquely determined without eliminating
any excess of the system’s DOF. However, it is important tonote that, even if there arises a small change of the initial
posture, say q0 + δq, it is possible to obtain the same desired
orbit xd (t) in E3 but the trajectory qd (t) of joints diﬀeres
slightly from that obtained when the initial posture is set as
q(0) = q0 and also the desired control signal vd (t) may diﬀer.
Another noteworthy characteristic of humanlike multijoint
movements is called “variability”, which was first pointed
out by Bernstein [2]. Latash [32] observed that in the caseof human skilled motion the grade of variability of each
endpoint trajectory is quite low relative to variable profiles
of joint responses at each trial of reaching. The mathematicalarguments in the previous section suggest that the main partof variability in joint space is caused by small fluctuations of
choice of an initial posture trial by trial and the others may
be noise.
We do not discuss the importance of redundancy of themuscles involved in a specific motion of multijoint reaching.
The central nervous system surely uses its own means ofcommunication with the muscles, which is not analogous to
the language of joint kinematics that specifies the trajectories
in individual joint. Instead of overcoming the problem of
redundancy of the muscles, we implicitly assume in thispaper that a desired torque of individual joint is finallygenerated by a total of forces of all the muscles involved inmovement of the joint. This premise may not be justified byany reason, but it is important to quote Jackson’s observation
as a neurophysiologist [41] (see Figure 12):“To speak figuratively, the central nervous system knows
nothing of muscles, it only knows movements.”“The highest centres represent all parts of the body, literally
all parts supplied by nerves.”

9. Conclusions
This paper discusses diﬃcult problems of control for humanlike robots with redundant DOFs from the standpoint of
Riemannian geometry. Irrespective of joint redundancy,
we have shown that there is no necessity to resolve theinverse kinematics problem by introducing any artificial
performance index and optimizing it in both the cases of(1) multijoint reaching movement with excess joints, and(2) handwriting through iterative learning control. The mostimportant conclusion is that Riemannian geometry does notcare about the redundancy of joints through which rigid
links are connected in series but it is powerful in establishinga correspondence between working-point trajectories inthe external three-dimensional space (E3 ) and movement
trajectories in a space of joint angles (a configuration space
Rn or a Riemannian manifold {M, gi j } as a set of whole
possible postures).

14

References[1] M. Brady, Robotics Research: The First International Symposium, MIT Press, Cambridge, Mass, USA, 1948.
[2] N. A. Bernstein, “On dexterity and its development,” in
Dexterity and Its Development, M. L. Latash and M. T. Turvey,
Eds., pp. 3–275, Lawrence Erlbaum Associates, Mahway, NJ,
USA, 1996.
[3] N. A. Bernstein, The Coordination and Regulation of Movements, Pergamon, London, UK, 1967.
[4] R. Murray, Z. Li, and S. S. Sastry, A Mathematical Introduction
to Robotic Manipulation, CRC Press, Boca Raton, Fla, USA,1994.
[5] G. Hinton, “Some computational solutions to Bernstein’s
problems,” in Human Motor Actions: Bernstein Reassessed, H.
T. A. Whiting, Ed., pp. 413–418, Elsevier, Amsterdam, The
Netherlands, 1984.
[6] A. G. Barto, R. S. Sutton, and C. W. Anderson, “Neuronlike
adaptive elements that can solve diﬃcult learning control
problems,” IEEE Transactions on Systems, Man and Cybernetics, vol. 13, no. 5, pp. 834–846, 1983.
[7] J. Morimoto and K. Doya, “Reinforcement learning ofdynamic motor sequence: learning to stand up,” in Proceedingsof the IEEE/RSJ International Conference on Intelligent Robots
and Systems, pp. 1721–1726, Victoria, Canada, October 1998.
[8] B. Sicilliano and O. Khatib, Eds., Springer Handbook of
Robotics, Springer, New York, NY, USA, 2008.
[9] J. C. Latombe, Ed., Robot Motion Planning, Kluwer Academic
Publishers, Norwell, Mass, USA, 1991.
[10] V. I. Arnold, Ed., Methematical Methods of Classical Mechanics,
Springer, New York, NY, USA, 2nd edition, 1989, translated by
K. Vogtmann and A. Weinstein.
[11] M. Takegaki and S. Arimoto, “A new feedback methodfor dynamic control of manipulators,” Journal of Dynamic
Systems, Measurement and Control, vol. 103, no. 2, pp. 119–
125, 1981.
[12] F. Bullo and A. D. Lewis, Eds., Geometric Control of Mechanical
Systems: Modeling, Analysis, and Design for Simple MechanicalControl Systems, Springer, New York, NY, USA, 2000.
[13] W. M. Oliva, Ed., Geometric Mechanics, vol. 1798 of Lecture
Notes in Mathematics, Springer, Berlin, Germany, 2002.
[14] S. Arimoto, M. Yoshida, M. Sekimoto, and K. Tahara, “Ariemannian-geometry approach for dynamics and control of
object manipulation under constraints,” in Proceedings of theIEEE International Conference on Robotics and Automation, pp.
1683–1690, Kobe, Japan, May 2009.
[15] S. Arimoto, “Modeling and control of multi-body mechanicalsystems part 1: a Riemannian geometry approach,” International Journal of Factory Automation, Robotics and Soft
Computing, no. 2, pp. 108–122, 2009.
[16] M. Sekimoto, S. Arimoto, S. Kawamura, and J.-H. Bae,
“Skilled-motion plannings of multi-body systems based upon
Riemannian distance,” in Proceedings of the IEEE International
Conference on Robotics and Automation, pp. 1233–1238,
Pasadena, Calif, USA, May 2008.
[17] S. Arimoto, Control Theory of Nonlinear Mechanical Systems:
A Passivity-Based and Circuit-Theoretic Approach, Oxford
University Press, Oxford, UK, 1996.
[18] J. Jost, Riemannian Geometry and Geometric Analysis,
Springer, Berlin, Germany, 2002.
[19] S. Gallot, D. Hulin, and J. Lafontaine, Rimennian Geometry,
Springer, Berlin, Germany, 2004.
[20] T. Yoshikawa, “Manipulability of robotic mechanisms,” International Journal of Robotics Research, vol. 4, no. 2, pp. 3–9,
1985.

Journal of Robotics[21] Y. Nakamura, Advanced Robotics: Redundancy and Optimization, Addison-Wesley, Reading, Mass, USA, 1991.
[22] D. E. Whitney, “Resolved motion rate control of manipulatorsand human protheses,” IEEE Transactions on Man-Machine
Systems, vol. 10, no. 2, pp. 47–53, 1969.
[23] O. Khatib, “A unified approach for motion and force control ofrobot manipulators: the operational space formulation,” IEEE
Journal of Robotics and Automation, vol. 3, no. 1, pp. 43–53,
1987.
[24] J. M. Hollerbach and K. C. Suh, “Redundancy resolution ofmanipulators through torque optimization,” IEEE Journal of
Robotics and Automation, vol. 3, no. 4, pp. 308–316, 1987.
[25] V. Potkonjak, S. Tzafestas, D. Kostic, G. Djordjevic, and
M. Rasic, “The handwriting problem,” IEEE Robotics andAutomation Magazine, vol. 10, no. 1, pp. 35–46, 2003.
[26] P. Morasso, “Spatial control of arm movements,” Experimental
Brain Research, vol. 42, no. 2, pp. 223–227, 1981.
[27] T. Flash and N. Hogan, “The coordination of arm movements:
an experimentally confirmed mathematical model,” Journal of
Neuroscience, vol. 5, no. 7, pp. 1688–1703, 1985.
[28] Y. Uno, M. Kawato, and R. Suzuki, “Formation and control
of optimal trajectory in human multijoint arm movement:
minimum torque-change model,” Biological Cybernetics, vol.
61, no. 2, pp. 89–101, 1989.
[29] S. Arimoto, M. Sekimoto, H. Hashiguchi, and R. Ozawa,“Natural resolution of ill-posedness of inverse kinematics
for redundant robots: a challenge to Bernstein’s degrees-offreedom problem,” Advanced Robotics, vol. 19, no. 4, pp. 401–
434, 2005.
[30] S. Arimoto, H. Hashiguchi, M. Sekimoto, and R. Ozawa,“Generation of natural motions for redundant multi-joint
system: a diﬀerential-geometric approach based upon theprinciple of least actions,” Journal of Robotic Systems, vol. 22,
no. 11, pp. 583–605, 2005.
[31] S. Arimoto and M. Sekimoto, “Human-like movements ofrobotic arms with redundant DOFs: virtual spring-damper
hypothesis to tackle the Bernstein problem,” in Proceedings ofthe IEEE International Conference on Robotics and Automation(ICRA ’06), vol. May 2006, pp. 1860–1866, Orlando, Fla, USA,2006.
[32] M. L. Latash, Neurophysiological Basis of Movement, Human
Kinetics, New York, NY, USA, 1998.
[33] J. Nakanishi, R. Cory, M. Mistry, J. Peters, and S. Schaal,“Comparative experiments on task space control with redundancy resolution,” in Proceedings of the IEEE/RSJ International
Conference on Intelligent Robots and Systems, pp. 1575–1582,
Edmonton, Canada, August 2005.
[34] J. M. Wintess and L. Stark, “Muscle models: what is gained
and what is lost by varying model complexty,” Biological
Cybernetics, vol. 55, no. 6, pp. 403–420, 1987.
[35] E. Bizzi, N. Hogan, F. A. Mussa-Ivaldi, and S. Giszter, “Doesthe nervous systems use equilibrium-point control to guidesingle and multiple joint movements,” Behavioral and Brain
Sciences, vol. 15, no. 4, pp. 603–613, 1992.
[36] M. Kawato and H. Gomi, “A computational model of four
regions of the cerebellum based on feedback-error learning,”
Biological Cybernetics, vol. 68, no. 2, pp. 95–103, 1992.
[37] N. Hogan, “The mechanics of multi-joint posture and movement control,” Biological Cybernetics, vol. 52, no. 5, pp. 315–
331, 1985.
[38] S. Arimoto, H. Hashiguchi, and R. Ozawa, “Simple control
method coping with a kinematically ill-posed inverse problem
of redundant robots: analysis in case of a handwriting robot,”
Asian Journal of Control, vol. 7, no. 2, pp. 112–123, 2005.

Journal of Robotics[39] S. Arimoto, M. Sekimoto, and S. Kawamura, “Task-space
iterative learning for redundant robotic systems: existence of atask-space control and convergence of learning,” SICE Journalof Control, Measurement, and System Integration, vol. 1, no. 4,
pp. 312–319, 2008.
[40] M. L. Latash, “The Bernstein problem: how does the central
nervous system make its choices,” in Dexterity and Its Development, pp. 227–303, Lawrence Erlbaum Associates, Mahway,
NJ, USA, 1996.
[41] M. L. Jackson, “Observations on the physiology and pathologyof hemi-chorea,” in John Hughlings Jackson: Selected Writings,
J. Taylor, G. Holmes, and F. M. R. Walshe, Eds., vol. 2, p. 400,
Hodder and Stoughton, London, UK, 1932.

15