Feb.18-Feb.20, 1996, B-Con Plaza, Beppu
Oita,JAPAN

Editor : Masanori Sugisaka
ISBN 4-99004ti2-6-9

Proceedings of International Symposium on

ARTIFICIAL LIFE AND ROBOTICS{AROB)
1st

Feb.18 - Feb.20, 1996, B-Con Plaza, Beppu
Oita, JAPAN

Editor: Masanori Sugisaka
ISBN 4-9900462-6-9

INTERNATIONAL SYMPOSIUM
ONARTIFICIAL LIFE AND ROBOTICS(AROB)ORGANIZED BY
Department of Electrical and Electronic Engineering
Oita University
Under The Sponsorship of Ministry of Education,
Science, Sports, and Culture, Japanese Government
CO-SPONSORED BY
Santa Fe lnstitute(USA)
SICE(Japan)
CO-OPERATED BY
ISICE
RSJ
IEEJ
EICE
IEEE CSS Tokyo Chapter
JRA
SUPPORTED BY
Oita Prefectural Government
Oita Municipal Government
Oita Chamber of Commerce and Industry
Beppu Municipal Government
Oita System Control Society
NHK-Oita
OBS Broadcast Company
TOS Broadcast Company
OAB Broadcast Company
Asahi Sinbunsya
Nikkankougyou Sinbunsya
Oitagoudou Sinbunsya
MITI(under negotiation)
Science and Technology Agency

INTERNATIONAL ORGANIZING COMMITTEE
Prof. J.L.Casti (Santa Fe Institute, USA)
Prof. C.G.Langton (Santa Fe Institute, USA)
Prof. W.B.Arthur (Santa Fe Institute, USA)
Prof. J.M.Epstein (Brookings Institution, USA)
Prof. S.Rasumussen (Santa Fe Institute, USA)
Prof. T.S.Ray (ATR, JAPAN)
Dr. T.Gomi(AAI,Canada)Prof. M. Raibert(MIT, USA)
Prof. C. Looney(Univ. of Nevada-Reno, USA)
Prof. A.P.Wang (Arizona State Univ., USA)
Prof. H.H.Natsuyama(California State Univ., USA)
Prof. R.E.Kalaba (Univ. of Southern California, USA)
Prof. W.R.Wells (Univ. of Nevada-Las Vegas, USA)
Prof. D.J.G.Jame:; (Coventry University, UK)
Prof. W.R.Wells (Univ. of Nevada-Las Vegas, USA)
Prof. Y.G.Zhang (Academia Sinica, CffiNA)
Prof. J.J.Lee (KAIST, KOREA)
Prof. G.I.Marchuk (Russian Academy of Science:;, RUSSIA)
Prof. S.Ueno (Kyoto School of Computer Science, JAPAN)
Prof. S.Fujimura (Tokyo University, JAPAN)
Prof. H.Miura (Tokyo University, JAPAN)
Prof. S.Arimoto(Tokyo University, JAPAN)
Prof. Y.Nishikawa (Kyoto University, JAPAN)
Prof. S.Kitamura (Kobe University, JAPAN)
Prof. K.Tsuchiya (Kyoto University, JAPAN)
Prof. T.Jinzenji (Tohoku University, JAPAN)
Prof. K.Abe(Tohoku Uµiversity, JAPAN)
Prof. H.Hagiwara (Kyoto School of Computer Science, JAPAN)
Prof. H.Tanaka (Tokyo Medical & Dental University, JAPAN)
Prof. T.Mushya (Tokyo Science University, JAPAN)
Prof. T.Fukuda (Nagoya University, JAPAN)
Dr. K.Mastuno(MITI, AIST, JAPAN)
Dr. K.Tamura(MITI, AIST, JAPAN)
Dr. Y.Tokura(ATR, JAPAN)
Dr. K.Shimohara(ATR, JAPAN)
Dr. K.Kyuma(Mitsubishi Electric Co., JAPAN)
Prof. T.Yamakawa (Kyushu Institute of Technology, JAPAN)
Prof. T.Nagata(Kyushu University, JAPAN)
Prof. M.Nakamura (Saga University, JAPAN)
Prof. H.Kashiwagi (Kumamoto University, JAPAN)
Prof. M.Sugisaka (Oita University, JAPAN) (Chairman)

STEERING COMMITTEE
Prof. J.L.Casti
Prof. S.Ueno
Prof. S.Fujimura
Dr. Shibata
Dr. Nanjo
Prof. M.Sugisaka (Chairman)
Oita University Steering CommittePROGRAM COMMITTEE
Prof. S.Fujimura(Chairman)
Prof. J.L.Casti
Prof. S.Ueno
Prof. K.Aihara
Prof. T.Ohmori
Prof. K.Shimohara
Prof. S.Arimoto
Prof. H.Tanaka
Prof. Y.Nishikawa
Prof. T.Fukuda
Prof. K.Tsuchiya
Prof. S.Ando
Prof. T.Nagata
Prof. T.Yamakawa
Prof. S.Kitamura
Prof. I.Harvey
Prof. P.Husband
Prof. R.Pfeifer
Prof. T.Nanya
Prof. M.Sugisaka

TOPICS
Hardware Oriented Topics are welcomein the fields given by
Artificial Brains
Artificial Intelligence
Artificial Life
Artificial Livings
Artificial Mind
Chaos
Cognitive Science
Complexities
Fuzzy Control
Genetic Algorithms
Innovative Computations
Neural Networks
Neurocomputers
Neuro Computing Technologies
and Its Application for Hardware
Robotics
Virtual Reality
Related Fields

COPYRIGHT
Copyright@ 1996 by the International Society for Artificial Lifeand Robotics(ISAROB). All right reserved, Reproduction in any
form without the written consent of ISCA is prohibited.

All correspondence relating to the symposium
should be addressed to:Prof. Masanori Sugisaka
Chairman of International Symposium on Artificial Life and Robotics(AROB)
Dept. of Electrical and Electronic Engineering
Oita University
700 Dannoharu, Oita 870-11
JAPAN
TEL 001-81-975-54-7831
FAX 001-81-975-54-7620
EMAIL msugi@cc.oita-u.ac.jp

PREFACE
Masanori Sugisaka
Chairman of ARO B(Professor, Oita University)

It is my great honor to invite you all to The First International Symposium on Artificial Lifeand Robotics(AROB 1st), organized by Oita University under the sponsorship of Ministry of
Education, Science, Sports, and Culture, Japanese Government and co-sponsored by Santa Fe
Institute(SFI), USA and SICE, Japan. This symposium invites you all to discuss developmentof new technology concerning ALife and Robotics using new devices and technologies such asneurocomputer etc., based on simulation and hardware in the field of Microworld Simulation
and Realities in 21st century.
This symposium is also financially supported by not only Ministry of Education but also
Oita Prefectural Government, Oita Chamber of Commerce and Industry, and other privatecompanies. Prof. Casti introduced SFI research fields during our discussions on joint research
two years ago. This symposium was motivated by the discussions. I'd like to express my sincere
thanks for Prof. J. L. Casti and also, S. Fujimura, S. Ueno, SFI Professors and all people who
contributed to the symposium. I hope that you all will enjoy staying in Beppu, Oita, profit
from AROB 1st, and look forward to meeting you in Beppu.

INTERNATIONAL SOCIETY FORARTIFICIAL LIFE AND ROBOTICS(ISAROB)
Dept. of Electrical and Electronic Eng.
Oita University
700 Dannoharu, Oita 870-11, Japan
Ph. +81-975-54-7831 Fax +81-975-54-7820

The First International Symposium
on Artificial Life and Robotics(AROB 1st)
Feb. 18-20, 1996, B-Con Plaza, Beppu, Oita, JAPANTECHNICAL PAPER INDEXPlenary Session: Invited Thlks
The Science and the Surprise of Artificial Worlds
J. L. Casti(Santa Fe Institute, USA, Technical University of Viena)
The SWARM simulation system for modelling complex systems.
C. G. Langton(Santa Fe Institute, Director, USA)
Numerical Analysis of the State of Mind
T.Musha(Brain Function Laboratory, Inc., President
and Keio University, Special Invited Professor, Japan)
Evolution of Digital Organisms
T. Ray(ATR Human Information Processing Research Laboratories,
Japan and USA)
Insect-Model Based Microrobot
H.Miura(University of Tokyo, Japan)
The Economy as an Ocean of Co-Evolving Beliefs
W. B. Arthur (Santa Fe Institute, Dean of Economics,
Stanford University, USA)
Growing Artificial Societies: The Sugarscope Model
J.M. Epstein(Santa Fe Institute, The Brookings Institution, USA)

Dynamical Hierarchies
S. Rasmussen (Santa Fe Institute, Los Alamos National
Laboratory, USA)

Session Al: Artificial Life
Al-1 Computational Approach to Consciousness: Continuous Self-Monitoring, Identification,
Evaluation and Decision
Y.Ishida(Nara Institute of Science and Technology, Japan)
Al-2 A Robot with a Personality-a fuzzy adaptive model of emotion and personality for
an autonomous robotK. H. Ho, N. Ohnishi(The Institute of Phisical andChemical Research(RJKEN), Japan)
Al-3 Study of Loop Structure on Abstract Rewriting System and its Application
to Artificial Life
Y. Suzuki, S. Tsumoto, H. Tanaka(Tokyo Medical and Dental University, Japan)
Al-4 An Advanced Simulation Scheme for Electric Railway Power Systems
Based on Artificial Life Approach
T. Jinzenji, M. Sasaki(Tohoku University, Japan)
Al-5 Adaptive Block Dividing and Quantization for Image Compression using
Artificial Breeding
H. Hiramatsu, S. Kit�ura(Kobe University, Japan)
Y. Kurematsu(College of Industry Technology, Japan)

Session A2: Neural Network
A2-1 Human 'l'ype Controller Using Combined Neurons of Different 'l'ype
T. D. Eom, J. J. Lee(Korea.Advanced Institute of S cience. and Technology(KAIST), KQrea)
A2-2 Neurocomputer Control for '.lracking of Moving Objects
M. S ugisaka(Oita University,.Japan)
A2-3 Reinforcement Learning of Dynamic Behavior by using Recurrent Neural Networks
A. Oi;tat,. H. I<:ita, Y. Nishikawa(Kyoto Uajversity, Japan)
A2-4 Pattern Recognition Via Neurocomputer
M. S ugisaka(Oita University, Japan)

A2-5 Classification of Landsat TM Data Using Neural Network App roaches
Y. Kawata, Y. Yashima(Kanazawa Institute of Technology, Japan)
A2-6 Fault Diagnosis of Logical Circuit by use of Pseudorandom signal and
Neural Network

H. Kashiwagi, M. Sakata(Kumamoto University, Japan)

Session Bl: Robotics-GeneralBI-I A Language for Describing Motions of Mechatronic Systems :A Nonlinear
Position-Dependent Circuit Theory

S. Arimoto(University of Tokyo, Japan)
BI-2 Aspects of Non-Cartesian Robotics
T. Gomi(Applied AI Systems, Inc., Canada)BI-3 Evaluation of Robust Control by Universal Leaming Network
M. Ohbayashi, K. Hirasawa, J. Murata (Kyusyu University, Japan)
BI-4 Cooperative Behavior of Interacting Robots
K. Sugawara, M. Sano, Y. Sawada (Tohoku University, Japan)
BI-5 A Step Toward Human-Robot Cooperative System
M. Ishii, H. Saita(Fukuoka Institute of Technology, Japan)
BI-6 A Method of Gait Coordination of Hexapod Robots Using Mutual-Coupled
Immune Networks

A. Ishiguro, S. Kuboshiki, S. Ichikawa, Y. Uchikawa(Nagoya University, Japan)

Session B2: Genetic Algorithm
Chair: J. J. Lee(KAIST, Korea)

B2-I A Control Design Method of Robotic System by Genetic Algorithm
T. Tsuji, T. Hachino(Kyusyu Institute of Technology, Japan)
R. Oguro, N. Umeda(YAS UKAWA Electric MFG Co., LTD, Japan)
H. Takata(Kagoshima University, Japan)
B2-2 Robot Path Planning through Some Places by Genetic Programming
H. Yamamoto(Wakayama University, Japan)
B2-3 A Local Motion Planner for Unicycle-like Vehicles:
Guaranteeing the Collsion Avoidance Even in the Unknown Environment

S. G. Hong, K. B. Park, S. W. Kim, J. J. Lee(KAIST, Korea)

B2-4 An Optimization Method of a Combinatorial Optimization Problem usinga Replicator Equation ,K. Tsuchiya(Kyoto University, Japan)
K. Tsujita, Y. Inoue(Osaka University, Japan)
B2-5 Nonlinear Numerical Optimization with Use of a Hybrid Genetic Algorithm
Incorporated the Modified Powell Method
M. Okamoto, T. Nonaka, S. Ochiai, D. Tominaga(Kyushu Institute of Technology, Japan)
B2-6 A Pamllel Genetic Algorithm for Matching and Similarity Measure of Graphs
Y. Wang, N. Ishii(Nagoya Institute of Technology, Japan)

Session A3: Fuzzy and Neural Network
A3-1 A Fuzzy Classifier System for Evolutionary Learning of Robot Behaviors
Y. Iwakoshi(Chyubu Hitachi Electric Co., Ltd, Japan)
T. Furuhashi, Y. Uchikawa(Nagoya University, Japan)
A3-2 A Fuzzy-Logic Approach to Industrial Control Problems
D. J. G. James, K. J. Bumham(Coventry University, UK)
A3-3 Computational Analysis of Acquired Dyslexia of Chinese Characters based on
Neural Networks
S. Tsumoto, H. Tanaka(Tokyo Medical and Dental University, Japan)
A3-4 The Improved Radial Basis Function Neural Network and Its Application
X. Wang, H. Shao(Shanghai Jiao Tong University, China)
A3-5 A Temporal Radial Basis Function Neuro-'l'racker
C. G. Looney(University of Nevada, USA)

Session A4: Artificial Sensory Organ
A4-1 Artificial Retina Chip
K. Kyuma, J. Ohta(Mitsubishi Electric Corporation, Japan)
A4-2 Vision Chip Architecture With Light Adaptation Mechanism
H. Kobayashi(Yokogawa Electric Corp., Japan)
T. Matsumoto, K. Tanaka(Waseda University, Japan)
T .. Yagi(Kyusyu Institute of Technology, Japan)
A+3 Artificial Passive Vision Using High Speed Pixel Matching
H. Hanaizumi, T. Tsumashima(Hosei University, Japan)
F. Fujimura(University of Tokyo, Japan)

A4-4 Artificial Active Vision Using Spatial Coding by m-sequence
S. Fujimura, S. Kiyasu(University of Tokyo, Japan)
A4-5 Building Ears for Robot : Sound Localization and Separation
J. Huang, N. Ohnishi, N. Sugie(RIKEN, Japan )A4-6 Artificial Plant Trees Based on IFS
X. Li(University of Aeronautics and Astronautics, China)
M. Sugisaka(Oita University, Japan)
Y. G. Zhang(Academia Sinica, China)

Session B3 : Mobil Robot and Vehicle
B3-1 Dynamic Behavior Arbitration for Autonomow Mobile Robots
Using Immune Networks-Evolution of Affinities among Antibodies­

A. Ishiguro, T. Kondo, Y. Watanabe, Y. Uchikawa(Nagoya University, Japan)
B3-2 Auto Navigation on the wheel Chair
H. Wang, C. Kang, T. Ishimatu(Nagasaki University, Japan)
T. Ochiai(Ube Technical College, Japan)
B3-3 An Emergence of Fuzzy Control Rules for Mobile Robots
wing DNA Coding Method

T. Yoshikawa, T. Furuhashi, Y. Uchikawa(Nagoya University, Japan)
B 3-4 Experimental Study on Autonomow Mobile Robot

Acquiring Optimal Action to Avoid Moving Obstacles

T. Aoki(Nagoya Municipal Industrial Research Institute, Japan)
T. Oka, T. Suzuki, S. Okuma(Nagoya University, Japan)

B3-5 The Development of A Practical Control Method foran Intelligent Mobile Vehicle

M. Sugisaka, X. Wang(Oita University, Japan)
J. J. Lee(KAIST, Korea)
B3-6 Development of A Stair Climbing Mobile Robot for Disabled and Elderly People
T. Mabuchi, T. Yamada, K. Shiraki, K. Awa, T. Nagasawa(Kansai University, Japan)

Session B4: Bio and Bio-Mimetic
B4-1 Linear Systems Analysis of Amino Acids Production
H . Hirayama, Y. Ishikawa, T. Nozawa, K. L. We, Y. Fukuyama(Asahikawa Medical College, Japan)

B4-2 Optimization of Growth of Peptide Chain by Ribosomes on a Messenger RNA
H. Hirayama, Y. Fukuyama(Asahikawa Medical College, Japan)
B4-3 Optimal Control in Active Transport in Biological Membrane
H. Hirayama, Y. Fukuyama(Asahikawa Medical College, Japan)
B4-4 Identification of a Transport System
A. P. Wang(Arizona State University, USA)
S. Ueno(Kyoto School of Computer Science, Japan)

Session A5: Evolution
A5-1 The Artificial Evolution Robot Control Systems
P. Husbands(University of Sussex, UK)
A5-2 Hardware Evolution -a real "life on the silicon"
H. Hemmi, K. Shimohara(ATR Human Information Processing Research Labs, Japan)
A5-3 Adaptive Evolution of Halon Networks by an Autonomous
Decentralized Method

N. Honma, K.Abe(Tohoku University, Japan)
M. Sato(Tohoku Institute of Technology, Japan)
H.Takeda( Tohoku Gakuin University, Japan)

A5-4 CAM-BRAIN The Evolutionary Engineering of a Billion Neuron Artificial
Brain by 2001 which Grows/Evolves at Electronic Speeds inside a Cellular
Automata Machine(CAM)

H. D. Garis(ATR Human Information Processing Research Labs,
Japan)

A5-5 Evolution of Locally Defined Learning Rules and Their Coordinationin Feedforward Neural Networks

H. Murao, S. Kitamura(Kobe University, Japan)

A5-6 Artificial Evolution and Real Robots
I. Harvey(University of Sussex, UK)Panel Discussion A6
Title: Coexistance between human being and robots
including artificial life
Chair: M. Sugisaka(Oita University, Japan}
J. Casti: Santa Fe Institute(Prof. Technical Univ. of Viena)
C. Langton: Santa Fe Institute
W.R. Wells: Univ. of Nevada-Las Vegas (Dean of Engineering)
J. M. Epstein: Santa Fe Institute (The Brookings Institution)
S. Rasumussen: Santa Fe Institute (Los Alamos National Laboratory)
F. Ota: Toshiba Corporation Oita Works(General Manager)
H. Miura: University of TokyoT. Yamakawa: Kyusyu Insutitute of Technology (Dean of Faculty of Information
Engineering)
S. Ueno: Kyoto School of Computer Science Information Science Laboratory(Emeritus Professor of Kyoto University)

Session B5: Manipulation and Autonomous-' System
B5-l Fine Motion Strategy for Skill-Based Manipulation
A. Nakamura, T. Ogasawara, H. Tsukune(Electrotechnical Laboratory, Japan)
T. Suehiro(Real World Computing Partnership, Japan)
B5-2 Artificial Realization of Human Skill for Robot Manipulator Control
M. Nakamura, S. Goto(Saga University, Japan)
N. Kyura(Kinki University, Japan)
B5-3 Spontaneous Behavior for Cooperation of Distributed Autonomous Robots
T. Shibata(Mechanical Engineering Laboratory, AIST, MITI, Japan,
Artificial Intelligence Laboratory, MIT, USA)
K. Ohkawa(Tsukuba University, Japan)(Mechanical Engineering Laboratory, AIST, MITI, Japan)
B5-4 Design Principles of Autonomous Agents: A Case Study of Classification
R. Pfeifer(U niversity of Zurich-Irchel, Switzerland)
B5-5 Distributed and Autonomous Sensing based on Immune Network
Y. Ishida(Nara Institute of Science and Technology, Japan)
B5-6 Force/Position Control of Robot Manipulator via Motion Dynamics
S. Jeong, K. Im, J. Kim, W.Ham(Chonbuk National University, Korea)

Session B6: Chaos and Robots
B6-1 Associative Readout from a Chaos Network enjoying Chaotic ltinerancy
T. Yamakawa(Kyusyu Institute of Technology, Japan)
B6-2 Chaotic Local Search Algorithm
C. Choi, J. J. Lee(KAIST, Korea)
B6-3 Chaos Control on Multi-Branch U.L.N.
M. Koga, K. Hirasawa, J. Murata, M. Ohbayashi(Kyusyu University, Japan)
B6-4 Autonomous Micro-Robot "Kity" for Maze Contest
J. H. Kim, M. J. Chung, H. S. Shin, S. W . Lee(KAIST, Korea)
B6-5 Deadlock Resolution Using Hand-to-Hand Motion in Group Robots
H. Ishihara, T. Fukuda, F. Arai, N. Hiraoka(Nagoya University, Japan)

Session Cl: Self-Organization
Cl-1 Self-Assembly Method for Mechanical Structure
S. Murata, H. Kurokawa, K. Tomita, S.Kokaji(Mechanical Engineering Laboratory, AIST, MITI, Japan)
Cl-2 A Self Organizing Evolutionary Algorithm
I. K. Jeong, J. J. Lee(KAIST, Korea)
Cl-3 A Self-Organization Leaming Algor ithm for Visuo-Motor Coordinationin Unstructured Environments

H.B. Zha, T. Onitsuka, T. Nagata(Kyusyu University, Japan)
Cl-4 Proposal of Decentralized Consensus-Making Mechanisms Based on
Immune System

A. Ishiguro, Y. Watanabe, Y. Uchikawa(Nagoya University, Japan)
Cl-5 Towords Designing Artificial Neural Networks by Evolution
X. Yao, Y. Liu(University of South Wales, Australia)

Would-Be Worlds: Toward a Theory of Complex Systems
John L. Casti
Santa Fe Institute1399 Hyde Park Road
Santa Fe, NM 87501, USA
Abstract
By their very nature, complex systems resist anal­
ysis by decomposition. It is just not possible to study,
say, the human immune system or a stock market,
by breaking it up into individual parts-molecules ortraders-and looking at what these parts do in isola­tion. The very essence of the system lies in the inter­action among all its parts, with the overall behaviorof the system emerging from these interactions. So by
throwing away the interactions, one also throws away
any hope of actually understanding the workings ofthe system. The problem is that until very recently,
there was no way of studying these sorts of systems ascomplete entities, since to do experiments with stock
markets, immune systems, rainforest ecosystems andthe like was either too expensive, too dangerous or justplain too difficult. But the arrival of cheap, powerful,widespread computing capability over the past decadeor so has changed the situation entirely.
This talk will examine the way in which the ability
to create surrogate versions of real complex systems
inside our computing machines changes the way we doscience. In particular, emphasis will be laid upon theidea that these so-called "artificial worlds" play therole of laboratories for complex systems, laboratories
that are completely analogous to the more familiar lab­
oratories that have been used by physicists, biologists
and chemists for centuries to understand the workingsof matter. But now we have laboratories that allow us
to explore information instead of matter. And since
the ability to do controlled, repeatable experiments isa necessary precondition to the creation of a scientifictheory of anything, the argument will be made thatfor perhaps the first time in history, we are now in aposition to realistically think about the creation of atheory of complex systems.
These philosophical points will be illustrated by on­
going work with artificial road-traffic networks, as wellas with systems for studying social and cultural phe­nomena.

1

Introduction

By more-or-less common consensus, Galileo is cred­ited with ushering-in the idea of controlled, repeatable,
laboratory experiments for the study of physical sys­tems. And as such experiments are an integral part ofthe so-called scientific method, it's no exaggeration
to say that Galileo's work formed a necessary precon­
dition for Newton's creation of a workable theory ofsystems composed of interacting particles, a theory
that formed the basis for much of modern theoreti­
cal science. But Newton's particle systems are what
in today's parlance we would term "simple" systems,
since for the most part they are formed of either a verysmall or a very large number of interacting "agents"(i.e., particles) interacting on the basis of purely local
information in accordance with rigid, unvarying rules.
Complex systems are different.
Typically, complex systems like a stock market or aroad-traffic network involve a medium-sized number of
agents (traders or drivers) interacting on the basis oflimited, partial information. And, most importantly,
these agents are intelligent and adaptive. Their be­
havior is determined by rules, just like that of planetsor molecules. But the agents are ready to change theirrules in accordance with new information that comes
their way, thus continually adapting to their environ­ment so as to prolong their own survival in the system.
At present, there exists no decent mathematical the­ory of such processes. One part of the argument tobe made here is that a major stumbling block in thecreation of a theory of complex, adaptive systems has
been the lack of ability to do the kind of controlled,
repeatable experiments that led to theories of simple
systems. The second-half of our argument is that the
microsimulations, or "would-be worlds," presented at
this meeting constitute nothing less than laboratories
for carrying out just such experiments. So for the firsttime in history, we have the experimental tools withwhich to begin the creation of a bona fide theory ofcomplex, adaptive systems.

2

Theories, Experiments, and "Big
Problems"

To see the role that microsimulations will play inthe creation of a theoretical framework for complex
systems, it's instructive to examine briefly the historyof theory construction for several major areas of mod­ern science.
Typically, a theory of something begins its life withwhat I'll call a "Big Problem." This is some question
about the world of nature or humans that cries out
for an answer, and that seems approachable by theconcepts and tools of its time. Just to get a feel forwhat such questions are like, here is a rather eclectic
list of Big Problems from a few areas of natural andhuman affairs:• Biology: The Structure of DNA-What is thegeometrical structure of the DNA molecule, and how
does this structure lead to the processes of heredity?
• Astrophysics: The Expanding Universe-ls
the Universe open or closed, i.e., will it continue toexpand forever, or will a phase of contraction back toa "Big Crunch" occur?
• Economics: Equilibrium Price&--ln a pure ex­change economy, does there exist a set of prices atwhich all consumers and suppliers are satisfied, i.e., isthere a set of prices for goods in the economy at which
the supply and demand are in balance?
• Physics: Stability of the Solar System-Does
there exist a finite time in the future at which either
there will be a planetary collision, or at which someplanet attains a velocity great enough to escape thesolar system?
So what we have here are four questions about thereal world, each of which arises pretty much from
opening our eyes and looking around. And each ofthese questions has given rise to a theoretical frame­
work within which we can at least ask-if not answer­
the question. But these theoretical frameworks, bethey the theory of knots for studying the geometryof DNA or the fixed-point theories of economics thattell us about prices, have each come about as the out­
growth of experiments with the system of interest.
For example, it was only by having access to the x­
ray crystallographic studies by Rosalind Franklin that
James Watson and Francis Crick were able to uncover
the double-helix structure of DNA. Similarly, observa­
tions by Edwin Hubble using at the Mount Palomar
Observatory showed the expansion of the universe, anempirical fact that has led to current theories of dark

matter for answering the question of whether or not
this expansion will continue indefinitely.
These examples-and the list could be extended
almost indefinitely-illustrate the so-called scientificmethod in action. It consists of four main steps:-+ observation -+ theory -+ hypothesis -+ experiment -+This diagram makes the importance of experimen­
tation evident; in order to test hypotheses suggested
by a theory, we must have the ability to perform
controlled, repeatable experiments. And this is ex­
actly where the microsimulations possible using to­
day's computing machines enter into our discussion.
In contrast to the more familiar laboratories of thechemist, physicist or biologist, which are devoted to
exploring the material structure of simple systems,
the computer-as-a-laboratory is a device by which we
can probe the informational structure of complex
systems. Let me look at this point just a bit further.

3

Information versus Matter

For the past 300 years or more, science has fo­
cused on understanding the material structure of sys­tems. This has been evidenced by the primacy ofphysics as the science par excellence, with its con­cern for what things are made of. The most basic
fact about science in the 21st century will be the re­placement of matter by information. What this meansis that the central focus will shift from the material
composition of systems-what they are-to their func­
tional characteristics-what they do. The ascendancyof fields like artificial intelligence, cognitive science,and now artificial life are just tips of this iceberg.
But to create scientific theories of the func­
tional/informational structure of a system requires
employment of a totally different type of laboratory
than one filled with retorts, test tubes or bunsen burn­ers. Rather than these labs and their equipment de­signed to probe the material structure of objects, we
now require laboratories that allow us to study the way
components of systems are connected, what happens
when we add/subtract connections, and in general, ex­periment with how individual agents interact to create
emergent, global behavioral patterns.
Not only are these "information labs" different fromtheir "matter labs" counterparts. There is a further
distinction to be made even within the class of in­formation labs. Just as even the most well-equipped
chemistry lab will help not one bit in examining thematerial structure of, say, a frog or a proton, a would­
be world designed to explore traders in a financial mar­ket will shed little, if any, light on molecular evolution.
So let me conclude this short discussion by consider­ing some would-be worlds, each each having its own
characteristic sets of questions that it's designed toaddress.

4

Would-Be Worlds

In the past few years, a number of electronic worlds
have been created by researchers associated with the
Santa Fe Institute to study the properties of complex,
adaptive systems. Let me cite just three such worlds
here as prototypical examples of the kind of informa­tion laboratory we have been discussing.
• Tierra-This world, created by naturalist Tom
Ray [1], is populated by binary strings that serve aselectronic surrogates for genetic material. As time un­folds, these strings compete with each other for re­sources, with which they create copies of themselves.
New strings are also created by computational coun­terparts of the real-world processes of mutation andcrossover. Over the course of time, the world of Tierra
displays many of the features associated with evolu­
tionary processes seen in the natural world, and hence
can be used as a way of experimenting with suchprocesses-without having to wait millions of years tobring the experiment to a conclusion. But it's im­
portant to keep in mind that Tierra is not designed
to mimic any particular real-world biological process;rather, it is a laboratory within which to study neo­
darwinian evolution, in general.
• TRANSIMS-For the past three years, a teamof researchers at the Los Alamos National Laboratory
headed by Chris Barrett has built an electronic coun­terpart of the city of Albuquerque, New Mexico insidetheir computers. The purpose of this-world, which iscalled TRANSIMS, is to provide a testbed for study­
ing the flow of road traffic in an urban area of nearly
half a million people. In contrast to Tierra, TRAN­
SIMS is explicitly designed to mirror as the real worldof Albuquerque as faithfully as possible, or at least to
mirror those aspects of the city that are relevant forroad-traffic flow. Thus, the simulation contains the en­tire road traffic network from freeways to back alleys,
together with information about where people live andwork, as well as demographic information about in­comes, children, type of cars and so forth. So here wehave a would-be world whose goal is to indeed dupli-

cate as closely as possible a specific real-world situa­tion.
• Sugarscape---Somewhere in between Tierra andTRANSIMS is the would-be world called Sug­arscape, which was created by Joshua Epstein and

Rob Axtell of The Brookings Institution in Washing­
ton, DC. This world [2] is designed as a tool by whichto study processes of cultural and economic evolution.
On the one hand, the assumptions about how individ­
uals behave and the spectrum of possible actions attheir disposal is a vast simplification of the possibilities
open to real people as they go through everyday life.
On the other hand, Sugarscape makes fairly realistic
assumptions about the things that motivate people toact in the way they do, as well as about how they go
about trying to attain their goals. What is of con­siderable interest is the rich variety of behaviors thatemerge from simple rules for individual action, and theuncanny resemblance these emergent behaviors haveto what's actually seen in real life.
The main point of bringing up Tierra, TRAN­
SIMS, and Sugarscape is to emphasize two points:(A) We need different types of would-be worlds tostudy different sorts of questions, and (B) each of theseworlds has the capability of serving as a laboratory
within which to test hypotheses about the phenomena
they can represent. And, of course, it is this latter
property that encourages the view that such computa­
tional universes will play the same role for the creationof theories of complex systems that chemistry labs andparticle accelerators have played in the creation of sci­
entific theories of simple systems. For a fuller accountof the technical, philosophical and theoretical prob­
lems surrounding the construction and use of thesesilicon worlds, see the author's volume [3] which willappear in the fall of 1996.

References[1] Ray, T., An Approach to the Synthesis of Life," inArtificial Life-II, C. Langton et al, eds. Addison­
Wesley, 1991, pp. 371-408.
[2] Epstein, J. and R. Axtell, Growing Societies,
MIT Press, to appear 1996.
[3] Casti, J., Would-Be Worlds, John Wiley & Sons,
to appear 1996.

Numerical Analysis of the State of Mind
Toshimitsu MUSHA
Brain Functions Laboratory, Inc.
KSP Bldg. E-211, Sakado, Takatsu, Kawasaki, Kanagawa-ken, JAPAN

Abstract
The state of mind is supported by thebrain activity, and hence features of thestate of mind appear in the scalp potentials
or
EEGs(Electroencephalo-gram).
Therefore, it is possible to estimate thestate of mind by proper mathematical ma­
nipulations on spatio-temporal behavior of
EEGs. The emotional state is decomposed
into more elementary states. Currently tenelectrodes are used and the four elemen­tary states, anger, sadness, joy, and re­laxation, are adopted. More electrodes
would allow more elementary emotional
states to be included in the analysis. The
features are found in the cross-correlation
coefficients of 45 pairs of EEG channels in
the theta (5-8 Hz), the alpha (8-13 Hz),and the beta (13-20 Hz) bands. The totally
135 variables are obtained and they arelinearly combined into four components
which indicate levels of the four elemen­tary emotional states. The maximum time
resolution of the emotion analysis is 0.64
second and it is done in real time. Thisnew technique has a wide variety in the
medical and non-medical areas. This newtechnology suggests a possibility of directcontrol of systems by the human emotional
state.
1. Introduction
The scalp potential (electroencephalogram, EEG) is generated byelectric activities of neurons in the brain. It
is rich in information about the state of thebrain, or in other words about the state ofmind. Features of the state of mind, how-ever, are buried in the spontaneous EEG
generated by the other brain activities, andsophisticated mathematical manipulations
will be required to pick up the requiredfeatures from the unwanted ones. Such
event-related potentials are obtained bysynchronous averaging of EEG by repeat­
ing the same events. However, synchro­nous averaging is not possible in estimat­
ing the state of mind or the emotional statebecause always we have to analyze single
events. The feature extraction of the emo­
tional state need to be done through single
events ..
We have solved this problem as de­scribed in Sec. 2. The emotional state isdecomposed into four elementary states asanger, sadness, joy, and mental relaxa­
tion. This technique is similar to spectral
analysis and we have given to it the nameof Emotion Spectrum Analysis, or ESA inabbreviation. The present method ex­presses the state of the human emotion
numerically, which would allow manipula­tion of a non-human machine according to
the state of mind.
2. Method of Analysis
Ten disk electrodes are placed on thescalp at positions FP1, FP2, F3, F4, T3, T4,
P3, P4, 01, and 02 according to the Inter­national 10-20 Standard, and scalp poten­
tials are recorded with a reference elec­
trode on the right ear-lobe. The electric
potentials are sampled at 100 Hz, and thenseparated in the theta, alpha and beta fre­quency bands by means of FFT. Values of
the cross-correlation coefficients on 45 (=10C2) channel pairs are evaluated in every

5.12 seconds (it is possible to shorten thistime down to 0.64 second); totally, 135such variables are obtained. The set ofthese 13 5 variables is the input vector y;this is linearly transformed to a 4-vector z= (z1, z2, Z3, z4) by operating a transforma­tion matrix Cony. Magnitudes of thesecomponents indicate levels of these ele­mentary emotional states. We call C theemotion matrix, and z the emotion vector;
they are related as

C.y + d =z(1)where dis a constant vector.
Numerical values of the emotion ma­
trix elements were obtained in the follow­
ing way. Seven people who have been welltrained in imaging participated in preparing
the emotion matrix. They first imaged
anger, and 5.12-sec EEG segments were
cut out from their ten-channel EEGs; thisprocess was repeated for all the other ele­mentary emotional states. The numericalvalues of the matrix elements were deter­
mined in such a way that z = (1, 0, 0, 0), (0,1, 0, 0), (0, 0, 1, 0) and (0, 0, 0, 1) for thefour emotional states, respectively, and (0,0, 0, 0) for the control state in which no
special emotion was activated; numericalsolutions are obtained in either under­
determined or over-determined condition.
These elementary emotional states areapproximately orthogonal, or in otherwords they are almost independently gen­erated. Each vector component is an index
of the related emotional level.
If the emotion matrix is prepared for aparticular person, its applicability is limited
to this particular person only, losing gen­eral applicability because the matrix in­cludes features of the emotional states aswell as personal characters. Therefore,
personal characters must be smeared out
by preparing the emotional matrix based
many subjects.

3. Noise Reduction
The emotion vector should have posi­tive components only according to thedefinition. In the real situations, however,it shows negative as well as positive value.
There are many other emotional states
which the four elementary ones cannot
cover. The emotion matrix cannot sup­press their appearance in the 4-dimensionalemotion space; their appearance in the 4dimensional emotion space will be ran­dom, making noise. The noise level isestimated as standard deviation cr of thenegative contribution in each vector com­ponent.
The factor ft..z) is multiplied to thederived emotion vector components,
which is.f(_z) = tanH...l..J =. �4dexplt;i:L- exp(- zj4
exp(t;) + exp(r t)(2)for z > 0 andft..z) = 0 for z < 0. The smaller
part of the emotion index that is compara­
ble to the noise level is suppressed by thisfactor. On the other hand, however, theindex value itselfis influenced by the noise.
This effect is removed by smearing theindex values by t_E-e moving average. The
smoothed index zn at the nth time point iscalculated as

When a subject has anger in mind, theindex of anger acquires a high level.
However, the index of anger does notalways mean that the subject has anger inmind but that the subject has mental stressin general. In the same way the index ofsadness means that the subject is in a de­pressed state. To avoid misunderstanding,the indexes for anger, sadness, joy andrelaxation are named as N1, N2, P1 and R.

4. Emotional State and EEG cross-correlation
Each emotion matrix element is concerned with a pair of EEG channels in one of thethree frequency bands. A larger matrix element in magnitude (regardless of plus or minus)
plays a more important role in extracting features of the emotional components. Fig. I
shows magnitudes of emotion matrix elements in reference to respective EEG channels
indicated by crosses. Open and dark circles refer to positive and negative cross­correlation coefficient, respectively, and the radius represents the magnitude of thecoefficient in four steps.

Fig. 1-1
•

0

0

I

•

THETA

I
Fig. 1-2

Fig. 1-3

5. Results of Emotion Analysis
A female subject did algebraic tasks for an hour. After quitting this work the emotion
analysis started and the result is shown in Fig. 2. This figure shows magnitudes of the fourindexes, N1, N2, P 1 and R, from the top, and each bar corresponds to 5 .12 sec. Initiallythe subject showed signals mainly in NJ and N2. NJ indicates mental stress or excitation;
when the subject is psychologically tired this index is usually accompanied by N2. When,on the other hand, the subject is excited with interest in something, NJ is accompanied by
P1. In the present case, the subject was tired in mathematical work. Then she began tolisten to her favorite music; immediately these two indexes decreased to low levels, and at
the same time index P1 (characterizing joy) increased which, however, was reduced after
two or three minutes probably because of habituation. Interesting to say, indexes NJ andP 1 were fluctuating in the opposite phase. After the music was over, the same state con­tinued.
The second example is shown in Fig. 3. Two male subjects A and B were doingalgebraic tasks. Subject B finished the work earlier than subject A and told the examinerthat he had finished it. Subject showed relatively high relaxation index R and low stressindex N1 before subject B finished his work. When he found that he was behind his partnerin finishing the work, his relaxation level lowered and the stress index increased.
6. Conclusion
Our technique of making numerical evaluation of the state of mind at high time
resolution is new, and it will contribute to artificial intelligence, robotics, and so on.
after calculation

after music

during music (Chopin)

N 1
N2
p 1
5

R

0 '------'------''-----'---__..J.LD.____.___------'------'-----_J__J__,L--___,__.
(time)

Fig, 2

partner finished his calculation
L

N1
N2
p 1

R
(time)

Fig. 3

Evolving Complexity

Thomas S. Ray
ATR Human Information Processing Research Laboratories
2-2 Hikaridai Seika-cho Soraku-gun
Kyoto 619-02 Japan
81-7749-5-1063 (phone), 81-7749-5-1008 (fax)
ray@hip.atr.co.jp ray@santafe.edu ray@udel.edu
http://www.hip.atr.co.jp/ ray/
Abstract
Humans have been practicing applied evolution
since the dawn of agriculture, long before the devel­
opment of the theory of evolution. The domestication
and breeding of plants and animals is based on the ap­
plication of artificial selection to captive populations.
However, our management of evolution has taken placeat the "micro" level, the alteration of existing species.
We have never been able to harness and manage themore creative properties of evolution: the origin of
new species, and the emergence of complexity itself.
We are able to guide the evolution of poor quality wildcorn into high quality domestic corn, however, we can
not guide the evolution of algae into corn. To manage
complexity increase in synthetic evolutions requires anentirely new approach: the creation of "natural" eco­logical communities where complexity increase can oc­
cur through evolution by natural selection.

1

Synthetic Evolution

It has been said that "life is art". If life is art, evo­lution is the artist. The creative products of evolution
include the huinan body and mind, the cheetah run­ning down its prey, the mahogany tree, the humming
bird pollinating a flower. These living works of art
exceed in beauty, and depth of structure and process,anything produced by the best of human artists. In
fact, human artists themselves are products of evolu­tion.
Human artists express themselves in many media:
oil paint, clay, stone, music, cinema. What is themedium of expression of evolution? On Earth, evolu­tion is best known for its works in the medium of car­bon chemistry. However, it has recently been demon­
strated that evolution can work in other media. In

the last few decades, evolution has begun to express
its creative potential through the digital medium.
Much of the work with evolution in the digital
medium has had clear engineering objectives, but somework has simply been an exploration of the process ofevolution in an unfamiliar medium. In this latter case,the conditions are created for evolution by natural se­
lection to take place among self-replicating computer
programs. The result is a diversifying phylogeny of"digital organisms", in a "natural" ecological commu­
nity within cyberspace.
To the extent that an ecological community of freely
evolving self-replicators can be considered alive, thisrepresents another instance of life, but a very alien
life. If we could travel to other planets and observe
life there, it would probably be organic life, and tothat extent, somewhat familiar. But life embedded in
the radically different medium of digital computation
would be even more alien than extra-terrestrial life.
Freely evolving digital organisms are an instance ofevolution, not a model or simulation of organic evo­lution. Like a chemostat full of evolving bacteria andviruses, a computer full of evolving digital organisms
could be thought of as a model system for the experi­mental study of evolution. However, the evolving dig­ital organisms are no more a simulation of evolutionthat are the evolving bacteria and.viruses.

2

Complexity Increase

Earth's most creative evolutionary transitions werereviewed recently [3]. They seem to occur relatively
abruptly, compared to the background rate of evolu­
tionary change. Some of the major transitions noted
were: origin of chromosomes, origin of eukaryotes, ori­
gin of sex, origin of multi-cellular organisms, origin ofsocial groups.

Of these major transitions, perhaps the most dra­
matic, and best known, was the rapid origin anddiversification of large multi-cellular organisms frommicro-scopic single celled ancestors, in what has cometo be known as the Cambrian explosion of diversity[1]. It has understandably been called evolution's "bigbang", when there was a dramatic inflation of com­
plexity of organisms, and species diversified rapidly
into an ecological void.
Because the Cambrian explosion generated thelargest of organic life's complexity increases, it is in­teresting to consider what its digital analogue may be.
At its most fundamental level, the Cambrian explosion
arose out of the transition from single to multi-cellularorganisms. The digital analog would be a transition
from serial to parallel processes.
If we make an analogy between the cell and theprocessor, then modern multi-cellular organisms areparallel programs on a scale of complexity that vastly
exceeds any existing computer software. In organic
life, the program is the genome, based on nucleic acidsequences. In humans this program has roughly threebillion bases. However, no individual cell expresses
all the genes in the genome. Each cell expresses asmall subset of the genes, and this subset defines which"cell type" the cell is. It may be a skin cell, liver cell,brain cell, etc. depending on what subset of genes it
expresses.
The human body is thought to have several hun­dred distinct cell types, with a total of trillions of cells.
This corresponds to the two main types of parallelism
in computer software: SIMD and MIMD. In SIMD
parallelism there is a single instruction pointer sharedby all of the processors, so every CPU executes thesame code in absolute synchrony. In MIMD paral­
lelism, each CPU has its own instruction pointer, so
each processor is capable of executing a different setof code.
SIMD parallelism corresponds roughly to multiple
cells of a single cell type, in that in both cases, thesame genetic code is being expressed. MIMD paral­
lelism corresponds to multiple cells of different cell
types in that different cells are expressing different
code. Large modern multi-cellular organisms combine
both SIMD and MIMD parallelism on a massive scale.
However, most parallel computer software is primarilyof the SIMD type. The reason is that SIMD soft­
ware is approximately the same as serial software, but
executed simultaneously on many processors and pre­sumably operating on different data. However, exist­
ing MIMD software is much simpler that the genomesof organic multi-celled organisms. Such software isjust too complex to write. While existing MIMD com­puter hardware has the capability of having hundreds
or thousands of processors each executing different
code, while all cooperating on a single task, there does
not exist a human art for writing such software. It isbeyond the capability of programmers to write code
involving more than a few distinct processes.

3

TierraMy own work with digital evolution is an attempt
to generate a free process of evolution by natural se­
lection. I have set up a system of self-replicating com­puter programs, where the only clearly defined "fit­ness" criteria imposed on the system is the same aswhat is found in organic evolution: the ability to repli­
cate, or transmit genetic material to future genera­tions. My system, called Tierra (Spanish for Earth)creates the basic Darwinian scenario: self-replicating
entities in a finite environment with heritable genetic
variation, inside the computer.
In Tierra, the self-replicating entities are executablemachine code programs, which do nothing more thanmake copies of themselves in the RAM memory of thecomputer. Thus the machine code becomes an ana­
logue of the nucleic acid based genetic code of organic
life. The machine code programs occupy space in the
RAM memory, thus the memory provides an analogueof the physical space of organic life. Each program"owns" the block of memory that it occupies, and has
an exclusive privilege of writing on its own memory
blocks. However, any process may read, or executethe machine instructions in any part of memory. Thus
the partial (only write) privilege surrounding the space
occupied by the program is thought of as analogous
to a semi-permeable membrane surrounding organic
cells, partially protecting the internal chemistry from
disruption from the surroundings.
Genetic variation is introduced into the populationof replicating programs by randomly flipping bits inthe machine code. This is analogous to mutations in­volving substitutions of nucleic acids in the DNA se­quence of organic life. Additional noise is introduced
into the system in the form of occasional errors in thecomputations performed by the CPU (central process­ing unit).
The replication of the programs is brought about
through their execution by the CPU of the computer.
Thus CPU time provides the analogue of the energythat drives the metabolism of organic life.
The results of the evolutions that have been ob­
served so far represent a variety of solutions to theproblem of self-replication of machine code programs.
We might consider these solutions to fall into two
broad classes: "ecological solutions", and "optimiza­tions". Ecological solutions involve interactions be­
tween the programs sharing the memory, whereas op­
timizations involve innovations. within the individual
algorithms that result in faster replication.
We know that evolution embedded in the mediumof caroon chemistry is capable of generating parallel
software combining both SIMD and MIMD parallelism
on an astronomical scale. However, we don't know
the capabilities of evolution when embedded in themedium of digital computation. The experiments thathave been conducted so far show a surprising abilityof evolution to reorganize and improve machine code.
However, organic evolution was able to increase thecomplexity of the replicators by many orders of mag­nitude. So far digital evolution has shown small in­creases of complexity, perhaps by a factor of as muchas two or three. Is digital evolution capable of verylarge spontaneous complexity increases, and if so, un­der what conditions? The answers to these questions
can only be determined by experimentation.
An effort is underway to create conditions thatmight provoke evolution to generate substantiallygreater complexity in digital replicators. I am prepar­
ing what I call the "biodiversity reserve for digital or­
ganisms." This will be based on a networked version of
the Tierra software. We will attempt to get thousandsof people to run network Tierra on their computers,
and all of these computers will be connected into avirtual sub-net of the internet, within which digitalorganisms will be able to move freely from computer
to computer.
This configuration offers two advantages over theoriginal single-computer version of Tierra: greater
size, and greater complexity of temporal and spatial
patterns of resources.
There will be a dramatic increase in the size of thespace, and the amount of CPU cycles available to thedigital organisms. The single-machine version is gen­erally run with a memory space of one or two hundred
thousand bytes. However, a single complex digital or­
ganism could exceed that size.
It is believed that evolution can only generate greatcomplexity in the context of an evolving ecologicalcommunity, as biotic evolutionary interactions are an
important driving force in evolution. Thus we need aspace that can hold entire populations of many speciesof large organisms. Suppose that we want to support,
minimally, a hundred species, each with populationsof five hundred individuals, with individuals of ten

thousand bytes in size. This would require a spaceof about five hundred megabytes. While this space
could be achieved on a single computer, there would
be a very bad ratio of memory to CPU power. Dis­
tributing the memory over many computers provides
a better memory/CPU ratio.
While greater size is an absolute requirement, large
size alone would probably not provoke evolution to­wards greater complexity. However, the distributedmodel provides a great complexity of temporal andspatial patterns of resources. It is felt that this com­
plexity could provide selective pressures for evolution
to create complex adaptations to those patterns.
The network Tierra software will be run as a low­
priority background process, like a screen saver. Thismeans that when the user is actively using the com­puter, the Tierra software will sleep, receiving no CPU
cycles, the energy source for the digital organisms.
Any digital organisms who are present on the machineat that time will be frozen, unable to metabolize or re­produce.
This should create a strong selective pressure forindividuals to move about the net, avoiding sleeping
Tierra programs, and seeking those with a rich sup­ply of CPU cycles. Evolution may generate behaviors
that respond to temporal patterns in the availability of
CPU cycles. For example, there is likely to be a daily
cycle, with generally more free cycles at night whenpeople are sleeping. So some digital organisms mightevolve the behavior of migrating around the planet ona daily basis staying on the dark side of the planet.
The approach for managing the evolution of com­
plexity that is being advocated here is to generate a"natural" ecological community of freely evolving dig­ital organisms. The environment in which they live
should include sufficient complexities to provide someselective pressures for increasingly complex behaviors
to evolve. Once a significant impulse in the directionof complexity has occurred, the hope is that selective
forces arising from interactions among the digital or­
ganisms can lead to an auto-catalytic increase in com­
plexity.
It appears that this is what happened in organic
evolution. In the Amazon region, there are rain forestson white sand soils, where the physical environment
consists of clean white sand, air, falling water and sun­shine. Embedded in this physical environment is themost complex ecosystem on earth, the tropical rain
forest. In this ecosystem there are hundreds of thou­sands of species. These do not represent hundreds ofthousands of adaptations to the physical environment,
but rather, most of the adaptations of these species arewith respect to the other living o�ganisms that theyinteract with.
Life transforms the environment, such that the liv­ing component of the environment comes to predomi­
nate over the physical environment, after which mostevolution involves adaptations to other living organ­
isms. Thus the complexity of the living component of
the environment comes to greatly exceed the complex­
ity of the physical environment that it is embedded in.
The first evolution observed in Tierra was the originof ecological interactions, which were based on adap­
tation to the presence of other digital organisms in theenvironment [2]. Thus this dynamic has been present
in Tierra from the beginning. It is hoped that withthe help of some impulse towards greater complexity,
this dynamic can lead to a large spiraling upwards incomplexity.

4

Practical Applications

How might we work with digital evolution to pro­
duce useful products? Although we have a well es­
tablished practice of plant and animal breeding, ourrelationship to digital evolution is quite different. Ourancestors were able to go out into nature and observe
many highly evolved and complex organisms. They
found uses for some of these, such as the ancestorsof, rice, corn, wheat, chickens, pigs, dogs, etc. They
then bred them to produce the much improved domes­
ticated plants and animals that we know today.
However, in the case of digital evolution, we arestarting with very simple organisms that have not yetachieved the complexity to be useful, so our first ob­
jective is to evolve complexity. We have no prior ex­perience with managing the evolution of complexity.
Probably any attempt to guide the evolution of al­
gae to become corn, through artificial selection in thecontext of a breeding program, would prevent such atransition from occurring. I believe that we will never
be able to guide the evolution of complexity by the useof artificial selection. To facilitate complexity increase
we need a new and different approach.
I suggest that the most likely way to achieve com­
plexity increase in digital evolution is through evolu­tion by natural selection in an ecological community.
No attempt should be made to provide fitness func­
tions, or artificial selection, to guide evolution towards
useful products. Rather, evolution should be free ex­plore the possibilities without the burden of human"guidance".
Traditionally we have managed evolution throughmanipulating selective forces. In this new approach,

our role is to create the conditions for complexity in­crease, rather than trying to guide it through artificial
selection. This is an· interesting scientific challenge,
as the conditions that generate complexity increase
are unknown. However, the process of attempting to
generate a complexity increase, amounts to an exper­
imental approach to the problem. We can try manydifferent approaches, and if we are successful, we will
be able to experiment with the system until we canfind the minimal conditions that generate the behav­
ior.
If a digital analog to the Cambrian explosion canbe achieved, then it should be possible to establish
the same kind of relationship to digital evolution that
we have with organic evolution. We can go out into
digital nature, and observe the complex products ofevolution. While most digital organisms will have noapplication, it is likely that some will. We can observe
them for interesting and potentially useful information
processes. When we identify potentially useful digitalorganisms, we can capture them and subject them toselective breeding to enhance their performance on theapplication, and inhibit unruly wild behavior. Even­
tually the product can be neutered and sold to the enduser.
We have seen the tremendous creative potential ofevolution when expressed through the medium of or­
ganic chemistry. We do not yet know the full poten­
tial of evolution in the medium of digital computa­tion. However, the initial experiments have been very
promising, suggesting that it is worthwhile to make theeffort to push digital evolution to its limits. If digital
evolution has even a small fraction of the potential oforganic evolution, it could result in information pro­
cess of a complexity far beyond anything that we haveexperience with today. While there are many poten­
tial obstacles and technical problems along the way,
the possible rewards for success make the risk worth
taking. Yet it is a venture into the unknown for which
we can not estimate the likelihood of success.

References[1] Gould, Steven J. 1989.Wonderful life.W.W. Nor­ton & Company, Inc. Pp. 347.
[2] Ray, T., An Approach to the Synthesis of Life," inArtificial Life-II, C. Langton et al, eds. Addison­
Wesley, 1991, pp. 371-408.
[3] J. Maynard Smith and E. Szathmary, The Major
Transitions in Evolution (Oxford: Freeman, 1995).

INSECT-MODEL BASED MICROROBOT
Hirofumi Miura, Takashi Yasuda, Yayoi Kubo Fujisawa, Yoshihiko Kuwana and Isao Shimoyama
The University of Tokyo, Mechano-Informatics, 7-3-1, Bunkyo-ku, Hongo, Tokyo 113, JAPANe-mail: miura@leopard.t.u-tokyo.ac.jp
URL: http://www.leopard.t.u-tokyo.ac.jp

Abstract
It should be understood that the robot is not a simple
automatic machine, but instead has certain level ofintelligence. Many kinds of intelligent robots have beendeveloped in the author's laboratory during the past 15
years. These robots perform many kinds of games likethe cup & ball game, top-spinning, walking on stilts,
etc. These robots apparently look intelligent, but are
they really and truly intelligent? There is one opinion
that these robots are no more than simple automatic
machines which are controlled by a computer withsophisticated programs. If so, then what is actual robot
intelligence? The author is trying to construct a newrobotics --- insect-model based microrobotics --- in orderto get a new concept of robot intelligence.

1 WHAT IS ROBOT INTELLIGENCE?
Intelligence for the robots shown in Fig. 1 is discussed
below.
(1) INTELLIGENCE FOR FAST AND ACCURATE
MOTION (Cup & ball game robot)
When the motion is slow, PTP(point to point) control
works well. But for the fast motion, PTP control yields
position error because for fast motion, the inertial forces(centrifugal force, Colioris force, etc. which act on the
joint motors) are considerably large and feedback control
is disturbed by them.
So-called " inverse dynamics " is a very effective control
scheme for fast motion. The cup & ball game cannot be
implemented by the usual industrial robot employing
PTP control scheme. On the other hand, by employing
the inverse dynamics, the robot played cup & ball gamewith a 95% success ratio. Inverse dynamics can be robot
intelligence for fast and accurate motion.
(2) INTELLIGENCE FOR LEARNING (Inverted
pendulum)
The ability to learn is one of the most challenging
subject in robotics. The inverted pendulum robot wasdeveloped as an example of a robot which has learning
ability. If the geometrical dimensions (length, weight,

position of center of gravity, moment of inertia, etc.) are
not given, learning control must be employed. During
the first several minutes of the experiment, a person's
hands help the pendulum to maintain the vertical attitude,
but after several minutes the pendulum has learned tokeep the vertical attitude all by itself.
(3) INTELLIGENCE FOR DYNAMIC BALANCE(The biped and the quadruped)
In the author's laboratory many kinds of bipeds havebeen developed. The biped (stilt type) is staticallyunstable but can be balanced dynamically. It has
intelligence for dynamic balance[!]. Another type of
biped (human type) has knee joints and ankle joints like ahuman. Eight motors are mounted in this robot in total,
and it can walk more slowly than the stilt type. It alsowaiks dynamically.
The quadruped also walks dynamically. Real animals
support their bodies with two legs (not three legs) andswing the other two legs in the air even during a slow
walk. At low speed, one fore leg and one hind leg on thesame side are in the air together. This gait is called"pace". When the walking speed of an animal gets faster,
the gait changes to "trot (diagonal two legs are in the air
together)". The gait is selected by a minimum energy
consumption criterion[2],[3].
(4) INTELLIGENCE FOR LEARNING THE ROPE
(top-spinning robot)
The author can spin a top. However, it is very hard toteach another person how to spin a top. This means thatwriting a program for the top-spinning robot is also very
hard. How fast should the arm move? At which position
should the top be thrown forward? How strongly should
the string be pulled back? The author learned thesetricks by watching his father's play. For the robot, thetrajectory of a human arm and a top during human top­
spinning play was input into the computer through twovideo cameras (one camera in front, and one over thehead) and the computer generated a program to reproduce
the same motion of the arm and the top from humanplay. With this program the robot spins a top veryskillfully. This example may be called " teaching byshowing."Top-spinning
robot

Biped
(still)Cup & ball game Inverted pendulurn robot
robot

Quadruped

Fig. 1 Some Examples of Intelligent Robots which havebeen developed at the author's lab.

2 CAN A MACHINE HAVE ITS OWN WILL?
Several examples of robot intelligence were presented
above. These robots complete dexterous tasks like ahuman does. However, the author is not sure whether
these robots are as intelligent as a human, because allalgorithms expressed in the program are developed by ahuman investigating the result of analysis of humanplay. The robot itself is not intelligent and it only
follows the program given to it by a human. The robot
has no will to play these games better and better.
On the other hand, the author practiced hard on cup &ball game, top-spinning, etc., so that he could play better
than his friends during childhood.
Can a machine have will? This is a big question in
robotics and AI technology. The author is trying to find a
new way for a small but steady step to answer thisquestion[4].

3 INSECT-MODEL BASED ROBOTICS( Microrobot )An insect looks much more intelligent and more lively
than robots introduced above. The insect looks to have awill. If an insect robot could be developed, it is possible
to make it look truly intelligent. The author's laboratory
started developing insect-model based robots about five
years ago when looking for a new and different way toapproach intelligent robots. Since the late 1980s,
microtechnology has been highlighted as a promisingtechnology for the development of very small sizedmechanical systems. Fabrication of micromechanisms ona silicon wafer using IC process may be the key

technology for developing mm sized rnicrorobots.
Millimeter sized insects are all around us. The author
considered that rnicrotechnology should be applied todeveloping an insect-model based robot. Although there
are still many problems which must solved to build real
insect-model based rnicrorobots, there are basic lines ofsolution for some problems with experimental results.
This paper argues that insects will be good models forrnicrorobots for design, control, actuation, etc., just as
human beings or mammals are good models for normal­
sized robots. In addition, an insect has only 105 neurons.
The motion of an insect is produced by simple
mechanisms such as "a reflex act". But it is interesting to
the author that this motion looks intelligent. This is why
the author is trying to develop an insect-model based
robot.

4EXTERNAL SKELETON
For developing microrobots, new design concepts must
be constructed. For instance, a frictionless structure must
be designed. Rotating joints must be avoided because at
all rotating joint, there exists friction. Frictional forces
are proportional to the sliding surface area size(L2).
Weight and inertial forces, however, are proportional tovolume(L3). As the size gets smaller, the frictional force
exceeds the other forces and governs the motion ofsystem. In an extreme case, motion cannot be realized
when the system is subjected to normal actuation forces.

Fig.2 Cross section of an insect thorax. Distortion of
the thorax causes beating of wings

Fig. 3 Basic model of the external skeleton. Three
dimensional structure is constructed by bending
along the polyimide hinges.

We argue that knowledge about insects may be useful fordeveloping microrobots. The insect has many interesting
features, such as an external skeleton, elastic hinges
Goints) , contracting-relaxing muscles, etc. These
characteristics suggest basic design principles for
microrobots.
The external skeleton of an insect consists ofelastic(hard) cuticles connected by elastic (soft) hinges. In
general, the elasticity of the cuticles is higher than theelasticity of the hinges.The motion of body parts likewings is based on the deformation of these elastic
structures. Sliding friction does not exist there.
The cross-section of an insect (fly) thorax is shown in
Fig. 2(5], illustrating the beating mechanism of thewing. The muscles are inside the skeleton, while muscles
are outside the skeleton in humans.The downward
movement of the wing is produced by distortion of thethorax caused by contraction of the dorsal longitudinal
muscle. The upward movement results from distortion of
the thorax produced by the dorso-ventral muscle.
Elasticity of the thorax plays an important role in thefriction-free, high-speed wing movement. For mostinsects, the beating frequency coincides with thestructural natural frequency of the thorax. Mechanicalresonance is a good way to get large deformation withsmall external forces. This may be applicable for
actuation in microrobots.

5 ORIGAMI STRUCTURE

a hinge can be set at any angle by heating after bending itto the desirable angle. This structure can be fabricatedeasily by IC processes.
First, the development surface figure is made on asilicon wafer, and the folding lines are made of polyimide
as shown in Fig. 3. The folding operation is done by ahuman with a microprobe under a microscope.

6 MICROFLIGHT MECHANISM
The authors have developed the beating micromechanism
actuated by electrostatic forces shown in Fig. 4. If anelectric voltage is applied between Al plates and the base(silicon wafer), the plates move towards the base and thepolysilicon wings bend up. When the frequency of thealternating voltage coincides with the natural frequency of
mechanical vibration for the system, the beating
amplitudes resonate. Including this example, several
kinds of microflight mechanism have been developed.
Magnetic force is also applicable for actuation of beatingor microflight mechanisms, and a flying microrobot is
now under development[6][7]. Fig. 5 shows a micro­
flight mechanism[8]. It consists of three layers:
polyimide, nickel, and polyimide. Nickel is sandwichedby two polyimide layers so as not to be attacked by HF
while etching takes place to remove a sacrificial layer
under polyimide. Both polyimide layers are spin-coated to
1 µm in thickness, and nickel is sputtered to 0.1 µm inthickness.

Since the silicon IC process is planar, a three
dimensional structure is difficult to make. The author's
group is proposing a folding process like paper folding(ORIGAMI in Japanese) to make a 3D microstructure. To
build an external skeleton, the author's group uses
polysilicon as the rigid plate, and polyimide as the elastic
joint. The basic structure of an external skeleton model isshown in Fig. 3. Polyimide is a thermosetting resin, and� �

1\.1..___+ downstroke:'\.."l+�....---st \J,.___
upstroke

Fig. 5 Structure and dimensions of microflightmechanism

7 MICROANT-ROBOTS
Fig. 4 Schematic figure of beating mechanismIn the author's laboratory a microant-robot has beendeveloped, as shown in Fig. 6 and 7(9). The gait for thisrobot is not the same as the real ant. The two middle legs
kick the ground to move forward, while the other fourlegs touch the ground at all times to support the body.
The ground vibrated with a very small amplitude( lessthan lµm) and this vibration is transmitted to the middle
leg through the ground. If the frequency of the vibration
coincides with the natural frequency of the leg, it
resonates with a large amplitude, providing the driving
force by kicking the ground. The natural frequency of aleg is set by the length of the polyimide spring, so thetwo middle legs can have different natural frequencies.

humans. It can be said that they are no more thansimple automatic machines which are controlled by acomputer. The author is constructing a new technology -­- the insect-model based microrobot --- to look forsomething new in robot intelligence.

◄------

5mm

4cm

AntennaFig. 7 Pheromone sensor

Reference
Fig. 6 Photograph of the fabricated microrobot alongside
a sewing needle.

The robot can go straight, and tum to the right or to theleft.

8 HYBRID INSECT ROBOTA male silk moth pursues a female by following a phero­mone. This action can be caused by only a few moleculesof pheromone which arrive at the antenna of a male silk
moth. A biological sensor was constructed as shown in
Fig. 7. Two sensors were attached to a simple wheeledmobile robot to determined the direction of a pheromone
trace. The robot followed the pheromone trace like a realmale silk moth.

9 CONCLUSIONS
Several intelligent robots which have been developed in
the author's laboratory have been introduced, but theauthor is not sure that these robots are truly intelligent
because all control schemes have been constructed byhumans and all computer programs have been written by[1] Miura, H; Shimoyama, I. Robotics Research-The
First International Symposium:MIT Press,1984, pp.303325.
[2] Miura, H et al., Robotics Research-The Second
International Symposium: MIT Press, 1985, pp.317324.
[3] Shimoyama, I; Miura, H et al., Robotics Research­
The Third International Symposium: MIT Press, 1986,
pp.357-363.
[4] Miura, H et al., Robotics Research-The 6thInternational Symposium: The International Foundation
for Robotics Research,1994, pp. 489-495[5] Chapman, R. F., THE INSECTS-Structure andFunction, Harvard Press, 1982, ch.XI, p. 216.
[6] Kubo, Y; Shimoyama, I; Miura, H. Proc. of IEEE
International Conference on R & A, 1993, pp. 386-391.
[7] Shimoyama,I; Kubo, Y; Kaneda, T; Miura, H. Proc.
of MEMS '94,1994, pp. 148-152.
[8] Shimoyama,I; Kubo, Y; Kaneda, T; Miura, H.
Robotics Research-The 6th International Symposium:
The International Foundation for Robotics Research,
1994, pp. 497-502.
[9] Yasuda, T. Shimoyama, I; Miura, H. Proc. of
Sensors and Actuators, 1994, pp.366-370.

Artificial Societies and Generative Social Science
Joshua M. Epstein and Robert Axtell
The Brookings Institution and Santa Fe Institute1775 Massachusetts Avenue, NW
Washington, DC 20036
Abstract
What is an ar tificial society?
What can such models offer the social
sciences in particular? We address thesedrawingbrief
generalquestions,
illustrations from the specific artificial
society we call "Sugarscape."

any time the interacting agents differ in
myriad ways--by age, by culture, bywealth, by vision, by economic tastes, byimmunocompetence, and so forth: artificial
societies are full of diversity.

1.2

Environment

An artificial society is a computermodel consisting of (i) a population ofautonomous agents(ii) a separate
environment and (iii) rules governing theinteraction of agents with one another, theinteractionofagentswith
theirenvironment, and the interaction ofenvironmental sites with one another. Let
us discuss each of these ingredients in turn.

Artificial social life unfolds in anenvironment. The Sugarscape, as the namesuggests, is a landscape of generalized
renewable resource (sugar) that agents liketo eat; indeed they metabolize sugar andneed it to live.
An artificial society
environment is often spatial, such as a two­
dimensional lattice, but can be a more
abstract--and dynamic--structure, such as
the Internet. The point is that it is anexternal medium with which the agents
interact and over which the agents"navigate."

1.1

1.3

1

What is an Artificial Society?

AgentsAgents are the "people" ofartificial societies. An agent is a data
structure--in programming parlance, an"object''--that can change, or "adapt," overtime. Each agent has "genetic" attributes,"cultural" attributes, and various operating
rules governing its interactions with theenvironment and with other agents.
Genetic attributes are "hard-wired," fixed
for the lifetime of the agent.
In
Sugarscape, an agent's sex, metabolism, andvision, are genetic. Cultural attributes, bycontrast, are not hard-wired; they aretransmitted "vertically" from parents tochildren, but then change "horizontally"
through contact with other agents. In
individual
Sugarscape,economic
preferences are culturally determined-­they can change as agents move around andbump into agents with different tastes. A t

Rules

Finally, there are rules of behavior
for the agents and the environment. First,
there are rules coupling every site of theenvironment to its neighbors, as in cellular
automata. For example, the rate at whichsugar regenerates at a feeding site could bea function of the sugar levels at neighboringsites. Second, there are rules coupling theagents to the environment. The simplest
movement rule for Sugarscape agents is:look around as far as your vision permits;
find the site richest in sugar; go there andeat the sugar. Of course, movement under
this rule may bring the agent into contact
with new neighbors, which brings us to thethird set of rules, those governinginteragent interactions.
In Sugarscape,there are rules governing sex, combat, trade,
culturalandtransmission,diseasetransmission between neighbors.

2

Social Structures Emerge

In a typical artificial society
experiment,
we release
an i n itial
population of agents into the simulated
environment
and
watch
for
self­
organization into recognizable macroscopic
social patterns. The formation of tribes orthe emergence of certain stable wealthdistributions are examples.
Indeed, t h edefining feature o f an artificial society i s
preciselythat
fundamentalsocialstructures and group behaviors emerge fromthe interaction of individual agents
operating in artificial environments undersimple local rules-rules that place onlybounded demands
on eachagent'sinformation and computational capacity.
The shorthand for this is that we "grow"the collective structures "from the bottom
up".

Our
Sugarscapemodel-forthcoming en CD-ROM [1 ]--integrates
population dynamics, migration, combat,
trade, cultural transmission, genetics,
environmental interactions, immunology,
and epidemiology
inaspatially
distributedofartificial
societyheterogeneous adaptive
agents withlimited information, bounded computing
capacity, evolving preferences, and otherrecognizably
human
attributes
andlimitations. Our broad aim is to begin thedevelopment of a unified evolutionarysocial science subsuming-and extending-­such fields as economics and demography.
The general point, however, is t h a t
artificial
societies can function aslaboratories--CompuTerraria--where
we"grow" fundamental social structures insilico, thereby revealing simple micro­
generators of the macro-phenomena ofinterest. This is a central aim. As social
scientists, we are presented with "alreadyemerged" collective phenomena--such as
settlement patterns, fertility rates, or
wealth distributions--and we seek simple
local rules that can generate them. We ofcourse use statistics to test the match
between the true, observed, structures andthe ones we grow; but the ability to growthem-greatly facilitatedby modernobject-oriented programming-is what isnew. Indeed, it holds out the prospect of a
new kind of social science.

3

Generative Social Scienceanparticular,Infromepistemological standpoint, what "sort ofscience" are we doing when we build
artificial societies? Clearly, agent basedsocial science does not seem to be deductive
or inductive in the usual senses. But thenwhat is it? We think generative is anappropriate term. The aim is to provide
initial microspecifications (initial agents,
environments, and rules) that are sufficientto generate the macrostructures of interest.
We consider a given macrostructure to be"explained" by a given microspecification
when the latter's generative sufficiency
has been established. We interpret thequestion "can you explain it?" as asking"can you grow it?"In effect, we areproposing a generative program for t h esocial sciences and see "the artificial
society" as its
principal
scientificinstrument.

References[1] Joshua M. Epstein . and Robert Axtell,Growing Artificial Societies: Social
Science from the Bottom Up, The
Brookings Institution and MIT Press,
1996[2] On questions of validation, see Robert L.
Axtell and Joshua M. Epstein, "Agent­
Based Modeling: Understanding Our
Creations," The Bulletin of the Santa
Fe Institute, Winter 1994, Vol. 9, No.
2.

CO-SPONSORED BY
Asahi Agency Advertising Ltd.
Beppu Chamber Of Commerce And Industry
Chiyoda Corporation
Daikin Industries, Ltd.
Daihen Corporation
Daihen Technology InstituteFuji Electric Co., Ltd.
Kawasumi Laboratories, Inc.
Kyudenko Corporation
Kyushu Electric Power Company Incorporated Oita Branch Office
Kyushu Oil Company, Ltd.
Matsushita Research Institute Tokyo, Inc.
Meidensha Corporation
Mitsubishi Heavy Industries, Ltd.
Mitsui Engineering Shipbuilding Co., Ltd.
Nippon Steel Corporation
Nishikawa Keisoku
Nishi Nippon Electric Wire Cable Co., Ltd.
Nisshinbo Mee Inc.
Oita Canon Inc. Information Systems Research Center
Oita Electronic Computer Center Co., Ltd.
Oita Gas Co., Ltd.
Oita Prefectual Goverment
Omron Corporation
Ricoh Company, Ltd.
Satogumi Co., Ltd.
Shintsurukai Kosan Co., Ltd.
Showa Denko K.K.
Sony Oita Corporation.
Sugahara Industry Co., Ltd.
Sumitomo Chemical Company, Limited Oita Works
Toshiba Corporation Oita WorksThe Oita Chamber Of Commerce And Industry
Yanai Denki Kogyo Co., Ltd.
Yatsushika Sake-Brewing Co., Ltd.
Yoshino Plastics Corporation, Fukuoka Plant

Author Index........................... AS-3
Abe,K.
........................... B3-4
Aoki, T.
........................... B6-5
Arai, F.
Arimoto, S. .......................... Bl-1
Arthur, B. ............................ PS-3
Awa, K.
........................... B3-6
Burnham, K. J. .................... A3-2
Casti, J.
........................... OP........................... B6-2
Choi, C.
Eom, T. D. ......................... A2-l
Epstein, J. ........................... PS-4
Fujimura, S. ........................ A4-3
Fujimura, S. ........................ A4-4
Fukuda, T. ........................... B6-5
Fukuyama, Y. ..................... B4-2
Fukuyama, Y. .... ................. B4-l
Fukuyama, Y. ..................... B4-3
Furuhashi, T. ...................... B3-3
Furuhashi, T. ...................... A3-l
Garis, H. D. ....................... AS-4
Gomi, T. ............................. Bl-2
Goto, S. .............................. B5-2
Hachino, T. ........................ B2-l
Ham, W. ............................. B5-6
Hanaizumi, H. .................... A4-3
Harvey, I. ............................ AS-6
Hemmi, H. ......................... AS-2
Hiramatsu, H. .................... Al-5
Hiraoka, N. ........................ B6-5
Hirasawa, K. ....................... B6-3
Hirasawa,K. ....................... Bl-3
Hirayama, H. ...................... B4-3
Hirayama, H. ...................... B4-l
Hirayama, H. ...................... B4-2
Ho, K. H. L. ....................... Al-2
Hong, S. G. ......................... B2-3
Honma, N . ......................... AS-3
Huang, J. ............................ A4-5
Husbands, P. ...................... AS-1
Ichikawa, S. ........................ Bl-6
Im, I(. ................................. B5-6
Inoue, Y. ............................ B2-4
Ishida, Y. . ........................... B5-5
Ishida, Y. ............................ Al-1

Ishiguro, A. ......................... C1-4
lshiguro, A. ......................... B 1-6
Ishiguro, A. ......................... B3-l
Ishii, M. .............................. B 1-5
Ishii, N. .............................. B2-6
Ishikawa, H. ....................... B6-5
Ishikawa, Y. ........................ B4-l
Ishimatsu, T. ...................... B3-2
lwakoshi, Y. ....................... A3-l
James, D. G. ...................... A3-2
Jeong, I. K. ......................... Cl-2
Jeong, S. ............................. B5-6
Jinzenji, T. .......................... Al-4
Jung, M. J. .......................... B6-4l(ang, C. ............................. B3-2
Kashiwagi, H. ..................... A2-6
Kawata, Y. .......................... A2-5
IGm, J. ................................ B5-6
IGm, J. H. ........................... B6-4
IGm, S. W. .......................... B2-3
IGta, H. .............................. A2-3
IGtamura, S. ....................... Al-5
IGtamura, S. ....................... AS-5
IGyasu, S. ........................... A4-4
Kobayashi, H. .................... A4-2
Koga, M. ............................ B6-3
Kokaji, S. ........................... Cl-1
Kondo, T. ........................... B3-l
Kuboshiki, S. ...................... Bl-6
Kurematsu, Y. .................... Al-5
Kurokawa; H. ..................... Cl-1
Kyuma, K. .......................... A4-l
Kyura, N. ............................ B5-2
Langton, C. G. .................... PS-1
Lee, J. J. .............................. A2-l
Lee, J. J. .............................. B6-2
Lee, J.. J. .............................. Cl-2
Lee, J. J. .............................. B2-3
Lee, J. J. .............................. B3-5
Lee, S. W. ........................... B6-4
Li, X. .................................. A4-6
Liu, Y. . ............................... Cl-5
Looney, C.G. ...................... A3-5
Mabuchi, T. ....................... B3-6
Matsumoto, T. ................... A4-2

PS-2
Miura, H.
A5-5
Murao, H.
Murata, J. ........................... B6-3
Murata, J. ........................... BI-3
Murata, S. . ......................... CI-I
Musya, T. ........................... PS-I
Nagasawa, T. ...................... B3-6
Nagata, T. .......................... CI-3
Nakamura, A. ..................... B5-I
Nakamura, M. .................... B5-2
Nishikawa, Y. ..................... A2-3
Nonaka, T. ......................... B2-5
Nozawa, T. ......................... B4-I
Ochiai, S. ......... ,................. B2-5
Ochiai, T. ........................... B3-2
Ogasawara, T. ..................... B5-I
Oguro, R. ........................... B2-I
Ohbayashi, M. .................... B6-3
Ohbayashi, M. .................... Bl -3
Ohkawa, K. ........................ B5-3
Ohnishi, N. ........................ A4-5
Ohnishi, N. ........... , ............ Al-2
Ohta, J. ................... , .......... A4-I
Oka, T. ................. , ............ B3-4
Okamoto, M. ..................... B2-5
Okuma, S. .......................... B3-4
Onat, A. ............................. A2-3
Onitsuka, T. ....................... CI-3
Park, K.B. .......................... B2-3
Pfeifer, R. ........................... B5-4
Rasmussen, S. ............ ........ PS05
Ray, T. S. ........................... PS-2
Saita, H. ............................ BI-5
Sakata, M. ......................... A2-6
Sano, M. ............................ BI-4
Sasaki, M. .......................... AI-4
Sato, M. ............................. A5-3
Sawada, Y. .............• ............ BI-4
Shao, H. ............................ , A3-4
Shibata, T. .......................... B5-3
Shim, H. S. ......................... B6-4
Shimohara, K. .................... A5-2
Shiraki, K. .......................... B3-6
Suehiro, T. ......................... B5-I
Sugawara, K. ...................... BI-4
Sugie, N. ........................... A4-5
Sugisaka, M. ...................... A4-6

Sugisaka, M.
B3-5
A2-2
Sugisaka, M.
Sugisaka, M. ...................... A2-4
Suzuki, T. .......................... B3-4
Suzuki, Y.
Al-3
Takata, H.
B2-1
Takeda, H.
A5-3
Tanaka, H.
A3-3
Tanaka, H. ......................... Al-3
Tanaka, K. ......................... A4-2
Tanie, K. ............................ B5-3
Tominaga, D. ..................... B2-5
Tomita, K. ......................... CI-I
Tsuchiya, K. ....................... B2-4
Tsuji, T. ............................. B2-I
Tsujita, K. .......................... B2-4
Tsukune, H. ....................... B5-1
Tsumashima, T. ................. A4-3
Tsumoto, S. ....................... A3-3
Tsumoto, S. ........ , .............. Al-3
Uchikawa, Y. ..................... B3-3
Uchikawa, Y.
A3-I
Uchikawa, Y.
CI-4
Uchikawa, Y.
BI-6
Uchikawa, Y.
B3-I
Ueno, S. ............................ B4-4
Umeda, N. ......................... B2-I
Wang, A.P. ........................ B4-4
Wang, H. ........................... B3-2
Wang, X. ........................... A3-4
Wang, X. ............................ B3-5
Wang, Y. ............................ B2-6
Watanabe, Y. ..................... CI-4
Watanabe, Y. ..................... B3-I
We, K.L. ............................ B4-I
Yagi, T. .............................. A4-2
Yamada, T. ........................ B3-6
Yamakawa, T. .................... B6-I
Yamamoto, H. ................... B2-2
Yao, X. ............................... CI-5
Yashima, Y. ....................... A2-5
Yoshikawa, T. .................... B3-3Zha, H. B. ......................... CI-3
Zhang, Y. G. ....................... A4-6

Artificial Realization oi Human Skill for Robot Manipulator Control
Masatoshi Nakamura and Satoru Goto
Department of Electrical Engineering
Saga University
Honjomachi, Saga 840, Japan

Nobuhiro Kyura
Research Laboratory
Yaskawa Electric Co., Ltd.
2-1 Kurosakishiroishi, Yahata.nishi-ku
Kitakyushu 806, Japan

Abstract

the human skill artificially by use of the mathematical
tools, clarification of the procedure of the human skill
is crucial. The procedure for the artificial realizationof human skill is summarized as follows:

A methodology for artificial realizatinn of the hu­
man skill of the experts was described. The artificial
human skill was realized in the problem of contour
control of mechatronic servo systems including robot
manipulators and machine tools. The merits of theartificial human skill thus obtained were discussed.

1l. Understanding of objective and visible humanskill
It is required for the designer to understand theobjective of the problem and the causes of theexisting problems clearly as the same level as theexpert through the intimate discussion with theexperts or by t,1e experimental tests as to workout the problem. The designer should also learn
the human skill through the experts.

Introduction

Through the collaboration works with the re­searchers of university and industry, the authors havetheoretically solved many problems which had beenencountered in the actual field of industry. Key ideas
for the solution to the problems were intuitively in­structed by the experts who have practiced in theworking field and dealt with those problems. Those
human skills were mathematically analyzed and thenthe solutions to the problems were attained throughthe long term collaboration work. This paper pro­
posed a methodology for artificial realization of hu­
man skill. The modified taught data method formechatronic servo system [l, 2] was interpreted under­
neath as a successful work of the artificial realization
methodology.

2(Invisible to Others)(Visible to Others)

JOB

Objective
Problem

Understanding of Objective and Human Skill
Discussion with Expert
Experimental Tests

Mathematical Interpretation

Methodology for Artificial Realiza­tion of Human Skill(Visible to Others)
Mathematical Interpretationof Human Skill

Experts in the industry are engaged in a job for along time. Their jobs are skillful based on the know­how of the specific field. Their skills are mainly gained
through a cut and try technique and intuitive man­ner (see Fig. 1). Only the final technique of humanskill is visible and recognizable to others but not theprocedure behind the development. Moreover, the ex­perts themselves can rarely explain the procedure how
to attain the skillful techniques. In order to realize

Artificial Realization of Human Skill
Method for the Objective

Figure 1: Methodology for artificial realization of hu­
man skill12. Mathematical interpretation
The procedure of the human skill is clarified
through the understanding of the objective andhuman skill. Thus kind of task can be understoodas an identification problem of the invisible pro­cedure of the human skill and can be referred tothe mathematical interpretation of human skill.
'.3. Artificial realization of the human skill
If the cause of the problem is clarified mathemat­
ically, the realization of the human skill by useof mathematical tools is rather straightforward
through the selection and the development of themathematical tool box, such as signal processing,system identification, control theory and so on.
The artificial realization of the human skill thus
obtained should be discriminated from a mere im­itation of the human skill.

3
3.1

Artificial Realization of Human Skillin Mechatronic Servo System
Mechatronic servo system

Many kinds of mechatronic servo systems such asindustrial robot manipulators and NC (numerical con­trol) machine tools are used in actual production lines
and assembly lines, etc. The industrial mechatronic
servo systems are constructed by the mechanical con­nection of the servo systems for each axis and the servo
systems are controlled for ea.ch axis, independently.
The angle and the angular velocity of the motors ofthe mecha.tronic servo systems are measured by using
sensors such as potentiometers, tachogenerators andencoders which are attached to the motors. These
motors are controlled by servo controllers based on themeasurement data of the motors. Thus, the servo con­trollers of the industrial mechatronic servo systems do
not directly control the position and the velocity of themechanism but they control the angle and the angu­
lar velocity of the motors. Usually, the position loop
and the velocity loop of the constant feedback gainof the servo controllers are constructed. The control
performance of the mechatronic servo systems mainly
depend on the servo systems for each axis.

3.2

Artificial realization of human skill forcontour control of mechatronic servo
systems

The artificial human skill for mechatronic servo sys­
tems such as robot manipulators is realized following- 2

Following
Modified�ective
Input
Trajecto� Modification
Mechatronic Trajectory_ Servo System
�
Te1m
R(s)

F(s}

U(S)

.

G(s)

Y(s)

Figure 2: Block diagram of modified input data formechatronic servo systems
the methodology described in the previous section:l. Understanding of the objective and visible humanskill
The objective of the contour control of the mecha­tronic servo systems is that the 'following trajec­
tory' of the end effector of the system exactly fol­lows the 'objective trajectory.' In the industrial
applications, working speed of the mechatronic
servo systems is also required to be high for ahigher working efficiency. However, at high speed
operation of the mechatronic servo systems, thecontour control performance of the mechatronic
servo systems often deteriorates in the following
trajectory. One technical expert in the industry
had found a technique for modifying the input
signal to the servo system of the robot manipu­
lator to achieve an accurate following trajectory
for a given objective trajectory. The length of theextension of the modified input signal had been
determined appropriately in the trial and error
manner.
2. Mathematical interpretation
The dynamics of the mechatronic servo system isrepresented by the mathematical model of a lin­
ear dynamical equation. The main reason why thedeterioration at the high speed operation occurs
is found to be the dynamical delay of the servo
motors of each axis in the mechatronic servo sys­tems. The human skill can be understood as amethod for modifying the input signal from theobjective trajectory to compensate the delay ofthe dynamics.
3. Artificial realization of the human skill
Figure 2 shows the block diagram of the modified
input signal for contour control of each axis of themechatronic servo system. As shown in Fig. 2,
the dynamics of the mechatronic servo system is
described for each axis independently asY(s) = G(s)U(s)(1)

where U ( s) is taught data for the input trajectory,Y ( s) is the following trajectory of the mechatronic

servo system and G( s) is the transfer function of
the mecha.tronic servo system. The taught dataU ( s) a.re modified from the objective trajectoryR(s) through a modification term F(s) , i.e., thetaught data U(s) can be expressed as

U(s) = F(s)R(s).

Input layer

The modification of the input signal is accom­
plished in two ways, i. e., the first one is based on
the pole assignment regulator of open loop type,
and the second one is based on the Gaussian neu­
ral network. The realization of the methods is
described in the following section.

4

Methods of Modification of Input
Data

4.1

Mathematical model of mechatronic
servo system

The mecha.tronic servo systems a.re opera.tee! under
the working condition such that the defection of the
disturbance such as the reaction force of the mech­anism and the influence of the composed a.xis is ig­
nored. Under the above condition, the dynamics ofthe mecha.tronic servo system can be treated indepen­dently for ea.ch axis and the problems of the control
performance of the mecha.tronic servo system can beanalyzed for ea.ch a.xis, independently.
The mecha.tronic servo system of the ea.ch axis 1s
described as a simple first order model [3]Kr,

s + ft p(3)

where J(P is the known position loop gain. For gen­eral working condition of the industrial mechatronic
servo systems where the speed of the motors are
1/20 ~ 1/100 times of a rated speed, the model (3)well describes the input and output relationship of themechatronic servo system including its dynamics andmechanical construction. The model (3) also implies
the dynamics of the industrial articulated robot ma­nipulators within the linearizable region [4].

4.2

Output layer(2)

The modification term F(s) should be designed
based on the mathematical model G(s) of themecha.tronic servo system.

G(s) =

Hidden layer

Pole assignment regulator of open
loop typeThe modification term F(s) in (2) is constructedby using the pole assignment regulator of open loop

LinearGaussianLinearFigure 3: Construction of Gaussian network
type based on the state space representation of themechatronic servo system (3). The modification termF(s) is derived [l] as

F(s) = -,(s + Kp)

(4)

Ep ( s - 1)

where I is the pole of the regulator .
The allowable limitation of the servo motor speed
incorporated in the actuators must be taken into ac­
count to use the modified taught data method for
actual mecha.tronic servo systems. The restriction is
given as(5)where Vma:r is the maximum speed of the servo system.

4.3

Gaussian network

The modification term F(s) can be determined byuse of another way in which the Gaussian neural net­
work is adopted [2]. Figure 3 shows the structure of athree-layer Gaussian network composed of two-input,
one-output, and one hidden layer with four Gaussianunits, which is used for the construction of the inverse
dynamics of the mechatronic servo system (3).
Two-input one-output Gaussian network is definedbyq;( ;r.) =L W;-i/J; (a::)
4

(6)

i=I

where q;(x) is the network output, x= (x1,x2f =(1',rf is the input vector, w; is the weight of ith unit.
The function V'i ( x) is a Gaussian unit defined by'1/J;(a:,) = e xp

(tt
j=l k= 1c};'i (xjJk- m;)(x,, - mt)/2)(7)where rnj is the jth element of the mean, a-J isthe marginal standard deviation, hh is the correla­tion coefficient and //,. = 1 if j = k and jh i.k [ <
J•·
. -

- 3 -

J

I

l otherw-it;e [5]. The parameters ( mj, o-J, hj ,J are
trained based on actual data for operation of themechatronic servo system by use of the Back Prop­
agation algorithm [6]. The parameters in the Gaus­
sian network are updated for each sampled teachingpattern and the algorithm stops when the parame­ters converge to constant values. After the training of
the network, the neural network realizes the actual in­verse dynamics of the mechatronic servo system and isadopted as the modification term for the mechatronic
servo system.

5
5.1

-

10-

- - - - - - - · Objective Locus- - - - - Conventional Method--- Pole Assignment Regulator

Experirnental Result
0-Pole assignment regulator of open
loop typeThe effectiveness of the proposed method of mod­ified input based on the pole assignment regulator
was assured by applying them to an industrial artic­
ulated robot manipulator. For the industrial articu­
lated robot manipulator, the dynamics can be approx­
imated for each axis in the working coordinates, inde­
pendently [4]. Hence, the proposed method was ap­
plied to each axis in the working coordinates of the ar­ticulated robot manipulator, independently. The posi­tion loop gain for each axis was same as Kp = 15[1/s].
Figure 4 illustrates the experimental results of theindustrial robot manipulator by using the conventional
method, which means the objective trajectory wasused as the input signal, and the proposed method ofmodified input based on the pole assignment regulatorof open loop type. The trajectory of the conventional
method deteriorates roundly at the corner because of
the time delay of the mechatronic servo system. How­ever, the trajectory of the proposed method of modi­
fied input based on the pole assignment regulator has
sharp edge corner. The results show the effectivenessof the proposed method of modified input based on
the pole assignment regulator of open loop type.

5.2

I

Gaussian network

Start
I

0

I

X-axis [cm)10

Figure 4: Experimental results based on the pole as­signment regulator of open loop type for the articu­
lated robot manipulator
was learned by using the teaching pattern of the ac­tual data of the X-Y table. The teaching pattern was
adopted 100 times for the learning of the parametersin the Gaussian network.
Figure 5 illustrates the experimental results of theconventional method and the proposed method ofmodified input based on the Gaussian network. As
shown in Fig. 5, the maximum position error of theconventional method is 0.,5[mm] and that of the pro­
posed method is 0.l[mm]. The result of the proposed
method is significantly better than that of the conven­tional method. The results prove that the proposed
method compensates the delay of the X-Y table mo­tion appropriately by using the modification term of
the Gaussian network.

5.3

Discussion

The merits of the artificial human skill for the con­tour control of the mechatronic servo systems are sum­
marized as follows:

The secondly proposed method of modified input
based on the Gaussian network was applied to the X-Ytable which had 2 degrees of freedom in the movement
and the Y axis is placed on the X axis. The position
loop gain for each axis was same as Kp = 5 [1/s]. The
proposed method was applied for X-axis and Y-axis of
the X-Y table, independently. The Gaussian network- 4 -a. Generalization of the human skill
The expert's technique must adjust the modifica­tion value for each objective trajectory, however,the proposed method of the artificial human skill
need not require such adjustment. The expert's
technique is only applicable to the simple rectan­
gular objective trajectories. However, the artifi-

model because of the learning of the Gaussian network.
However, the designing procedure of the modifica­tion term is laborious compared with the first method
based on the pole assignment. In both proposed meth­ods, the first order model (3) for the mechatronic servo
system is not crucial, but the higher order model of thesystem can be used for the designing procedure.

2
7'

1/

1/
''/

'I

Objective Locus
Conventional Method
Gaussian\�6

� .. _______ _.....

-2
-2

0
X-axis [cm]2

Figure 5: Experimental results based on the Gaussian
network for X-Y tablecial human skill can be applicable to any shapeof trajectories. The original human skill was de­veloped for the specific industrial robot manipu­
lators, however, the artificial human skill can beapplied to any kind of mechatronic servo systems.
b. Acceptability in industrial field
The method for the modification of the input data
is similar to the expert's technique and the mod­ification term can be automatically determined if
the value of J{P in (3) or the actual input/output
data of the system are given. Human skill of theexperts can be replaced by the artificial humanskill.
c. Objective evaluation
The performance of the methods of the artificial
human skill for modification of input data is eval­uated theoretically because the mechatronic servo
system is described by the mathematical equa­tions of (1).
The both of the proposed methods of the artificial hu­
man skill have their own merits. The first method
based on the pole assignment regulator is simple andeasy to calculate the modified input signal, how­ever, the method requires the exact knowledge of themodel for the mechatronic servo systems. The sec­
ond method based on the Gaussian network is adapt­able for any system without a priori information of theConclusion

A methodology for artificial realization of humanskill of the experts was investigated. The methodology
was realized in the contour control of the mechatronic
servo systems. The two methods of modified input
for the contour control of the mechatronic servo sys­
tems were proposed. In order to realize the methodol­
ogy the following points are required for the designers:
1) exact understanding of the objective and the humanskill, 2) long and intimate discussion with collabora­
tors, 3) good human relationship between collabora­
tors.

References[l] S. Goto, M. Nakamura and N. Kyura, "Method for
Modifying Taught Data for Accurate High Speed
Positioning of Robot Arm," SICE Trans., 27-12,
pp. 1396-1404, 1991 (in .Japanese).
[2] S. Goto, M. Nakamura and N. Kyura, "Accurate
Contour Control of Mechatronic Servo Systems byUsing Gaussian Networks," IEEE Industrial Elec­tronics, 1996 to be appeared.
[3] Yaskawa Electric Mfg. Co., Ltd., Introduction toServo Techniques for M echatronics, Nikkankogyo­
shimbunsha 1986 (in .Japanese).
[4] S. Goto, M. Nakamura and N. Kyura, "Proprietyof Linear Model of Servo System for Industrial Ar­
ticulated Robot Arms and Evaluation of Its Lin­earization Error," ISCIE Trans., 7-3, pp. 103-112,
1994 ( in Japanese).
[5] S. Lee and R. M. Kil, "A Gaussian Potential Func­tion Network with Hierarchically Self-Organizing
Learning," Neural Networks, 4, pp. 207-224, 1991.
[6] B. Widrow and M. A. Lehr, "30 Years of Adaptive
Neural Network: Perceptron, Madaline, and Back­propagation, in C. Lau (Ed.), Neural Networks,
Part 2, pp. 27-53, New York, IEEE Press, 1992.

- 5 -Computational Analysis of Acquired Dyslexia of Kanji Characters
based on Neural Networks
Shusaku Tsumoto and Hiroshi Tanaka
Dept. of Information Medicine, Medical Research Institute, Tokyo Medical and Dental University
1-5-45 Yushima, Bunkyo-ku Tokyo 113 Japan
TEL: +81-3-5803-5840 FAX: +81-3-5803-0247
E-mail:tsumocom@cmn.tmd.ac.jp, tanaka@cim.tmd.ac.jp

Abstract
Acquired dyslexia of Kanji characters is one of themost interesting research areas in the neurophysiologi­
cal field. Although dyslexia of alphabets is known as themalfunction of gyrus angularis {classical hypothesis},reading and writing of Chinese characters (Kanji char­acters} are intact in Japanese patients who sufferedfrom cerebral infarction of gyrus angularis. Thus, it
has been pointed out that another group of neurons,
which functions as integration of shape information,
should play an important role in recognition of Kanjicharacters (Iwata's model). In this paper, two neu­
ral network models, based on classical hypothesis and
Iwata's model, are introduced to examine the validityof these two hypotheses. The computational experi­ments gives the following three results: first, Iwata's
model learns Kanji characters much faster than clas­
sical model. Second, Iwata's model is more robustwith respect to the malfunction of neurons. Finally,
third' Iwata's model simulates the characteristics of
two types of Japanese dyslexia.

Keywords: Acquired Dyslexia, Kanji characters,
Neural Modeling

1

Introduction

The Japanese language uses three kinds of char­acters, Hiragana, Katakana, and Kanji. The former
two sets of characters are used as syllabogram, eachof which corresponds to a Japanese syllable. For ex­,,
ample, "\t , and " 1" corresponds to a syllable, "i".
The main difference between Hiragana and Katakana
is that the former set is used to represent ordinary
sentences, while the latter one is used to describe loan
words, such as " 7 :Ji"", which is borrowed from a
English word, "radio".
On the other hand, Kanji is used as the combina­
tion of morphogram and syllabogram. For example,"i::." means a dog, which corresponds to "\t '�", rep­resented by Hiragana. The reason why we use Kanjicharacters is that Hiragana or Katakana characters are
too weak to represent the meanings of Japanese words.
Since Japanese syllables are restricted to only 50 kinds,
the Japanese language has many homonyms. For ex­
ample, ""'�" has the following two meanings: one isa dog, and the other one is a verb, "leave". Japaneseuses Kanji characters to differentiate these two mean­
ings: "i::." and "¾�". In this way, Hiragana andKanji characters have different functions, which causes
acquired dyslexia of the Japanese language to be moredifficult than other languages.
Although dyslexia of alphabets is known as the mal­function of gyrus angularis (classical hypothesis)[2],
reading and writing of Chinese characters (Kanji char­acters) are intact in Japanese patients who sufferedfrom cerebral infarction of gyrus angularis. Thus, it
has been pointed out that another group of neurons,
which functions as integration of shape information,
should play an important role in recognition of Kanjicharacters (Iwata's model)[3].
In this paper, two neural network models, whichcorresponds to classical hypothesis and Iwata's hy­
pothesis, are introduced to examine the validity ofthese hypotheses. The computational experiments
gives the following three results: first, Iwata's model
learns Kanji characters much faster than classicalmodel. Second, Iwata's model is more robust with re­spect to the malfunction of neurons. Finally, third,
Iwata's model simulates the characteristics of two
types of Japanese dyslexia syndromes.
2

Acquired Dyslexia

Classical Hypothesis: Classical hypothesis forreading and writing characters is illustrated in Fig. 1,
where S, A, V and GA denotes Somatosensory area,
Auditory Language area, Visual cortex, and Gyrus
Angularis, respectively. In reading process, first, in-- 6 -whose T region is damaged can read Kanji charactersand Hiragana, but cannot understand these meanings,
but a patient whose GA region is damages can read
Kanji characters and understand these meanings, butcannot read Hiragana. Thus, for reading Kanji char­acters, two pathways should be considered, which isillustrated in Fig. 2. As to the phonological aspects,
the reading process is the same as that in Hiragana.
On the other hand, concerning the semantic aspects,
visual information is transmitted from visual cortex toT region. Then, integrated information is sent from T
region to auditory language area (V-+ T-+ A).

Reading: V-GA-A
Writing : A -GA-S
Figure 1: Classic Hypothesis for Pure Dyslexia{Left Posterior Inferior Temporal Gyrus)
Kanji Reading: V...OA-A {Phonological)
V - T - A (Semantic)
Kanji Writing : A...GA-s

A -r-v...oA-s

Figure 2: Hypothesis for Japanese Dyslexia
formation on characters is captured by Visual cor­tex. Then, the obtained information is transmitted to
Gyrus Angularis, which integrates all the informationof characters. Finally, integrated information is sent
to auditory language area, which is closely connectedwith speaking languages. This hypothesis is proposedby Geschwind[2], which explains clinical cases on puredyslexia reported by Dejerine[l] very well.
However, it is notable that this hypothesis is intro­
duced in order to explain pure dyslexia with respect
to alphabets, a kind of syllabogram. Recently, it is re­ported that this hypothesis does not explain dyslexia
syndromes in Japanese whose patients cannot read Hi­
ragana but can read Kanji characters, or cannot read
Kanji characters but cannot read Hiragana[3].
Hypothesis for Japanese Language: Iwata pro­poses a new hypothesis for Japanese reading andwriting process, based on clinical cases on Japanesedyslexia syndromes[3]. Close clinicopathological ex­
aminations show that left posterior inferior temporalgyrus(T) is closely involved with integrating semantic
information on Kanji characters. That is, a patient

Dyslexia of Kanji Characters: In dyslexia of Chi­nese characters, the following four types of syndromesa.re reported. The first one is disorder of captur­
ing global structure, which means that a patient
can only read some part of Chinese characters. For
example, when "W" (blue) is shown, a patient misread
it as "Ji"(moon), which is only a lower part of "W".
This syndrome can be viewed as damage of integrating
partial information on Kanji characters.
The second one is confusion of morphology,
Whose patient confuses a Kanji character with a char­acter of the similar shape. For example, when "i!"is shown, a patient misread it as "i.R"(silver), whose
left part is the same as that of the character "i!"(key).
This syndrome can be viewed as damage of integrating
shape information on Kanji characters.
The third one is semantic misleading, where apatient misread a character as one whose meaning isvery similar, although each morphological character­istics are not similar. For example, when "Jt"(horse)is shown, a patient read it as ":X"(dog). The sharing
characteristic of both characters is that they describe
animals. This syndrome can be viewed as damage of
integrating semantic information.
Finally, the fourth one is disorder of choice ofauditory representation. In this syndrome, a pa­
tient makes an error in choosing phonological repre­
sentation of Kanji characters. For example, in " i3�", " i3" should be read as " -?: C" (koto), but a pa­
tient read it as "1-flv" (gen). This syndrome can beviewed as damage of integrating phonological informa­tion. Thus, these types suggest that dyslexia of Kanjicharacters is observed when neurons integrating infor­
mation on Kanji characters are damaged.

3

Computational Model

Computational models are based on three-layer
neural networks, whose learning algorithms are back­propagation[4].

- 7 -

!

( Letters )Letters
Input

f

Visual Representation
Kana.

Visual Representation
Kana

KanJi

Phenological athway

Auditory
Representation!

I

outputFigure 3: Classical Model�

Semantic PathwayII�

outputArticulation

For simplicity, we use the following ten Kanji char­acters for inputs:(tree), ;f:;f- (grove), � (forest),
� (cherry tree), t:JE (peach tree), tl (pine tree), f$:(plum tree), fit (bridge),(pillar), 1t (bamboo)", allof which are related with "wood". Each character isrepresented by a 16 to 16 square dot matrix. Further­more, to make a phonological representation of ea.ch
Kanji characters, we also use 16 Kana characters.
For outputs, each category is set to each Kanji char­acters. Thus, four bit strings is used to represent eachcategory, which corresponds to the meaning of eachcharacter. Furthermore, for simplicity, auditory rep­resentation is set to the same representation as theabove category. That is, four bit strings is also used
to represent pronunciation of each Kanji characters,
and constrains that this representation and semantic
representation is the same. This is a kind of simplifi­
cation of interaction between auditory representationand semantic representation.
Using these representations, we introduce two com­putational models to examine the effect of damage ofneurons on dyslexia syndromes as follows.

ttClassical Model: A computational model based onclassical hypothesis is illustrated by Fig. 3. Thismodel consists of one large component of neural net­
works. Visual representation corresponds to the in­put layer. Then, intermediate layer, corresponding to
GA region, are composed of 100 neul'.ons. Finally, au­ditory representation correspond to the output layer,
respectively. In each learning step, we give one Kanjicharacter and its corresponding Kana representation.
For example, whenis input, its corresponding
Kana representation "�"(ki) is given as an answer.
For each learning step, the output of neural network"*", Kanji

Auditory__., ( Meaning of a Word)
Representation .. - - - '-· ------'·

Articulation

*

Input

Figure 4: Iwata's Modelis compared with its answer, and total calculus is ter­minated after the total error rate is saturated.
lwata's Model: A computational model based on
Iwata's model is given in Fig. 4. This model con­sists of the following two neural networks: one de­scribes phonological pathway and the other represents
semantic pathway. For both models, visual represen­tation corresponds to an input layer. Then, phonolog­
ical pathway and semantic pathway are described byintermediate layers, each of which are composed of 50
neurons. Finally, auditory representation and mean­ing of a word correspond to the former and the latter
output layers, respectively. It is assumed that there
is no interaction between phonological pathway andsemantic pathway in this model.
In each learning step, one Kanji character is in­
put with its corresponding Kana representation andits category. For example, when a character
isinput for semantic pathway, its corresponding Kanarepresentation "�"(ki) is given as a input for phono­
logical pathway and its category "1" is given as ananswer. For each learning step, the output of neuralnetwork is compared with its answer, and total calcu­
lus is terminated after the total error rate is saturated.

"*"

Focal Lesioning: After learning steps, we make thefollowing "focal lesioning" steps for each model in or­
der to simulate the effect of damaged neu.rons on mis­reading. First, one neuron is randomly selected fromthe neural networks. And then, its neighbors (within k- 8 -Iwata's Model 4-­
Classical Model + Table 1: Learning Steps Needed for Termination
Model
Classical Model
Iwata's Modelp-value

Learning Steps
66700 ± 1319
56400 ± 795
0.00947

Error Rate
87.5 ± 8.5 (%)
89.2 ± 9.2 (%)
0.341

neurons) are set to the malfunctioned neurons. Then,we examine misclassification rate of this damaged net­
work by using the above ten Kanji characters for eachmodel. These two steps are iterated for 100 steps, andthen statistical significance is tested by t-test with re­spect to the error rate obtained.
In these procedures, first k is set to 1, and increased
with step one. Then the above test-procedures arerepeated until the misclassification rate is equal to 1.0.

4

Experimental Results

We make the following two experiments using the
aforementioned computational model. First, we com­pare learning steps needed for termination for eachcomputational model. For this experiment, we change
the order of Kanji characters, and start learning steps.
These steps are repeated for 1000 times, and the ob­tained steps are averaged, and tested using t-test. Sec­
ond, we compare the behavior of each network using
focal lesioning with respect to a parameter k. Fromthese experiments, the following three interesting re­sults are obtained.
The statisticsof this experiments are shown in Table 1. The conver­gence of Iwata's model is higher than that of classicalmodel, which is statistically significant with p < 0.01.
This result suggests that processing of Kana and Kanjishould be separated.
Learning Steps for Termination:

Behavior of Neural Networks: Phase-transition
behavior is observed with respect to k for each model:
when k is larger than the critical value, the error rateof a model suddenly becomes near 1.0 (Fig. 5). The
main difference between two models is that Iwata's
model is more robust than classical model.
Simulated Dyslexia Syndromes: The computa­
tional model based on Iwata's model can simulate two
types of dyslexia: disorder of capturing global struc­ture and confusion of morphology. For example,"�"is almost always misread as
when k is larger than"*"

0.9 -�;;:::-r-T7r-r--i-�r-i
0.8
0.7
0.6
0.5
Error Rate
0.4
0.3
0.2
0.1
0 L---'---'---'---'--...1,..._...L!.!.ftHtftHt-lU�
0 5 10 15 20 25 30 35 40 45 50
kFigure 5: Effect of Parameter k on Error Rate
30, and"�" is misread as"��" when k is larger than
20. Thus, those results, suggests that the former typeof dyslexia is usually observed when the focus of in­
farction is very large.
In summary, these three results suggest that Iwata's
model explains dylexia syndromes better than classicalone in the case of the Japanese language. However,this computational model can only explain two typesof Japanese dyslexia syndromes, which suggest that
our model is weak. Thus, it is our future work to refine
a computational model in order to explain other two
types of dyslexia syndromes.

5

Conclusion

In this paper, two computational models are intro­
duced to explain Japanese dyslexia syndromes. The
experimental results show that Iwata's model explain
dylexia syndromes better than classical model, al­though the computational model is a little weak to ex­
plain the whole aspects of acquired Japanese dyslexia.

References[1] Dejerine, J. Sur un cas de cecite verbale avec agra­
phie, suivi d'autopsie. CR. Soc . Biol., 9, 197-201,
1891.
[2] Geschwind, N. Disconnexion syndromes in animals
and man. Brain, 88, 237�294, 585�644, 1965.
[3] Iwata, M. Neural mechanism of reading and writ­ing in the Japanese language. Functional Neurol­ogy, 1, 43-52, 1986.
[4] McClelland, J., Rumelhart, D., et al.(eds.) Parallel
Distributed Processing, MIT press, 1986.

- 9 -Human-Type Control Using Supervisory Controller
Tae-dok Eom and Ju-Jang Lee
Department of Electrical Engineering
Korea Advanced Institute of Science and Technology
373-1 Kusong-dong Yusong-gu
Taejon 305-701 Korea
email: jjlee@ee.kaist.ac.kr
Abstract
Modeled from human neurons, various types of ar­tificial neurons are developed and applied to control
algorithm. As in the case that a man is trying to learna skill through training his own neurons, a teacher isneeded to suggest the artificial neural network a ref­erence signal and prevent it form getting unstable. In
this paper, supervisory control algorithm is devised tomeet these demands. It guarantees stability in thesense of the boundedness of tracking error and addi­
tionaly provides the thinning effect of network input
dimension so the reference signal is obtained. Simula­tion is also performed for simple two-link robot to show
the validity of the suggested algorithm.

1

Introduction

Since the artificial neural network was born as a re­sult of investigating and modeling of human neurons,
it has found wide applications due to the function ap­
proximation capability. Related to the control theory,
many researchers have tried to find the efficient way
of using neural network for control algorithm. Despite
the generic defects neural network bears, flourishing of
this field stems from a certain belief that the perfect
mimicry of human behavior will come true someday.
Though the modern technical improvement enables us
to precisely control the systems of severe nonlinearity,
no one doubts there are kinds of works in which man
is superior to the automatic controller. What we pur­sue through the study of neural network is to devise
the human-type controller that replaces the functions
which have been thought of man's own.
Just as the various kinds of human neurons incharge of various functions in different parts, there aremany kinds of artificial neural network. Each has theprominent figures and own applications. If we want

to design a human-type controller that has man's be­
havioral characteristics, it may be one promising way
to combine various kinds of neural networks and con­struct the whole system. Recently we suggested aconstruction paradigm[l] which employes feedforward
neural network, associative memory and chaotic neu­ron as its components. That is based on the conjecturethat these neurons take important parts in man's in­formation processing routine. In fact, associative rea­soning and chaotic phenomenon is the nature of, what
we call, the human.

In this paper, the supervisory control algorithm
which takes the role of 'teacher' is introduced. It pro­vides stability during the learning period of neural net­
work and extracts the reference signal - the functionof time - which should be approximated. This coin­cides with man's behavior of adjusting proper control
output as the function of time when he tries to learna skill.
We designed supervisory control algorithm for
robot dynamics to make an articulated robot whichlearns and acts like a human. For the robot dynam­
ics many researchers tried to apply neural network in
identifying the function in the dynamics [2] [3]. But
in this case of the robot dynamics the problem of sta­bility is serious and most of the researches use theway of robust control to stabilize the dynamics. The
supervisory control in this paper is adjusted parallelto the neuro-controller, so it makes the system stableand enables neural network to reduce the input dimen­sion. The primitive idea of the supervisory control is
initially proposed by Wang [4]. He developed it to sta­bilize the fuzzy adaptive controller. His supervisory
control algorithm is only useful for the single input
system but we extended it to be valid for the multi­
input system so it becomes applicable to the robot
dynamics.

- 10 -

2

Supervisory Control Algorithm for
Robot Dynamics

The robot dynamics can be described by the follow­
ing equation.
M(q)q + N(q,q) = T

Assumption 1 Inertia matrix M(q) is known, i.e.,
all uncertainties belong to N( q, q).
Assumption 2 The state vector q, q in (1) is mea­
surable.
Besides the assumptions above, robot dynamics has
following property.
Property 1 In robot dynamics described by {1), there
exist c 0, c 1 > 0 such that
2

Define F(q,q) = -M- 1 (q)N(q,q) and uthen the equation {1) is changed to,q = F(q,q) + u.

q = F(q, q) + (uc + u 8 ).

{ 2)
M -1 r,{3)(8)or equivalently in the other vector form,e = Ae + B(u +c(4)where eT = [qf qf] is the new error vector and K =[K1 K2 ] is chosen to make the matrix

( 9)
(10)

A= [

Define V = ½eT Pe where P is a symmetric posi­tive definite matrix satisfying the following Lyapunov
equation.

(11)

where Q is a positive definite matrix.
Now, the following theorem is derived based on the
Lyapunov's direct method.
Theorem 1 Consider the robot dynamics described by(1), satisfying Assumption 1- 2, and subject to control
given byM(q)uc if V < Vu
(12)
T M(q)u. if V > VM-{

andqd - Ke+ u s 1
Us 1

(13)

(eT PBflle T PBII·IIMII ·(co+ c1ll<ill 2 )(14)where VM > 0 is a constant specified by the designer.
Then, V < VM as t--+ oo.
Proof when V > VM,(5)
has the eigenvalues in the left half plane. Applying (4)to (3) renders the system asymptotically stable, but
this control input can not be implemented sinceF(q,q)
in u* is unknown. T his impractical input, u*, justhelps the conversion of (3) to tracking error dynamics.
Now, Suppose that the input, u, is the addition of
the neural network controller,Uc, and the supervisory
controller, u., that is
U =Uc+ Us

Us - u*)where

To induce the tracking error dynamics, define qd isthe desired trajectory, and let the error vector qe =q - qd, then the robot system can be controlled withthe following control input.
u* = -F(q,q) + qd - Ke(7)

By subtraction and addition of u*, the equation oferror is obtained:(1)

where M(q) is an inertia matrix and N(q,q) includes
centrifugal, Coriolis, and gravitational force. To apply
the supervisory control algorithm, the following as­
sumptions should be valid for the dynamics describedby equation {1).

N(q, q) :-:; co+ C1 ll<ill •Substituting (6) to (3), (3) becomes{6)- 11 -V=<_!eT Qe + eT PB(u - u*)2-½eT Qe+ eT PB(u s 1+F(q,q))-½eT Qe+ lle T PBII IIF(q,q)IIT+e PBu 81<- 1 eT Qe

2+lle T PBIIIIMll(IINII - {co +c1ll<ill 2 )}

< 0

(15)

□

3

Identification and Control Proce­dure+2lilg2 cos(q2)) + /2

Instead of the infeasible control input u* of (4), thecontroller which consists of neural network identifierand PD-type controller is used in our identification
procedure:(18)

m2(lg� + l1lg2 cos (q2 )) + 12

(19)

M12

(20)

m2lg� + /2

(21)

-m2l1lg2 sin(q2)(2q1cfo + q�)+m1glg1 cos(qi) + m2g(licos(qi)where Np takes the place of Fin (4). Of course any
kind of control scheme is available for the candidate ofUc, controller of (16) with simple FNN in it is chosen
in this paper. It is also obvious that with the help ofsupervisory controller, u 8 , the system controlled by Uc
does not lose stability even in the case the shape of
NF is far from that of F.
Another merit of using supervisory controller is thefact that neural network is trained within the specified
region about the desired trajectory. The identification
along the desired trajectory is very efficient for thetask of following the desired trajectory.
As iteration goes over and over, NF is trained by
the samples from F with back propagation algorithm,
and finally cancels out F. H the networks becomes
sufficiently accurate to verify the cancellation, thenthe error is described bye � AeSimulation Results

The simple two link robot is considered, which has
the following M(q) and N(q, q).

M11=m1li + /1 + m:i(l� + lg�

(22)m2lilg2 sin(q2)11f+m2glg2 cos(q1 + q2)(23)

where m1 = 2, m2 = 1, g = 9.8, lg1 = 0.5, lg2 = 0.5,
l1 = 1, l2 = 1, /1 = 0.5, /2 = 0.25, and q1 ,q2 denotethe joint angles.
The desired path in the cartesian coordinate is givenby (xd(t),Yd(t)) = (0.5 + 0.3sin(t), 0.5 + 0.3cos(t)).
qd1, qd2 is calculated by inverse kinematics. We choose
VM = 0.5, Q = diag[lO, · · ·, 10], c 0 = 10, c1 = 10
and K1 ,K2 are the diagonal matrices whose elementson the diagonal are 2. Two neural networks whichhave the structure of N1,10,6,2 are used to identify thefunction N(q, cj). The sigmoid functions for the output
nodes are scaled properly according to the variation
range of Ni( q, q), that is,(17), so that the system is controlled to be asymptoticallystable.
H VM is chosen as the value near zero and the tra­jectory in the state space is periodic, the time at eachcycle is sufficient as the input variable of neural net­
work. It takes many iterations to train neural networkwith the large input dimension. So the supervisory
controller brings about the additional effect of reduc­ing the computational burden.
H we set VM to zero, the error is not only bounded
but also asymptotically stable. However, this control
input resembles the relay control of VSC, so control
effort is excessive and the chattering occurs. Combin­
ing the supervisory control and PD-type control withneural network identifier, such problems can be elimi­
nated.

4+lg2cos ( q1 + q2))

S0 (x) = K tanh(x)(24)

where K is a scale factor.
The following figures show the results of the identi­
fication and control of the robot dynamics using neuralnetworks under the supervisory control while learning.
Fig.l depicts the tracking of the circle at the first rev­
olution. Fig.2 shows the tracking of the circle at the100th revolution.

5

Conclusion and Further Study

This paper is devoted to guarantee stability in theapplication of neural network to robot dynamics iden­tification. Developed supervisory control algorithm isuseful in both extracting the reference signal as thefunction of time and preventing neural network fromgetting unstable during the learning period. It takes
the role of the teacher just as a teacher maintains sta­bility and suggests a reference signal when a student
is trying to learn a skill. The whole human-type con­troller paradigm including supervisory control algo­rithm is now proceeding.

- 12 -

References

1.2 I[1] Tae-Dok Eom, Sung-Woo kim and Ju-Jang Lee,
"Human-type controller using various kinds ofneural networks," submited to !CNN '96, 1996.
[2] Man Zhihong, A. P. Paplinski and H. R. Wu,"A sliding mode control scheme using neural net­
works for robotic manipulators," ICARCV '94,
pp. 1738, 1994.
[3] Akio Ishiguro, Takeshi Furuhashi, Shigeru
Okumil and Yoshiki Uchikawa, "A neural networkcompensator for uncertainties of robotics manip­
ulators," IEEE Trans. Industrial Enectronics, vol.
39, no. 6, pp. 565-570, 1992.
[4] L. Wang, "Stable adaptive fuzzy control of non­linear systems," IEEE Trans. Fuzzy Systems, vol.
1, no. 2, pp. 146-155, 1993.

7

J

o.
:
r
0.6:-

0.
4
r
0.2 -

>.

o!

-0.2
-1

-0.5

0

0.5

1

X

Fig. 1 : Tracking control at the first revolution.(- : desired trajectory)

1.2,-----,-----r-----�----

o.:�

0.61
i
0.
41
0.2

l
..:

1

i

0 t"""--------"-----------!

-0.2 _______
-1

-0.5

....1,._________

0

0.5

1

X

Fig. 2 : Tracking control at. the 100th revolution.(­Jesired trajectory)- 13 -

CHAOTIC LOCAL SEARCH ALGORITHM
Changkyu Choi and Ju-Jang Lee
Department of Electrical Engineering
Korea Advanced Institute of Science and Technology
373-1 Kusong-Dong Yusong-Gu, Taejon 305-701, KOREA
Fax. : +82-42-869-3410
E-mail : jjlee@ee.kaist.ac.kr
Abstract
This paper describes a chaotic local search algorithmof an unstructured search space. The problem is: given
the quality function Q(X), where X is a vector of ad­
justable parameters x1, ..., Xnj find the value of X thatminimized Q. The proposed algorithm started basicallyfrom the gradient search technique but at prescribed
points, i.e. local minimum points, chaotic search algo­rithm is introduced by the dynamics of chaotic neuronmodel. It is systematically combined with the gradi­
ent search technique to automatically detect that points
and toggle its state between two search techniques. A
parallel formulation of this search algorithm is devel­
oped to determine a solution which represent the best
improvement among the feasible solutions from chaotic
neuron models each equipped with different parameter
set. Computer simulations for the application of the
motion planning problem of a mobile robot are pre­sented.

1

Introduct ion

Many problems of both practical and theoreticalimportance concern themselves with the choice of abest configuration or set of parameters to achieve somegoal. Over the past few decades a hierarchy of suchproblems has emerged, together with a correspond­
ing collection of techniques for their solution. At one
end of this hierarchy is the general nonlinear program­ming problem. There are some problems for which
no efliecient algorithm is known and these cannot be
solved by any known polynomial algorithm. By theway, there grow other techniques which are motivatedby nature. Genetic algorithm is a typical example.
Our method is based on a chaotic dynamics which look
like random.
In general, local optimality does not imply global
optimality. Algorithms may get stuck in local min­ima, thus providing suboptimal solutions. For somesystems, this suboptimal solution is enough in a view
point of the system performance as a whole. Howeverin other cases only global optimality makes sense. For
example, in robot motion planning problems the sub­optimal solution ( i.e. local minima) is meaningless.
Though the value of quality function increases, it's
better to move to any direction than to stay there.
The organization of this paper is as follows. Chaotic
neuron models are discussed in section 2. In section 3,
a chaotic search technique is proposed. A parallel for­mulation of this algorithm is also given. Application
to the motion planning problem of a mobile robot isinvestigated in sections 4. Finally, we draw some con­clusions and discuss the proposed search algorithm..

2

Chaotic Neuron Models

2.1

I-dimensional case

Aihara et al.[1] suggest a chaotic neuron model
based upon Caianiello's neuron equation. The chaotic
neuron model which includes conventional models ofneural networks reads+ 1) = f(A(n) - a L k g(p(n - r)) - 8),
n

p(n

r

r=O

(1 )

where p(n + 1) is the output of the neuron at the dis­crete time n + 1, which takes an analog value between
0 and 1; f is a continuous output function, e.g. thesigmoid function; and g is a function describing therelationship between the analog output and the mag­nitude of the refractoriness to the following stimula­tion. A(n) is the strength of the input at n, a is apositive parameter and k is the damping factor of therefractoriness, which takes a value between O and 1.
By defining the internal state q(n + 1) as follows,- 14 -

q(n + 1) = A(n) - a L k g(p(n - r)) - o,
n

r

r=O

(2)

(1) can be simplified as (3) and (4):p(n + 1) = f(q(n + 1)),(3)

q(n + 1) = kq(n)- ag(f(q(n))) + a(n),(4)a(n) = A(n)- kA(n- 1)- (1- k)o.

(5)

where

The variables p, q, a and A are 2-dimensional vectors,
corresponding to p, q, a and A of 1-dim. case, respec­tively, and /2x1 is [1 1j T . According to the above mod­ifications, the output function need not to be changed,
while the refractory function takes the form of 2-dim.
Gaussian distribution function (Fig.3):2g(p(n)) = {IT the output function, f, and the refractory function,g, take the following forms (Fig.1):
f(q) =pp
2
1 + e-q/11?

(6)

(7)
where 4> and u controls the shape of the functions /and g, respectively, and p the magnitude of the out­
put function. From Fig.1 the refractory function g isa kind of hysteresis function. As the input increases,
the upper Gaussian function is traced. So, it gives a
directionality to q and it is analogous to the momen­tum. The internal state q( n + 1) is drastically changed'
from A(n) when q(n) is near 0. Thus the output un­dergoes unstable motions near the origin. Fig.2 shows
the patterns of output p with respect to a(n). Whenthe magnitude of input is larger than 0.2, the output isalmost the same as the input. But the input is near O,
the output oscillates chaotically. It has the tendency
to wander positive region when the input is positive
while it wanders negative region when the input 1s
negative due to the hysteresis-like refractoriness.

2.2

if p(n)- p.(n- 1) � 0

2
2_ _!_ e-llP(n)ll / 2 a2,ra 2otherwise •(11)

where 4> and u controls the shape of the functions /and g, respectively, and p the magnitude of the out­
put function as in the case of 1-dim. chaotic neuronmodel. Bifurcation diagram of 2-dim. chaotic neuron
resembles the motion of Fig.2 when p(n) is near 0.
When only one component of p(n) is near O, it does
not reveal chaotic motion. Extensions to the higher
dimensional neurons are straightforward.

3

Chaotic Search Technique

3.1

Algorithm

byConsider a configuration vector X which is update(12)

Basically, AXn is a negative gradient vector. But
naive usage of it to (12), makes that X n is vulnerable
to get stuck in local minima. To avoid this situation,a negative gradient vector is calculated and it serves
as an input of chaotic neuron model, so a(n) = AXn.
Then, (12) becomes:(13)2-dimensional case

By defining the internal state q(n + 1) similar tothat in (2), (3) and (4) can be modified as (8) and (9):i = x, y,(8)q(n + 1) = kq(n)- ag(p(n))l2x1 + a(n),(9)

p,;(n + 1) = f(q,;(n + 1)),2_l_-llP(n)ll /2a2,ra 2 ewherep(n + 1) = [p,., (n + 1) p11 (n + l)f,

For larger values of AX, p(n) is almost the same as
AX. However as seen in Fig.2, there are infinitely
many values of q(n + 1) and thus infinitely many val­ues of p(n + 1) corresponding to AX near the origin
exist. Therefore we need a selection mechanism. For
simplicity, we choose values of p(n) randomly and cal­culate its improvement. After several trials, all thetrials are compared and the only one which produces
the best improvement among these is selected.
Algorithm 1 : Single Processorq(n + 1) = [q,., (n + 1) q11 (n + l)f
anda(n) = A(n) - kA(n - 1)- (1- k)ol2x1•(10)- 15 -

Repeat until goal reached or global time-out
1. gradient descent until local minima
It repeat several times
choose a value in the chaotic motion

tributed. The probability 1 - Pk(t) that the solution
time on k processors will exceed t is just the probabil­
ity that none of the k processors will find a solution
within time t:

calculate its improvement
choose one for the best improvement
among these

apply it to {19}

If still in local minima, GoTo 1!

(14)

GoTo 1

The above algorithm, however, cannot guarantee thatthe configuration vector can produce the global op­
timality. There is no justification to use the specificvalues of p, u, <l> and so on. Varying the parameters,
a, k, p, <l> and u, the chaotic neuron models reveal
rich dynamics. In a local minima with very high po­
tential, it is desirable for the width and the height ofregions, which reveal chaotic dynamics, to be longer
than those of regions in a local minima with lower
potential. Hence, it is further needed to refine thisalgorithm.

3.2

Parallel Formulation

Assuming that a k-processor parallel search exists.
Each processor is equipped with a different set of pa­
rameter values, p, "I, <l> and so on. Thus, the regionsthat reveal chaotic dynamics are different from pro­
cessor to processor. Some parallel search processor
may have wide chaotic motion regions than those ofanother. An outline of the algorithm is shown below.
Algorithm 2 : k-processor parallel search
Repeat until one of the processors find the goal
or global time-out
1. gradient descent until local minima
2. for each. k-processors, repeat several times
choose a value in the chaotic motion
calculate its improvement
choose one for the best improvement
among these
apply it to {19}
If still in local minima, GoTo 2
GoTo 1

We can give a brief theoretical explanation for the
success of parallel search schemes. Let P 1 ( t) be theprobability that a single processor will find a solution
within time t, and let Pk(t) be the probability thata k-processor parallel search scheme will find a solu­
tion within time t. Assume that the probability P1 (t)is identical to every k processors and let the randomvariable T1i be time it would take processor P to findia solution, if allowed to run to completion; Since thisis equivalent to running multiple trials on a single pro­
cessor, the T 1 i's are independent and identically dis-

To interpret this formula, suppose a single processor
has only a 10% probability that it can solve the prob­
lem within a given time t10%• Then a 32 processor
system has over a 96%probability of finding a solution
within t10%. A 64 processor -system has over a 99%
chance of doing the same.

4

Motion Planning Problem

Virtual Force Field (VFF) method is widely usedin motion-planning problem because of its simplicity.
The total virtual force F t is thought to be a negative
gradient of the electrostatic potential field. A virtual
attractive force, F a , of constant magnitude is applied
to the mobile robot pulling it toward the target and all
virtual repulsive forces from obstacles are added up toyield the resultant repulsive force, F r . H the total force
happens to be zero, then the robot cannot move andthe velocity of the robot becomes zero. In this case, therobot fails to find the target. In order to overcome thissituation, our algorithm can be applied. Simulation
results are not given here for the page limitaion, but
you will find similar successful results in [21[3].
Our method differs from previous heuristic ap­
proaches(see references mentioned in [2] and [31), in
which they determined whether the trap or local min­ima situation occurred, and then changed its mode toone of the case-by-case heuristic algorithms. Although
our method is similar, in that they are a kind of detec­tion method, we systematically combine the detection
stage and the solution of trap situations. Using Paral­
lel search algorithms ( ONM,) each equipped with dif­ferent parameter sets, P,, to unknown environments,
our path-planner can be flexibly applied to various sit­uations and is superior to other methods. This is themain difference between ours and other methods.

5Conclusions

We suggests a 2-dim. as well as 1-dim. chaotic
neuron model equipped with Gaussian function having
hysteresis as a refractoriness. Due to the hysteresis­
like refractoriness, chaotic dynamics appears near the- 16 -

ongm. Also, it has a directionality so that chaotic
motions are confined to specific regions.
By utilizing a chaotic search algorithm, the local
minima problem is solved. and the parallel search
scheme enhances the probability to place the configu­
ration vector to global optimality. Application to the
motion-planning problem shows the effectiveness ofthe proposed algorithm. Hence, the use of intentional
chaotic dynamics might improve the performance ofthe system of interest.

Output function, f

1

0.5
0
-0.5
-1
-5

References

05

Refractory function, g
4 ,------,-----,[1] K. Aihara, T. Takabe and M. Toyoda, "Chaotic
Neural Networks," Physics Letters A, Vol. 144,
No. 6, pp. 333-340, 1990.

2[2] C. Choi, S.-G. Hong, J.-H. Shin, I.-K. Jeong and
J.-J. Lee, "Dynamical Path-Planning Algorithm
Using Chaotic Neuron Model," in Proc. IROS,
Vol. 2, pp. 456-461, Pittsburgh, PA, Aug. 5-9,
1995.

0 I----<,
-2

\

\\

/

I

,>,------,

,'

I

/

\

\' I
','

-4�---�----'-1[3] C. Choi and J.-J. Lee, "Dynamical Path-Planning
Algorithm of a Mobile Robot : Local Min­ima Problem and Nonstationary Environments,"
Mechatronics, 1995. (in press)

Fig.1

0

1

Functions used in 1-dim. CNM.

Ou lru l,r

1 �---�--------r----.-----,
0.8
0.£,

0.4
0.2
0

-0.2
-0.4

.......

-0.£,

-0.8

'�
•

...

-1L-----'--------'--------''------'
-1

Fig.3

-0.5

0

0.5

Fig.2 Output, p of 1-dim. CNM.

Refractory function, g of 2-dim. CNM.

- 17 -1

A Self Organizing Genetic Algorithm
II-Kwon Jeong and Ju-Jang Lee
Department of Electrical Engineering
Korea Advanced Institute of Science and Technology

373 - 1 Kusong-dong, Yusong-gu, Taejon 305 - 701, Korea
FAX: +82-42-869-3410 E-mail: jik@odyssey.kaist.ac.kr

Abstract
A genetic algorithm(GA) has some parameters thatshould be determined before execution. We propose aself organizing genetic algorithm(SOGA) which sets
GA parameters such as population size, crossoverprobability and mutation probability adaptively dur­ing the execution of a genetic algorithm. In SOGA,
GA parameters change according to adaptation rules.
We discuss about SOGA and other approaches foradapting operator probabilities in GAs. The validityof the proposed algorithm will be verified in a simula­tion example of system identification.
Keywords Genetic algorithm, Self organizing

1

Introduction

Genetic algorithms(GAs) are robust search and op­
timization techniques which are based on the natural
selection and genetics. GAs are applied in a num­
ber of practical problems nowadays. GAs are differ­ent from conventional optimization methods in several
ways. GA is a parallel and global search method which
searches multiple points so it is more likely to get the
global optimum. It makes no assumption on the search
space so, it can be easily applied to various problems.
In control area, it has been applied to identification[1], adaptation and neural network controller [3] [4].
However GAs are inherently slow and not good at finetuning of the solutions.
The GA may be thought as an evolutionary pro­
cess where a population of solutions evolves over asequence of generations. During each generation, thefitness(goodness) of each solution is calculated, andsolutions are selected for reproduction based on theirfitness. The probability of survival of a solution is pro­portional to its fitness value. This process is based on
the principle of 'Survival of the fittest'. The repro­
ducted solutions then undergo recombination which

consists of crossover and mutation. Genetic represen­tation may differ from the real form of the param­
eters of the solutions. F ixed-length and binary en­
coded strings have been widely used for representing
solutions since they provide the maximum number ofschemata and as they are simple to implement [2].
In this paper we describe a self organizing genetic
algorithm(SOGA) for multimodal function optimiza­tion. The choice of the crossover probability, P c andthe mutation probability, P m is known to critically af­
fect the behavior and performance of the GA. Thougha number of generalized guidelines exist in the litera­ture for choosing P c and P m , these guidelines are in­adequate as the choice of optimal P c and P m becomes
specific to the problem under consideration. The sizeof a population is another important parameter that
affects the performance of the algorithm. In our al­
gorithm, P c , P m and the size of the population aredetermined adaptively by the GA itself to realize thetwin goals of maintaining diversity in the populationand sustaining the convergence capacity of the GA.
Recently, several works have been done for adapt­ing P c and P m · Operator probabilities vary according
to the population maximum fitness and the popula­tion mean fitness in [6]. But, their adapt,, rules are
not general as ours. Bryant used the relat1 credit ofeach operator over some generations [7]. However, hisalgorithm needs more memory and computation timecompared to simple GA. The paper is organized as fol­lows. In section 2, a simple genetic algorithm is briefly
described. Section 3 describes our algorithm, SOGA.
In section 4, we present simulation results. The con­clusions are presented in section 5.

2

A Simple Genetic Algorithm

GA is a search method based on the natural
selection and genetics. GA is computationally sim­
ple yet powerful and it is not limited by a.:.sumptions- 18 -

about the search space. The most important goal of
optimization should be improvement. Although GA
cannot guarantee that the solution will converge tothe optimum, it tries to find the optimum, that is, itworks for the improvement.
Following the model of evolution, GA establish a
population of individuals, where each individual cor­
responds to a point in the search space. An objec­
tive function is applied to each individual to evaluate
their fitness. Using genetic operators, a next genera­tion is formed based upon the survival of the fittest.
Therefore, the evolution of individuals from genera­tion to generation tends to result in fitter individuals,
solutions, in the search space. Empirical studies haveshown that genetic algorithms do converge on global
optima for many problems including NP-hard ones.
Simple GA has three basic genetic operators: Re­production, Crossover and Mutation. There are threedifferences of GA from random search. F irst, the ex­istence of the direction of search due to the selection
probability. Second, the fact that the better strings
make more offsprings and finally, being likely to be
improved in average fitness over generations.

3

Self Organizing Genetic Algorithm

In optimizing unimodal functions, it is importantthat the GA should be able to converge to the opti­
mum in as few generations as possible. For multimidal
functions, there is a need to be able to locate the re­gion in which the global optimum exists, and then con­verge to the optimum. GAs possess poor hill-climbing
capabilities and, they are vulnerable to getting stuckat a local optimum especially when the populations
are small. The significance of P c and P m in control­ling GA performance has long been acknowledged in
GA research. The higher the value of P c , the quicker
are the new solutions introduced into the population.
As P c increases, however, solutions can be disrupted
faster than selection can exploit them. Large valueof P m transform the GA into a purely random search,while some mutation is required to prevent the prema­ture convergence of the GA to suboptimal solutions.
The population size also affects the GA performance.
Premature covergence may occur when the popula­tion size is small, while a large population size makes
the algorithm slow. Usually, the choice of P c , P m andpopulation size is left to the user to be determinedstatically prior to the execution of the GA.
To overcome the above-stated problem of difficultyin choosing the GA parameters, we suggest the follow-

ing expressions which are main components of SOGA.
P c = k1(fma:,; - J')/(fma:,; - fm i n ) + k2

(1)

k1 + k2 � 1
Pm=

k3(fma:,; - /)/(!ma:,; - fmi n ) + k4
k3 + k 4 � 1{2)

where Ima :,; is the maximum fitness value and !mi nis the minimum fitness value. f' is the larger of thefitness values of the solutions to be crossed. k3 andthe population size, Npop are changed adaptively us­
ing the following procedure.
1. Initialize k3•
2. i +-- i + 1, generation i.
3. If the fittest is the same for nre • et generations, thenNpop +-- Npop + n1 and go to step 1.
4. Npop +-- Npop - n2 , k3 +-- c * k3, go to step 2.
where n 1 and n2 are positive constants of integers, andc < 1. As we can see from (1), P c is a linear function off' which varies from k1 + k2 to k2 as f' changes from/mi n to fma:,;. P m varies in a similar fashion. Thus, thehigher the fitness of a solution, the lower the probabil­
ity of crossover or mutation of the solution. Therefore,
we are able to preserve 'good' solutions of the popu­
lation while the low fitness solutions prevent the GA
from getting stuck at a local optimum. Note that k3
is designed to decrease exponentially over generations.
After few generations k3 vanishes to O from its initialvalue and the mutation operator becomes to behave
like a normal one. But, when the fittest is the samefor nre • et generations, that is, the GA is getting stuckat a local optimum, P m is enlarged to its initial value
to move the search to the global optimum and, thepopulation size Npop is increased to search wider re­gion of the search space. When the algorithm is in
its normal operation state, Npop is decreased at every
generations to speed up the algorithm.
There have been similar works to improve the GA.
[3] incorporates simulated annealing technique into theGA. [4] uses a fitness modification technique and an
adaptive mutation probability. A local improvement
operator is introduced in [5]. [6] proposes adaptive
probabilities of crossover and mutation. However, its
adaptive rules are not as general as ours and, the adap­
tation of the population size is not considered.

4

Simulation

The problem considered here is the same as thosein [1][3]. It is presented for comparison purpose. The- l9 -

5

object system is a discrete time system:

where q-1 is the backward shift operator and the ob­jective is identifing A(q- 1), B(q- 1) and delay d using
the given input u(t) and the output y(t). We define
the error sequence as
77(t) = y(t) - y(t)

(4)

with
The fitness function to be maximized isF(t) = 1/

L (11(t- i))2w

B(q- 1)1
1.0 - l.5q- + 0.7 q -2

b o (l.O + o.5q- 1 + o.oq- 2)[P1,P2] = [0.75,-0.37], [z1,z2] = [-0.25,0.25]

(7)
(8)
(9)

where b 0 is 1 and the delay d was set to 1. We apply
a simple GA and SOGA to identify p1, p2, z1, z2,b 0 and d. b 0 is assumed to be in [O,2] and the poles
and zeros in [-1, l]. We use binary encoding. 7 bits
were used for each parameter except for d (2 bits), so
the resolution is slightly smaller than 0.02. A string
consists of 37 bits. We used P c = 0.8, P m = 0.01,
Np ap = 100 and w = 30 for the simple GA. We used
k1 = k2 = 0.5, initial k3 = 0.9, k4 = 0.01, initial
Np ap = 50, nreaet = 5, n1 = 6, n2 = 2, C = 0.9 andw = 30 for SOGA. Input for the sample data isu(t) = sin(t) - sin(t/2.5) + random(-1A self organizing genetic algorithm(SOGA) was de­signed to prevent the premature convergence and tosustain the convergence capacity of the GA. SOGA
determines P c , P m and Np op automatically using its
adaptive rule so, we do not have to determine the val­ues for the parameter prior to the execution of GA.
Simulation results indicate that SOGA has adaptive
characteristics and improved hill-climbing capability
compared to the simple GA. Using adaptive popula­tion size, execution time of the algorithm is signifi­
cantly lowered. Furthur work includes the theoreticalanalysis of SOGA.

(6)

i=O

where w represents window size. The system polyno­
mials, poles and zeros in the reparameterized plane [1]are the following:
1
A(q- )

Conclusion~ 1) (10)We show the input and output used for the sample inFig. 1. One simulation was done using 200 samples
with 3 generations per one sample, that is, 600 gener­
ations. 10 simulations were done for each algorithm.
Fig. 2
4 show the average of the identification re­sults of the poles with simple GA. The true value of p2
is -0.371. But, the limitation on the resolution due tobinary coding makes pz equal to -0.375. Fig. 5 ~ 7show the average of the results with SOGA. It shows
the better hill-climbing and optimum finding capabil­
ity than simple GA. The average of the population
sizes of 10 simulations was found to be 43.4, which ismuch smaller than that of the simple GA though theperformance of SOGA is much better than that of thesimple GA.

~

- 20

References[1] K. Kristinsson and G. A. Dumont, "System iden­tification and control using genetic algorithms,"
IEEE Trans. Syst., Man, Cybern., vol. 22, no. 5,
pp. 1033-1046, Sep., 1992.
[2] D. E. Goldberg, Genetic Algorithms in Search, Op­
timization, and Machine Learning, Reading, MA:
Addison-Wesley, 1989.
[3] I. K. Jeong and J. J. Lee, "Genetic algorithms
and neural networks for identification and control,"

Proceedings of the First Asian Control Conference,

pp. 697-700, 1994.
[4] I. K. Jeong and J. J. Lee, "A modified genetic al­
gorithm for neurocontrollers," IEEE International
Conference on Evolutionary Computing, pp. 306311, 1995.
[5] J. A. Miller, W . D. Potter, R. V. gandham and C.
N. Lapena, "An evaluation of local improvement
operators for genetic algorithms," IEEE Trans.
Syst., Man, Cybern., vol. 23, no. 5, pp. 1340-1351,
Sep., 1993.
[6] M. Srinivas and L. M. Patnaik, "Adaptive prob­
abilities of crossover and mutation in genetic al­
gorithms," IEEE Trans. Syst., Man, Cybern., vol.
24, no. 4, pp. 656-667, April, 1994.
[7] B. A. Julstrom, "What have you done for me
lately?
Adapting operator probabilities in a
steady-state genetic algorithm," Proceedings of the
Sixth Int. Conj. on Genetic Algorithms, pp. 81-87,
1995.

0.75

pl

0.6

0.4

0.2
.5

-0.2
--,P2

Gcncraliun

Fig. 1. The sample input and output

Fig. 5. Identification of the poles using SOGA

0.6

0.4

0.2

-0.5

Gcncraliun

Fig. 6. Identification of the zeros using SOGA

Fig. 2. Identification of the poles using a simple GA

0.2
1.6

1l
;r
!!
<

1.4

-0.25

1.2

-0.4

-0.6

Gcncratiun

Generation

Fig. 7. Identification of b 0 and d using SOGA

Fig. 3. Identification of the zeros using a simple GA

1.4

Gcncratiun

Fig. 4. Identification of b 0 and d using a simple GA

21

Local Motion Planner for Unicycle-like Vehicle :Guaranteeing Collision Avoidance Even in Unknown Environment
Sun-Gi Hong, Changkyu Choi, Kang-Bark Park, and Ju-Jang Lee
Department of Electrical Engineering
Korea Advanced Institute of Science and Technology
373 - 1 Kusong-dong, Yusong-gu, Taejon 305 - 701, Korea
FAX: +82-42-869-3410 E-mail: sghong@odyssey.kaist.ac.kr

Abstract
This paper deals with the motion planning prob­
lem for a unicycle-like vehicle whose motion is con­fined by its own nonholonomic constraints. Gener­
ally, it is known that it is difficult to develop the feed­back control scheme of nonholonomic systems due tothe fact that nonholonomic system cannot be stabi­
lized to a given any configuration based on a smooth
time-invariant state feedback law. In this paper, we
exploit a simple feedback scheme for unicycle model,
based on an approximation of the desired configura­tion given by local holonomic planner that ignores mo­tion constraints. To prevent the approximation error
from causing the collision with obstacles, we propose
an inequality constraint, based on the analysis of ve­
hicle's motion , which is assumed to be governed by
the constant control input during the sensor's sam­pling time. Consequently, we formulate our problem
as the constrained optimization problem and the feed­back scheme based on local sensor information is es­
tablished by simply solving this problem. Throughsome simulations, we confirm the validity and effec­
tiveness of our algorithm.

1

Introduction

This paper deals with the problem of planning con­strained motions where the constraints are nonholo­
nomic in nature. Specifically, we focus on the motion
planning of the unicycle-like vehicle using the feedback
control scheme.
Generally, it is difficult for the nonholonomic sys­
tems to exploit the feedback control law due to thefact that nonholonomic system cannot be stabilized toa given any configuration based on the continuous andsmooth time-invariant state feedback law [1]. This mo­tivated some complex feedback laws, for example, thetime-periodic function based law [2] and so on. On

the other hand, these previous methods do not take
into account the presence of the unknown obstacles
and require a sequence of feasible targets to completea point-to-point motion such as a car parking problem.
In contrast to these nonholonomic planners, the lo­
cal holonomic planners furnish the simple and pow­
erful feedback scheme for the mobile robot navigating
through the partially known environment. Using thesemerits, A. D. Luca and his collegues [3] have proposeda feasible projection strategy to modify the output of
local holonomic planner. Their scheme provides a very
simple and powerful feedback motion planner in a local
sense. Instead, their method has a serious drawbackthat it cannot guarantee the collision avoidance withobstacles in spite of the collision avoidance being abasic and intrinsic demand for motion planning.
In this paper, we focus only on the goal position
reaching problem. The basic idea of this paper is on
the extension of A. D. Luca et al.'s work [3], in a sensethat our proposed algorithm approximates the desiredconfiguration given by the local holonomic planner.
We propose the inequality constraints to prevent theapproximation error from causing the collision withobstacles. Note that this is very different from A. D.
Luca et al.'s work [3]. For this purpose, we analyze thelocal motion of vehicle, which is assumed to be gov­
erned by the constant control input during the sensor's
sampling time. Finally, we formulate our problem as
the constrained optimization problem under the as­
sumption of the lower-bounded curvature due to themechanical restrictions. Through the computer simu­
lations, we will show the validity and effectiveness of
our algorithm.

2

Motion Planner for Unicycle Model

Consider a unicycle-like vehicle positioned on theplane !R 2 with respect to the base frame < b >, whose- 22 -

motion is governed by the combined action of boththe angular velocity w and the linear velocity vector
v. A linear velocity vector v is assumed to be always
directed as x axis of its attached frame < a > where its
origin is located at the center of vehicle, as depicted inFig. 1. Then, the kinematic model of the unicycle-likevehicle, which involves the vehicle's Cartesian position
x, y and its own orientation fJ, is known as follows [3]:where v is the linear velocity and w is the angularvelocity. The values of x, y, fJ are all measured withrespect to the base frame < b >. This model takes
into account the nonholonomic constraint
y cosfJ - x sinfJ = 0(2)that specifies the tangent direction along any feasible
path for the vehicle.
As stated earlier, A. D. Luca et al. [3] have pre­sented a direct projection strategy to modify the out­
put of local holonomic planner in the on-line manner.
Their approach generates the velocity level control in­puts that implement the desired motion, which is up­
dated by a local holonomic planner, in the least-square
sense. For any desired motion xd, their solution is
given by the pseudo-inversion, as follows:u

G# (x)xd[GT (x)G(x)i-1GT (x)xd

motion according to the analysis of the following equa­tion. Note that we denote this fixed frame as < /, a >through this paper.

( ;;:: ) ( :r:]::: �) ( ;:;,: )(x1, a (O),Y1 , a (O),fJ1, a (O))T = (O,O,O)T ,o $ t $ Ste(4)where Vf ,a and Wf , a have the constant values respec­tively and Ste is the time interval for the control input
updating, that is, the sensor's sampling time. The sub­script "f, a" denotes that the values of Xf , a ,YJ, a ,0 1 , a
are all measured with respect to the attached frame
which is fixed at the instant for the control input up­
dating ( that is, t = 0 in a local view). As shown in
Fig. 2, a constant control input Uf,a = (v1, a , Wf , a fyields a circular path with a turning radius p during
Ste . Note that the value of Vf, a is not negative at all
because a linear velocity vector v1 ,a is assumed to bealways directed as x axis of the attached frame.
Integrating the equation for the main direction ofvehicle yieldstlo

0 1 , a dTtlo

0 1 , a (t )WJ, at(0 1 , a (O) = 0)

(5)
(6)

where O $ t $ Ste . Then, the value of turning radiusp can be computed as follows:(3)

where x E !R is a configuration vector of the gener­
alized coordinate, u E !R n -m(n > m) is an admissible
control input vector, and G(x) is a uncycle model.
Although their simple and efficient local planner fur­nishes a feasible motion in most situations where in­cluding even the unknown environment, no guaran­tee for the collision avoidance can cause some seri­
ous problems. This is because the least-square error[xd -G(x)uf[xd -G(x)u] can drive the vehicle into
the obstacle region when the desired motion xd does
not hold the nonholonomic constraints.
Thus, now we require to analyze the vehicle's mo­tion, which is assumed to be governed by the constantcontrol input vector u = (v, wf during the time in­terval of the command updating. This will be helpful
to derive the constraints for preventing the approxi­
mation error from causing the collision. For simplicity
of analysis, we fix the attached frame < a > shown in
Fig. 1 at the command updating instant. And then,with respect to this frame, we can obtain the vehicle's

Wf , a dr

t

lo

n

p

I:: : I(d,a:� ) 2+ ( d: ) 2 dr (7)a(S)

Thus, for the given Vf, a and WJ, a , the vehicle tracks
the following trajectory during Ste .
XJ, a (t)=YJ, a (t) =sgn(w1, a )psinfJ1 , a (t)sgn(w1 , a )P(l - cos0 1, a (t))(9)

where the function of sgn( WJ, a ) returns the sign value.
To prevent the collision with obstacles, we shouldchoose the control input ur a = (v1, a ,w1 , a f so thatthe predicted locus in the e�uation (9) will not inter­
sect any edges of obstacles. For this purpose, we as­sume that the function of d( ,t, 1,a) represents the modelof vehicle's surroundings and it returns a proximity
measure in the direction of ,Pf, a with respect to the
fixed frame < f, a >. This model is updated per ev­
ery Ste . During Ste , the following constraint should- 23 -

be satisfied to guarantee the collsion avoidance for the
given constant control input Uf,a = (v1,a, WJ,a) T .
d(t/i1,a(t))>

JxJ,a(t) + YJ,a(t)o , a (t))I
2pl sin( ,2

2pl sin(t/i1, a (t))I

(10)
(11)

(12)

where VJJ,a(t) = w 2,. t , 0::; t::; 5tc and p = I��: I- Fig.
2 will be helpful for the comprehensive unders'tandingof the derivation for the above equations.
Based on the derived results so far, we can formu­
late our algorithm as follows:
Minimize [xd - G(x)uj T [xd - G(x)u]subject totd(t/i1,a( )) > 2plsin(t/i1, a (t))I

(13)

(14)
(15)
where O ::; t ::; 5t c . The solution of the equation (13)can be found based on the nonlinear programming
technique. The relationship between the base frame< b > and the fixed frame < f, a > can be easily im­
plemented through a simple transformation matrix.

3

Simulation Results

We demonstrated the proposed motion planner fora unicycle model in the presence with the circular ob­
stacles, based on the potential field approach as the
local holonomic motion planner.
Using the potential field approach, the desired ve­
locity vector <id = (:.i;d, Yd) can be obtained from thefollowing equation at every sampling time.

where Ua (·) is an attractive potential function andUr(·) is a repulsive potential function [4]. To com­plete finding the desired configuration, we require toassign the desired rotation of e d . For simplicity, wechoose the desired steering input as follows:.
1 Yd
0d = tan- -:- - 0
Xd

For the uncertain environment cluttered with circu­
lar osbtacles, we perform the simulation for the pro­
posed motion planner. All the parameter values usedin the simulation are given in Table 1. To solve theproblem in equation (13), we use the exhaustive searchtechnique because the serach domain is relatively small
and its method is very simple for the computer anal­
ysis. We believe that another nonlinear programming
techniques can also solve this problem in a short time.
In Fig. 3, we can show the successful result of thelocal holonomic planner. The nonholonomic motions
obtained for the initial 00 = 0 are shown in Fig. 4,
with the associated desired configuration. In Fig. 4,
we can see that the constructed path by the proposed
algorithm is different from the holonomic one. Thisis because our algorithm approximates the holonomic
path in local sense. Anyway, we confirm that the posi­tion error goes to zero in the terminal phase. We haveapplied our method to several other situations, andthe satisfactory results, which guarantee the collision
avoidance, were always obtained.

4

Conclusions

We have presented a efficient motion planner for a
unicycle-like vehicle in the presence of unknown ob­
stacles. Our feedback scheme can be utilized in a realtime, to prevent the vehicle from colliding with ob­
stacles. This makes our method be useful in a realapplication.
The proposed scheme has been applied to theunicycle-like vehicle. The potential field approach has
been used as the local holonomic motion planner. The
simulation results confirmed that the obtained config­
uration approximates the desired motion in a least­square and local sense and the proposed planner pro­vides the satisfactory results. Since the proposed plan­ner is based on the local strategy, even in the unknown
environment the vehicle system can reach the target
successfully.
Future research directions include a real applicationof our algorithm in a real world.

References(17)

where the function of tan- 1 ( ·) returns the radian value
from O to 2,r. Therefore, we can compute the desiredconfiguration xd for the equation {13).

- 24 -[1] R. W . Brockett, "Asymptotic Stability and Feed­back Stabilization," in Differential GeometricControl Theory, R. W . Brockett, R. S. Millmann.
and H. J. Sussmann (Eds.), Birkhauser, Bost01.,
pp. 181-191, 1983.

[2] J. -B. Pomet, "Explicit Design of Time Varying
Stabilizing Feedback Laws for a Class of Control­
lable Systems without Drift," System and Control
Letters, Vol. 18, pp. 139-145, 1992.
[3] Alessandro De Luca and Giuseppe Oriolo, "Local
Incremental Planning for Nonholonomic Mobile
Robots," Proc. of IEEE Con/. on Robotics andAutomation, pp. 104-110, San Diego, California,
May, 1994.
[4] 0. Khatib, "Real Time Obstacle Avoidance for
Manipulators and Mobile Robots," International
Journal of Robotics Research, Vol. 5, No. 1, pp.
90-96, 1986.

00Fig. 3. A holonomic motion with potential fieldapproach

Unicycle-like robot

T

v•

G

8
<b>

X

Fig. 1. A unicycle-like robot w.r.t the base frameFig. 4. A nonholonomic motion with potential fieldapproach

Table 1. All the parameter values used in thesimulations<{,a>

0 ... ---,-.,,1..1) ---,

Environment size
Radius of circular robot
The number of sensors
The time interval forthe command updating ot c
Maximum speed Vmax
Sensor range:,

Fig. 2. A motion analysis during the time interval ot c

- 25 -

300m x 300m
3m
36
0.5sec2.0m/sec

20.0m

The Improved Radial Basis Function Neural Network and Its Application
Xudong Wang Huihe Shao Zhongjun Zhang
Automation Department
Shanghai Jiaotong University
Shanghai,200030,People's Republic of China
Figure 1 is the structure ofthe RBFNN.
Input
Hiddenlayer
layer

Abstract
Compared with other feed forward neural networks.theradial basis function neural network (RBFNN) has a lot ofadvantages which make it properer for modeling thenonlinear process,so it has recently gained considerable
attention. In this paper, the RBFNN is introduced and its
problems in modeling the nonlinear system are analyzed.
In order to solve these problems, the structutre of the
RBFNN is improved and the training method is modified.
With the improved RBFNN an industrial high purity
distillation column is modeled. Because of thecomplexities of the nonlinear system, a real time model
correction method is neccessary. In this paper the Givens
transformation is used to correct the model. Simulationshows that the modeling is successful.

1 Introduction¢, (-)X-, �fn ( X)
t

0on 0

Fig. I. The structure ofRBF neural network
Such a network implements a mapping J1 : Rn ➔ R

according to

0;1 ¢(11 X -C; II) 1 s j s n0
flX) = 001 + L
i=In,where X eRn is the input vector, ¢(.) is a kind ofMultilayer feed forward neural network with one ormore hidden layers can be used to approximate acontinuous nonlinear function. So in recent years many
feed forward neural networks including the radial basis
function neural network have been researched. Although
the radial basis function neural network is also a kind of
feed forward neural networks, its structure and training
method show the advantages such as the linear-in-the­parameters [l] [2] [3] and efficient training procedure.
Based on the variational principle and regularization
formulation, the RBFNN proves to be the best multilayer
neural network to learn the input-output mapping[4].Because of the good characteristics of nonlinear
mapping, the RBFNN is used in this paper to model thenonlinear industrial system such as a high purity
distillation column. But the RBFNN has several problems
in its structure and training algorithm. So the RBFNN
should be improved. Using the improved RBFNN, thehigh purity distillation column is modeled. Because theworking conditions of the industrial system may exceed
the training field, on-line training or real time correctionofRBFNN model is necessary.

2 The improved radial basis function neural network

nonlinear function from R+ to R, 11-11 denotes the

Euclidean norm, fl;;, 0 s i s n , I s j s n0 are theweights or parameters, c; E Rn , I s i s n, are the RBF

centers and n, is the numbers of centers. ¢(.) is radialbasis function which make the RBF networks have good
approximation capabilities.
There are a lot of methods to train the RBFNN.
Different method results in different training process of
RBF network. The following four training methods are
the most important and useful: 1. Mooky and Darken's
algorithm [3]; 2. Local training method[5]; 3. Orthogonal
least squares (OLS) algorithm [l] [2]; 4. Recursive Givens
transformation method [6]. Usually method 3 and 4 ismore efficient. But there are several problems should be
solved.
First is how to calculate the centers of the radial basis
function and determine the original number of neurons in
the hidden layer. The rival penalized competitive learning(RPCL) can cluster a set of data [6]. This method do notneed the selection ofan appropriate number ofclusters. So
in this paper RPCL is used to calculated the RBF
centers. [7]- 26 -

Second is that only one kind of radial basis function is
not sufficient to approximate the given input-output
mapping. Improving the ability of nonlinear
approximation requires the modification in the structureof the RBF neural network. In order to solve this problem,
two sub-RBFNNs are combined into one RBFNN. It is
important that the radial basis functions in the two sub­
RBFNNs are different. Because the solution of thisproblem is same as that of the third problem, it isdescribed in detail in the third problem.
The third problem is the computation problem inorthogonal least squares (OLS) algorithm which is used in
the training procedure of RBFNN. When the vector
orthogonalizing proceeds in the OLS algorithm [ 1], themodule of w k is very small. Sometimes the value ofllwk ll/llw1 II is too small to keep it in the group oforthogonal vectors. If such orthogonal vector is selected,
the generalization of RBF network will deteriorate. Thisphenomena is just same as overfitting. If the problem isneglected, the performance of the network is very bad. Inorder to avoid overfitting, the module of w k had better beverified. w k will not be selected if its module is too small.
Although this method can avoid overfitting problem, it
usually can not satisfy the training accuracy. So when themodule verifying is used, the training procedure of RBF
network can not finished. In fact this problem is caused by
the ill-conditioned data{¢, D} which are the output of
the hidden layer of the RBFNN and the desired output
respectively. From the hidden layer to the output layer issuch a linear equation D = rjB. Usually the equation isan inconsistent equation. Only the approximate solution
0 can be found so that trace( D - ¢B) is sufficientAA

small. So the matrix ¢ should be chosen properly. But ¢is the output of the hidden layer. In order to decrease thecontradiction of the equation D = ¢B , the nonlinear
characteristics of the RBF neural network should bechanged. So one RBFNN is decomposed into two sub­
RBFNNs which have different radial basis function.
Figure 2 is the structure of the improved RBF neuralnetwork.
Structure (1) and structure (2) are two sub-RBFNNs
which have the same topology. It is important that theradial basis functions in two sub-RBFNNs are different.
The training procedure of this RBF network has five steps
which are discribed in detail in [8].
Step l:Use the improved OLS algorithm with the input­
output sequence {u, y} to train structure (1).

y

u

y

e

Fig. 2. The structure of the improved
RBF neural network

Step 2:
If the training accuracy is sufficient, theprocedure can be terminated and structure (2) is not useful.
If the training accuracy is not sufficient and theorthogonal vector w k can not be selected then go to step 3.
Step 3:Calculate the output sequence {y}of structure.

(1) : j7 = J; ( u) , and the error sequence { e}: e = y - y
Step 4:
Use the improved OLS algorithm with input­
output sequence {u, e} to train structure (2).
Step 5:After structure (2) is trained the accuracy of thefull network (structure (1) and (2)) is usually
sufficient. The output of the RBF network is y.

3 Modeling the industrial system: high purity
distillation column
In industry process the system is usually nonlinear andsometimes the knowledge that we can get from the system
is only the input-output data. So structure and parameter
identifying is very difficult .
It is well known that the multilayer network has thenonlinear mapping ability. So It is advantageous to use
the network to identify the nonlinear system. But most of
the multilayer networks need modify- 27 -

overheadvapor

condensor
condensor
holdupD
distillate

/

p
Ti
feed

--·

tower
B

tower
A

boilupTz

refluxboilupreboile/bottom productFig 3 The distillation column
because of complexities of nonlinear system. So the
RBFNN is improved to model a high purity distillation
column which is an nonlinear system.
Figure 3 is the distillation column
Where overhead vapor is mainly composed ofpropylene and propane,bottom product is mainlycomposed of propane, propylene, and isobutane.
The input-output equation of the system is
YB = J; (p,½ ) yD = f2 (p, ½ ) (1)

Ti ,

Ti ,

When the RBF network learns an industrial system it
can only approximate the nonlinear system in the learning
field. In fact when the system is working, its working
conditions maybe exceed this field. This is usually causedby the disturbance or time varying parameter andenvironment. When the working conditions exceed thelearning field, the RBF network usually can not work well.
Even if the working conditions return into the learning
field, the RBF network can not approximate the system.

Where YD is the concentraion of the propylene in theoverhead vapor, y 8 is the concentraion of the propane inTi,

the bottom product.
I;_ are the differential temperature.
p is the pressure.
The input-output equation of RBF network is as
equation 2.
y (t) f, (y (t- I),y0 (t-l),p(t- l),I; (t-1),T, (t-1)) (2){ 8 : 8y 0(t) -f, (y,(t-l),y0(t-l),p(t-l), I; (t-1), T, (t-1))

Figure 4 is the curve of the input

Ti, Figure 5 is the1

output y8 of the distillation column. Figure 6 is theoutput y8 of the RBFNN. Comparing figure 5 with figure
6, the RBFNN model almost has the same charecteristics
as the distillation system.
If the RBFNN is not improved, either the training
accuracy is not enough or the performance of the RBFNN
model is not good.

4 Real time correction of RBF neural networkmodel

0 .____.__.____.____._____,
300
0
200
40[
100

time (2h)Fig 4 The curve of input Tl

In order to solve this problem the recursive Givens
transformation is used to correct to parameter of RBFNN
in real time.
Givens transformation is very complex. it is describedin detail in Gentleman's paper [9].
Using the recursive Givens transformation, the model
correction strategy can been put forward . The strategy is
as following.

- 28 -

tested after a large delay. This may cause the difficulty tocontrol the system. If an accurate model of the system canbe gotten,not only the system can be easily controlled but
also the system output can be tested in time so that theworking condition of the system can be monitored.
Although the RBF network training method has been
improved in this paper,there are some problems need tosolve. For example, how to improve the generalization
ability of the RBF network is an important step because
the generalization ability determines the property of
performance. Also how to train the RBF network so that it
can predict the practical process needs more research.

Before the correction proceeds, the parameter 0(t)should be stored as 0 1 . After correction, the parameter0(t) should be stored as 0 2 • If the conditions of theindustrial system return into the training field the-100�-��-�-���

40

References

20�-�--�--�-�
200
300
40(
0
100
time (2h)Fig 5 The output Y.B of the distillation column
.-100

-� 80
Cl)

u
u

60
40
20�-�-�-�-�
0
100
200
300
40(
time (2h)Fig 6 The output y of the RBFNN
D

correction procedure can be terminated and the parameter0(t) should be replaced by 0 1 .If the conditions ofsystem is still out of the training field, the correction
procedure should continue and at that time the output of
RBF network should be calculated with the parameter 0 2 .

5 Conclusions
Nonlinear system modeling is an important problem
because the model of the nonlinear process can be used todesign the control strategy such as predictive control,
adaptive control or internal model control. Also nonlinear
system model is very useful in soft sensor.In practice theoutput of the system often can not be tested or can only be[l] S.Chen, C.F.N.Cowan, P.M.Grant, "Orthogonal Least
Squares Leaming Algorithm for Radial Basis Function
Networks," IEEE Transactions on Neural Networks,
Vol.2, No.2, pp. 302-309, March, 1991.
[2] S.Chen, P.M.Grant, C.F.N.Cowan, "Orthogonal Least
Squares Algorithm for Training Multioutput Radial
Basis Function Networks," IEE Proceedings-F,
Vol.139, No.6, pp 378-384, December, 1992.
[3] Moody and C.Darken, "Fast Leaming in Networks ofLocally-tuned Processing Units," Neural Computation,
Vol.I, pp 281-294, 1989.
[4] S.Y.KUNG,Digital Neural Networks. PTR Prentice­Hall lnc.,1993.
[5) Tyler Holcomb and Manfred Morari, "Local Training
for Radial Basis Function Networks: Towards Solving
the Hidden Unit Problem," American Control
Conference, pp 2331-2335, 1993.
[6] S.Chen, S.A.Billings and P.M.Grant, "Recursive
Hybrid Algorithm for Nonlinear Identification Using
Radial Basis Function Networks," International
Journal of Control, Vol.55, No.5, pp 1051-1070, 1992.
[7] Lei Xu, Adam Krzyzak and Erkki Oja, "Rival
Penalized Competitive Learing for Clustering Analysis,
RBF net, and Curve Detection," IEEE Transactions on
Neural Networks, Vol.4, No.4, pp 636-649, July, 1993.
[8] Xudong Wang, "Nonlinear System Modeling andControl Using the Radial Basis Function Neural
Networks", Master Dissertation of Shanghai Jiao Tong
University, 1995.
[9] Gentleman, W.E., "Least Squares Computations by
Givens Transformations without Squares Roots,"- 29 -

Journal of Institute of Mathematics and Applications,Vol.12, pp 329-336, 1973.

Artificial Plant Trees Based On IFS
Y.G.ZhangInstitute of Systems Scienceof Academia Sinica,
Beijing, China, 100080

X.J.Li

Department of Computer Science,
Beijing University of Aeronautic andAerospace, Beijing, China, 100083

ABSTRACTIn this paper we explain the notion of Iterated
Function System(IFS) and created the growing units
based on IFS. Then a artificial plant tree was evoluted.
The transmission and control of growing information issimilar to the RNA in Biological science. The comparison
with L-system and future jobs for us was discussed. Three
pictures showing the computational result are presented in
the last part of this paper also.
Key words: Artificial Life, Genes, Fractal, Iterated
Function System(IFS), L-system.

1. Introduction
In 1987, more than hundred scientists and engineers
gathered in Los Alamos, New Mexico, to establish the
new science of Artificial Life. In recent years theenthusiasm to study the Artificial Life is arising andsome very interesting results were reported. Obviously,
this is a very new field and it involves a deep philosophy.
Some books have already been presented to introduce
this new science systematically [1,2,3]. Artificial Life, orA-life, is devoted to the creation and study of lifelikeorganisms and systems built by humans. The stuff of thislife is nonorganic matter, and its essence is the manner of
information processing, computers are the kilns fromwhich these new organisms emerge. By simulating a kindof life different from that with which we are familiar, A­
life scientists seek to explore paths that no form of life in
the universe has yet taken, the better to understand theconcepts and limits of life itself. The earlier works in thisfield is very heuristic [4,5]. Lindenmayer' s original
work[4] showed a modeling plant growth using cellular
automata. Dawkins' work[5] showed a vaguely lifelike
creatures called Biomorph Land. He introduced theelements called "genes" into his program for biomorphic
development. These genes influenced things like angle of

M.Sugisaka

Department of Electrical andElectronic Eng., Oita University,
Oita, Japan, 870-11

branching, the length of a branch, the number ofsubbranches, and so on. [6] has showed more new works
that to grow Artificial Brain by using of cellular automata
in work station computer. Now a very relative research
field named Evolutionary Computation is growing anddeveloping. On the other hands, the Fractal studies have
got many results in recent years[7,8,9]. Some of themwere used to generate so-called plant at computer screens.
Of cause, it is rather different from the Biomorphy Landand L-system. It is based on the principle of self­similarity and Iterated Function System. In spite of thatsome beautiful plant pictures were showed at computer
screens, but they have not any meaning of Life. They are
the same thing as the plants in 3-D cartoon, all of them isthe computer graph skill only. Note that, some results ofFractal studies were used to explain the phenomena in thenature including the growing of alive organisms. We try
to show another way to generate the plants with A-life
meaning by using IFS we defined in this paper. This iswhat we have done in this paper.

2. IFS and Growing units
2.1 IFS in Fractal Field
In 1982, Mandelbrolt defined the word Fractal andused it to describe the divert world. Fractals are the sets
whose Hausdorff dimension, which is generally a real
number, differs from their topological dimension. An
important features of fractals is that to describe a object
only a little information is needed for their specification,even though the object looks like very complicated. It canbe generated by a simple iterate rules. These rules are
named IFS(Iterated Function System).
IFS in Fractal is defined as following:
2
In R , an affine transformation W is a linear map from2
R to R2 :- 30 -( x i, x 2)

la=leHere, a,b,c,d,e,f are some suitable real number. We canwrite W(X)=AX+B. A 2-dimensional affine transfor­
mation is thus completely specified by six real numbers.
For an affine transformation and arbitrary x and ybelonging to R2, certainly at least one nonnegtive real
number S can be found such that
IIW(X)-W(Y)II � SIIX-YII,
the minimum of all S that is suitable to above inequality
is called as Lipschtz index SL . If SL < 1, then we call thisaffine transformation as a compressive affine transfor­
mation.
, 2 , •.• ,W0} be n compressive affine transfor­
Let {W 1 W
mations and {P 1, P2 , ..• , P0} is a probabilistic distribu­
tion on it, that is P; > 0, i=l,2,... n, and � P-1= I. If(S )Pl( S )P2•••(S )Pn < 1, then we say that {W;, P;,12
0
i=l,2, ...,n}is a IFS codes. We can use this kind of codes
to draw some Fractal graph, for example, fem plant atcomputer screen. It is not difficult to see that there is no
any A-life meaning inside. We modified IFS to meet therequirements in A-life meaning.

2.2 Growing Units with genetic information
First of all, we try to generalized the concept of IFS. In
fact, any unit which can be operated iteratively based on
certain information series, whatever it is a set of function
or a group of rules, could be called as a Iterated Function
Unit. When we are going to simulate certain life
phenomena, for example, the growing process of a tree,
several kind growing units are needed. One is for the tree
trunk, second for the branches, third for the subbranches
and four for the leaves, above are the minimum. We
designed four growing units:
T=T(high; diameter; growing direction; branch control;year),
B=B(length; diameter; growing direction; subbranchcontrol; year),
SB=SB(length; direction; growing direction; leaves
control; year),
L=L( size; shape control; direction; year).
They operate on the growing of the Trunk, Branches,
SubBranches and Leaves respectively. To give them theA-life meaning, the manner of information processing
must include the encoding, transcription, translation,
decoding and growing etc.. The genes decide what kindof species will grow. This manner could be comparedwith the one for the processing in DNA. The information
stored in DNA were encoded and transcribed into RNA,

usually the RNA is a single-stranded molecule. In fact,
the RNA sequence is the base sequence to control
growing. During the second step, the ribosome makes use
of genetic codes to translate a section of the RNA strand
in order to form a string of amino acid(a protein). So, thegrowing units we have mentioned in the previous section
are similar to the ribosome, the growing control
sequences to be introduced in next section are similar to
the RNA sequence, the growing units read the growing
message from the control sequences and translate it into
the growth of the tree body. The group of the growing
units to proceed a growing process are still called as
Iterated Function System(IFS) here. Obviously, the IFS
defined here is clifferent from that one in Fractal field.
The gene� m the IFS defined here store the certain
genetic information which distinguish one to others. In
this paper we just considered very simple genetic
information: shape(high and diameter), the position of
branches starting, the direction of growing, and growing
speed which is obey certain growing curves
y= C/[1+ A exp(-Bk)]
where A,B,C are the given constants and k is the discrete
time variable(year). The constant A is the final size when
the time variable tends to infinity.

2.3 Control Sequences of Growing
As what we have mentioned in last paragraph we needa control information which is similar to the RNA
sequence to control the artificial tree's growing . We do
not pay more attention to the transcription from theinformation like DNA to the RNA, shortly assume that
we have already got the similar RNA sequence--Growing
Control sequence. In fact, this sequence is a zero-one
sequence extended with time variable :0000101011101...
where "O" means growing only, without new branch in
the time k, "1" means growing and generating a new
branch in this time k. In the place where "1" appeared a
new control sequence will be embedded by using thesymbol"{" and"}". The sequence embedded in {} is alsoa zero-one sequence which is read from certain section of
the original control sequence. Similarly, "1" in theembedded sequence has the same meanµig as one in theoriginal control sequence. Then a new embedded
sequence in {} is generated and denote it as [ ]. Of cause,in this sequence a new sequence can be also embedded if"l" appeared in the previous embedded sequence anddenote it as ( ). all of them is read from a section in the- 31 -

original control sequence. The different control sequence
will give different tree in shapes, but not in specie,
because the specie is decided by the genes in the growing
units.

2.4 The IFS in A-life Field
When we have defined the growing units and control
sequences, a new Iterated Function System was establi­
shed. That is IFS in A-life field : {Growing units +Control sequences in a embedded form + Probabilistic
structure}. The probabilistic structure means the genetic
information are randomly taken from a probabilistic
space, and the control sequences is disturbanced. Basedon it, we can start the works of growing of artificial planttree.

3. Growth of the Artificial Plant Tree
After we had the definition of IFS and explained how
to control the growing of artificial plant tree, we could
make computer program to draw the picture according to
the genetic information including in growing units. Now
the genes are stored in growing units. For example, if wewant to generate an artificial tree, we need to form thegenes in trunk, branches, subbranches and leaves inadvance. From these genes, we can generate the tree
which we want.
The growth of each things which is vital in the naturedepends on their own genes such as their size, structure,color, behavior etc.. For synthesized biology this is theprocess of generating artificial genes. For example, trunk
genes control the tree's length and diameter. Different
genetic information grows different tree, someone has
good status, but someone is not. So suitable selection ofgenes information are necessary. In order to simulate thenatural selection in the nature world we should have a
population of certain specie of artificial tree, which has
the same kind of genes but there is mutation on it. Thenthe Genetic Algorithm (GA) is used to get someone thathas good status. GA is a search algorithm based on themechanism of natural selection and natural genetics. A
simple genetic algorithm that yields good results in many
practical problems is composed of three operators, that is
reproduction crossover and mutation. These operators, ofcourse, are artificial versions of natural selection andevolution. The figures attached in the end of this paper
show the some results of ours. The work has done on the
SGI work station computer.

4.
Discussion and Comparison with Lsystem
To establish more reasonable A-life theory andapplication, we should have more inside on the process oflife information such as the encoding, transcription,translation and decoding etc.. In the "wet" life studies
scientists pay more attention to the synthesized of protein
subject. They simulate the real alive basic subjects in thereal world through many very difficult and exploratory
experiments. In the view of point of A-life scientists
should pay more attention to the information processingin the real alive subjects. The more complicated the realsubjects are, the more complicated the information
processing are. Some simulations look like the growing ofplant, but it is not artificial A-life study because themanner of information processing is rather different fromthe real process. Lindenmaryer[4] introduced the notionof parallel rewriting grammars for the modeling ofdeveloping biological systems, that is L-system. It needs
many rules in L-system. In our work, we use IFS to growtrunk, branches, subbranches and leaves through thegenes, and the growing is controlled by the control
sequences. The rules in L-system are replaced by genes.
We think that the growing of tree based on known L­
system should not be A-life. Mainly, because the growing
can not be controlled. The sequence in L-system
consisted of zero, one,[ and ] is not the control sequence,but a result of growing only.

5. Conclusion
In this paper we redefine the Iterated Function
System(IFS) which is different from that one in Fractal
field. We try to follow the study on DNA to simulate thegrowing of plant tree. The IFS defined here has two mainparts: one is the growing units with the genetic
information, the another is the control sequence withembedded form. The artificial plant tree based on IFS
defined here is different from L-system graphics andFractal fem plant graph. We can get many different trees
from the different genetic information. The processing of
information in this paper is more close to the process inDNA. This paper is our starting work in A-life field, thecomputer experiments are rather simple. In the near future
we will present more works in this field.

- 32 -

6. References[I] Casti,J.L., "Reality Rules: Picturing the World in
Mathematics", JOHN WILEY & SONS, INC. 1992
[2] Levy,S.," Artificial Life--a report from Frontier
Where Computers Meet Biology", Vintage Books,
Random House, Inc. 1993
[3] Langton,C.,"Artificial Life" and" Artificial Life--11",
Addison-Wesley, Redwood City, CA, 1989 and 1992
[4] Lindenmayer,A., "Mathematical Models for cellularInteractions in Development, Part 1 and Part 2", Journalof Theoretical Biology, 18, pp. 280-315, 1968
[5] Dawkins,R., "The blind Watchmaker", Longman,
London, 1986
[6] Lay,T.,"Artificial Brain",[7] Bamsley,M.F., Ervin,V., Hardin,D. and Lancaster,J.
,"Solution of an Inverse Problem for Fractals and Other
Sets", Proceedings of the National Academy of Science,
Vol.83, April, 1985
[8] Smith,A.R., "Plants, Fractals, and Formal Languages",
Computer Graphics, 18(3), July, 1984
[9] Mandelbrot,B., "The Fractal Geometry of Nature",
W.H.Freeman and Co., San Francisco, 1982

(a)Fig. I Growing of Artificial Tree based on IFS(a) 6 years (b) 10 years (c) 12 years(c)- 33 -

Fault Diagnosis of Logical Circuit by use of Pseudorandom Signal and Neural
Network
H. Kashiwagi and M.Sakata
Department of Mechanical Engineering, Faculty of Engineering
Kumamoto University, Kumamoto 860, Japan
Tel: +81-96-342-3742, Fax +81-96-342-3730
E-mail: kashiwa@gpo.kumamoto-u.ac.jp
Abstract
This paper describes a new method of pseudoran­
dom testing of a digital circuit by use of correlation
method and a neural network. The authors have re­cently proposed a new method offault diagnosis oflog­
ical circuit by applying a pseudorandom M-sequence
to the circuit under test, calculating the crosscorrela­tion function between the input and the output, andcomparing the crosscorrelation functions with the ref­erences. This method, called MSEC method, is fur­ther extended by using a neural network in order tonot only detect the existence offaults but also find theplace or location ofthe faults. The author investigatedthe effects ofusing a part offault patterns for training
the neural network on the ability of fault detection of
the neural network. It is shown that we can obtain
more than 95 % of fault detection ration even when
we use only the half of the possible training data.

1

Introduction

A functional logic testing of logical circuits is now
an important technique, since the information de­vices including digital logical circuit are increasing so
rapidly. For example, fault diagnosis of logical circuits
boards is required in the process of producing any in­formation devices.
There are mainly two methods in detecting faults
of logical circuits. One is to analyse the circuits under
test (CUT) logically and to design an optimal input
sequence for CUT. D-algorithm, for example, belongs
to this group. This method is called "deterministic
method" and usually requires a large amount of com­putation time, and the design must be done for eachkind of circuit.
The second method is to apply a random input tothe circuit and the corresponding output is comparedwith the output offault free circuit[3][4]. This method

is called "random testing method" and has several ad­
vantages: simple hardware, low cost and applicability
to any circuits.
In this connection, we have recently proposed a new
method[l] offault diagnosis oflogical circuit by apply­ing a pseudo-random M-sequence to CUT, calculating
the crosscorrelation function between the input andthe output, and comparing the crosscorrelation func­
tions with the references. This method is called M­
SEquence Correlation (MSEC) method, and it has avery small probability that we overlook any faults inthe circuit.
In this paper, MSEC method is further extended
by using a neural network in order to not only detect
the existence of faults in the circuit but also find theplace or location of the faults. A simple digital cir­
cuit is taken as an example, and artificial faults areinserted to the circuit. MSEC method is then applied
to this circuit to obtain the crosscorrelation functions
between the inputs and outputs. Since the crosscorre­
lation functions contain the information about faults
in the circuit, they are then applied to a neural net­
work. After training the neural network by use of thedata on the faults and resulting crosscorrelation func­
tions, the ability of the neural network to detect thelocation of faults is checked.
The results ofthe experiment show that the trainedneural network has an ability not only to detect theexistence of faults but also to locate the faults in thecircuits.

2

Brief description of MSEC method

The basic diagram of MSEC method is shown in
Fig.l. The input to the CUT is an n-th degree M­
sequence a; (a;=0 or 1, i=l,2,... N, N=2n -l) which is
known as one of the pseudo-random sequences. The
output sequence b; (b;=O or 1) of the CUT is crosscor-- 34 -

related with ai. The crosscorrelation function between
ai and bi is obtained as follows.
N4> a b(r)=N-2(LaiEBbi+T)
i=l

(1)

where EB denotes Exclusive OR (EOR) operation andis the arithmetic operation. The crosscorrelation
functions 4> a b(r1), cl> a h(r2), .. are then compared withthe correct values of 4> a b(r)'s which are the crosscor­
relation functions in case of fault-free circuit. W hen
any of 4> a b(r) differ from the correct values, we cansay that there must be some fault in the CUT.
The structure of MSEC method is simple, requiring
only EOR circuit and counters. In addition to this,MSEC method has a very small probability that wemiss any faults in the circuit.
The undetected fault ratio of MSEC method, thatis, the expected value of the probability that we over­look any faults in the CUT, is shown to beE(2)
where r is the number of delay bits we use.
The undetected fault ratio of MSEC method IS
shown to be much smaller than the conventional one's
counting, transition counting, or even CRC(cyclic re­dundancy check method which is rather well known
as Signature Analysis method developed by Hewlett­Packerd Co. USA).

3

Data for training neural network

As is shown above, MSEC method has a very small
probability that we miss any faults in the circuit, so
it is very efficient to detect faults in a logical circuit.
However, MSEC method can detect whether there ex­
ist any falts or not, but it cannot find the location of
fault in the circuit. Since we can think that the cross­correlation functions 4> a b(r) 's have some informationon the state of the CUT, we can extract the informa­tion about· the location of fault from those crosscorre­
lation values by applying the values of the crosscorrela­tion functions to a neural network. The training data
for the neural network are obtained from the following
experiment.

4

Experiment

As an example of CUT, a simple digital circuit (2bit binary adder SN 7482) is chosen where number of

input is 5, number of output is 3(Fig.2). In this circuit,
14 switches are inserted as the simulation of possible
faults. So the total number of the fault patterns is
2 14 = 16384. Let us denote the fault pattern as (1,2,3)
in case where switches no.l, no.2, and no.3 are faulty.
M-sequence of 5 degree (N=31) is applied to the input
C0 , its one bit shifted veision to A1, and likewise to
B1,B2 ,A2.
We have measured the crosscorrelation functions
4>co-a1( r), 4>co-a2( r), and 4>co-c2( r), 31 values foreach(total 93 values) for one fault pattern.
In order to simplify the neural network which will
be used in practice, the correlation data are converted
to binary ones according to whether each correlation
value is positive or negative(Fig.4). Thus the corrre­
lation values of total 93 bits are applied to the neuralnetwork.
After measuring the correlation functions for 16384
fault patterns, it is noticed that we obtain 372 different
patterns of correlation functions. That is, one corre­
lation pattern correponds to some fault patterns. For
example, the crosscorrelation pattern(Fig.3) obtainedfor fault (3,13,14) is the same as for (13,14), but dif­ferent from those of all other fault patterns. There­
fore when we obtain the correlation pattern as shownin Fig.3, we can say that switches no.13 and no.14 are
faulty with 100 % probability and switch no.3 is faulty
with 50 %. These probability is called here as theo­
retical fault probability. Thus we can calculate thetheoretical probability for each correlation patterns.
These data are used for training the neural network.
The neural netwok which we used has three layers: 93
nodes for input layer, 23 nodes for intermediate layerand 14 node for output layer. The algorithm usedin this neural network is back propagation algorithm.
The example of the output of the neural network isshown in Fig.4.

5Effects of using partial data

One of the properties of neural network is that once
all the possible inputs are used for training and theneural network converges well, the network has theability to detect any fault with high probability. How­ever, when only a part of possible input data are used
for training, what happens about the ability of thefault detection of the neural network ?
Fig.5 shows an experimental result on the fault de­tection ratio vs. the number of input data for training.
From this figure, we can say that we obtain more than
95 % of fault detection ratio when we use only the halfof the possible input data for training.

- 35 -

References

We see from these results that the trained neuralnetwork has enough ability to find the location faults
in the circuit.

6[1] H. Kashiwagi and I. Takahashi; A New Methodof Fault Detection of a Logical Circuit by useof M-sequence Correlation Method, Trans. SICE,
Vol.23, No.9, pp.993-997, 1987.

Conclusion[2] H. Kashiwagi and I. Takahashi; Fault Detectionof a Logic circuit by use of M-sequence Correla­tion Method, Proc. China-Japan Reliability Sym­
posium held in Shanghai, pp.149-157, September
1987.

A new method of fault diagnosis of logical circuitis proposed by use of pseudorandom M-sequence as
its input, crosscorrelation functions between the in­put and output, and a neural network. This method
can not only detect whether or not there exist somefaults in the circuit with very small probability that
we overlook any faults, but also find the location of
the detected faults in the circuit.
The experiment show that we obtain more than 95% of fault detection ratio when we use the half of thepossible inpout data for training the neural network.
This is very effective in practice, since it will be very
difficult to provide the complete training data for all
possible faults in the circuit. Some sort of statistical
data such as those concerning the frequently occurring
fault points would be useful for training the neuralnetwork.

INPUT
ai

CIRCUIT[3] T.Ohteru et.al.; Digital Circuit Test System Us­
ing Statistical Method, Proc.lOth ISFTC, pp.1979181, 1980.
[4] K.D.Wagner, C.K.Chin and E.J.McClusky; Pseu­
dorandom Testing, IEEE Trans., Vol.C-36, No.3,
pp.332-343, 1987.
[5] H.Kashiwagi and M.Sakata; Fault Diagnosis of
Logical Circuit by use of Correlation and Neural
Network, Proc. '92KACC held in Seoul, Korea,
pp.569-572, 1992.

OUTPUTbi

EXPECTEDVALUES
OF <Pab(T)'Pab(T,)Fig.l Basic diagram of fault diagnosis system- 36 -NEURALNE1WORK11.'/l.
''

I

-

I

't SW12f SW3

t::::n
r-1)
I

�·

I

f

.,

'

./

,L)

-ITT

1bJ
-

SW14
SW9

SW4

SW6

SW13'l�r;=OSW
tof-LJ---1=D7

�
SW1

,._

SW5

----

_,

/

SW11 sw·7

Tdlf

L

,/

/

:_ )o--

_,

....,

/

l

........----++5SW8

Fig.2 An example of CUT.
30
20

,e,

<I

10

·■■ ·I
I■· · · · · · ■- ■·■·■■•■· •■· · ■I0I· . ■ .

-10

-20

-30,
0

5

0

5

15

10

20

25

30

20

25

30

INTERMBDIATB

�1 I n I I fulJ Lf.
Delay

10

15
Delay

Fig.4 An example of output of the neural network

Fig.3 Difference of crosscorrelation values and its
binary version

100.-----r----.----,-----.-----,---....----...-,
90

80
70
60 1.-___,___......._____.._____.___.____...,____

0

50

100

OU11'111'

150

200

250

300

Number of Training data

Fig.5 Probability of fault detection vs. number oftraining data

- 37 -

350

Computational Study on Cyclic Structureusing Abstract Rewriting System on Multisets
Yasuhiro Suzuki, Shusaku Tsumoto, and Hiroshi Tanaka
Department of Information Medicine
Medical Research Institute, Tokyo Medical and Dental University
1-5-45 Yushima, Bunkyo-ku Tokyo 113 Japan
Email: {suzukcom, tsumocom}@cmn.tmd.ac.jp, tanaka@cim.tmd.ac.jp

Abstract
One of the most essential temporal structure in life
systems is a cycle, which can be observed in any hierar­
chy of living things, such as TCA cycle in cytoplasmic
level, the cell cycle in cell division, and the life cyclesof living things. Although the importance of this struc­ture has been pointed out by several researchers, they
do not focus on the conditions under which such cy­
cle structures will emerge and become stable. In thispaper, an abstract rewriting system on a multiset is
introduced to a model chemical reaction as a symbolic
rewriting system acting on a multiset, which can beviewed as a chemical reaction system in a test tube.
By use of this model, the condition of a cycle emer­
gence, its robustness and how this system works under
the probabilistic condition are examined. The results
show not only that a cycle will emerge even under asimple initial condition, but also that complex behaviorof a cycle, .mch as fusion of several cycles, is observed
when input randomness is introduced.

Keywords: Abstract rewriting system, Cyclic
structure, Emergent computation.

1

Introduction

One of the most essential temporal structure in life
systems is a cycle, which can be observed in any hi­
erarchy of living things. For example, in metabolic
pathways inside the cell, TCA cycle is used to extractenergy from organic chemicals, which plays an impor­tant role in cell metabolism[6]. Also, the cell cycle incell division is important for the growth of life systems,
and the life cycle can be observed in all living things[6].
Thus, a cycle can be viewed as a universal structure
in all the hierarchies of complex systems, especiallyin living things, whose importance has been pointed
out by several researchers of complex system, suchas Eigen's hypercycle[2], Maturana's autopoiesis[5],Kauffman's NK network[4], and Fontana's Algorith­
mic Chemistry[3]. However, these researches do not
focus on the conditions under which such a cycle will
emerge and the process in which it becomes stable.
In this paper, we introduce a new abstract rewriting
system on multiple sets (ARMS) in order to examine
the conditions of cycle emergence and the algebraic
characteristics of the obtained cycle. In this model, amultiset 1 is taken as a role of a test tube, which con­tains "symbols", which correspond to chemical com­pounds. Then, rewriting rules, corresponding to chem­
ical reaction formulae, act on this multiset, and theapplied symbols are transformed into other symbols,
corresponding to chemical reactions. Furthermore, we
assume that the specific symbol a is input to this mul­tiset. Interestingly, it is easily shown that even sucha simple structure can emerge a cycle, although thestructure of rules is strongly constrain�d to do that.
Then, we further examine the condition of emergenceof a cycle, such as the effect of inputs and that of theorder of rules. The results show not only that a cycle
will emerge even under a simple initial condition, butalso that complex behavior of a cycle, such as fusion ofseveral cycles and period-doubling, is observed when
the initial condition is perturbed during the simula­tion. These results suggest that randomness of input
is one of the principal factors on the complex behavior.

2

ARMS

Our computational model is based on the abstract
rewriting system [1] because of the following three rea­sons. First, chemical reactions, which are importantin our model, can be viewed as rewriting rules. For
1 A multiset is defined as a set which is allowed to include
the same elements. For example, { a, a, b, c} is not a set, but amultiset, while { a, b, c} is both a set and a multiset.

- 38 -

example, ½0 2 + H2 -+ H20 is a kind of a rewrit­
ing rule. Thus, the abstract rewriting system gives aformal model of chemical reactions. Second, abstract
rewriting system does not need a specific architecture
like a turing machine [1] and cell automata[7). Finally,
third, we can apply the results of researches on the
abstract rewriting systems, such as those on rewrit­ing pathways, and rewriting rules. Using the concepts
of abstract rewriting system, we introduce a new ab­
stract rewriting system on multisets (ARMS) in order
to examine the conditions of emerging a cycle and thealgebraic characteristics of the obtained cycle. ARMS
is defined as follows.
Definition 1 (ARS on Multisets(ARMS))
Abstract rewriting system on multisets is defined asa quintuple (A,R,OR,C,I), where A, R, OR, C andI denote a set of symbols, a set of rewriting rules, ruleorder, a multiset of symbols, and a set of input strings,□
respectively.
In this model, a multiset C is used as "a test tube",
which contain "symbols" in A, which correspond tochemical compounds. Then, rewriting rules. R, corre­
sponding to chemical reactions, act on this multiset,and the applied symbols are transformed into othersymbols. Moreover, for simplicity, we assume that no
character outputs from a multiset C. In this paper,.
we also focus on the effect of finiteness of a test tube,
thus we assume that C is finite. Especially, for sim­
plicity, the cardinality of C, ICI is set to 10 (ICI = 10)
in our experiments. Then, normal form can be defined
as follows.
Definition 2 (Normal Form in ARMS) If
norule in R can be applied to C and if no string in I
can be input to C, then then it is said that C is in□normal form.
Thus, normal form corresponds to the steady state, orthe death of living things. Furthermore, for experi­ments, we specify the characteristics of ARMS in thefollowing way.
Initial Condition As to the initial condition, A and
I is set to { a, b , c, d, e} and {a}, respectively. Then,the cardinality of ICI is set to 10. Finally, rewriting isapplied for 5005 steps, which is equal to the total sizeof search space of multiple set C.
Initial State and Final State Initial state of C is
set to an empty set ( C = {}). On the other hand, final
state is defined as a normal form defined in the above
subsection. That is, if no rule in R can be applied to

C and if no string in I can be input to C, then thenC is in a final state.
Order of Rules In ARMS, we assume that one rule
is applied in each rewriting step unless no input isallowed based on a given rule order OR. For example,
let us consider the case when R = {r1, r2, r3} andOR = {r1 -+ r2 -+ r3}. Then, each rule is appliedin the following way. In Step 1, r1 is applied. In
the next steps, Step 2 and 3 , r2 and r3 are applied,
respectively. In Step 1, r1 is applied again. Thus,
transition of the order in each step can be viewed as a"shift" operation on a bit string.

3

Experimental Results

Throughout the experiments, R is set to{ r1, r2, r3, r4, rs, r5}, where each rule is described as
the following formulae.
a a a-+ b : r1,
c-+ d d : r4,

3.1

b a-+ c : r2,
d-+ e : r5,de-+ a : r3,
d-+ c : r6.

Rule Order and Emergence of Cycles

We, first, examine the relationship between rule or­der and emergence of cycles. In the above rule set R,
the total number of rule order is equal to 6! = 720. For
each order, we assume that a will be input after eachrewriting step and check whether cycles will emerge ornot.
Surprisingly , in only 9.4 per cent(68/720) of 720
kinds of order, cycles emerge, and in the rest, 90.6
per cent, rewriting calculus is terminated. In 68 kindsof order where cycles emerge, steps needed to emergea cycle are 15 to 22 steps, and its average and its
standard deviation is 19 and 3.3 steps, respectively.
The average of the length of cycles is 5.5 steps, whose
standard deviation is 0.49 steps.
As to terminated cases, the average of steps needed
for termination is 39 steps, and its standard deviationis 6.48 steps. For further examination on the effectof inputs and rule order, we adopt the following two
kinds of order, 0R1 and OR2,
OR1 = {r1-+ r2-+ r3-+ r4-+ rs-+ r5},
OR2 = {r3-+ r1-+ r2-+ r4-+ rs-+ r5},
because of the following two reasons:
The behavior of OR1 is a typical cycle structure in
68 cases. Second, the behavior of OR2 is completely
different from the above one, although ARMS applies
the same rules in these two cases.

- 39 -

3.2

Effect of Inputs and Rule OrderIn this subsection, we examine the effect of random
inputs. After that, we introduce randomness in ruleorder and examine the effect of rule order random­ness. In order to incorporate.randomness, we intro­
duce characteristic function as follows.
Definition 3 (Characteristic Function) Let x bea uniform random number selected from [0,1). Thena characteristic function h0 (x) is defined as:

Table 1: Relation between Frequency of Inputs (fJ) andTermination of Calculus
0.7 >
0
100

Decrease Frequency of Input, with Rule Or­
der Fixed In the above three kinds of experimental
results, the most interesting one is cases when the fre­quency of input is decreased, with the rule order fixed.
In the cases of rule order ORl, rewriting steps are notterminated in all the trials, and cyclic structure also
emerges. Interestingly, the length of cycle varies over 7
to 30 steps, and period-doubling of cycles is observed.
On the other hand, in the cases of rule order OR2,
unstable cycles emerge and rewriting steps are termi­
nated while the threshold is larger than 0.5. How­ever, when the threshold is equal to 0.4, almost the all

0.4
0
13

< 0.3
0
0

Table 2: Relation between Randomness of Rule Order(1 - fJ) and Number of Emerging Cycles
0.7
0.9
0.5
0.1
0.3
106(5)
106(5)
23(5)1300(5)325(5)
300(3)
223(3)
1118(5)335(3)
DR2
1269(11)
A(B) : A and B denote the number of emerging cycles,
and the median of the length of cycles, respectively.
1- 6
ORl

D

For example, when {J is equal to 0.9, then the fre­quency, with h 0 (x) positive, is equal to 0.9. Thus,
when this function is applied to inputs, a will be in­put after nine of ten rewriting steps, which also meansthat no input will be done in one of ten steps. On theother hand, when the characteristic function is applied
to rule order, the rule order will not be perturbed innine of ten rewriting steps, which means that rule or­der is perturbed in one of ten steps. For example,
let us consider a case of Om. When the sixth stepin ten rewriting steps is perturbed, rule is applied asshown in the following sequence: r1, r2, r3, r4, rs,r2(r5), r1, r2, r3, r4, where r5 is changed to r2. Thus,
both input randomness and rule randomness is equal
to 1 - h 0 (x). Using the above two kinds of order,
we make the following three kinds of experiments: (1)Decrease the frequency of input, with rule order fixed,(2) Introduce randomness in rule order, with inputs inevery step, and (3) Decrease the frequency of input,
and introduce randomness in rule order. In each case,
we perform 100 trials, each of which consists of 5005
rewriting steps, and measure the effect on emergenceof cycles, extinction of cycles, and steps needed fortermination.

0.5
0
98

Rule Order: Fixed

where {J denotes a threshold from the following set,

{0.1,0.2,0.3,0.4,0.5,0.6,0.7, 0.8,0.9}.

0.6
0
100

rewriting trials are not terminated, where cycles be­come stable. Moreover, at this point, the variabilityof the length of cycles is largest among the all trials.
The peaks of the length of cycles is located around11, 23, 34, 46, 57, and 67 steps, which shows period­
doubling. We closely examine these experiments in
Section 5, which shows that several kinds of cycle fu­
sion is observed. If the threshold is less than 0.3, all
the trials are stable, similar to the cases of rule order
0 Rl. These results are summarized into Table 1.
Randomness in Rule Order, with Inputs in Ev­
ery Step In these experiments, as to rule order
0 Rl, cycles become unstable and period-doubling isobserved in ORI when randomness of rule order islow. However, as randomness of rule order grows large,
the number of emerging cycles decreases exponentially.
Table 5 shows this tendency, the relation between ran­domness of rule order and the number of cycles emerg­
ing in 100 trials. When the randomness is larger than0.5, almost the all trials generate only one cycle ingeneral. On the other hand, the behavior of cycles in
OR 2 is different. As shown in Table 5, the number of
emerging cycles decreases exponentially, as random­
ness of rule order grows large. However, the behaviorof cycles when randomness of rule is higher than 0.5
is different from that when the randomness is lower
than 0.3: the median of the length of cycles in the lat­ter cases half smaller than that in ORl, while that in
the former cases is equal to or twice than that in ORl.
Interestingly, trials in the case of OR2 have two phasetransitions: one is between 0.1 and 0.2, and the other- 40 -

is between 0.3 and 0.5. It is notable that the cycles
whose length is 11.0 are dominant over those whose
length is 5.0.
Decrease Frequency of Input and Increase Ran­domness in Rule Order In the final cases when
the frequency of input is decreased and randomness
is introduced in rule order, the obtained results are
straightforward. In the cases when randomness is in­troduced in rule order with an input in every step, cy­cles become unstable in ORl, and only unstable cycles
emerge in OR2. However, in both cases, the numberof emerging cycles decreases exponentially, as random­
ness of rule order grows large. Interestingly, the be­havior of cycles in ORl is very similar to that in OR2,
except when the frequency of input is very high andrandomness of rule order are very low. In the excep­
tional cases, both cases are very similar to the cases in
Subsection 4.1, except for emergence of cycles in thecase of ORl.

4

5

Conclusion

In this paper, we introduce a new abstract rewrit­
ing system on multisets, and model chemical reactions
as a symbolic rewriting system which acts on a mul­tiset, which can be viewed as a test tube. Using thismodel, we examine the condition of emerging a cy­cle, its robustness and how this system works under
the probabilistic condition. The results show not onlythat a cycle will emerge even under a simple initialcondition, but also that complex behavior of a cycle,
such as fusion of several cycles, is observed when theinitial condition is perturbed during the simulation.

Acknowledgments
This research is supported by Grants-in-Aid for Sci­
entific Research No.07243203 from the Ministry of Ed­
ucation, Science and Culture in Japan.

References

Discussion

As discussed in Section 4, a special case in rule or­der OR2, with the frequency of input 0.4, generates
complex behavior of cycles. Closer examination showsthat a core cycle generates first and that other cycles
around this cycle fuse into a core cycle in rewriting
steps.The generated core cycle cg is given as follows:cs : 8-+ 9-+ 10-+ · · ·-+ 18-+ 8 : (llsteps),
where each number denotes the number of rewriting
steps. Around the cg, the following cycle c54 is also
observed:
c54 : 54-+ 55-+ 56-+ · · ·-+ 66-+ 54 : (13steps),

It is impossible to transit from State 8 to State 54
and State 66 to State 11, with input fixed, but it will
become possible when randomness of input is intro­
duced. Then, cs fuses into c11, and a new cycle will
be generated as follows:
Cc : 8-+ 54-+ · · ·-+ 66-+ 11-+ · · ·-+ 8 : (21steps)

Thus, fusion of cycles depends on randomness of in­
put: randomness of input influences the application ofrules, and then change of application of rules causes
the change of structure of a multiset, then it opens
the hidden pathway to fusion of cycles. Although themechanism of fusion of cycles is not fully studied, it
is notable that even such a simple structure emerges
a complex behavior. Further research on fusion, suchas classification of cycles, will be a future work.

- 41 -[1] N. Dershowitz and J. P. Jouannaud, Rewrite
Systems, in Handbook of theoretical computerscience 245-309, Elsevier. 1990.
[2]

M. Eigen and P. Schuster,Springer-Verlag, 1979.

The Hypercycle,[3] W. Fontana and L. W. Buss, The Arrival of thefittest: Toward a Theory of biological organiza­
tion, Bulletin of Mathematical Biology, Vol.56,
No.I, 1-64, 1994.
[4) S. A. Kauffman, The Origins of Order, Oxford
University Press, 1993.
[5) H. R. Maturana and F. J. Varela, Autopoiesis
and Cognition, D. Reidel Publishing Company,
1980.
[6] J. D. Watson, N. H. Hopkins at el, Molecularbiology of the gene, The Benjamin/Cummings
publishing Company, Inc, 1992.
[7) S. Wolfram, Cellular Automata and Complexity,
Addison Wesley, 1994.

Identification of a Dissipative Transport System
Alan P. Wang & Cecilia Y. Wang
Department of Mathematics
Arizona State University
Tempe, AZ 85287-1804
andSueo Ueno
Information Science Laboratory
Kyoto School of Computer Science
Kyoto 606, Japan

Abstract
The aim of this paper is to identify the propertiesof an unknown medium of a transport system frommeasurements of outputs at the boundaries. A new
method is developed here based on the inverse prob­
lem and the properties of a scattering matrix and its
generator.

1

Introduction

A great deal of interest has developed in recent
years in the identification of a transport system, thatis, to identify the physical properties of an unknown
medium based on measurements of outputs at theboundaries of a medium. Such identification problems
arise in many physical problems, such as transmission
lines, signal processing, acoustic propagation, radia­
tive transfer, etc.
A transport system consists of inputs and outputsof a medium. Inputs at the boundaries are combina­
tions of transmitted and reflected inputs. The physicalproperties of the medium which govern the inputs andoutputs are called medium coefficients.
It is well known [1], [3], [6] that the reflection andtransmission can be determined by solving a set of
Chandrasekhar's equations with known medium co­efficients. This is equivalent to s ay ing that outputs
are determined if inputs and medium coefficients are
known. This is usually called the direct problem.
The inverse problem considered here is that if in­puts and outputs are known at the boundaries, wewish to determine the medium coefficients inside themedium. In general, the medium is inhomogeneouswith anisotropic scattering.

The method presented here is based on comparisonand on the properties of a scattering matrix and its
generator. We establish relations between two scatter­ing matrices; one is known and the other is determinedby measurement outputs at the boundaries. The rela­tionship between these two scattering matrices leads
to that between the two media. Hence the unknown
medium is determined.
Research similar to this paper, in the sense of using
scattering matrices, is by Kailath [2] , Ueno and Wang[6], [8], [5]. The former developed the layer peeling �ndl ayer-adjoining methods. The latter used scattermg
matrices to solve the inverse problem which focusedon the determination of the boundary reflection. Bothhave many interesting applications.
The method developed here is surprisingly sim­ple and computational algorithms can be implement­
ed. We have not discussed numerical convergence andcomputational stability. Our main purpose is to de­velop this unique method with the hope that it will
have applications.
Since we use the scattering matrix, a section is de­voted for introduction to and review of some prop­
erties which are essential for our analyses. The fol­.lowing section is on the development of our identifica­tion method. Mathematical analyses are presented in
the appendix. The last section is for general remarks,
including the treatment of an identification problem
with homogeneous medium;

2

Scattering Matrix

In this section we give a review of the scatteringmatrix of a transport system. Certain properties are- 42 -

essential to our method and our analyses. A scatteringmatrix S = S(x, y) of a medium extended from x toy
is given by [1], [3],S = S(x,y) =(rt -rP).

(1)-rwhere t and are left and right transmission opera­
tors and p and r are left and right reflection opera­
tors. The scattering matrix maps inputs to outputs atboundaries, i.e.,(2)
inputsoutputs
Physical properties of inputs and outputs are de­pendent on the particular transport system under con­sideration. In radiative transfer they are specific in­tensities. For example, u(x) = f(x,n), n = (0,rp), tosignify that they are functions dependent on x and on
direction, rp. In n-port transmission line theory, they
are n-dimensional vectors of electric waves, etc. Op­
erators are linear mappings. If there is only an inputv(x) at boundary x, then tv(x) = v(y) which denotes
the amount of input v(x) transmitted to the output
v(y) at boundary y. Likewise, rv(x) = u(x) which in­dicates the amount of inputs v(x) is reflected to the
output u(x) at boundary x. Likewise, we have similarinterpretations for p and
The transport system satisfies the principle of in­variant imbedding [1]. That is, when two media areassembled, the individual medium properties are notaffected by the existence of the other. Then when
two media are combined, the inputs and outputs atthe boundaries of the combined media are governedby [3].

-r.

S(x, z) = S(x,y) * S(y, z).

(3 )The associative multiplication on the right of eq. (3)is not the usual matrix multiplication. It is called *product which is
( �

�

) * (

�:

�� )

The associative * product governs the algebraic
structure of the transport system at which the princi­ple of invariant imbedding holds. The generalizationof eq. (3) leads tonS(x, h) = S1 * S2 · .. *Sn �ITS;

where S(x, h) denotes the scattering matrix associat­
ed with a medium extended from x to h, h 2: x and(x, h) are partitioned into x = x0, x 1, x 2, ••• , Xn = h.
Each S; denotes the scattering matrix associated witha medium extended from x; to x;+1, x;+1 2: x;.
In the case where a medium extended from x toy
is imbedded in a free space, then when x = y there are
no reflections and all inputs are transmitted. That is,S(x,x) = ( � � )where S = S(x, y) and S 1 = S(y, z) for convenienceand E = identy , We assume there is no "critical case,"i.e., all operators are bounded.

(6)

It is well known that the scattering matrix satisfies
the following nonlinear system [3], [6], [7]_(b + pc)t a+bp +pd+pep!._) . (7)S(x,y) (ayrct-r(d+cp)
where a, b, c and d are medium coefficients. They are
related to the physical properties of a medium. There­
fore medium coefficients determine the scattering ma­trix by solving equation (8) with initial value (7). Thatis called the direct problem.

3

Identification

Let us consider two transport systems, one withknown medium coefficient M and the other with un­
known coefficient M, andandM(z) =(4)t 1 (E - pr1)- 1 t( r + -rr 1 (E - PT1)-1t
1
Pl+ t 1 p(E - r1 p)- -r1
1)-r(E - r1p - -r1

(5)

i=l

(b a)
c <i '(8)

where coefficients a, b, c and d arc identical to thosein eq. (8). The precise identification problem is that
we will determined M(z) for all x :S: z :S: y from theknowledge of outputs at boundaries, x and y. We
present here a step by step method to solve this prob­
lem. Some mathematical analyses are presented in the
appendix.
i) To determine S(x, y).

- 43 -

This is done by the direct method. For discussionof this, see [3], {6), (4). Because eq. (8) is nonlinear the
analysis is much more difficult than, say, a linear equa­tion. However, we present an approximation methodhere.
The approximation method takes advantage of thealgebraic structure of the* product, as in eq. (6). For
t!..x; small and eq. (7), we haveS(x;, Xi+1) - S(x;,x;)[b(x;)+ p(x;,x )c(x;)]t(x;,x;)�=(

By knowing inputs v;(:z:) and outputs vk(Y) and uk(:z:),t and fcan be obtained. Likewise, we can obtain f andp. Therefore, S(:z:, y) is determined. For the function
case, t, f, p and fare integral operators,l. v(x) =for some x; $ ii:; $ x;+1 by the mean value theorem.
Eq. (9) leads to
S; = S(x;, x;+1) = E + M(x;)6x;.

t(u1(Y),u2(Y), ...,un(Y)), etc. Then transmission ={t;k} and reflection operators f = { r;k} are n x nmatrices and(10)

And eq. (5) can be rewritten asjl(n - n')f(x, n') dn' = g(y, n)

where n = (0, cp) denotes the direction /(:z:, n') is the
input v(:z:) in the direction n', g(y, n) is the output inn, and t(n, -n') is the kernel. Knowing j and g, thetransmission operator can be solved by the convolution
theorem, i.e.,nS(x,y) = Il[E + M(ii:;)6x;]i:1

AI\ . S, --t S, - S; •

•

•

-

f,.

-

( r1
I

r,-ti i- t ;-1 Pi'T'j -

r; t i-1 p;

(17)

(11)

The choice of ii:; needs computation in the particular
problem under consideration. Even with a poor choiceof ii:;, or merely ·letting ii:; = x;, the convergence isuniform as n ➔ oo.
Eq. (11) involves many * product multiplications.
The following discussion is to facilitate the computa­tion.
Let us use Redhe:ffer's /\ operation [3), i.e.,

).

(12)
Eq. (12) requires t;- 1 to exist. This condition is always
satisfied for t!..x; small, see [7). Use the properties

where · is the usuai matrix multiplication. We obtain

Eq. (14) is easier to compute than eq. (5). Upon
substitution of eq. (11) for S;, we have solved forS(x, y). The conclusion is
Given M(z), for all x $ z $ y, we can obtain
S(:i:,y) or estimate Six,y).
ii) To determine S(x,y).
To determine S(:z:, y), the scattering matrix foran unknown medium, we shall consider whether in­puts and outputs are vectors or functions. In thevector case, v(:z:) = (v1(:z:),v2(x), . . ,,v n (:z:)), u(y) =(16)

where F is the Fourier transform. In practice one may
use Fast-Fourier series to speed up the computation.
Likewise for other operators. Therefore, S(:z:,y) is de­termined.
B y measurement., of inputs and outputs at bound­aries :z: and y, we obtained an estimate S(x, y).
We have used the fixed-point theorem by Wang [7]for the diagonalization of a dissipative scattering ma­trix. For application in radiative transfer, this con­dition is easily satisfied in case the volume attenua­tion is not zero. The other assumption we have usedthroughout this paper is that the transmission opera­
tor is nonsingular. This condition is always satisfied
for a medium width less than the distance to the con­jugate point [7]. In the case of radiative transfer, thecondition that t and 7" are. singular restricts us to con­sider a finite thickness medium if the attenuation is
not zero.
iii) Solution for unknown M(z).
In the appendix, we establish that if S(:z:, y) andS(:z:, y) are dissipative and transmission operators are
nonsingular, then there exists a constant nonsingularmatrix Q such that
S(x,y) = Q- 1 . s(x,y). Q.

(18)

Since we have obtained S and S, S and S, we merely
employ the /\ operations on them. The matrix Q canbe obtained by the standard numerical method at least
for the vector input and output case. And Q can be
estimated in the function case.

- 44 -

Once Q is obtained, equation (A-14) in the Ap,­pendix provides us a formula to compute the infinites­
imal generator N( z ) for all x ::; z ::; y. The M( z ) of
the unknown medium is solved,-M( z ) =

where

(E O)
· N- ( )EOz-

N( z ) = Q- 1 · N( z ) • Q.

(19)

(20)

In case Q = E = identity, then M(z )M(z ).
We can consider that the unknown medium coefficientM(z ) is a perturbation of M(z ) under the operation
Q. Letting
6.M(z )= M( )- M( )zz

6.M( z ) = JQ-1JMQ - M

(22)

-E

(23)

=
J

(

0

O
).
E

S(x,y) = S; · S; ...

s; = Sf.

Since S; = E + MAx, we obtain- a6.xb6.x)
S; = ( E cllx
E + d6.x

by eq. (13). If we choose n = 2k and assume S(x, y)
is positive definite, then S; and a, b, c, d can be solved.
This is a very simple method; however, it is rather
difficult to prove S(x, y) is positive definite based onsome physical reason.

(21)

and substituting M(z ) and M(z) into S(z , z + 6.z ) =Q-1,S(z, z +A z ) ·Q, and taking the limit as 6.z -+ 0,
we obtain the following perturbation equation

where

equal, i = 1, 2, ..., n. Then, by taking the A operation
on eq. (4), h =yin this case, we have

In case 6.M is small, we will expect Q to be near­ly equal to the identity. This perturbation of a near
identity also guarantees the existence of Q.

Appendix
If the scattering matrix is dissipative (the pre­cise definition of dissipative used here is that it isreflective-left and right-dissipative, see [7]) in ad­
dition to the assumption that t and rare nonsingular,then by Theorem 6 of [7], there exist two unitary ma­trices U1(z ) and U2(z ) such that
where D is a diagonalized scattering matrix_ ( to(x, z )D ( x, z ) -

(A-2)

O

4

General Remarksi)

The Method

The result presented here on identification of atransport system is new and unique. We believe ourmethod can be easily implemented.
The basic solution involves the following steps:a) F ind a medium coefficient M(z1 closely related tothe unknown medium coefficient M(z). b) Computethe scattering matrix S(x,y) associated with M(z) by
the direct method. c) Compute the scattering matrix
S(x, y) associated with .M(z) by measurements of in­puts and outputs at boundaries x and y. d) Use anumerical method to obtain Q, see eq. (19). e) Com­
pute N( z ), M( z ) by eq. (20) and (21).
ii) The homogeneous case
The homogeneous case is a special case consideredhere. The method outlined in i) applies. However,an alternate method can be used to solve this specialcase.
For the homogeneous case, we set the partitions on(x, y), having equal width, llxi = llx. Then all Si areThis scattering matrix has zero reflections. The /\product is well defined on U(z) since E" = E and(S 1 * S2)" = S1 · S2. Then it follows that t and r non­singular implies to and ro are nonsingular. It follows
that D has an inverse under the * product, and
Likewise, for S(x,y), we can solve for Sin terms of S,i.e.,
S(x, z) = P(x, z) * S(x, z) * Q(x, z )

(A-4)

where
--

-

1
P (x , z) = U1 (x, z ) * D(x, z ) * U1(x, z)andQ(x, z ) = U2(x, z ) * n- 1 (x, z ) * u; 1 (x,z).

As z -+ x, S(x, z) and S(x, z) both approach E, thatis,P(x, x) * Q(x, x ) = E.

- 45 -

and

(A-5)

References

under the usual matrix inverse.
Let N and N be the infinitesimal generators ofS"(x, z ) and S"(:z:,z ), respectively; thenoS , z )t= S(:z:,z)N(z)andoS( :z:,z)oz(2] A. Bruckstein and T. Kailath, "An Inverse Scat­
tering Framework for Several Problems in Signal
Processing," IEEE ASSP, Jan. 1987, pp. 6-20.

=- s ( x, z .)N- ( z ) .
_

(A-6)

By taking the /1. operation on both sides of eq. (A-4),
and differentiating with respect to z, we haveos(x,z)oz=AP( x,z)·oS(:z:,z ). A
Q( :c,z)OzoP(x, z)•S(:z:,z)•Q(:z:,z)(A-7)
ozoQ(x,z).
+P(x, z) · S(:z:,z) ·
oz
A

A

+

A

A

On the other hand, if N(z) satisfies the equationQ"(:z:,z ) • N(z) = N(z) ·Q"(:z:,z),(A-8)

this leads toP(:z:, z) • S(z,z) · Q(x, z ) · N(z)P(:z:,z) · S(z, z) · N( z ) · Q(x, z)oS(:z:,z)· Q(x, z) (A-9)
P(:z:,z) ·S(:z:,z) · N( z)

A

A

oz

=

oS( x,z)oz

Comparing eq. (A-9) and eq. (A-7), it is necessarythat the last two terms of eq. (A-7) be zero. Thiscondition is easily satisfied if P(:z:,z) and Q(:z:,z) are
independent of z. In view of their initial condition(A-5), we have**S(:z:,z ) = Q- 1 S(:z:,z ) Q(A-10)

where Q = Q(:z:) is independent of z. By taking the/\operation, this leads toS(:z:,z ) = Q- 1 · S(x,z) • Q(A-11)

for all :z: $ z $ y. For convenience we drop the /1. on
Q. Eq. (A-8) now has the formN( z) = Q- 1 • N( z )•Q(1] R. Bellman and S. Ueno, "Invariant lmbedding
and Chandrasekhar's Planetary Problem of Ra­
diative Transfer," Astrophys. Space Sci., Vol. 16,
1972.

(A-12)

for all :z: $ z $ y.

- 46 -[3] R. Redheffer, "On the Relation of Transmission
Theory to Scattering and Transfer," J. Mathe­
matics and Physics, Vol. XLI, No. 1, March 1962,
pp. 1-41.
[4] S. Ueno, R. Kalaba, H. Kagiwada and R. Bell­man, "Some Mathematical Aspects of Multiple
Scattering in a Finite Inhomogeneous Slab with
Anisotropic Scattering," Pub. Astr. Soc. Japan
Vol. 22, 1970, pp. 75-83.
[5] S. Ueno and Y. Kawata, "Earth-atmosphere Sys­
tem Identification of Optical Parameters," Con­cise Encyclopedia of Environmental System, P.C.
Young (Editor), Pergamon Press, Oxford, 1993,
pp. 163-168.
[6] S. Ueno and A. P. Wang, "Scattering and Trans­mission Functions of Radiation by Finite Atmo­
sphere with Reflecting Surface," J. Astrophysics
and Space Science, 23 (1973), pp.. 205-219.
[7] A. P. Wang, "Dissipative Properties of a Nonlin­
ear Differential Operator Equation," J. Mathe­
matical Analysis and Appl. Vol. 72, No. 1, Nov.
1979, pp. 75-88.
[8] A. P. Wang, "Searchlight on a Target with Diffuse
Background," J. Math. Physics, Vol. 34, 1993, pp.
878-884.

Robot Path Planning through Some Places by Genetic Programming
H. Yamamoto
Dept. of Opto-Mechatronics
Wakayama University
930, Wakayama-shi, 640, Japantiger@center.wakayama-u.ac.jp

Abstract

2 Next Generation FA FactoryThis paper describes the method to determine a
robot path that travels some places among machine
tools of a production line FA factory. This decision
is made by Genetic Algorithm with Lisp language
programming. In the algorithm, the building block
method to decide fitness is adopted. The method isapplied to FMS system that has four machine tools anda robot.
Keywords: Genetic Algorithm, Robot Path Planning,
FA Factory, Lisp Language
1 Introduction

As an example of the next generation FA factory, adecentralized autonomous production system isconsidered. In the next generation production system,
the machine tools and robots achieve the production
plan by changing their production configuration bygathering or dispersing. As the locations of themachine tools are not fixed and are changeable at a
moment's notice, it is useless for us to program therobots' path while developing the production system.
In this paper, in order to decide the moving pathof a robot operating by itself, the method by
Genetic Programming (GP) is described. By carrying out
genetic operations [1] such as crossover andmutation for the several elements of Lisp lists, theproposed 1ethod automatically finds a robot moving
path through some visiting points with a Lisp program.
During the genetic operations, as the original method
to calculate fitness, the building block method isadopted to reduce the searching time for the solution.
By applying the method to the FMS (Flexible
Manufacturing System) example where a robot moves
among four machine tools, the new method is
ascertained to be useful.

Many kinds of research for the next generation FA
have been proposed in recent years [2]-[4]. The
common element in such research is that FA systems
such as a machine tool and a robot have a knowledge
and make decisions or reason by themselves. As one ofthe prominent figures in the 21st century FA
factory, the following intelligent FA system isconsidered in this paper. In an intelligent system,
the machine tools and robots have different amounts
and level of knowledge and mobilities which they use
to interact with each other and determine a production
system. This is called an autonomous decentralized
production system. For example, in order to produce
some varieties of parts, several machine tools androbots communicate and negotiate among themselves.
As a result, an agreement is reached and they move toa certain place in the factory to begin production of
the parts.
Under these circumstances, the location of thegathered machine tools and robots, which corresponds
to the production line, cannot be forecasted
beforehand. That means machine tools and robots cangather and disperse in the factory any time. Becauseof the changeable situation in a factory, it isuseless for us to have the robot traveling path
programmed beforehand in order to supply parts and.
tools to the machine tools.
Figure 1 shows an example of the production line
which is used as the simulation example in this paper.
The production line consists of four machine tools anda robot. The robot travels, as shown in Figure 1 andthen returns to its starting point.
3 Program Of Robot Path Via
Some Visiting Points

3.1 Expression of Path Prograuing with Lisp

- 47 -

mf - nuve 0. 5m forwardmb -► DXJVe 0. 5m backward°ti -► tum left 30°
tr -► tum right 30
sOI -► minimum distance of obstacle from 0° to 30°
s02 --► minimum distance of obstacle from 30° to 60°
s03 --► minimum distance uf obstacle from 60° to 90°
s04 --► minimum distance of obstacle from 9()° to 120°s05 ·-► minimum distance of obstacle from 120° to 150°
s06 -- ► minimum distance of obstacle from 150° to 180°s07 - -► minimtnn distance of obstacle from 180° to 210°s08 ► minimum distance of obstacle from 210' to 2,1()"s0lJ ► minimtnn distance of obstacle from 2,10° to 270°
slO -► minimum distance of obstacle fnxn 270° lo 300°
sll - minimum distance of obstacle fnxn 300° to 330°
sl2 - minimum distance of obstacle fmn 330° to 360°d - ► dangerous distance

Mach
Machin½

MachinROBOT

Figure 2 17 Arguments

START and GOAL

Figure 1 FMS Production Line

Consider how a robot takes its next step. The next
step can be forward, backward, to the right or to theleft. If the four kinds of movements are combined, thepath locus of a robot can be expressed. For example,
if the Lisp function for moving forward is [fw],turning right [tr] and turning left [tl], the Lisp
program to express certain movements corresponds to((fw)(tr)(fw)(tr)(fw) (tl) (fw)). The Lisp program using
functions corresponding to the four kinds of movements
can express the factory space even if the space isfilled with machine tools and work tables.
3.2 Path Decision by GP

Because of developing the Lisp program repeatedly
to carry out the four kinds of Lisp functions, [fw],[tr], [tl] and [bw] ( which means moving backward ),the path between two points can be expressed. Randomly
developing this program, the solutions for the path
become limitless.
In order to find an efficient solution, thedecision method for a Lisp program by GB is adopted[5]. As "atoms" to express the Lisp program for thecycle movement of a robot, as shown in Figure 1, a setof arguments and a set of functions are used. A set ofarguments consists of 17 elements, as shown in Figure
2. Figure 2 includes each meaning of the arguments. A
set of functions consists of two functions, [IFLTE)
and [PROGN2]. The function [IFLTE] needs four
arguments. For example, it is expressed as (IFLTE abc d) and its meaning is " Evaluate a and b. If theresult is aSb, evaluate c. If the result is a>b,
evaluate d". The function [PROGN2] needs twoarguments. For example, it is expressed as (PROGN2 ab) and its meaning is " Evaluate a and b and return b
as the result of the evaluation". As the arguments ofthe two functions, the elements of an argument set areused.
3.3 Genetic Operations[Initial Population]
In order to create an initial population for Lisp
programs, after randomly selecting each element of anargument set, a function set and a pair ofparentheses, make a list of them. The example for anindividual can be described as (PROGN2 (MF) (IFLTE(PROGN2 (MB)(TR) (SOI)(MF)(TL))). Every individual has
the characteristic that it can be expressed with its
own tree construction.
[Crossover and Mutation]
A crossover is performed by using thecharacteristic of an individual, as mentioned above. A
crossover first randomly selects one point on thelist of a Lisp program as a crossover point. With anindividual by tree construction, the crossover is alittle different from the one-point-crossover which is
performed by a string type expression for anindividual. From the view of the tree construction,
the crossover finds a certain node in the tree andmoves the node and the sub-tree situated below thenode to another individual.
In the same manner, a mutation is carried out.
After the mutation point is randomly selected, theselected node and the sub-tree whose top corresponds
to the selected node are changed to new node and sub­tree. The new node and sub-tree are created just likean initial individual was created.

- 48 -

f'- .

� �\

:L4\

'
t'

LS

·--·
L2

: L3

__ ..

'

: L2

l ........ .. 1..___)

.,.,

L1

..

....L1----�....

'
'

'
'

''

}

RO

START md OOALSTART md OOAL

Figure 3 Ordinal method for fitness

Figure 4 Building Block Method

3.4 Fitness
Fitness shows how each individual adaptable is toan environment. In this paper, the degree ofadaptability to an environment is defined as how far arobot travels in one cycle movement. The ordinal
method for fitness chooses fitness as the distance
between the robot's starting point and the point
where the robot finally arrives by a generated Lisp
program. So, for example, Ll + L2 + L3 + L4 + L5 in
Figure 3 corresponds to total fitness.
On the contrary, tremendous amount of paths totravel between two points in two dimensional spaceare existing. That means it spends tremendous amountof time for the robot to search the path from thestarting point to the first traveling point in Figure
3. If the single searching time is limited, GA
operations generate lethel gene that most individuals
cannot arrive at the first traveling point.
In order to solve the above problem, the followingbuilding block method of stacking fitness is adopted.
The building block method splits up fitness for anindividual into fitness for each of the travelingpoints. The fitness for each traveling points means
the distance from each of the robot arrival points toeach of the traveling points (machine tools' ports).
Each distance is considered as a "block". A single
individual's fitness is decided by stacking theblocks. The split block has the time constraints forsearching.
In the building block method, a robot moves acertain distance in a predetermined time. So, in its
first phase, it moves from the starting point. For

example, one minute and then stops. The distance from
where it stops to the first traveling point is known
as its fitness of a single block ( which we will call
Ll, as shown in Figure 4). In its second phase, it
moves from the first traveling point a certain
distance and stops. The distance from the stopping
point to the second traveling point may be called L2.
And so on until it has visited all of travelingpoints and returns to its goal. The fitness in Figure
4 is expressed by summing up five blocks' fitness: Ll,
L2, L3, L4, and L5.
By calculating fitness with the building block
method, the individual that has the schemata
representing the distances between the arrival points
and traveling points can exist. Even if an individual
does not complete the cycle from start to goal, thebuilding block method allows it to survive to the next
generation. If the individual includes the block thatthe robot arrives at a traveling point, the block's
schemata has a high possibility to be inherited by thenext generation.
In this way, even though some individuals are notcomplete schemata as a whole, that is, they cannot
complete the cycle, but they include complete
schemata for a certain block, the individuals thathave complete schemata for all blocks can be finallygenerated by repeated GA operations such as crossoverand mutation.
4 Application Examples- 49 -The building block method is applied to a FMS thathas four machine tools and a robot, as shown in Figure
1. The robot travels to machine tools Machinel,
Machine2, Machine3, and Machine4 in turn and thenreturns to its starting point. The application example
adopts a population size of 400, a crossoverprobability of 88% and a mutation probability of 1%.
Figure 5 shows the best result acquired fromgeneration 154. Figure 6 shows the fitness curve of
the application example. The bold curve is the best
fitness among each generation, a big dotted line is
the fitness average among each generation and a small
dotted line is the worst fitness.
5 Conclusions

This paper describes the method to automaticallygenerate a robot path programing among some machine
tools called FMS. The method adopts Genetic
Programing. In the programing, a building block method
is used which allows individuals with partial
schemata to survive to the next generation. The
strategy of the building block method is " theindividual that partially includes a good schemata tosurvive".
After simulations of a robot path generated in
FMS, the best individual is found from generation 154.
References[l]D.E.Goldberg, Genetic Algorithm in Search,
Optimization and Machine Learning, Addison-Wesley,
1989.
[2]P.Valeckenaers et al., IMS Test Case 5: Holonic
Manufacturing Systems, Preprints of IMS'94 2nd
IFAC/IFIP/IFROS Workshop, 1994.
[3]L.Kavraki and J.Latombe, Randomized Preprocessingof Configuration Space for Fast Path Planning,
Proceedings of 1994 IEEE International Conferenceon Robotics and Automation, 1994, pp.2138-2145.
[4]H.Yamamoto, Simulator of Flexible Transfer Line
Including Genetic Algorithm and its Applications to
Buffer Capacity, Priprints of IMS'95 3rd
IFAC/IFIP/IFROS Workshop, 1995, pp.127-132.
[5]J.R.Koza, Genetic Programming, MIT Press, 1994.
best
worst
average

Y: fitness

28.00 -+------+-------1-----26.00 -+.-----+-------1-----24.00 --t:,\::--. --·-: ,· ---+---__:---.+.ff \'--,-,-22.00 -+--:-\-....-......... ,-;;-',. --"·.-...-.._-__.+-_.-=--• -..-...+,\---+-----J.--4\:·, -.{'--+20.00 ----f,-----,-,_.......
18.00 --+----_-,=-,- ----+-----J.-______:_
16.00 -+IL:::,,�.----'-'-d-••-------4---14.00 -t----\\,-----+-..:.....-,_-_ _ --+---12.00 -+-----'---.,--� -+---_="--<-_-jf---_-- --t==--=---+--'.-10.00 -t-\ ---l---'-_ :...._ -8.00 -+-----l---�
-\ -2.-_. ' _:___
6.00 -+-----t-----''---h_4.00 --t------+----H\-__________::_c"-.------L.. -2.00 -t------+-------+---::..
�
0.00 -+-----+-----+---_,J---i� ,---t,_..

+--,__....�... -�
..._-t-J. 1-___i---'i--,

.4�

100

Figure S Final Simulation Result120

X : generation

Figure 6 Fitness Curves- 50 -

140

A Radial Basis Function Neuro-Tracker
Carl G. Looney
Computer Science Department/171
University of Nevada
Reno, NV 89557 USA
email: Looney@cs.unr.edu

Abstract

Figure 1 - The RBF for Cluster Center vy

We use a radial basis function neural network tocontrol a video camera to track a target moving at highspeed. The results show considerable promise.

y = f(x;v)

1 Introduction to RBF Neural Networks
We investigate the application of radial basisfunction(RBF) neural networks to video tracking of a moving
target. RBFs originated in 1964 as potential functions in[1] and [2], and in [6] in 1968. Their use in neuralnetworks was due to [3] in 1988 and to [5] in 1989. The
tracking problem is taken from [7]. An RBF has theGaussian form [4]x ➔ f(x;v) = exp[- II x - v II 2/(2a2)]

Figure 2 -Architecture of The Tracking RBF Network(1)

Difference
VectorInputswhere x is an input feature vector, v is a fixed center
vector of the same dimension as x, and a2 is the spreadparameter. Figure 1 shows an RBF on the x1xz-plane. An(k)dx -----c><"---r---r---+=..t(k)dy

RBF and center vector belong to each hidden neurode.
Our N-M-J RBF neural network is shown in Figure
2. Let {(x<q),t(k(q)))} be a set of Q pairs of exemplar input

Controls

and output vectors for training, where x<q) =(x 1 <q),...,xN<q)) is the qth input feature vector, t(k(q)) =(t1 (k(q)),...,tl(q)) is the kth target output for input x<q) andthere are k = 1,...,K different output classes. We selecta number M of neurodes (neuronal nodes) in a single
hidden layer. For each m = l,...,M, there is an mth
center (weight) vector v<m) and a spread parameter um2

for the mth hidden neurode) that determine the RBF Ym

When x is close to v<m) then ym is high, but when x
is far from v<m) then ym is low. Each of the J perceptron

During operation, an input vector x is presented to
the network and processed separately at each mth hidden

type neurodes in the output layer computes a weighted
average of the RBF components of y = (y1,.•• ,yM) via= fm(x;v<m)) = exp[- II x -v<m) II 2/(2a2)] for it.

neurode to produce the output value Ym = fm(x;v<m)).

(2)- 51 -

where either S = M or S = y1 + ... + YM· The RBFs
partition the input space into small hyperball subclasses
so the output layer can join (OR) groups of them into
nonconvex classes for nonlinear separability.
The first stage of training selects the M centersm)<{v } and the spread parameter11. The quickest method
is to put M = Q and take each input exemplar feature
vector x<q) to be a center via v<m) = x<m>, m = l,...,Q.
This method requires only the training of the output
weights {�j}, which is extremely quick.
When Q is too large, we construct K different
vectors {y (k)} such that each one has a single high value
in the mth component of y(k> = (y/k>,...,yM(k>) and all of
the remaining components are zero and put M = K. The
weight (center) vectors {v(m)} are then drawn randomly
with components between O and 1, where v<m) =(v1 <m>,...,vN<m>). Next, we present the exemplar feature
vectors {x <V} to the hidden layer, one at a time, and findthe maximum RBF value ym• that determines the winning
m*-th neurode. We update the single winning neurode
with center v<m•) via

the results of training with the exemplars (0.3,0.3),(0.3,0.7), (0.7,0.3) and (0.7,0.7), which is more robust.

Figure 3 - llBF Contours for XOR Logic

Figure 4 - RBF Contours, New Centers(n = 1,...,N) where "I is the learning rate. We update
the other centers by subtracting the rightmost term.
Each exemplar is presented in tum about 12 times(12 epochs or 12Q iterations) to obtain good centers. The
value 11 = 0.6 usually suffices but may be smaller orlarger. The spread parameter ci1- about 0.01 to 0.8 for anormalized feature space of inputs, depending on M (alarger number of clusters M means a smaller ci1-).
The second stage trains the jth output weights {�j}over all Q inputs by steepest descent via

2

The output target t(k(q)) must be known for each input
exemplar vector x <V. The input vectors and output values
do not need to be constrained as in MLPs, but we preferto standardize (normalize) the inputs.
This training is extremely fast (up to 1000 times
faster than backpropagation). The difficult training of
hidden neurodes by gradieut methods is not done, which
circumvents the problem of local minima.
Figure 3 presents the case of XOR logic where thetraining uses the exemplars (0,0), (0,1), (1,0), and (1,1)
as neurode centers. In contrast to this, Figure 4 shows

The Tracking Application

Tracking is implemented with a planar video displaythat shows the object being tracked. The field-of-view(FoV) is a rectangular .screen with cross-hairs at thecenter. It is conceptually covered by M radial basis
functions whose one-sigma (o) contours are shown in
Figure 5. The idea is to drive the current errors dx anddy of the target position (dx,dy) to zero using the input
vector of feedback errors e = (dx(r), dy(r), dx(r-l ), dy(r-1),
dx<r-2>,dy(r-2)) on iteration r. The RBF network is shownin Figure 2. The outputs ex and Cy are the respective x
and y commands for the camera positioning motors.
An input positional error vector e activates one ormore RBFs to put out higher Y m values (1 S ms M).

- 52 -

The combination of outputs is fed into each of the two
output neurodes, where the values are multiplied by theweights {umj} (j = 1,2) and summed to yield the control
commands, ex and cy. The network is trained to map the

Figure 6 - The CCW Target Path (no control)

input error vector e into the commands ex and cy that areproportions of maximum turns 0xmax and 0ymax that the+camera's gimbal motors can make.

Figure 5 - The Field of View with RBF u-Contoursdy

Figure 7 - The CCW Path after Training 1

dx

y-am

3 Simulation Runs and Results
A simulated high speed target was run on both thecounter clockwise (CCW) path shown in Figure 6 andclockwise (CW) path that is the reflection of the path of
Figure 6 about the vertical axis. The RBF network
controlled the camera after training.
The centers of the nine circles shown in Figure 5
were taken as the initial center (weight) vectors of thehidden neurodes. The initial output weights were selected
randomly between -1 and 1.
The network was trained by adjusting both the hidden
and output weights, one at a time, to minimize the sumsquared error (SSE) over the paths (sum of dx2 and dy2
over the updates of an entire path). The camera tracking
system was modeled simply so that the commands ex andCy moved the camera through the angles ex and ey thatare proportions of 0xmax and 0ym ax·
We trained for 20 iterations, where a single iteration
adjusted each of the hidden and output weights in turnover a complete target path flyout to determine the sum-squared error. The spread parameter c?- = 0.4 yielded
more stable results than smaller values. The CCW target
path starts on the right, but the CW target path starts on
the left. Figures 7 and 8 show the target position relative
to the controlled camera on the respective CCW and- 53 -

x-axisFigure 8 - CW Path after Training 1
y-am- -- -- - --- ----- <±)x-axisCW paths under Training 1 runs that used the two inputs
dx(r) and dy(r). Figures 9 and 10 show the results ofTraining 2 runs that used the four inputs dx(r), dy(r), dx(r­l) and dy<r-l). The Training 2 errors here are muchsmaller.
It is clear that this system is equivalent to a fuzzy
rule-based control system where the RBFs are Gaussian
fuzzy set membership functions and the rules connect the
fuzzy set membership values to the defuzzified controls.
In both neural and fuzzy systems, the fuzzy membershipfunctions allow room for inexactitude.

Figure 9 - The CCW Path after Training 2
y-axisextrapolate to conclude that the use of twice delayederrors additionally would provide even more accuracy,
but we expect that more delayed errors might not
improve the control further or might even degrade it. The
size of the network is small and the training is quick andnonproblematic in contrast to that of backpropagation
multiple layered perceptrons.
It is clear that this network is equivalent to a fuzzy
rule based control system and has the advantages of i)
possible implementation in parallel for speed, ii)
simplicity of design in that the rules become built in by
the training, and iii) simplicity of execution because rules
do not need to be fired by antecedents that must be
fuzzified and consequents do not need to be defuzzified.
The fuzzification and defuzzification are built in also.
Once an RBF network is trained, the weights can be usedin on-line production systems for intelligent control.

References[1] M. A. Aizerman, E.M. Braverman, and L. I.
Rozonoer,(1964), "Theoretical foundations of thepotential function method in pattern recognition
learning, Automation and Remote Control 25, 821-837,
1964.

*············--····················

11[2] 0. A. Bashkirov, E. M. Braverman and I. B.
Muchnik, "Potential function algorithms for patternrecognition learning machines," Automation and Remote
Control 25, 629-631, 1964.

Figure 10 - The CW Path after Training 2
y-axis[3] D. S. Broomhead and D. Lowe, Multivariable
functional interpolation and adaptive networks, Complex
Systems 2, 321-355, 1988.
1111[4] C. G. Looney, Pattern Recognition with Neural
Networks, Oxford University Press, NY (in press, 1996).

·······"'·"······.........................."F�

[5] J. E. Moody, and C. J. Darken, "Fast learning in
networks of locally-tuned processing units," NeuralComputation 1, 281-294, 1989.
[6] D. F. Sprecht, A Practical Technique for Estimating
General Regression Suifaces, LMSC-6-7968-6,
Lockheed Missile and Space Co., Palo Alto, CA, 1968.

X-IIXIS

4 Conclusions
The use of RBF neural networks to control a video
camera to track a high speed moving target demonstrates
the flexibility of such networks. The use of feedback
errors delayed by one time increment in addition to thecurrent errors provided more accurate control. We may[7] Masanori Sugisaka, "Neural network control forrecognition and tracking of moving objects," Proc. /SCA
Fourth Golden West Int'l Conj. on Intelligent Systems,
San Francisco, 189-193, 1995.

- 54 -

Self-Assembly Method for Mechanical Structure
Satoshi Murata, Haruhisa Kurokawa, Kohji Tomita, Shigeru Kokaji
Mechanical Engineering Laboratory, AIST, MITI
1-2 Namiki, Tsukuba, 305 JAPAN

Abstract: A novel design method for a machine system
is proposed and a prototype model is constructed.
Conventional machine systems are composed ofvarious parts which are passive. In the new scheme,
they are made with only one kind of active am
intelligent units. The prototype of this kind of unit iscalled II Fractum 11, which has actuators sensors am
information processing functions. In the· system, there
is no concentrated part. These units are the same am
have equivalent capacity so that any of them canreplace any unit. The knowledge about the wholesystem is embeddedin every unit and this enables thegroup of these units to organize the whole system bythemselves and to repair it by themselves. 1healgorithm for the self-organization and self-repair isdesignedand tested by computer simulations.

1 Introduction- Distributed Mechanical Structure
Living organism has internal maintenancemechanism. It can repair itself and reproduce itself
without outside help. In contrast, most industrial
products or machines rely on outside help for theirmaintenance. As the complexity of the machineincreases, the burden of its maintenance becomes
tremendous. If we can develop built-in autonomy forthe machine, it will be of great use. The idea has been
tried, however, with rather ad-hoc methods inconventional machines. If a supervisory system tocheck the machine operation is added to the originalmachine, then it certainly improves the machine's
reliability. But this method is an automated external
help and has limitations. There is no guarantee of thenormal operation o f the additional system. 1headditional system may cause trouble for the wholesystem. In order to avoid this, another supervisorysystem should be introduced to monitor the additionalpart and this repeats endlessly.
A solution is possible by making the wholesystem distributed. In this method, the machine iscomposed of units compatible to each other. The design
information of the whole machine is stored in each unit
and these units work together to establish the total
system function.
There is, however, a difference between thisnew type of machine and a living organism. Since themachine unit cannot reproduce itself, enough units
must be supplied from the beginning, also it ts
necessary to develop a novel way of self-assembly.

2 Hardware of Mechanical Unit
We have built actual mechanical units for a
two dimensional distributed mechanical system [1,2].
The unit called "fractum" may be described as a"moving LEGO block." It i s a n autonomousreconfigurable unit, namely, it can change connection
with other units without outside help. Each unit has 6
connection arms, an onboard micro-processor which
controls these connection arms, and 6 bilateral opticalcommunication linkages between processors ofconnected units. The connecting arm uses a pair of
permanent magnets and an electro-magnet as an
actuator. When the arms are connected mechanically, theinformation linkage between the units is established at
the same time. Using this prototype, we confirmed its
basic functions such as changing configuration am
communication. We are now constructing 20 units.

- 55 -

Figure 1. Schematic view of mechanical units

o {K,K}
K{o,K,K,s}s {K,K,K,K,K,K}

3 Software for S�lf-Assembly
A specific feature of the homogeneous
structure of the system is that each unit can be· used at
an arbitrary position in the system. Thus, each unit
must have information on the whole system, and theinformation must be identical in all the units.
We have developed two types of algorithms
which enable "self-assembly" of the group of
mechanical units. The self-assembly is a process where
the group of units form a pre-determined particular
destination shape from an arbitrary configuration. Only
local communication between the adjacent units is
asswned.in these algorithms.

3.1 Algorithm for Simultaneous Assembly(Algorithm A [l, 1)
2

On the unit, only information on local
connections is available. We must describe the wholeshape of the system based on the local geometry of theconnection. There are 12 possible types of local
connections for the unit (see Figure 2). The typetransition diagram shows the 12 possible types am
defines the distance between different types by thenwnber of movement steps required to transit from one
type to another.

(1)

The set of the above 3 statements (1) is a
blueprint of the whole shape given to all the units. In
the statement, the first letter shows one of the goal
types which appears in the final configuration, and theletters in parenthesis are neighbor types of the goal
type.
Each unit evaluates the difference between.its
current situation and the statements in (1) based on thetype information obtained by local communication. If
the current situation is involved in (1), the unit does
not move. Otherwise, it moves randomly, but thefrequency of the movement is determined corresponding
to the difference between its current situation and thegoal.
For small scale self-assembly, this algorithm
has excellent performance (97% success in the case of
10 units). However, the larger the number of the units,
the worse of the convergence and the probability ofsuccess. Also the algorithm could not deal withcomplex shapes. In short, the reason is combinatorial
explosion. Namely, the movement of the whole systemdoes not cease until all the unit satisfy the goalstatements simultaneously, and such a situation
scarcely happens when the number of · units is large.

3.2 Algorithm for Developmental Assembly(AlgorithmB [31)Figure 2.. Type transition diagram
We developed a method to describe the wholeshape of the system by these types. For example atriangular configuration made of 10 uni ts is described asfollows.

In order to cope with the complexity of large
scale self-assembly, the following developmental
algorithm is devised. The algorithm is an analogy ofthe development processes of living organisms. By
developmental assembly we can reduce the complicated··large scale assembly to small sub-problems.
In the first step, which is a counterpart offertilization, the group of units selects one particular
unit by mutual voting process. The selected unit calleda "kernel" becomes the starting point of thedevelopment process. The kernel gathers surrounding
units to form the first stage. After the units in tlie firststage confirm that they have finished the first stage of
assembly, they transfer to the next stage. They gather
surrounding units to form the second stage and so on.
In this algorithm, the blueprint given to each
unit must describe not only the final shape of thesystem but all transient· shapes through thedevelopment process. We use the followitig lower- 56 -

triangular matrix t o describe the process ofdevelopment.
{{ Y,---,---,---,---,---,---,---,---,---},{ Y, e,---,---,---,---,---,---,---,---},{ s, o, 0,---,---,---,---,---,---,---},{ ---, K, K, e,---,---,---,---,---,---},{---,---, f,eps, 0,---,---,---,---,---},{---,---,---,---, K,eps, 0,---,---,---},{---,---,---,---,---, K,eps, o,---,---},{---,---,---,---,---,---, K,eps, o,---},{---,---,---,---,---,---,---, K,eps, o}} (2)them represents a part of structure. Structure of thewhole system is expressed by the combination of thesesubstructures. We can use the same description matrix .
for the same kind of parts, thus the size of thedescription can be much reduced when the system
includes repeatedstructure and high symmetry.
To connect different description matrix, a jumpcommand is embedded in the primal (order: 0)
description matrix. If a unit is hit by the command theunit becomes a kernel o f secondary (order: 1)
description matrix. Of course the secondary description
matrix may involve jump commands to any otherdescription matrices including itself. In the case ofjumping itself, the system reads a description matrix
recursively. The following example illustrates thisprocess. The primal matrix represents a large triangle,
and "1" in the last line is the jump command to thesecondary matrix, which represents small triangle. The
secondary matrix called recursively, thus 3 long arms
made of many triangles are formed (see Figure 4)
0: {{ s, ---, ---, ---, ---, ---, -},{ s,eps, ---, ---, ---, ---, --},{ s, K, o, ---, ---, ---, --},{ S, f, o, o, ---, ---, ---},{ s, s, K, K,eps, ---, --},{ s, s, S, K, K,eps,---},{ s, s, s, 1, K, K, o}}
1: {{ K, ---, ---, ---, --},s, o, ---, ---, --},s,eps, o, ---, --},s, K, K, o, --},s, K, 1, o, ---}}

{
{
{
{

Figure 3 Stages of development describedin (2)This example shows the development process of asystem made of 25 units which has 3 radial arms. Row
numbers indicate the stage, and the characters in therow are the connection types that must be involved in
the stage. The numbers of each types are not given inthis matrix, but are determined implicitly byconsistency of the configuration. Figure 3 shows thedevelopment process described in the matrix.

3.3 Hierarchical Description Method for Large
Construction

By introducing multiple description matrices,
we extend algorithm B to realize construction of larger
structure. In this extended method, several description
matrices with different order are introduced and each of(3)"

�, ii\\ ;;1
1---1-- -1_,\\/
1 \\11,_\\II

1- -1----1\\11 \\11l-- --1\\II

1- --1-- --1\\11 \\11
1-- --t\\II·-- -----...... - --1----0-- -------\\1/\\II \\II \\II \\II \\II·-- -... ---0-- --0---1-- ---\\II \\II \\II \\II \\II..- -a--0--..0--..(1\\11 \\11 \\11 \\111
1
�, ii \\ ii\\ � it\\ ii \\ ii�\ 1/·1- -1-- --1- -1-S
1- -1- -s- -1"ll"ll"ll

1.- -1- -1- -1"ll"ll"ll

I-- •-1-• •-1- --1l\\11 \\11 \\11\\ ii� it\\1-- --1- -- 1

1/\\11 \\11

1.- --l- 57 -

tl

l

1

1

-..o--"ll"llo-- _...

"llI"ll"ll"ll
1

1-- -s- --I-- -1"ll"ll"lll1- -1- ••l•• -1\\11 \\11 \\111

1

1

Figure 4 Example of recursive structurel\" ""\ ii

l

12
II \\
12-----12

system. In this procedure, we make two assumptions,
1) enough number of unit is provided at the initialtime, and 2) no faulty units remains after some units
are removed
In algorithm B, construction process is divided
into several stages, and the system converges to sub­goal in each stage and finally realizes the target
confi guration. Thus the self-repair can be naturally
performed by backtracking of stages. At first, the lostof parts is detected units by checking communication
channel. Then the units which detect the failure
broadcast degeneration signal to other remained units.
The degeneration signal includes information about alevel of degeneration which is determined by the
maximum stage number which is not affected by the
failure. When the unit receives the degeneration signal,
the connection type of the unit is reset to someprevious level, and resume construction JIOCedureagain. Figure 5 illustrates a simulation results of theself-repairing process.

11\\ II\\

12----12----12
II \\ II \\ II \\12----12----12----12

II \\ II \\ II \\ II \\12----12----12----12---... 12
II \\ II \\ II \\ II \\ II \\12----12----12-... --12...---12----12
II \\ II \\ II \\ II \\ II \\ II \\12----12----12----12----12----12----12

II\\

II\\ II\\ II\\ II\\ II\\ II\\

12----12----12----12----12----12----12----12
II\\ II \\ II\\ II\\ II\\ II\\ II\\ II\\
12----12----12----12----12----12----12----12----12
II \\ II \\ II \\ JI \\ II \\ II \\ II \\ II \\ II \\12----12----12----12----12----12----12----12----12----12

spare units

removed-\\ I
12----12----12
II \\ II \\ II \\I

12----12----12----12

II \\ II \\ II \\ II \\12----12----12----12----12
II \\ II\\ II\\ II\\ II \\12----12----12----12----12- ---12
II \\ II \\ II \\ II \\ II \\ II \\12----12----12----12----12----12----12
II \\ II \\ II \\ II \\ II \\ II \\ II \\12----12----12----12----12----12----12----12
II \\ II \\ II \\ II \\ II \\ II \\ II \\ II \\12----12----12----12----12----12----12----12----12
II \\ II \\ II \\ II \\ II \\ II \\ II \\ II \\ II \\12----12----12----12----l 2----12----12----12----12----12

6
0, ,, ,,
6-- --6
I \\ II\\degeneration0

6-- --6-- --6
II \\ II \\ II \\6-- --6-- --6-- --6
II \\ II \\ II \\ II \\6-- --6-- --6-- --6-- --6
II \\ II \\ II \\ II \\ II \\6-- --6-- --6-- --6-- --6-- --6
II \\ II \\ II \\ II \\ II \\ II \\6-- --6-- --6-- -�6-- --6-- _.;.6-- --6

•

/

\

resume construction\ II \\
8-- --8--

11\\

5 Future Study

11\\/

8-- --8-- --8-- --8-- --8-II \\ II \\ II \\ II \\ II \\8-- --8-- --8-- --8-- --8-- --8\\ II \\ II \\ II \\ II \\ II \--8-- --8-- --8-- --8-- --8--. 00\ II \\ II \\ II \\ II \\ II \\8-- --8-- --8-- --8-- --8-- --8-\

II\\

II\\ II\\ II\\ II\\ II\\0

8-- --8-- --8-- --8-- --8-- --8-- --8
I
I \\ II \\ II \I
00--8-- --8

I

12
11!-��12

11\\

11\\

12----12----120complete self-repair

II \\ II \\ II \\12----12----12---• 12
II \\ II \\ II \\ II \\12----12----12----12----12

The study of distributed intelligence will be
indispensable for the refinement of self-organization am
self-repair function. Together with these logical study,
we envisage that the physical realization of the unit
structure will be the key to the next generation
technology. The organ system which is composed clvast number of organ cells are still beyond om
complete understanding because of its. high complexity.
We hope the study of highly distributed system withactive unit structure, like our unit based machine
structure, might lead us to the better understanding.

JI \\ II \\ II \\ II \\ II \\12----12----12----12----12----12
II\\ II\\ II \\ II\\ II\\ II\\
12----12----12----12----12----12----12
II \\ II \\ II \\ II \\ II \\ II \\ II \\12----12----12----12----12----12----12----12
II\\ II \\ II\\ II\\ II\\ II\\ II\\ II\\
12----12----12----12----12----12----12----12----12

ll\\ 11\\

References11\\ 11\\ II\\ II\\ II\\ II\\ II\\

12----12----12----12----12----12----12----12----12----12

Figure 5 Self-repair by backtracking of development

4 Algorithm of Self-Repair c41
Another extension of algorithm B is self­
repaiJ: We have developed an basic procedure of self­
reconfiguration when some of the units are lost. It
enables recovery from lost of arbitrary part of the[1] S. Murata, et al.: Self-Assembling Machine, Proc.
IEEE Int. Conf. Robotics & Automation, San
Diego, pp.441-448 (19'J4)[2] S.Murata, et al.: Self-Repairable Machine, J. SICE
31-2 (19<J5) in Japanese.
[3] S.Murata, et al.: Design of Distributed Mechanical
System (3rd Report), Proc. ADS Simp. SICE,
Nagoya (19<J5) in Japanese.
[4] K.Tomita, et al.: Design of Distributed Mechanical
System (4th Report), Proc. ADS Simp. SICE,
Toyama (19<J5) in Japanese.

- 58 -

System Properties of Amino Acids Production.
Linear system analysis for Asparaginic acid-Isoleucine-Methionine-Ly�:ne.
H Hirayama, Y Ishikawa, T Nozawa, Ko Li We,

and Y Fukuyama.

Department of Public Health, Asahikawa Medical College
4-5 Nishi -kagura, Asahikawa 078, Japan
ABSTRACTThe system properties of amino acids production
initiatedfrom Asparaginic acid to lsoluecine,
Metionine and Lysine was investigated by applying
linear system.
By examining the eigen values, thesystem was highly stable anduncontrollable.
With the optimized control input that minimizes
the squares of changes of the concentrations of all thecomponents in the system and the control input, theamplitude of
impulse responses
and theamplitude of singular values were reduced particularly
when the metabolizing enzymes do not exist. Present
investigation is available
to evaluate the relative
magnitude of minimization of each component
involved in the amino acids production system.
Amino acid, feed back, stability,
key words :controllability, singular value, optimization.
[ INTRODUCTION]
Amino acids are the building blocks of proteinsand peptides [1]. They
serve as precursors of many
important molecules that have essential biologicalactivities.
Therefore a proper control to produce
amino acids is indispensable to maintainthephysiological operation , namely, Homeostasis in
all the biological systems. · The processes of amino
production have well known as the mostacids
typical biochemical systemcharacterizedbymultiple
negative feed back inhibition [ 1 ][2].
Disturbances of the feed back mechanism result inserious pathological states, such ascongenital
enzyme deficit diseases[2].
The system analysis focused on the amino acidproduction which is initiated from Asparaginic acidto produce lsoluecine, Mehionine and Lysine to
examine, 1) the stability and controllability, 2) the
effects of the optimized
control input on theimpulse responses, on the temporal behaviors of thesystem and 3) singular values of the system.

[ MATHEMATICAL METHOD ]is shown
The amino acids production system
in Fig 1 [1][2]. Asparaginic acid is converted to
produce Lysine,
to Metionine and to lsoleucine.
Applying the mass-action law to the reactions in Fig 1,
following 21 linear rate equations are obtained
Xl'= -k12 Xl + k-12 X2 + k12 ( kllO XlO+ k114 X14 + k116 X16 + k117 X17 )X2' = k12 Xl - k-12 X2 - k12 ( kllO XlO + k114 X14+ k116 X16 + k117 X17 ) - k23 X2 + k-23 X3
X3' = k23 X2 -· k-23 X3 - k34 X3 + k-34 X4 - k311 X3+ k-311 Xll + k34 k310 XlO + k311 ( k314 X14+ k316 X16 + k317 X17 )X4' = k34 X3 - k-34 X4 - k4S X4 + k-4S XS- k34 k310 XlO
XS' = k4S X4 - k-4S XS + k-S6 X6 - k56 XS
X6' = k56 XS - k-S6 X6 + k-67 X7 - k67 X6
X7' = k67 X6 - k-67 X7 + k-78 X8 - k78 X7
X8' = k78 X7 - k-78 X8 + k-89 X9 - k89 XS
X9' = k89 X8 - k-89 X9 + k-910 XlO - k910 X9XlO' = k910 X9 - k-910 XlO -klOs XlO
XU' = k311 X3 - k-311 Xll - k311 ( k314 X14+ k317 X17+ k316 X16 )- klllS Xll+ k-lllS XIS + klllS k1116 X16- k1112 Xll + k-1112 X12 + k1112 k1114 X14
X12'= k1112 Xll - k-1112 X12 - k1213 X12+ k-1213 X13 - k1112 k1114 X14
X13'= k1213 X12 - k-1213 X13 - k1314 X13+ k-1314 X14
X14'= k1314 X13 - k-1314 X14 - k14s X14
XIS'= klllS Xll - k-lllS XIS - k1S16 XIS+ k-1S16 X16 - klllS k1116 X16
X16'= k1Sl6 XIS - k-1S16 X16 - k1617 X16+ k-1617 X17 + k1617 k1621 X21
X17'= k1617 X16 - k-1617 X17 - k1718 X17+ k-1718 X18 - k1617 k1621 X21
X18'= k1718 X17 - k-1718 X18 - k1819 X18+ k-1819 X19- 59 -

Fig 1. Schematic illustration of amino acid production initiatedfrom Asparaginic acid.
1'

k110

l ------...J

Asparagi n ic acidkl2--------------'--_-_-....:::-....::.-:1',fi k23

Asparacyl Phosphate

..

k117

k34
Dihydro Picophosphalclk

Tetora45
hydropicolinalc!k56
N-succinly £ kcto
a amino pymcrinalc
k67

I

,..

N-succinyl L a E
diamino pymcrinatc
k78

l

1

L - a - E diamino
pymcrinatc
k89

1

klll2
Cystationinck114

j kl213�1l1�klll5
Homoscrinc
Phosphate

l

l

k1516
Threonine -----'
kl617

Homocyctcin

a-keto1------ Butyric acid
k1718

l

kl314
·---- Methionine

k1621

a - fl di hydroxy
fl methyl valcicacidmesa a t: diamino
pymerinatc
k910

1

I

k*110=k12 kll0
k*114=kl2 kll4
k*ll6=k12 k116
k*ll 7=k12 kll 7

k1920

a -keto f3 methyl..___ Lysine
klOs
Sacctaropinla accto hydroxy
Butyric acid
k1819

c

k2021� k::a-kcto r
methyonylc
Butyric acid....... Isolcucinc

k*310=k34 k310
k*314=k311 k314
k*316=k311 k316
k*317=k311 k317

- 60 -

k*1114=kl112 k114
k*ll16=kll15klll6
k*1621=k1617kl621

fl methyl
Butyrly-CoA

XI9'= kI8I9 XI8 - k-I8I9 XI9 - kI920 XI9+ k-I920 X20
X20'= k1920 XI9 - k-I92 0 X20 - k202I X20+ k-202I X2I - k20s X20
X2I'= k202I X20 - k-202I X2I
where
the variables expressing the concentrations
correspond to the chemical species as shown in Table I
and kn denotes the rate constants.
TABLE 1. List of the variables corresponding
to the chemical species in the system.
XI: Asparaginic acid ,X2: Asparatil phosphate
X3: Asparaginic acid semialdehyde
X4: Dihydro di picolinate , XS: Tetra hydropicolinate
X6: N-succinyl E keto a di amino pimelate
X7: N-succinyl - E - a di amino pimelate
X8: Di amino pimelate
X9: Meso e - a di amino pimelate
Xll: Homoserine
XlO: Lysine
X1 3: Homocycteine
X12 : Cystathininie
XIS: Homoserine phosphate
XI4: Methionine
XI7: a keto butyric acid
XI6: Threonine
XI8: a aceto a hydroxy butyric acid
XI9: a- f3 dihydroxy /3-methyl valerianic acid
X20: a keto /3-methyl valerianic acid
X2I: lsolycine.
Table 2. The rate coefficients ( mM )1. The rate constants for the pathway to produce
LysinekI2=4.7 k23 =0.I6
k-23=2.6 k34=0.13
k45=0.10I
k56=1.0I
k67=5.2 k78=1.3
k89=6.7
k-89=100k910=1.7 kll0= 1.106 k310=1.101 klOs= 12
2. The rate constants for the pathway to produce
Methionine
k1112=4.1 k1213 =1.6 k-121 3=11.1
k311=2.3
k1 3 I4=0.022 k114= 1.04 k3 14=0.467
kI114= 0.98 k14s=0.I8
3 . The rate constants for the pathway to produce
Isoleucine
k1115=0.6
kI516=1.02
kI617=4.1
kI718=3.25
kI819=0.7 3
kI920=0.54
k202I=5.I k316=1.051
k1116= 1.01
k1621= 1.01
k317=1.01
k116= 0.984
k117= 1.1
k20s=l.0
The physiological values of kn ( mM ) are
shown in Table 2 [2][3]. klOs, kl4s and k20s are
the rate constants for catabolyzing Lysine, Methionine
and lsoluecine. The state equations are expressed invector form
X' =AX+ BU,
Y=C XAsparaginic acid Xl, input was set to afford only

The stability of the total reaction of the
to XL
system was investigated by analyzing the eigen
values of the eigen equation of the system.
The
controllability wasjudged by the rank of the matrix. The
number of uncontrollable variables
were calculatedbylength (A) - rank (Co )where
Co = [B AB A2B A3B

A4B

To calculates the optimal feed back gain matrix K
such that the feed back law U = -KX minimizes thecost function,J=f[ X Q X +U RU] dt

0
The Q and R are the weighting coefficients
vectorsThe
systemsupplied bytheoptimized control input is obtained by solving
X' = ( A - B K ) X.
The initial conditions are set to be, Xl =1.0 ( mM )and Xn =0.0 ( 2 < n < 21 ). The impulse responseof each component were compared with and without
the optimized control input. The impulse responses
the metabolizing enzymes forin which
Lysine,Methionine and lsoleucine do not exist (k10s=kI4s=k20s= 0.0 ) were compared inthestates with and without the
optimized
control
input. The singular values of the system wereobtained by calculating the complex matrix,
C(j w 1-A) -l B
as a function of frequency ( rad/sec ). The singularvalues were analyzed for the normal and the congenital
enzymes deficit ( k10s=k14s=k20s=0.0) states withand without the optimized control input.

[ RESULTS]
I .Stability and Controllability of the system.
Under the normal conditions ( table 2 ), the 21
eigen values were composed of 17 negative reals
and 4 sets of complexes with negative real parts.
Thus the system was judged to be stable. The
system was judged to be uncontrollable
with 17
uncontrollable
variables that were insensitive to
10 fold increases in kn.
2. Theimpulse responses with and without optimized
control input.
Fig 2 shows the comparison of
the impulse
responses of X3 ( Fig2-a ) , XlO ( Fig 2-b )between those without the optimized control input( Fig 2- left column ) and those with them ( Fig
2 right column ). With the optimized
control
input, the maximum amplitude of each impulse
response wasdepressed
and the responses- 61 -

Fig3-aFig2-aut3

lnutlOu

0.03

0.02

11.04

] 0.01] 0.02CJ] 0.02

:a

:a_

:g_

�

E

E 0.01

<

<

200
100
1imc (secs)

300

-0.01

20
10
iunc (secs)

030

xl0-6 In_ul 1 Ou ul 10

0
0

..,2

20

10

] 0.01
0

-0.02
0

0.05

-0.01

5000

0
iunc (secs)

30

In ul 1 ·out t 10

..,.,

-0.0S

0

l.S

ci.

�

5000

0
0

Fig 4-a , 4-b, 4-c

Fig 4. The singular value of the system.

I 'Fig 4-d, 4-e, 4- f

..

IOI

50

15

Methionine and lsoleucine
exist ( Fig 4-a , Fig 4b, Fig 4-c ) and do not exist ( Fig 4-d and Fig 4-e,Fig 4-f ). With the optimized· control input, theamplitude of singular values (dB ) were reduced
particularly when the metabolizing enzymes do not 20
exist ( Fig 4-d to Fig 4-e ). Under the optimized
control input, there was little difference ofthesingular values
betweenthose
when themetabolizing enzymes exist ( Fig 4-b ) and do notexist ( Fig 4-e ).

I,

tOI

JO

0.5

Fig3. The impulse responses when the metabolizing
enzymes do not exist ( k10s=k14s=k20s=0.0 )J .·.",

5

xl0-3 In ut1 Ou ut10

Fig2. The impulse responses of the system.

.I "..

so

.il

0

:g_

�

300

J

1imc (secs)

u

E
200

!l

Fig3-b2

100

Inut l Out t3

0.02
CJ

0\

Fig2-bJ

Input1 Output3

25

�I

UH

111-1

-IOI

191

1f..1

'8-1

terminated faster than those without
control input.

10,1

-lOf

IOI

,.

..

the optimized

3. The impulse responses when the metabolizing
enzymes do not exist ( k10s=k14s=k20s=0.0 )but thesystem was supplied by optimized control input.
Without the optimized control input, the impulse
responses oscillated ( Fig 3-left ) and lasted several
1000 secs while those with the optimized control
input, these oscillations
deaeased
and terminatedthe optimized control
earlier than those without
input. These responses were quite resemble to thosewhen the metabolizing enzymes existed and controlled
by the optimized control input ( Fig 3- right column ).
4. The singular value of the system.
Fig 4 shows the singular values of the systemwith ( Fig 4-b, Fig 4-c, Fig 4-e, Fig 4-f) and without( Fig 4-a, Fig 4-d ) the optimized control input
when
the metabolizing enzymes
for Lysine,
]5
20
j(I

[DISCUSSION ]Present analysis has shown that the amino acids
production started from Asparaginic acid is stable but
uncontrollable. The optimized control input
for
Asparaginic acidshown inthis study could
reproduce normal physiological impulse responses
even in the state of the congenital enzymes deficit.
Thus for the treatment of the congenital disease, suchan input as determined by U = - K X must be available.

30[ OONCLUSION ]The amino acids production processes started from 35
Asrapaginic acid isstablebut uncontrollable.
The optimized
control input diminished theoscillative impulse responses and negative valuesof the temporal behaviors of the system particularlyin the state of enzymes deficit state. Present
investigation affords important information for the 40
medical treatment of congenital metabolic enzyme
defects.
[ REFERENCES ][l]Stryler R. Biochemistry. Chapter 24.
1988.[2]Lecture on Biochemical Experiments. vol 11. . Tokyo
Biochemistry Co.Limt.1974. [3]Data book of biochemistry.Chapter 13.Tokyo. Biochemical corporation. 1979.

- 62 -

Optimal Control in Active Transport in Biological Membrane
Number of Sites as the Control Input.
H,Hirayama., Y, Fukuyama.
Department of Public Health Asahikawa Medical College.
4-5. Nishi Kagura, Asahikawa city 078 Japan.
ABSTRACTTo analyze the transient behavior of the particles
passing through biological membrane and its control
principle, we have utilized optimal control theory to
the 3 layers model proposed by Hill. The densities ofparticles in the boundary layers between the out sideof the cell changed rapidly than thosewithinmembrane. Changes in reaction coefficients within themembrane influenced not only the intra membranes
particle density but also those at the boundary surfaces.
The 3 layers model of membrane was insensitive tochanges in the weighting coefficients involved in theperfonnance function of optimal control of membrane
function.
[ INTRODUCTION ]Hill , T, L and Kedem , 0 proposed a theoreticalmodel for reversible thennodynamics regarding withactive transport across membrane. Since this work,
many investigations have been perfonned. However,
the mechanism by which ions or molecules penneate
through biological membrane had been yet not welJ
understood. Particularly, the transient phenomenons of
substrate transport through membrane is still unknown.
The most critical problem has in relation to create
artificial membrane is what is the control principle
or control strategy of transport across the biologicalmembrane. What do they economize or minimize tooperate more efficiently.
In the present study we analyze the simplest
membrane model proposed by Hill, T, L to showtransient change in the state of molecules at each layer ofbiological membrane and optimal control principle.
[ MATHEMATICAL PROCESS ]We set folJowing coefficients. a a : the rate of active
adsorption at the boundary layer facing to the bath A. ab : the rate of active adsorption at the boundary layer
facing to the bath B. f3 a: the rate of active desorption atboundary layer A. f3 b: the rate of active desorption atboundary layer B. kn ; the rate of exchange of molecules
between adjacent layers within the membrane. B : thenumber of sites in each layer. Nj : the occupation densityof the j th layer. The Nl is the boundary layer facing to
the bath A and N3 is another boundary layer facing to
the bath B. The probable situations are shown in Fig 1.
The system equations relating with Nj ared Nl/dt= a a ( B- Nl)- /3 a Nl+kl( N2- Nl}--(1)(2)
d N2 /dt= kl( Nl- N2 )+k2 ( N3- N2 )d N3 /dt= a b ( B- N3 )- /3 b N3 +k2 ( N2- N3 )-(3)Tenns such as k ( N2- Nl) require some explanation.
Let N12 be the number of units with layer 1 and layer 2
sites both occupied . Then the rate at which layer 2 tolayer 1 is k ( N2- N12 ) and the rate at which layer 1 tolayer 2 is k ( Nl- N12 ). Hence the difference is N2 NI. Above equations are rearranged toNl'= aaB+Nl(- aa- f3a - kl)+klN2
N2'= Nlkl+ N2 (-kl- k2 )+N3 k2
N3 =' a b B+N2 k2 + N3 (- a b- /3 b - k2 )Fig 1 : model of membrane transport of molecules. The
layer 1 faces to the bath A while layer N faces to the layer
B. The direction of arrows shows the way in which thereactions operate.
aA

l

i

�h�

•

0

•

0 0

•

•

0

•

•

0

•

0

•

•

•

•

•

0

- 63 -

jk

Layer

2

!
.
I
�
/3

n-1

n

By setting adequate coefficients

N3' = NI ( a7 dll ) +N2 ( a7 d12 +a8 )+N3 ( a7 d13(11)+a9)+pl ( a7 d14)+p3 ( a7 d15).

NI' = al B +a2 NI +a3 N2 ----(4)N2' = a4NI +a5 N2 +a6N3 ---(5)N3 ' = a7 B +a8 N2+a9N3 ----(6)We put B as the optimal control variable.
performance function we propose is

There is no change in N2. The differential equations forthe co-state variables areThete

J = f [ a I ( NI)"2+ a 2 ( N2)"2+ a 3 ( N3)"2] dt
0--1(7)The biological significance of the performance function is
to minimize the time dependent changes of concentration( density ) at each layer of membrane. By expanding
d J / dt = a I [ al"2 B"2+a2"2 NI"2+a3"2 N2"2
+2 ( al a2 B NI +al a3 B N2+a 2a3 NI N2 ) ]+a 2 [ a4"2 NI "2+a5"2 N2"2+a6"2 N3"2
+2 ( a4a5Nl N2+a4a6NI N3+a5 a6N2 N3 ])+a 3 [ a7"2 B"2 +a8"2 N2"2 +a9"2 N3"2
+2 ( a7 a8 B N2+a7 a9B N3+a8 a9N2 N3 ])p2'= - d H / dN2 = - [ 2 d3 N2+d6 B +d8 NI +dlO N3+a 3pl +a5p2+a8 p3 ]=NI ( -d8 - d6 dll +) N2 ( -2 d3 - d6 d12 )) p2 ( -a5 )+N3 ( -dlO - d6 d13)+pl ( -a3 - d6 d14 ++p3 ( -a8 - d6 d15) -----(13 )p3' = - d HJ dN3 = - [ 2 d4 N3 +d7 B +d9 NI + dlO N2+a6p2+a9p3]= NI ( -d9 - d7 dll ) + N2 ( -dlO - d7 d12 ) +N3 ( -2 d4- d7 d13 )+pl ( - d7 d14)+p2 ( -a6)
+p3 ( -a9- d7 dl5 ) -----------(14)Then , the integrand can be rearranged intodJ / dt = B"2 ( a I al "2 + a 3 a7"2 ) +NI"2 ( a I a2"2
+ a 2 a4"2 )+N2 ( a I a3"2 + a 2 a5"2+a 3 a8"2 )+N3 ( a 2 a6"2+ a 3 a9"2 )+2 B NI ( a I al a2 )+2 B N2 ( a I al a3+ a 3 a7 a8 )+2 B N3 ( a 3 a7 a9 ) +2 NI N2 ( a I a2 a3+ a 2 a4a5)+2 NI N3 ( a 2 a4a6)+2 N2N3 ( a2a5a6+ a3 a8a9 )Thus NI, N2, N3, pl,p2 and p3 express the optimal
state equations. For the simplicity we set two boundary
conditions such thatNl(0) =0.01,
N2(0) =0.01,
N3(0) =0.01,

Nl(te ) =0.8
N2(te)=0.9,
N3(te) =1.0 ---- (15 )This mathematical problem was solved numerically bymultiple shooting method. For the standard conditions,
we set, aI=a2=a3=1.0, aa=ab=0.5, ,Ba=,Bb=0.l,
kl =k2 =0.1.

The Hamiltonian function is
H = dl B"2+d2 NI "2+d3N2"2 +d4 N3"2+d5 B NI
+d6 B N2 +d7 B NB3+d8 NI N2 +d9 NI N3+dlO N2 N3+pl ( al B +a2 NI+a3 N2 )+p2 ( a4 NI +a5 N2 +a6N3 )+p3 ( a7 B +a8 N2+a9N3 )--------(8)The optimal control can be obtained by differentiating
Hamiltonian with respect to B.
d H / d B = 0.

pl'= - d H / dNl = - [ 2 d2 NI+ d5 B+d8 N2+d9 N3+a2 pl+a4p2]= NI ( - 2 d2 - d5 dll +) N2 ( -d8 - d5d12 )+N3 ( -d9- d5d13 )+pl ( -a2 - d5 d14)+p2 ( -a4)------(12)+p3 ( -d5d15)[N1,N2,N3]--e--cont-n18--

cont-n3--tt--

cont-n2eThen

2 dl B + d5 NI + d6 N2 +d7 N3 + al pl + a7 p3 =O-----t9)Thus
B = dll NI +d12 N2+d13 N3+d14pl +d15p3.
Putting this optimal B into state equations of NI and N3
NI' = NI ( al dll+a2 +) N2 ( al d12+ a3 ) +N3 ( al(10)d13 )+pl ( al d14 )+ p3 ( al d15)

4

2

time

0 ---=9c:=::::::;:._-,---.--,-...---.--.-�-.--,
0.0
0.2
0.4
0.8
0.8
1.0
1.2

Fig 2 express the transient response of occupation density( Nns). The reaction time was normalized to 1.0 sec.

- 64 -

Fig 3-aFig 3-cFig 3-b

I Nl I----

cont-n1--

k1c1.0-n1-----

k2:1.0-n1

I N2 I[N3J-e--

tc1 .. 1.0-n2--&-

cont-n3-k2=1.0-n2-

k1•1.0-n3--e--cont-n2----

k2:1.0-n3

o.a

1.0

-2

Um•

·•-+-���������-�
o.o

0.2

o.•

o.e

o.a

1.0

1.2

t1111e11111•
o.o

0.2

o.•

o.e

o.o

1.2

0.2

o.4

o.e

o.a

1.0

1.2

Fig 3. The effects of 10 fold increases in the kl and k2 on the temporal changes in Nl ( Fig 3-a ), N2 ( Fig
3-b ) and on N3 ( Fig 3-c ). In each figure, cont-nm indicates the standard state.
[RESULTS ]1. The total behavior of Nl N2 and N3.

Fig 2 shows time dependent changes in [ Nl ] [N2 ]and [N3] at control level. The [Nl] and the [N3] showedparabolic pattern with definite peaks while [ N2 ]increased monotonously.

2. Effects of reaction coefficients kl and k2. Fig 3.
To disclose the effect of kl and k2 , these parameters
were increased ten times from kn =0.1 to 1.0. The [ Nl ]changed into bi-phasic pattern ( sinusoidal pattern)followed by negative values with increase in kl and k2.
A pronounced change was manifested with changes in
k2. [ N2 ] showed a definite peaks with increase in kl
and k2 while at control level [ N2 ] increased
monotonously. A marked increase occurred with increasein k2. [ N3 ] also showed bi-phasic pattern but negativevalues. A larger hump manifested at the initial quarter ofreaction time. Then a second hump developed at near theend. These changes are evident with change in k2.

3. The effects of weighting coefficient a 1 , a2 and a3
Fig 4 shows the effect of 100 times increase in 15
weighting coefficients of a n. With increase in an, thetime to the peak was shortened, the peak magnitudes weredecreased and the curves changed total gradual. Thesechanges, however, were small
4. The effects of reaction coefficients a and f3.

20

Fig 5 express effects of adsorption, desorption
coefficient at both sides of boundary layer. With increasein any kinds of reaction coefficients, the ( Nl ] showedbi-phasic pattern like sinusoidal wave. A prominentchange occurred with increase in ab from 0.5 to 1.0.
Since the a b is the adsorption coefficient of boundary
layer facing to bath B which is opposite to A, the 25
changes in [ Nl ] seem to be interesting. Anotherinfluential one was the f3 b which is also related withb�th B. Thus the [ Nl ] ( boundary layer facing to thebath A ) is more affected by reaction coefficients relating

Fig 4. The effects of 100 fold increases in the weighting coefficients, a 1, a2, a3 on the temporal changes inNl ( Fig 4-a ), N3 ( Fig 4-b ). In each figure, cont-nm indicates the standard state.
[ Nl]
8

Fig4-a[ N3]---a--

cont-n1---al2-100-n1

al1-100-n1

8

---a-

cont-n3---al1-100-n3
al2-100-n3

time

time
1.2

Fig4-b

1.2- 65 -

Fig5-cFig5-b

Fig5-aI NI I
20

I N2J

----

[N3 I

alB0.9-n2----

COfll-n1---+---;

al80.9-n1----

betaA0.5.ft"

�belaB0.5..i1---+-

bMaBD.5-1'13:,----'IA-O.l•n1�■IA·0.9-n3----+-

blltaA0.5-n2----

betaB0.5-n2

40

----

cont-n3----+--

aJB0.9-n3--

�1aA0.5-n3

30

10

20

10

-10

11111•-20+-�����-��--,-�-,

0.0

0.2

0.4

0.8

0.8

1.0

0.0

1.2

0.2

0.4

0.8

0.8

1.0

1.2

0.2

o.-t

o.•

a.a

1.0

1.2

Fig 5. The optimal time courses of concnetrations in Nl ( Fig 5-a ), N2 ( Fig 5-b ) and N3 ( Fig
5-c ) when the coefficients were increased to a A = 0.9, a B = 0.9, f3 A = 0.5 , f3 B = 0.5
independently while all other system parameters were at the standard level.

to bath B. [ N2 ] increased with increase of any kindsof reaction coefficients. Here also coefficient relating tobath B had larger effect than those of bath A. What wasanother finding was that [ N2 ] changed its pattern frommonotonous increase to have a definite peaks. [ N3 ]changed its pattern from parabolic pattern to bi-phasic
pattern with definite two peaks with the first the larger.
In this case also, the reaction coefficient relating to bath
B had larger effects than those relating to bath A.
[ DISCUSSION ]1. The behavior of the occupation density
There was little difference between [ Nl ] and [ N3 ]which are the boundary layers facing to different baths.
Fig 2 showed transient pattern of optimized behavior of
density of molecules. Indeed rapid changes were limited
only to the boundary layers while a monotonous increase
manifested at the middle layer. These changes of density
at em:.h layer must in part be dependent on the setting of
the boundary values at the terminal of the reaction ( t = te). The essential pattern, however, seemed to be describedon this Figure.
The changes in reaction coefficient kns influenced
on density [ NI ] and [ N3 ] in the boundary layers. The
times to the peak of two boundary layers were shortenedwith increase in k2 ( fig 3 ). This indicated that thechanges developed within the membrane ( deeper layer )affected not only the layer within which it was involved
but also, boundary layers facing to the outer world.
Another importance was that the calculated density
resulted in the negative values at near the terminal ofreaction time. We could not offer any comprehensive
explanation of this result. This result may be originatedin the setting of boundary values or in the limit of thenumerical calculation. ( multiple shooting method ).
However, both kl and k2 influenced much on the twoboundary layers, it seemed likely that changes developed
15

tO

within the membrane surely would affecton theproperties of the membrane surface.
Almost similar pattern of densities could be observed
when the rate of adsorption and of desorption have beenchanged. A particular interest was that the density
showed always larger magnitude with increase in rate 20
coefficients relating to the bath B ( Fig 5 filled diamonds
and open diamonds ). These asymmetricity may beoriginated in the mathematical structure of the biologicalmembranes but comprehensive explanation have to wait
the experimental data.
Unexpectedly there was little effect by the weighting 25
coefficient.
This must be 1), the magnitudes ofweighting coefficientwere too small or 2) themathematical constitution itself was insensitive tochanges in weighting coefficient in another word, the 3layers biological membrane are created optimally fromthe initiation of the evolution.
30[ Conclusion ]I.Densities of molecules in the boundary layers changed
rapidly than molecules within membrane.
2. Changes in reaction coefficients within the membrane
influenced not only the intra membranes molecular 35
density but also those at the boundary surfaces.
3. The 3 layers model of membrane was insensitive tochanges in the weighting coefficients involved in theperformance function of optimal control of membrane
function.
[ REFERENCES ]1. HILL, T,L., KEDEM, 0. J. Theoretical Biology.
Vol. 10. pp 399-441.
2. M.S. Seshadri,: J. Theoretical Biology. Vol. 31. pp245-253. 1971.
3. Frehland, E and Langer P. J. Theoretical Biology.
Vol. 47. pp 189-207. 1974.

- 66 -

40

Optimization of Growth of Peptide Chain by Ribosomes on
a Messenger RNA.
H,Hirayama., Y, Fukuyama.
Department of Public Health Asahikawa Medical College.
4-5. Nishi Kagura, Asahikawa city 078 Japan.
ABSTRACTA theoretical approach was challenged to disclose theproperties of temporal changes in poly
peptidechain growth on a messenger RNA when more than
2 ribosomes work on.
The system was stable andcontrollable. There was only one singular value which
was depressed by the system optimization process.
The impulse response terminated to O value earlier
with smaller magnitude in the optimized systemthan that calculated without optimization. Present
model is available to evaluate the peptide synthesis
on a messenger RNA.
key words :. Peptide
chain, Messenger RNA ,Ribosomes, Stability, Controllability, Singular value,
Optimality.

[ INTRODUCTION ]The ultimate nature of biological system isdetermined by DNA and resulting protein synthesis
on a messenger RNA. The biological information is
finally converted to be a kind of protein which acts
as the executor of thecell.
Yet, what is theorganization strategy
of peptide synthesis
on amessenger RNA is still unclesr. Because to know thecontrol strategy of peptide synthesis is available toevaluate the functional properties of DNA, messenger
RNA and peptide synthetic process. More over it will
be possible to diagnose path physiological change andrelation to hereditary disease in protein synthesis on
the messenger RNA level.
In the present investigation, we analyze thesequential processes of peptide change elongation
mechanism by linear system analysis. Moreover wehave analyzed the effects of system optimization.

has progressed in a certain distance along the messenger
RNA from its left tip, p amino acids residues will
be akled to the ribosome. After that a second
ribosome can attach to the messenger RNA.
This
process continues until the full complementofribosome per messenger RNA is achieved. When the
production of each peptide chain is complete, thepolypeptide and ribosome are liberated from the right­
hand end of the messenger RNA while the remaining
bound ribosomes continue to moveon themessenger RNA. Another ribosome will attach to theleft end of the messenger RNA.
Messenger RNA is linked to ribosome with timedependent rate g(t). The free messenger RNA ( M )is catalyzed with the rate of kd while with the rate of
km when the messenger RNA is bound to ribosome.
Synthesis of peptide is started by binding a ribosome( R )to a strand of free messenger RNA.
Aminoacyl-S-RNA
compounds( AS )
isindispensable for the this linkage process. ThereforekO

M + R + AS --- M-R-AS ----(1)Growth of the peptide requires adequate adhesion ofamino acyl-S-RNA on the messenger RNA ( M ) ribosome (R ) complex ( M-R ) at a position adjacent--oMessenger­
RNA

70s0

Ribosome

I

ti�

�N

ProteinAS(Aminoocyl-5-RNA)[ Modeling of poly-ribosome transition]

Fig 1 shows a model for the poly-ribosome
mechanism.
Amino acid residues in the form ofaminoacyl-S-RNA compounds ( AS ) are delivered to
the growing peptide chain. When the first ribosome

Fig 1
Present model of generalized peptide chain
synthesis on a messenger RNA. Amino acid reside isexpressed by A.

- 67 -

to the growing peptide site. Amino acid residue (A) is then irlJed to the peptide by non specific aminoacid transferase.
The amino acid is linked throughthe carboxyly group to the soluble RNA ( S ) and theformation of the peptide bond is achieved by thetransfer of the entire peptide chain to the amino acidgroup of AS.
The growth step is expressed bybinding at the active site followed by a transfer
reactionkl
M-R-AS + AS ------- M-R-AS.AS -----(2)k-1
kM-R-AS.AS --- M-R-A2S+ S. ---(3)In (2), the AS to the right of the dot expresses thebound species. This sequential reactions are repeatedduring growth of the peptide. Thus generally
M-R-Ap-1 S + AS ------ M-R-Ap-1 S. AS ---(4)M-R-Ap-1 S. AS -------- M-R-Ap S + S ----(5)When the first ribosome has translated , p aminoacids in the peptide chain is produced. Then anotherribosome can be irlJed and thereby start the second
growing peptide chain.
The reactions subsequent toequation (5) can be expressed by
M-R-ApS + AS + R ( the second ribosome)--- M ( R-AS) ( R-ApS) ----(6)Iteration of reactions expressed by equation (1) to (6)results in polyribosome
M ( R-AilS) ( R Ai2 S ) - ( R Aiq S)
We express the adsorbed AS on the binding site of aribosome( il, i2, i3, --- iq/ 1 ).
For thesimplicity of analysis, we assume that addition of theamino acid residues occurs in a sequence initiating withthe ribosome which locates farthest to the right, themost left sided near to the lastly irlJed ribosome.
Then(il, i2 ---iq-1, iq) --> ( il i2 iq-1, iq/1) ---> ( i1 i2
iq-1, iq+l }----> ( il i2 iq-1 + 1 , iq + 1)----> ( i1 + 1 , iq-1 + 1 , iq + 1).
The messenger RNA is divided into q regions. As theribosome that has linked with some specific region on
the messenger RNA moves along the region, the pamino acid residues are irlJed to the growing peptide
chain so that p*q peptide residues are irlJed in all.
Generally, following 4 rules operate1). When the most left site on the messenger RNA isempty, the newly arrived AS occupies the most left
side even though other sites are empty.
( 0, 3, 0) ---> ( 1,3,0 ) and not (0,4,0)2). When all other sites are empty except the first
first site is yet fulfilled, the newly
site, and thearrived AS is irlJed to the first site.
( 1,0,0 }--->( 2,0,0)--> ( 3,0,0 }--->( p,0,0,)3). When a give site has been fulfilled by the amountof the product equivocal to p ( amino acids residues)*q (regions ), the fulfilled ribosome moves on to thenext site ( q + 1) and at there the amio acids attains
p*q + 1. After this transition, the q th site becomes
empty.
( 2,4, 0/1 }----> ( 2,0,5 )( 2, 0, 0/1 }----> ( 0,3,0 )( 2, 0, 5 }---->( 0,3,5 )4). When all the sites are not fulfilled, the newly
arrived AS ia irlJed from the right of the total sequence( 1,3,5 ) ------> ( 1,3,6 )( 1,3,6 ) ------>( 1,4,6 )( 1,3,0 ) ---->( 1,4,0 )Thus generally forq (regions) each of which has
the capacity of p ( amino acids residues) ,( 0,0,0--- , 0)--> ( l,0,0-- ,0 ) --> ( 2,0,0,--,0 ) ---> ( p1,0,0--, 0) -------> ( p,0,0 -- , 0).
( p,0,0 -- , 0/1 ) ---> ( 0, p+1,0,0--, 0)---> ( 1, p+1,0,0,--, 0) -----> ( 1, p+2,0,0,--, 0)( 1, p+2,0,0,--, 0/1 }----> ( 1, p+3,0,0,-- , 0 )(1, p+ p -1 ,0,0,--, 0/1 }----> ( 1, 2p,0,0,0)( 1, 2p,0,0,0/1 }----> ( 2, 2p,0,0 0)----> ( 3, 2p, 0, 0-- 0 }-->-->---> ( p, 2p, 0, 0 0)
( p, 2p, 0, 0 0 /1) ---> ( p, 0,2p+1, 0,0,0 )( p, 0,2p+1, 0,0,0 }--->( 0, p+1, 2p+1, 0,0, )( 0, p+l, 2p+l, 0,0, }--> ( 1, p+l, 2p+l, 0,0, )( 1, p+1, 2p+1, 0, 0,0 ,0/1 }-->( l,p+l,2p+2,0,0 )( l,p+l,2p +2,0,0 }---> ( 1, p+1, 2p+3, 0,0 }-->-->-->( 1, p+l, 3p, 0,0,0 }-->( l,2p,3p,0,0,-- 0 )Then the differential equations of this system ared (0,0, • •,0) / dt = g(t) - ( k0 r u + km ) ( 0,0,***, 0)d ( 1,0,*••,0 ) / dt = k0 r u ( 0,0,***, 0) - ( kl u + kd )( 1,0,***,0) + k-1 ( 1,0,***, 0/1)
d ( 1,0,•••,0;1) / dt = kl u ( 1, 0,***,0) - ( k-1 + k+ kd) ( 1,0 , ***, 0/1)
d ( 2,0,*••,0 ) / dt = k ( 1,0, *** 0/1) - ( kl u + kd)( 2,0,*••,0 ) + k-1 ( 2,0, ***, 0/1)

••

• •

•

•

•

•

•

•

d ( p,0,* • •, 0) / dt = k ( p-1,0,*••, 0/1) - ( kl u + kd)( p,o:••,o)d ( 0,p+l,0,***, 0) / dt = k ( p, 0,***, 0/1) - ( k0 r u+ kd) ( 0, p+1, • • • ,0)- 68 -d ( 1, p+1, 0,** *,0) /dt = k0 r u ( 0, p+1,0,***,0)- ( klu + kd) ( 1, p+l, 0,***,0) + k-1 (1, p+l, 0,***,0)d ( 1,p+1,0,***,0/1 )/dt = kl u ( 1, p+1, 0,**,0) - ( k-1+ k + kd) ( 1, p+l,0,***,0/1)d( 0,p+l, 2p+l, 0, ***,0) /dt = k ( p,0, 2p+l,0,***, 0/1 }- ( k0 r u + kd) ( 0,p+l, 2p+l, 0, ***,0)d(l,p+l,2p+l,0,***,0)/dt=k0ru ( 0, p+l, 2p+l,0,***,0/1) - ( kl u + kd) ( l,p+l, 2p+l, 0, ***,0 )+ k-1 ( 1, p+l, 2p+l,0,**,0/1)
d ( p, 2 p,***, (q-l)p,0 ) / dt = k ( p-l,2p, ****, (q­l)p,0/1) + kf ( p,2p,***,qp)- ( kl u + kd) ( p, 2 p,* **, (q-1)p,0/1)
d ( p, 2 p,***, (q-l)p,0/1) / dt = kl u ( p,2 p,***, (q­l)p,0) - ( k-1 + k + kd)( p, 2p ,* ** ,( q-1)p, 0/1 )d( p,2p,***,qp) I dt = k ( p-l,2p, **** qp/1) - ( kd +kf) ( p,2p,***,qp)
dAn/dt = kf ( p, 2 p, ***, qp) - kp An
d P n I dt = kp An
1n the present investigation we analyze the casewhen there are q ( = 3 ) ribosomes each of which
occupies a space on the messenger RNA corresponding
to the designation of p ( =2) amino acids. Thus fullspace state is expressed by vector form of ( 2,4, 6)
and null state is by ( 0,0,0) respectively.
The state equations are expressed in vector form
Y = CX
X' =AX + BU,
where X is the state vector ofconcentration ofeach component of the sequences reactions. U is thecontrol input. Because, the sequential reactions arewasinitiated from the state of (0,0,0), input g(t)set to operate only to this state. Y is the out put
vector. A is time invariant coefficient matrix ofthe system.
B is the matrix
for the control
variable.
The stability of the total
reactionof thesystem was investigated by analyzing the eigen
values of the eigen equation of the system.
The
controllability was judged by the rank of the matrix. The
number of uncontrollable variables
were calculatedbywherelength ( A) - rank (Co )A22 B]
Co =[B AB A2B A3B A4B
To calculates the optimal feed back gain matrix K
such that the feed back law U = -KX minimizes thecost function,
' t
J=

l�

X QX +U R u l di

We have utilized the optimal control theory (4 ].
The Q and R are theweightingcoefficients
vectors composedof
23* 23elements tocharacterize therelativemagnitudeof theminimization of each component X and the input
U
in the cost function.
For the simplicity ofanalysis, all the elements were set to be unity at thestandard level.
To show the effects of relative
magnitude of minimization for Xn, the elements of theweighting vector, all the elements of the matrix Q
were increased to 10. The
system supplied by
the optimized control input is obtained by solving
X'= (A - BK)X
The initial conditions
are set to be, Xl =1.0 (number for a given unit volume of a cell ) and
Xn =0.0 ( 2 < n < 23 ). The singular values of thesystem were obtained by calculating the complex
matrix,C( j w l - A ) -l B
as a function of frequency ( rad/sec).

[SETTING PARAMETER S]
The numerical values relating to the sequential
transient movements of ribosome are still unclear
because of little kinetic study. For the simplicity
of analysis, we set following values of systemparameters as the standard physiological state.
u=l.0 r=l.0 k0=l kl=0.5 kml=0.01
kf=0.5 kp=0.1
kdn=0.1 ( n=l to 22)
k3 4 =0.5
k56=0.5k89=0.5k1415=0.5
kl011=0.5 k1213 =0.5
k2122=0.5
kl718=0.5 k1920=0.5
where u and r denote corresponding concentrations of
The kdn are those describing the
AS and R .
catabolic process of each state and knm are thetransitional rate constants.
[R ESULTS]
1. Stability and Controllability
Eigen valuesof the sequential process are all
negative without 0 value. Thus the system wasstable. Surprisingly the rank of the matrix revealed
that the system is controllable because of 0
uncontrollable variable.
2.

The Singular values of the system.
Fig 2 shows the singular value of the system asfunction of frequency rad/sec.
There was only one
singular value. It showed a positive value below
frequency 0. 3 ( Fig 2-a ).
Fig 2-b expresses the
singular value when the system was optimized byminimizing the cost function and the feed back input
was optimized. The singular value was depressedconsiderably. Fig 2-c shows the singular value when
all the elements of the matrix Q were increased to
10. Further depression of the singular value wasobserved and it became independent from frequency.

- 69 -

Fig 2. The singular value of the system
Fig 2-aFig 3. The impulse response of the system.

10

!ll

�
:a

'a o.oi

5

�

0
-5

1 -10

Fig 3-b

-15

-20

10-3

Fig 2-b

500
Time (secs)

1000

�

!

l

0

Fig 3-c

g -10

�:a

0

500

1000

�

0

�

10-2

10-1

10•

100

Frequency (rad/sec)

Fig 2-cFig 3-d0500Time (secs)

1000

\

!;l

g -10
:a 15

1;;;

-20
10-•

10-2

Fig 3-e

101

100

Frequoncy (-)

3. The impulse response of the system.
Fig3 shows the example of impulse responses ofeach state of the system. Fig 3-a expresses thetransient change on ( 2,0,5 ) in response of impulse
input. Fig 3-b, Fig 3-c, Fig 3-d and Fig 3-e show
those of ( 0,3,5 ) ( 1,3,6 ), ( 1,4,6) and the final
product respectively. The left column of the Fig 3
expresses those when the system was not optimized
while the right column of the Fig 3 shows thosewhen the system is optimized.
The impulse
responses with optimized system terminated earlier
and their magnitude were lower than those of thenon optimized impulse responses.
[ DISCUSSION ]Present investigation showed that the sequential
processes in growth of peptide change on amessenger RNA is stable and controllable. The
impulse response showed that the processes of
peptide synthesis works effectively without retention
or depletion of intermediate products. Almost all theprocesses terminated within the same duration indicatesthat the synthesis process is driven to operate parallelat every step of the total processes. The biologicalsignificance to mmumze
the cost function isbased on the concept that. any biological system must
operate so as to minimize the deviation from the target
10

15

0

50

100

]

�

-5

�

Time (secs)

-ll

0

100

]

0.

-20

50

Tune (secs)

i

1 -15

100

i

Time (secs)-550
Tune (secs)

-ll

I

10•

100

!;l

-25
)0-3

0
0

-ll
10-1

10-2

Fr<quoncy (,ad/s,c)

-25
)0-3

lL=:J
!C:' l□l□ 1CE:J'.[�"''°��w]'fKJr.""'f���l[!SJ

Fig 3-a j"OOCJ

15

0

500

1000

� 0.5

0
0

100

-ll

I

10 0050o50Tune (secs)

Time (secs)

500
Time (secs)

1000

�

0

50

100

Time (secs)

state simultaneously with minimize the consumptionof the energy to drive the system. These features aredisclosed explicitly in the Fig 4 in which theweighting matrix for the changes in each state of
peptide synthesis were reinforced by
10 fold
minimization ( setting all the elements of the matrix Q
to be 10 ). Thus by regulating elements in thematrix Q, simulation for optimal state in peptide
synthesis on a messenger RNA must be possible.
[ CONCLUSION ]1. Protein synthesis on the messenger RNA isstable and controllable.
2. The singular value of the system was sensitive
to the optimization of the system in which the feedback input minimizes the deviation from the target
state and consumption of energy to drive the system.
3. The impulse responses of the optimized systemterminated earlier with smaller magnitude thanthose of non optimized system.
[REFERENCES ]1. Gerst I and Levine S,N. 1965. J. Theor. Biol.
vol 9. pp 16-36. 2. Jacob, F and Mond J. 1961. J.
mol. Biol. vol 3. pp 318. 3. Levinthal C., Fan, D.P.
Higa., Zimmerman RA. 1963. Cold. Spr. Harb. Symp.
quant Biol. XXVII. p 183.

70 -

Adaptive Evolution of Holon Networks by an Autonomous Decentralized Method
N. Honma*
K. Abe**Dept. of Electrical Eng.
Tohoku Univ.
Sendai, 980-77, .JAPANl\f. Sato
Communication Eng.
Tohoku Inst. of Tech.
Sendai, 982, .JAPANKey words: Evolution, Autonomous decentra.lized
system, Holon, Self-organize, Edge of chaos, Recurrent
network.

AbstractvVe present a holon network that is a new modelof complicated systems and their complex phenom­
ena, such as complex ad(iptive systems or artificial life
and their evolution. Rolon networks have a hierarchi­
cal construction where each member of this hierarchy,
called holon, is a network composed of lower members.
The networks are able to self-organize their struc­ture by changing their parameters according to the en­
vironment. This is achieved by an evolution methodwith growth of the networks where all members, on
whatever level, have their respective autonomous cri­
teria ba5ed on properties of holons. In this method theinitial holon network comprises only a few holons andthe number of holons increases gradually and adap­
tively as occa5ion demands.
Some examples show that this method can lead toa network structure which has sufficient flexibility andadapts well to environments.

1

Introduction

Necessity to deal with large-scale and nonlinear
complicated systems increases in many engineering
fields, for instance in control and system engineering
fields[6]. It is important to study what model is effec­
tual for the problem of grasping the behavior of thelarge-scale nonlinear complex systems.
One of difficulties of building complex system mod­els is caused by the variety of the system dynamics
with extraordinary complexity[l2, 3, 1]. The variety
appears to be given rise to the various relations be­
tween the elements and the whole system. How do we
describe such various relations in a "unified form" ?
The ability to explain these complex relations by re­ductionistic approaches may be limited. On the other

H. Takeda
Dept. of Electrical Eng.
Tohoku Gakuin Univ.
Tagajo, 985, .JAPAN

hand, the notion of holon yields a new approach, be­yond the reductionism and holism, which incorporates
valid aspects of both[8]. An another concept of repre­
senting the property of complex systems using a holon
character has been proposed as holonic control andholonic loop[l3). However, this holonic theory is not
yet scientifically established.
In a previous paper we have proposed a model ofthe complex systems, called holon network, aiming
at constructing a general system architecture basedon the input-output relation with complicated in­teraction. Rolon networks are able to self-organize
their structures and adapt their parameters to theenvironment[4).
In this paper we present a new evolution method
to achieve the evolution by adapting the number ofholons comprising the network to the complexity ofenvironments. It is shown that this method leads to
network structure which adapts well to changes of en­
vironment and acquires the environmental dynamics
for several estimation problems.

2

Holon network

Rolon networks provide a common architecture ofcomplex systems and a nature of components in the ar­chitecture. Fundamental properties described below,
therefore, are ubiquitous in biology, psychology, ecol­ogy and wherever we find complex hierarchic systems,
especially in organic systems[8).

2.1Fundamental properties of holon net­
work

First, the architecure of a holon network is open­
ended multilevel hierarchical construction where eachmember of this hierarchy, on whatever level, is a sub­whole, called holon, composing a network within thelevel(see Fig. 1).
Second, holons are Janus-faced. The face turned
upward, toward the higher levels, is that of a depen-- 71 -

binary variables, namely each variable takes the value

either O or 1. B; are Boolean functions, w;j connection
weights and w;,, weights of external inputs.
\Ve consider the higher level composed of only one
holon which has an static input-output relation given
as a nonlinear function of feedforward neural networks
FNN. The output of the holon fl is defined by

Fig. 1: open-ended multilevel hierarchical construc­tion.

dent part; the face turned downward, towards its own
constituents, is that of a whole of remarkable self­
sufficiency. This implies that every holon is possessedof two opposite tendencies or potentials: an integra­
tive tendency to function as part of the larger whole,and a self-assertive tendency to preserve its individualautonomy[8].
Finally, functions of holons are evolved by dynami­
cal balancing the dual aspects of its holons, as wholes
and as parts.

2.2

This means that the outputs of the lower level layer
binary holons y; are inputted to the higher level holonas arguments of the function FNN(see Fig. 2). In thisfigure the function of each neuron fN N is defined by!NN(x) =x;(t + 1) = B;(t) (w;1 (t)y1 (t), · · ·W;N(t)yN(t), w;,, (t)u;(t)),y;(t) = x;(t),

(1 )
(2)

i = 1, • • •, N. N is the number of binary holons, andexternal inputs u;, outputs y; and states x; are all(4)

where u 0 is a constant.
low level layer
I \

I

I

high level layer

'

\

I
I

\

\

u

Holon networks for an estimation
problemIn this paper we will consider an estimation prob­
lem whose task is estimating time sequences from an
unknown binary dynamical system described in sec­tion 5. For this, two leveled holon networks are used;
the lower level layer approximates arbitrary time se­quences from unknown dynamics, and the mapping
function of the higher level layer integrates the lower
level dynamics.
\Ve employ binary holons for members of the lower
level. Binary holon networks constructed by one level
layer are similar to random Boolean networks[7] or cel­lular au.tomata[9] in respect of both of structure andfunction. The states of binary holons x; are governedby1
l + exp(2x/uo)'II(dynamics)

- --

I

I

(mapping)

Fig. 2: Architecture of a neural holon network.

3
3.1

Dynamics of binary holon network
Parameters of the network dynamics

To parameterize dynamics of binary holon net­
works, we have presented two parameters, k; andFr[5], where k; corresponds to parameter I( in random- 72 -

Boolean networks[7] and F,. corresponds to parameter>. in cellular automata[9].
The individual connection intensities k; and the av­
erage f.; are defined by
k; =

L
N

i = l, 2, · · ·, N,

Wij,

j=l

1
K= N

Fr=10 •

:r:

.,

(5)

Lk;.
N

N

..
.
. ..

0.2

In this paper, we choose one of four Boolean func­
tions for B;, namely, B; E {AND, OR, XOR, XOR}.
Then, the parameter F,. , implying heterogeneousness
of Boolean functions in the network, is defined by

L F;(B;),

0.6
0.4

i=l

F,. =

0.8

♦

10

15

20

Average connection intens�yFig. 3: Average single holon entropy fl over K space
for approximately 2000 binary holon network runs.

(7)

Quantitative overview of binary holon
network dynamics

Here, for the networks with N = 20, stepping
through the ranges 0 ::; K ::; N and 0 ::; F,. ::; N indiscrete steps, we prepare binary holon networks foreach (Il,Fr) point by randomly choosing Wij and B;.
Then we run these networks with (K, F,.), collecting
data on the following two measures of their dynamical
behavior. Finally, we investigate the relation between
the behavior of these measures and each of two pa­
rameters.
vVe measure the complexity of the binary holon
network dynamics using Shannon's entropy H andpse11.do Lyapunov exponent >.' introduced in our pre­vious paper[5].
Regarding the state transitions of each binary holon
as a discrete process of two states, the entropy H; andthe average fl are given by

L Pr(:i:; = s) log Pr(x; = s),... . ...

•

5

0

•

where F;(B;) = 1 when B; is either XOR or XOR,
and F;(B;) = 0 when B; is either AND or OR.

H; = -•♦

O lilllll!lllt�o!:-�-_:•�·�· ��:....___L____:____,___..!_• _· :...· ·�-

i=l

3.2

♦

♦

0.8

..

:r:

0.6

<(

0.4

.,

• ••
•

•

0

..

•

•
•.
i

♦

.

♦

♦

•
♦

0.2

.
0

5

10
Fr

15

20

Fig. 4: Average entropy fl over F,. space.

4

2
C:

0

>

-2

..

0

C:

(8)

.

•••
I I t •

.....

-4

0

s=O,l

-6

(9)
fl as a function of K and F,. are shown in Fig. 3 andFig. 4, respectively. Each point represents a distinct
network structure indexed by the parameters: K andF,. .

- 73 --8-10 ���· ·�·�---�----�--�
20
15
10
0
5
Fr

Fig. 5: Pseudo Lyapunov expnent >.' over F,. space.

In Fig. 3, we can see a remarkable feature in thecase of around k = 2. The resemble result to this fea­
ture has been reported in random Boolean networks[7].
It suggests that binary holon networks with aroundk = 2 exhibit spontaneous collective order and that
networks with large k are stochastically disordereddifferently from small k networks.
On the other hand, in Fig. 4, note that as F,,
increases, fl also increases stochastically. And thesparsely populated gap over O :::; F,. < 10 and between
0.0 < fl ::; 0.4 is observed. This distribution suggests
the presence of a phase transition[9].
Pse·1ulo Lyapunov exponent >..' measures an anothercomplexity,which is one with respect to chaotic lwhav­
ior, given by computing Hamming's distances between
state transitions from an initial state and an anotherinitial state added a minimum perturbation[5]. Fig. 5
shows pseudo Lyapunov exponent >..' as a function of
Fr . Note that Fig. 5 bears a resemblance to Fig. 4.
Also we observe a similar resemblance with respect
to parameter k. The relation between two measures
shown in Fig. 6, however, we can conjecture an inter­esting fact that binary holon networks around >..' = 0,
called the edge of chaos, where the phase transition be­
tween ordered and chaotic dynamics is observed, areable to give rise to various dynamical behavior fromhigh entropy( chaotic or disordered behavior) to low
entropy(simple or ordered behavior).

. • � .�. e\... ... .
...
.. ......

0.8_,.o i• •;.
..., 0

:r:

.,

0.6

<C

0.4

Cl

0.2
0
-10

..

-5

<It•-

0

method, as one of main strategies for adaptation to en­
vironments, we will make use of the feature of binary
holon network dynamics controlled by the parameters.

4

Autonomous decentralized evolution\Ye propose an autonomous decentralized evolu­tion method in order to estimate time sequences of
unknown environments in desired accuracy. In thismethod neural networks giving the input-output rela­tion of the higher holon are learned by the backprop­
agation algorithm, and the initial low level holon net­
works comprise only a few holons and they are growngradually as occasion demands.
This evolution of low level networks is achieved bychanging the two parameters of network structure ac­cording to two criteria based on the holon's dual char­acter; fitness criterion given by estimation error, andau.tonomou.s criterion in order to keep the network dy­
namics the edge of chaos.
1fore details with respect to exerting of the dual
characters of a holon, namely its integrative tendency
and its self-assertive tendency, are discussed below.
Throughout all level layer holons, the integrative ten­dency of a holon is affected by the self-assertive ten­dency of its higher level holon, while the two tenden­cies of a holon are independent each other[8].
In each generation T = 1,2,•••,the change,denotedby 61.:;, of connection intensity k;, is calculated bywhere a; is the backpropagation error from fitness de­fined by (15) on the i-th low level holon. This equationis also based on the result of section 3: As .k or Fr in­creases, the complexity of the network dynamics also
increases stochastically.
The integrative tendency of the higher holon works
to improve fitness to the environment. This work has
influence on 6K in (10) as follows.

• ·,.

5

Pseudo Lyapunov exponent

10

(11)

Fig. 6: Average entropy fl versus pseudo Lyapunov
expnent >..'.
Therefore, if we keep the dynamics of binary holon
networks to the edge of chaos by changing parameters!.:; and Fr suitably, then we can expect high efficient
adaptation ability to wide range processes with com­
plexity from high to low. In the following evolution

where C is a proportional constant.
The self-assertive tendency of this higher holon has
influence on the integrative tendency of its lower level
layer holons as f>., ( >..') = ->..' in ( 10). This term a5
autonomous criterion acts in order to keep the network
dynamics around the edge of chaos.
On the other hand, the self-assertive tendency of
low level holons works to maintain the present confiq­
uration; values of k;. The term of these works in tJ.u)- 74 -

fk (k;) is given as
1 + k; - kc;
fk (ki ) = { 1 + k·c;(>.' < 0),(>.' � 0),

where kc; is defined by

L Wi F (B )­
N

kc; =

i=I

j

j

(12)

j

Finally, we present an evolution method:

step 1: Set a low level layer holon network at an initial

generation, by taking N = 1, initializing B1 to a ran­
domly chosen Boolean function out of the four Booleanfunctions, and initializing w11 and :i;� to a randomly
chosen binary number.
step 2: In each generation, calculate fitness to the en­
vironment, and then give the backpropagation errora; to the i-th low level layer holon.
step 3: If the fitness is successively greater than thedesired value for generations fixed beforehand, thenexit, otherwise go to step 4.
step 4: For all i = 1, 2, • • •, N, execute step 5 and step
6.
step 5: Compute 6k:i(r) by (10).
step 6: If 6k; is greater than O and if kci+ok; is greaterthan Fr , or if ok; is less than O and if k; - kc; - ok:;
is greater than N - Fr , that is the change cannot heachieved present network configuration, then generate
one low level holon with the following Boolean function
decided by sign of ok;:

XOR or XOR (6k; > 0),

BN+I = {
OR

or

AND

(ok i � 0).

Then, change N to N+l,else change Wi j as follows: If 6k; is greater than 0,then let kc; increase, else let k; - kc; increase.
step 7: Go back to step 2 at next generation.

5

Simulation results

We have tested the evolution method for holon net­
works on an output estimation problem of the follow­
ing discrete time nonlinear systems with single input
and single output.

x(t + 1) = / (x(t), u(t)) ,y(t) = g (x(t), u(t)),

(13)
(14)

where u(t), y(t) and :z:(t) denote the external input,
the output and the state vector at time t( = 1, 2, ...),

respectively. For simplicity, we shall consider binary
systems where each variable takes the value either 0
or 1: The input u, the output y and the componentsof the state vector :z: are binary variables. The compo­
nents of transition function f = [Ii, h, · · ·, fnt f andoutput function g are all unknown Boolean functions.
The task of solving the problem requires the net­
work to learn to configure itself so that it may store im­portant information obtained from the input sequenceat earlier times to help determine the output at latertimes. In other words, the network is required to ac­
quire approximately the dynamics of unknown syste. J
to accomplish the task.
For the above task, the evolution was made bymeans of a real-time estimation method that is not
an iterative learning method feeding repeatedly thesame pattern at every generation. We set the input
signal u(t) at each time t = 1, 2, • · • a randomly cho­sen binary value and the length D = 100 ( see [5]). The fitness to the environment at rth generation,fit(r) (0 � fit� 1), is defined by the estimation er­
ror, e(t) = y(t) - y(t), as follows:fit(r) = 1 - �L le(t)IT(15)

t=r-D+l

Here we suppose that the unit interval of the systemtime t is equal to the generation's unit interval.
We investigated adapting ability and the effect of
the autonomous criterion on the simulation experi­
ment where the environment was changed every 100
generations: the transition functions and output func­
tions of the environmental system are changed. The
learning by the backpropagation algorithm was done
every generation except for the generation when thefitness at the previous generation was greater than 0.9.
The fitness as a function of evolutional generations
by using the proposed evolution method is shown by
the solid line in Fig. 7, and the broken line shows thefitness by using a method excepted for the term of
the autonomous criterion effect f>..,(>.') in (10). This
exception of the effect was made by ignoring the valueof >.' and letting f>.., be 1 every generation.
Despite the changes of environment, the fitness ismaintained effectively in desired values, namely notless than 0.9, for several successive generations. This
fact suggests that the holon network acquires approx­
imately not only static mapping from an input to an
output sequence of the systems at a certain generation
but also dynamics of the systems. Also the efficient
adaptation to changes of environment suggests thatthe proposed self-proliferative evolution method can
reduce the expensive computation time in comparison- 75 -

different from the change required by the fitness crite­
rion. However, it seems that this self-inconsistency
makes the evolution of holon networks be rich andflexible since the autonomous requirements are com­
promised with the total requirements by balancing thetwo faces of each holon. In fact, if the changes by thetwo criteria were always consistent, the autonomous
criterion is not needed for the evolution. This notionof systems will be useful for the autonomous decentral­ized system theory and the emergent system theory[6].

With autonomous criterion Without autonomous criterion ----

0.8
Ql
C

0.6
0.4
0.2

References

0'------'---�--��--�--�
0
200
400
600
800
1000
Generations

Fig. 7: Fitness as a function of generations for adapta­
tion problem to various sequential circuits dynamics.
Solid line shows evolution with autonomous criterion,
and broken line shows evolution without the criterion.

with our former evolution method in which the mnn­
ber of low level layer holons was fixed[5].
As is clear from the results, to have the appropri­ate autonomous criterion independent of the fitness
criterion is efficient for adaptation to changes of envi­
ronment.

6

Conclusion

It is shown that holon networks are useful for mod­eling of complicated nonlinear dynamical systems. By
using the presented evolution method, holon networks
are able to have a great deal of flexibility and adapt­ability to changes of environment, and it is possible toreduce the expensive computation time since the mun­ber of low level layer holons is increasing gradually andadaptively as occasion demands.
The main concept of the method is to keep bal­ance of two independent criteria; the fitness criterion,
namely total criterion, and the autonomous criterion.
This concept can be incorporated into other meth­ods, such as gradient decent methods[ll, 10, 14] forrecurrent neural networks and genetic algorithms[2]for optimization problems. Indeed, we have proposeda method in which an genetic algorithm is combinedwith an evolution method where the number of lowlevel layer holons is fixed, and illustrated the effects ofthe evolution method(5].
In several situations, the change of network config­
urations required by the autonomous criterion may be[1] P.Bak and K.Chen, "Self-Organized Criticality",
Scientific American, January, pp. 26-33, 1991.
[2] D.E.Goldberg, Genetic Algorithm in Search,
Optimization and Machine Learning, Addison­
Wesley, 1989.
(3] .I.Horgan, "From Complexity to Perplexity", Sci­
entific American, June, pp. 74-79, 1995.
[4] N.Honma, M.Sato, K.Abe and H.Takeda, "Identi­
fication of Nonlinear Dynamical System with Au­
tonomous Decentralized Adapting Methods using
Holon Network Model", Prep. of The 36th .Japan.Joint Automatic Control Conference, pp. 141-142,
1993.
(5] N.Honma, M.Sato, K.Abe and H.Takeda, "On
Evolution of Rolon Network by using Au­
tonomous Decentralized Adaptation Method",
Trans. of SICE, vol. 32, No. 7, pp. 908-915, 1995.
(6] S.Kitamura, At the starting point of the research
project on "System Theory of Function Emer­
gence", Proc. of the 34th SICE Annual Confer­ence, pp. 875-876, 1995.
[7] S.A.Kauffman, "Antichaos and Adaptation", Scientific American, August, pp. 64-70, 1991.
[8 A.Koestler, .Janus, Kousakusya, 1983.
[9 C.G.Langton, "Computation at the Edge of
Chaos: Phase Transitions and Emergent Compu­
tation", Physica D vol. 42, pp. 12-37, 1990.
[10] B.A.Pearlmutter, "Learning State Space Trajec­
tories in Recurrent Neural Networks", NeuralComputation, vol. 1, No. 2, pp. 263-269, 1989.
[11] F. .J.Pineda, "Generalization of Back-Propagation
to Recurrent Neural Networks", Physical Review
Letters, vol. 59, No. 19, pp. 2229-2232, 1987.
[12] R.Ruthen, "Adapting to Complexity", Scientific
American, January, pp. 110-117, 1993.
[13] H.Simizu, Life and Place, NTT PublishingCo.,Ltd., 1992.
[14] R..J.Williams and D.Zipser, "A learning Algo­rithm for Continually Running Fully Recurrent
Neural Networks", Neural Computation, vol. 1,
No. 2, pp. 270-280, 1989.

- 76 -jDevelopment of A Stair Climbing Mobile Robot
with Legs and Wheels
Toshihiko Mabuchi,
Tomoharu Yamada

Takeshi Nagasawa,Keizou Awa, Kazuhiro Shiraki and

Department of Mechanical Engineering
Kansai University
3-3-35 Yamate-cho, Suita, Osaka 564 Japan
abstract
This paper deals with the development ofa stair climbing mobile robot with legs andwheels. The main technical issues indeveloping this type of robot are the stability
and the speed of the robot while climbing
stairs. The robot has two wheels in the frontof the body to support its weight when it
moves on plain terrain, and it also has arms
between the wheels to hook onto a tread ofstairs. These are two pairs of legs in the rearof body. Using not only the rotational torqueof the arms and the wheels but also the forceof the legs, the robot goes up and down stairs.
It measures the size of stairs when going up
and down the first step, therefore themeasurement process does not cause thisrobot to lose any time. The computer which
controls the motion of the robot needs nocomplicated calculations as other legged
robots do. The mechanism of this robot andthe control algorithm are described in thispaper. This robot will be developed as awheelchair with a stair climbing mechanism
for disabled and elderly people in the near
future.

1 Introduction
In Japan there are more than one million people whohave disability with lower limb in 1993, and thenumber of the people is going up in this rapidly aging
society. Due to the shortage of helpers in the near
future, society will have a certain need of robots as a
substitute for human beings that offer to support
disabled and elderly people. To develop this kind ofrobots there are many problems which must be solved.
The most important and difficult thing in designingthese robots is how to make the disabled and elderly
people who use robots in their daily life comfortable.
On the other hand, many technical problems must also
be solved.
It is not easy to move with wheel chair even at
public place where all citizen lead a life because ofsteps. Then elevators or slopes should be equipped atthese places, but under the actual situation in Japan
there are few places. The ratio of stations equipped
elevator is a very low: There are only 2.5 percent at JR,
5.4 percent at private railroads in 1994.
In the social background we develop a new stair
climbing mobile robot. The real technical issues indeveloping a stair climbing robot are how simple therobot can be made and how fast it can move withstability when climbing stairs. There are many kinds ofstair climbing robots: for the maintenance of nuclear
reactor or other plants,[1][2] for disabled and elderly
people as wheelchairs,[3]. There are three basic types ofmobile mechanism: a wheel type, a crawler type and aleg type. In the wheel type an energy efficiency ishigher to move the plain terrain than the others and thistype has been the most popular mechanism.[4][5] In
crawler types a terrain adaptivity is higher than thewheel type and this type is most useful to move on sandor mud place, but not suitable for plain terrain.[6]Legged type has the highest adaptivity to rough terrain- 77 -

and can move on stair or mountain with stability,
hecause contact points of ground where feet support the
body would be safely selected. But this mechanism has
problems : load weight, energy efficiency and speed of
movement so far.[7]18)[9] Then we will adopt the newmechanism with both legs and wheels.

Fig.3 Foot

2 Construction of The Robot

2.2 Mechanism Arm
Figure 1 shows the structure of this robot and therobot is 1 /2 scale model. Two special mechanisms to go
up and down stairs are equipped with this robot.
Clinometer

Wheel

Fig. I Structure of SCARAB

Each of the two arms with a free roller at the top of
itself is equipped between two wheels in front of body.
And each of the arms has a touch sensor to detect thetread of the step. When the robot goes up stairs, these
two arms rotate clockwise and hook onto a corner of thesteps, then these two arms can help the robot to climb
stairs as in figure 3. These arms also support the bodyof the robot in such a way as not to slip down, while
the robot climbs down stairs. The position of therotational axis and the length of the arms must be
determined carefully, because the desired torque of the
motor and the height which arms can lift the body are
determined by them, though it is difficult to find themost suitable point and the angle for all stairs.

2.1 Mechanism of Leg
This robot has two pairs of two legs in the rear of
body. Each pair of rear legs has two degrees of freedom
in 135mm as vertical and 225mm as horizontal direction
and composed of three parts: leg(h), leg(v) and foot asshown in figure 2. The position of each leg is detectedfrom rotary encoders of DC motor. The push and pullforces of the legs helps the robot to go up or down
stairs and prevents it from slipping down. Figure 3
shows each pair of legs has four touch sensors and anangle sensor. Touch sensor detects the leg contactingwith the proper place of the tread, and the angle sensor
measures distances between the leg and the riser of thestep.

Leg(h2)

Fig.2 Legs

Fig.4 Arm
2.3 Movement of Stair climbing
The robot with this mechanism can climb according
to the size of stairs : from 130 ~ 200mm in the horizontal
direction and from 0~ 100mm in the vertical direction
These ranges under which the robot can be controlled are
decided with the measurements of stairs in about 300
places, and information from the Building Standard Act
and six major railway companies in Japan. Stairclimbing maneuver may divided into three categories
according to robot's control algorithm: low part, middle
part and high part. While the robot steps up the stair, astep is first detected with arms and the size of the steps
at the low part. Then the robot goes up at middle part ofthe stair. When the arms can not detect the tread of thestair, the wheels are at the top of the steps. The robot
goes down backwards and legs detect a steps instead ofarms while goes down stairs.

- 78 -

Figure 5 shows how the legs can be moved invertical and horizontal direction to step up or down at
the middle part of the stair. Most of legged robots steps
up or down stairs one by one. Crawler type robots moveat a low speed limits because of the friction coefficient
between stair and crawler. But the legs with thismechanism step up and down every two steps. Going up
and down stairs, one pair of legs stays on a tread ofstairs to keep hold on the body. At the same time, theother pair of long legs steps up or down to the next
stair. Therefore this mechanism can go up and down
faster with stability than the others.

4. Measurement of Stairs
When the robot climb up steps, the height of a stairis measured by the arms as shown in figure 6.

h = t- a cos a

Fig.6 Measurement of stair height
A wide of the stair is calculated with following
equations.

a2 = r 2 + ( f cos p + s 2°51

w={a -(h+r} } " -s 1
2

2

Fig. 7 Measurement of stair width
When the robot climbs down stairs, the height andwidth of the stair are measured by shown in figure 7.
Figure.5 Movement of stair climbing

3 Control
A position control for legs and a torque control forwheels and arms are adopted respectively. The positionsof legs are first estimated roughly and then decide aproper place by both external and internal sensors.
Measured data of a stair include errors, and the same stair
has also about 10mm difference of the height and thewidth. Touch sensors and an angle sensor of each footwork as external sensor to prevent legs slipping down,because of these error and differences. The distance
between legs and the riser of the step is controlled by
the angle sensor. The touch sensors beneath the footdetect the foot being on the tread in safety. Not only therotation of wheels and arms but also movements of legs
must be synchronized. While the robot goes up anddown stairs, the rotational angle of wheels is determined
depending on the position of the legs. Then the slip ofwheel can be measured by comparing a calculated
rotational angle of wheel and a measured angle of thatby rotary encoder.

Fig.8 Measurement of stair size

5 Experimental Results
The experimental results are conducted in order totest the performance of the system. It is much difficult
for mobil robots to go down stairs with stability than to
go up. The both pitch and roll angle of the body andchanges of Leg(h) and Leg(v) motors torque are recorded,
while the robot goes down three steps at the middle part.
Experimental results with this robot are shown in if
figure 10, 11 and 12. The trial robot is controlled bymeans of computer under these conditions: the stairheight is 75mm, the width is 150mm and the number ofsteps is 10. Figure 10 shows the both roll and pitch
angle and these angles do not cause a change abruptly.
This means the robot goes down the stair in safe. The
relation between the torque of mortars and the horizontal- 79 -

displacement of the legs are shown in figure 11 and 12.
As it is clear from these figures above mentioned, thedriving torque change of the legs corresponds the
movement of each leg.

..

- leg (vi)······ leg (v2)

0.

=>
� 0.

,_

0

0 L-4-�---''--..___,,+,--..._..,_�-,.h,--'0400 [mm]200
Horizontal Displacement
Fig,12 Torque of Leg(v)6 Conclusion
We proposed the prot type of this mechanism andthe control algorithm in this paper. The effectiveness of
this mechanism is recognized from the experimental
results. We wilt develop the robot with this mechanism
for disabled and elderly people.

References
Fig.9 SCARAB...

Pitch Angle:;; 20

<

C
0

u
C

Ro11 Angle
o� �

0

200
400 [mm]Horizontal Displacement

Fig. IO Inclination angle of the robot..

0.

- leg (hi)······ leg (h2)

=>

� 0.

,_

.0

0. 1

Fig. I I Torque of Leg(h)[I] Y. Maeda,"Motion Analysis of Obstacle Striding
for the Multinational Robot vehicle",]. of Robotics
Society of Japan, Vol.7, No.2, pp.13-19, 1989.
[2] J. D. Martens,"Stabilization of a Mobile Robot
Climbing Stairs" ,Proc. of IEEE Int. Conf on
Robotics Research, pp.2501-2507,1994.
[3] T. Komeda and H. Funakubo, "Wheelchair andWalking Aid Machine", J. of Robotics Society ofJapan, Vol.11, No.5, pp.44-48, 1993.
[4] J. E. Braes,"Amber: A Walking Robot for Autono­
mous Planetary Exploration", Proc. of Int. Symp.
on Artificial Intelligence.Robotics and Automationin Space, pp.127-130, 1991.
[5] S. Hirose,"A study of design and control of aquadruped walking", Int. J. of Robotics Research,
Vol.3, No.2, pp.113-133, 1984.
[6] D. R. Pugh,"Technical description of the adaptive
suspension vehicle", Int. J. of Robotics Research,
Vol.9, No.2, pp.24-42, 1990.
[7] S. Hirose,"Design of Stair-Climbing Vehicle (Intro­
duction of New Wheels and Modified RC
Mechanism)", Proc. of The 13th Annual Conf of
RSI, pp.153-154, 1995.
[8] M.Yamasina and T. Mabuchi,"Terrain Adaptive
Wheel Vehicle: ALSSION",Proc. of The 71st
JASME Spring Annual Meeting, pp.468-470,1994[9] E. Gut,"Behavior Control for Robotic Explorationof Planetary Surfaces", IEEE Transactions on
Robotics and Automation, Vol.10, No.4, pp.490503,1994.

- 80 -

CAM-BRAIN"
ATR's BILLION NEURONARTIFICIAL BRAIN PROJECT
A Three Year Progress ReportII

Hugo de Garis
Brain Builder Group, Evolutionary Systems Department,
ATR Human Information Processing Research Laboratories,
2-2 Hikaridai, Seika-cho, Soraku-gun,Kansai Science City, Kyoto, 619-02, Japan.
tel.+ 81 7749 5 1079, fax.+ 81 7749 5 1008
degaris@hip.atr.co.jp
Abstract
This paper reports on progress made in the first 3
years of ATR's "CAM-Brain" Project. which aims touse "evolutionary engineering" techniques tobuild/grow/evolve a RAM-and-cellular-automata
based artificial brain consisting of thousands ofinterconnected neural network modules inside special
hardware such as MIT's Cellular Automata Machine"CAM-8", or NTT's Content Addressable Memory
System "CAM-System". The states of a billion (latera trillion) 3D cellular automata cells, and millions ofcellular automata rules which govern their statechanges, can be stored relatively cheaply ingiga(tera)bytes of RAM. After 3 years work, the CA
rules are almost ready. MIT's "CAM-8" (essentially a
serial device) can update 200 million CA cells asecond. It is likely that NTT's "CAM-System"(essentially a massively parallel device) will be able
to update a hundred billion CA cells a second. Hence
all the ingredients will soon be ready to create arevolutionary new technology which will allow
thousands of evolved neural network modules to beassembled into artificial brains. This in tum willprobably create not only a new research field, but
hopefully a whole new industry, namely "brain
building". Building artificial brains with a billion
neurons is the aim of ATR's 8 year "CAM-Brain"
research project, ending in 2001.

Keywords
Artificial Brains, Evolutionary Engineering, Neural
Networks, Genetic Algorithms, Cellular Automata,Cellular Automata Machines (CAMs), Nano­
Electronics, Darwin Machines.

1.

Introduction

ATR's CAM-Brain project resulted from theexperience of the author's thesis work, in which heevolved neural net modules (using concatenated bit­string weights) to control the behavior of a simulated

quadruped called "LIZZY", which could walk straight.
tum left. tum right, peck at food and mate [de Garis
1994]. Each of these behaviors was controlled by thetime varying outputs of a single evolved neuralnetwork module, and applied to the angles of the leg
components of LIZZY. (As far as he is aware, theauthor was the first person to evolve neural net
dynamics [de Garis 1991], (in the form of walking
stick-legs "Walker")). Switching between behaviors
involved taking the outputs from one neural net
module and feeding them into the inputs of the next
module. The next step was to evolve neural net
detectors, e.g. for frequency, signal strength, signal
strength difference, etc. Finally, neural net"production rule" modules were evolved which could
map conditional inputs from detectors to output
behaviors. Thus an "intelligent" artificial creature
was built, which could detect prey. mates andpredators, and then approach and eat or mate, or tum
away and flee. Virtually every neural net that theauthor tried to evolve, evolved successfully. The

evolution of these fully connected neural network
modules proved to be a very powerful technique. This

success made a deep impression on the author,
reinforcing his dream of being able to build much
more complex artificial nervous systems, evenartificial brains. However, every time the author
added a neural net module to the Lizzy simulation,
its speed on the screen was slowed (on a Mac II
computer). Gradually, the necessity dawned on theauthor that some kind of evolvable hardware solution[de Garis 1993] would be needed to evolve large
numbers of neural net modules and at great speed (i.e.
electronic speed) in special machines the author calls"Darwin Machines" [de Garis 1993]. Evolving
artificial brains directly in hardware remains theultimate future goal of the author, but in themeantime (since the field of evolvable hardware(EHW, E-Hard) is today only in its infancy), theauthor compromises by using cellular automata togrow/evolve neural nets in large numbers in RAM,
which is cheap and plentiful. (In a year or so, it will
be quite possible to have a gigabyte of RAM in one's
work-station). By using cellular automata based- 81 -

neural nets which grow and evolve in gigabytes ofRAM, it should be possible to evolve large numbers(thousands) of neural net modules, and then assemble
them (or even evolve their interconnections) to build
an artificial brain. The bottleneck is the speed of theprocessor which updates the CA cells. State of the art
in such processors is MIT's "CAM-8" machine,which can update 200,000,000 CA cells a second.
Very recently, it has been suggested by the author's
ATR colleague Hemmi, that NTT's Content
Addressable Memory System (CAM-System) might
be able to update CA cells at a rate thousands oftimes faster than the MIT machine, i.e. at a hundred
bi llion CA cells per second. NTT's machine ismassively parallel. Hemmi and his programmer
assistant Yoshikawa are now (December 1995) busily
engaged in writing software to convert the author's
CA rules (in 2D form) into Boolean expressions
suitable for the NTT machine. If they succeed inapplying this machine to CAM-Brain, then a new eraof brain building can begin, because the ability to
evolve thousands of neural net modules would
become realistic and very practical (for example, toevolve a neural net module inside a cubic space of amillion CA cells, i.e. 100 cells on a side, at ahundred billion cells a second, would take at mostabout 500 clock cycles, i.e. about five milliseconds.

So the evolution of a population of 100
chromosomes over 100 generations could all be donein about one minute.) All the essential ingredientsfor brain building would be available (lots of RAM,
the CA rules, and fast CA processors). Even if
Hemmi does not succeed, then a new machine can be
designed to be thousands of times faster than the
CAM-8 machine. The author believes the CAM­
Brain breakthrough is either less than a year away,
or at most only a few years away (the time necessary
to design and build a "Super-CAM" machine,
probably with the help of NTT).
The above gives an overview of the CAM-Brain
research project. What now follows is a more detailed
description of CAM-Brain, showing how one grows
and evolves CA based neural net modules in 2D and
3D. We begin with the essential idea. Imagine a 2D
CA trail which is 3 cells wide (e.g. Fig. 2). Down
the middle of the trail, send growth signals. Whena growth signal hits the end of the trail, it makes thetrail extend, or turn left, or right, or split etc.,
depending upon the nature of the signal (e.g. see Fig.
3). It was the author who hand coded the CA rules
which make these extensions, turns, splits etc.
happen. The CA rules themselves are not evolved. It
is the sequence of these signals (fed continuously
over time into an initialized short trail) that is
evolved. This sequence of growth signals is the"chromosome" of a genetic algorithm, and it is thissequence that maps to a cellular automata network.
When trails collide, they can form "synapses" (e.g.
see Fig. 5). Once the CA network has been formedin the initial "growth phase", it is later used in asecond "neural signaling phase". Neural signals movealong CA-based axons and dendrites, and acrosssynapses etc. The CA network is made to behave likea conventional artificial neural network (see Fig. 5).
The outputs of some of the neurons of the complex
recurrent networks which result can be used tocontrol complex time dependent behaviors whose
fitnesses can be measured. These fitness values canbe used to drive the evolution. By growing/evolving
thousands of neural net modules and theirinterconnections in an incremental evolutionary way,
it will be possible to build artificial brains.
According to the CAM developers at MIT, it islikely that the next generation of CAMs will achieve
an increase in performance of the order of thousands,
within 5 years. However, to be able to evolve abillion neuron artificial brain by 2001 (ATR's goal),
it is likely that a "nano-CAM" machine (i.e. one
which uses nano-scale electronic speeds and densities)
will need to be developed. To this end, we arecollaborating with an NTT researcher who has
developed a nanoscale electronics device, who wants
to combine huge numbers of them to behave as
nano-scale cellular automata machines.
In the summer of 1994, a two dimensional CAM­
Brain simulation was completed which required
11,000 hand crafted CA state transition rules. It was
successfully applied to the evolution of maximizing
the number of synapses, outputting an arbitrary
constant neural signal value, outputting a sine wave
of a desired arbitrary period and amplitude and to theevolution of a simple artificial retina which could
output the vector velocity of a "white line" which"moved" across an array of "detector" neurons. Workon the 3D simulation should be completed by early
1996, and is expected to take about 150,000 hand
crafted CA rules. The Brain Builder Group of ATR
took possession of one of MIT's CAMS machines in
the fall of 1994. At the time of writing (December
1995) the porting of the 2D rules from a Sparc20
workstation to the CAMS is nearing completion. If
the porting of the rules of the 3D simulation to thismachine is not possible, then a "SuperCAM"
machine will be designed specifically for CAM-Brain,with the collaboration of the Evolutionary
Technologies (En group of NTT, with whom ourBrain Builder group of ATR's Evolutionary Systems(ES) group, collaborates closely. The complexity of
CAM-Brain will make it largely undesignable, so a(directed) evolutionary approach called "evolutionary
engineering" is being used. Neural networks based on
cellular automata [Codd 1968], can be grown andevolved at electronic speeds inside state of the art
cellular automata machines, e.g. MIT's "CAMS"
machine, which can update 200 million cells per
second [Toffoli & Margolus 1990]. Since RAM ischeap, gigabytes of RAM can be used to store thestates of the CA cells used to grow the neuralnetworks. CA based neural net modules are evolved ina two phase process. Three cell wide CA trails aregrown by sending a sequence of growth signals(extend, tum left, tum right, fork left, fork right, Tfork) down the middle of the trail. When an
instruction hits the end of the trail it executes its- 82 -

function. This sequence of growth instructions istreated as a chromosome in a Genetic Algorithm[Goldberg 1989] and is evolved. Once gigabytes of
RAM and electronic evolutionary speeds can be used,genuine brain building, involving millions and later
billions of artificial neurons, becomes realistic, andshould become concrete within a year or two. The
CAM-Brain Project should revolutionize the fields ofneural networks and artificial life, and in time help
create a new specialty called "Brain Building", with its
own conferences and journals.

2.

Cellular Automata Based
Neural Networks

Building an artificial brain containing billions ofartificial neurons is probably too complex a task to
be humanly designable. The author felt that brain
building would be a suitable task for the applicationof evolutionary engineering techniques. As
mentioned briefly in the introduction, the key ideas
are the following. Use evolutionary techniques to
evolve neural circuits in some electronic medium, so
as to take advantage of electronic speeds. The
medium chosen by the author was that of cellular
automata (CA) [Codd 1968], using special machines,
called "Cellular Automata Machines (CAMs)", which
can update hundreds of millions of CA cells a second[Toffoli & Margolus 1990]. CAMs can be used to
evolve the CA based neural networks at electronic
speeds. The states of the cellular automata cells canbe stored in RAM, which is cheap, so one can havegigabytes of RAM to store the states of billions ofCA cells. This space is large enough to contain an
artificial brain. Mirs Information Mechanics Group(Toffoli and Margolus) believe that within a few
years it will be technically possible to update atrillion CA cells in about 0.1 nanoseconds [p221,
Toffoli & Margolus 1990]. Thus, if CA statetransition rules can be found to make CA cells
behave like neural networks, and if such CA based
networks prove to be readily evolvable, then apotentially revolutionary new technology becomes
possible. The CAM-Brain Project is based on theabove ideas and aims to build artificial brains before
the completion of the project in 2001. The
potential is felt to be so great that it is likely that a
new specialty will be formed, called "Brain
Building".

immediately after the 2D version was ready. Proper
evolutionary tests will be undertaken once the 3D
version is ready, which should be by early 1996. To
begin to understand how cellular automata [Codd
1968] can be used as the basis for the growth andevolution of neural networks, consider Fig. 1 whichshows an example of a 2D CA state transition rule,and Fig. 2 which shows a 2D CA trail, 3 cells wide.
All cells in a CA system update the state of theircells synchronously. The new state of a given celldepends upon its present state and the states of its
nearest neighbors. Down the middle of the 3 cellwide CA trail, move "signal or growth cells" asshown in Fig. 2 As an example of a state transition
rule which makes a signal cell move to the right
one square, consider the right hand most signal cellin Fig. 2, which has a state of 5. The cell
immediately to its right has a state of 1, which wewant to become a 5. Therefore the 2D state transition
rule to tum the 1 into a 5 is 1.2.2.2.5-->5. Thesesignal or growth cells are used to generate the CA
trails, by causing them to extend, turn left or right,
split left or right, and Tsplit. When trails collide,
they can form synapses. It is the sequence of thesesignal cells which determines the configuration of the
CA trails, thus forming a CA network.
Tap

9.18.16.11.5 -> 4

CTRBL -> Cnext
Fig.1

For the first 18 months of the CAM-Brain
Project, the author simulated a two dimensionalversion of CAM-Brain on a Spare 10 workstation.
This work was completed in the summer of 1994.
The 2D version was used briefly (before work on the
3D version was started) to undertake someevolutionary tests, whose results will be presented
in the next section. The 2D version served only as afeasibility and educational device. Since trails are
obliged to collide in 2D, the 2D version was nottaken very seriously. Work was begun rather quickly
on the more interesting 3D version almost- 83 -A 2D CA State Transition Rule

2

2

2

2

2

2

2

2

2

StGNALORECTICN

Fig.

2

Signal Cells Move Alonga Cellular Automata Trail.. .' . ..
. . . . 'I
..
' •I

s

11

Fig. 3 Extend the Trail

IMIT Press, Cambridge, MA. 1987; andCellular Automata Machines. in Lattice Gas
Methods for Partial Differential Equations,
SFISISOC. eds. Doolen et al. Addison­
Wesley. 1990.

It is these CA trails which later are used as neuralnetwork trails of axons and dendrites. Neural signals
are sent down the middle of these CA trails. Thus
there are two major phases in this process. Firstly.
the CA trails are grown. using the sequence of signal
cells. Secondly. the resulting CA trail network isused as a neuralnetwork. whose fitness at
controlling some system can be measured and used to
evolve the original growth sequence. To make thismore explicit. it is the sequence of growth cells
which is evolved. By modifying the sequence. one
alters the CA network configuration. and hence thefitness of the configuration when it functions as aneural net in the second phase. From a genetic
algorithm (GA) point of view. the format of the GA"chromosome" is the sequence of integers whichcode for the signaling (growth) instructions. By
mutating and crossing over these integers. one
obtains new CA networks. and hence new neuralnetworks. By performing this growth at electronic
speeds in CAMs. and in parallel. with one CAM per
GA chromosome. and attaching a conventional
programmable microprocessor to each CAM tomeasure the user defined fitness of the CA basedneural circuit. one has a means to evolve large
numbers of neural modules very quickly. Using
CAMs to evolve neural circuits. is an example of a
type of machine that the author labels a "Darwin
Machine". i.e. one which evolves its own structureor architecture. A related idea of the author concerns
the concept of "Evolvable Hardware (EHW)" [deGaris 1993) where the software instructions used toconfigure programmable logic devices (PLDs) aretreated as chromosomes in a Genetic Algorithm[Goldberg 1989). One then rewrites the circuit foreach chromosome.

Fig. 4.
:

3D CAM-Brain
Non-Synaptic Growth •.'References[Codd 1968) E.F. Codd. Cellular Automata,Academic Press. NY. 1968.
[de Garis 1991) Hugo de Garis. "Genetic
Programming", Ch.8 in book Neural andntelligent Systems Integration, ed. Branko
Soucek. Wiley, NY, 1991.
[de Garis 1993) Hugo de Garis. "Evolvable Hardware: Genetic Programming of a Darwin
Machine". in Artificial Neural Nets andGenetic Algorithms, R.F. Albrecht. C.R.
Reeves. N.C. Steele (eds.). Springer Verlag.
NY. 1993.
[de Garis 1994) Hugo de Garis, "Genetic
Programming : Evolutionary Approaches toMultistrategy Leaming". Ch.21 in book
"Machine Learning : A Multistrategy
Approach. Vol.4", R.S. Michalski & G.
Tecuci (eds). Morgan Kauffman, 1994.
[Goldberg 1989) D.E. Goldberg, Genetic Algorithms
in Search, Optimization, and Machine
Learning, Addison-Wesley. MA. 1989.
[Toffoli & Margolus 1987. 1990) T; Toffoli & N.
Margolus, Cellular Automata Machines.

•:iii
.
.
.
.
------,. ·1'' ''' ''•·---o1- '--'f.illl! ! .
:••••••'•,;

o

I

I

O

I

I

... 1

I

'
I

I
I

''

.. ••-••�..,,...,.,

Fig. S 2D CAM-Brain Neural Signaling- 84 -

•

_.

1 -�

Spontaneous Behavior for Cooperation ofDistributed Autonomous Robots
Takanori Shibata · 1 •2, Kazuya Ohkawa •3, Kazuo Tanie ·2•3
*1 Artifcial Intelligence Laboratory, MIT
545 Technology Square, NE43-814
Cambridge, MA 02139
shibata@ai.mit.edu*2 Mechanical Engineering Laboratory, MITI
1-2 Namiki, Tsukuba 305
tanie@mel.go.jp*3 Tsukuba University, okawa@melcy.mel.go.jp

Abstract

This paper proposes an algorithm for action
selection as production of strategies for
coordinated behaviors. Each robot has somepatterns of behaviors correspond to somestrategies. The algorithm selects the mostappropriate strategy to the situation. Even thoughhuman operator gives an evaluation function for
relations between robots, the robot learns a modelof behaviors of other robots for strategy selection.
When the selected strategy does not fit to its
environment, the robot feels frustration andrecognizes that. In this case, the robot changes its
strategy to change its situation actively. Thischange may generates a new situation and a new
relation between robots. The proposed method isapplied to a task for clearing up empty cans in aroom by two robots. Simulations are simple butshow the effectiveness of the proposed method.

This paper deals with spontaneous behavior forcooperation among distributed autonomous
robots. The robot determines its behavior
depending on its experience and behaviors ofother robots. Each robot feels frustration when its
determination does not suit to its environment.
Then, it changes its behavior to change its
situation actively. Human gives the robots anevaluation function of relation of cooperation
among robots. The robots acquires a model ofbehaviors of the other robots through learning.
This model is considered as the experience.

1. Introduction
In biological societies including human beings,
there are many coordinated behaviors without asupervisor for optimization. Each determines its
own behavior while seeing behaviors of others
and its situation. This behavior is referred to as
spontaneous coordinated behavior in this paper.
Coordinated behaviors are important for multiple
robots to accomplish tasks effectively. The
coordinated behaviors means that the robots work
together or share tasks cooperatively. We assume
that there is no explicit communication amongrobots and no supervisor.
There are many research works on
coordination among robots [1-5]. In the mostresearch, rules and strategies of behaviors are
given by a human operator for effective movement.
There are cases that the rules and strategies do
not suit to the robots' environments. Therefore,
modification of them are necessary in such cases.
Strategies must be produced depending on therelations for cooperation among robots and theirenvironments.

2. Spontaneous Behavior
Fig. 1 shows an algorithm for generation of the
spontaneous behavior. A robot has someelements of behaviors. The meaning of behaviors
are vary in some research from simple action, suchas move forward, to complex action, such as moveto a goal avoiding obstacles. In this paper, thebehavior means the later and can be seen as a
strategy.
The robot is given some strategies a priori,
though each strategy should be a combination of
the simple behaviors. The robot has behavior
rules to perform along with the strategy.
When a main goal or task is given to a robot, it
selects a strategy which suits to its situation. Whenthe selected strategy does not suit to its situation,
the robot changes its strategy. While changing,
the robot accomplish the main goal. The situation
includes the environment and- 85 -

Frustration
Strategy I
Strategy 2
Ot herRobcts'Behaviors
I

New
Behavior

Strategy N
Action-----------------------------------!

Environment•

Fig. 1 An Algorithm for Generation of Spontaneous Behavior
motion of the other robots. The robot must havesensory system to know its situation. When there
is only one robot in a work space, the robot selects
its behavior depending on its environment. On
the other hand, when there are multiple robots in awork space and they have the same main goal, therobot should take into account the other robots.
The robots should work cooperatively while
assisting other robots, avoiding conflicts, sharing
tasks and so on. They may have interaction among
themselves. As we assume there is no supervisor,
they should determine their own behavior
efficiently. Also, we assume there is no explicit
communication among robots; each robot only
indicates the current strategy and other robots cansense this.
As we assume there are multiple robots in thesame work space, there are interactions among
them. They must share some resources toperform selected strategies. This means theymust behave cooperatively. The best strategy for
one robot may influence other robot unfavorably.
This depends on the aim of the robot and its
situation, especially the strategies of the otherrobots.
We give the robot a learning capability tomemorize the relation of cooperation among therobots and to use it as a criterion for the action
selection. In order to generalize the criterion, thislearning does not concern with the environment,
just relation among the robots.

3. A Task and Its Evaluation
A main task is decomposed into some sub­tasks. In order to carry out one sub-task, the robot
has one strategy which consists of behavior rules.

Multiple robots work on the sub-tasks. Each robot
selects a strategy and executes it to accomplish
the corresponding sub-task. During the execution,
the situation of the robot changes. Depending on
the situation, the robot selects the next sub-task.
Continuing this sequence, the robots accomplish
the main task. In this case, even though one robot
can complete the task, multiple robots share thetask for efficiency.
We assume that each robot has an evaluation
function between its behavior and other robots'one. In this paper, human operator gives theevaluation function for the relation between sub­tasks. While using this function, the robot acquiresa relation model of other robots. This model isused to select the new strategy. Followings
explain the evaluation method for the task which isfor clearing empty cans in a room.
The task is decomposed into two sub-tasks.
One is that the robot collects empty cans into thecenter of the room. The other is that the robot
disposes of the collected cans by using a dust­pan. The sub-task of collection of the empty cansare named T1. The other is T2. We assume that
there are two robots in the room and there is one
dust-pan. Then, there may be conflicts between
the robots if both robots select the sub-task T2.
We assume that the T2 is preferable for the
robot to T1 and the relation of evaluation value ofthe sub-tasks is T2 > T1. This assumption
determines order of the evaluation values which
expresses cooperation and conflict between
robots. While considering the order, we assigned
the evaluation values as follows:(strategy (sub-task) of itself, strategy of the otherrobot) ---> Evaluation value(T1, T1) ---> p (equal cooperation)(T1, T2)---> -q (unfavorable cooperation)- 86 -

(T2, T1 )---> q (favorable cooperation)(T2, T2)---> -p (conflict)(1)
where q > p >0.
Each robot is given this evaluation function.
The robot can select its strategy (sub-task)
arbitrarily though the evaluation value isdetermined by the relation between the other anditself. It is desirable to gain higher evaluation value.

4. Learning for Strategy Selection
In order to get high evaluation value, therobot had better select a strategy while predicting
the other robot's behavior. For this purpose, Q­
learning is applied for modeling relation to theother robot's starategy [6]. This chapter explains
the method.
A is a set of strategies for each robot. In thispaper, A is (T1, T2). X is a set of finite discrete
state of the robots. In this paper, X is the selected
sub-tasks by the robots. The strategies of therobots at time t are expressed as a state Xt­
Depending on Xt, the robot selects its behavior
whose Q value is the maximum. The robot selects
a strategy at and recognizes the state of the otherrobot. As an assumption, each robot has a sign
which indicates the state of its behavior, so that
other robots can recognize it. Q value is modified
depending on the evaluation function (1) as
learning as follows:
if Ct= at,
Qnew (xt ' ct ) f- Qo lix t ' ct ) + LlQ
else,
Qnew (x1, c1)t--Q,1ixt' c1)-

1

Q
n-ILl

(2)

where c1 is the possible strategy to be selected attime t and n is the number of the possible
strategies. i1Q is calculated as follows:�Q = a{maxE( a .,b,) + r m ax Qo ,Ax,+l' a 1+1)- Qo,Ax., a ,)}hl EA
a1+J EA(3)

where a is learning rate and y is discounting factor.
The evaluation value E(at, bt) is given by eq. (1).

5. Frustration of Robot and New
Action to Its Environment
The previous chapters described theevaluation method for relation between robots
concerning with cooperation and conflict, and its
usage for learning the model of the other robots.
The model is used for the behavior selection.
However, this selection does not consider robots'
environments. In human beings, he may befrustrated when his decision does not suit to his
situation. We give a similar function to the robots
to take into account their environment in thestrategy selection.
As a model of the frustration, we defined amonotonous increasing function. tt the selectedstrategy is not accomplished for a certain time, theselection is considered that it does not fit to theenvironment. In this case, the robot changes toother behavior which has the second high Q value.
This function means that the robot acts to its
environment
tochangeits
situation
spontaneously.

6. Simulations
Simulations were carried out to show theeffectiveness of the proposed method. It wasapplied to a task for clearing up a room by two
robots.
Fig. 2 shows the initial state of robots and theirenvironments. There are two robots at left side.
Small cylinders are the empty cans. At upper rightside, there is a dust-pan.
Each robot has two sub-tasks. One is tocollect the empty cans in the center of the room(T1). The other is to dlspose of the collected cansto the lower right of the room by using the dust­pan (T2). The robot does not know the locationsof cans but knows that of the dust-pan and thedisposing place.
At first, each robot learned the behavior of theother robot while changing its behavior. Theyobtained Q values in the learning process which is
on-line process. Then, they learned to collect theempty cans cooperatively as seen in Fig. 3. For awhile, each robot engaged in the collection task,
T1, as equal cooperation. In Fig. 4, the right robot(Robot 2) could not find empty cans and it was- 87 -

frustrated. Then, it changed its strategy to take thedust-pan for disposition. Fig. 5 shows that the
robot got the collected cans by the dust-pan andmoved them to the appropriate place. After all, thetask was carried out.

7. Conclusions
This paper proposed an algorithm for the
spontaneous behavior of the autonomous robot.
The robot learns relations of cooperation amongrobots by using Q-leaming to obtain models ofother robots' behaviors. Also, the robot has afunction of frustration which works as a fitness
function to the environment. In the case that the
robot is frustrated, the robot changes its behavior
to change its situation actively. The algorithm isapplied to distributed autonomous robot system.
The simulation was simple but the result showedthat robots worked cooperatively as coordinatedsystem even though there was no supervisor.
As the future work, the proposed method will
be applied to more complicated tasks where many
robots works together in the same work space. A

generation method of the evaluation function willbe also investigated.

Reference[1] H. Asama, et al., Collision Avoidance among
Multiple Mobile Robots Based on Rules andCommunication, IEEE IROS'91, pp. 1215-1220(1991)[2] S. Yuta, et al., Coordinating Autonomous andCentralized
Decision
Making•oActiveCooperative Behaviors Between Multiple Mobile
Robots, IEEE·IROS'92, pp. 1566-1574 (1992)[3] Y. Kuniyoshi, et al., Cooperation be
Observation, -The Framework and Basic Task
Patterns, IEEE ICRA'94, pp. 767-774, (1994)[4] D. McFarland, Towards Robot Cooperation,
From Animal to Animat 3, lnt'I Cont. on SAB, pp.
440-444 (1994)[5] K. Sekiyama and T. Fukuda, Adaptability
Enhancement of Multiple Robots in Dynamical
Environment, Proc. of the IEEE ICRA'95, pp.
1656-1661 (1995)[6] C. Watkins, Leaming from Delayed Rewards,
PhD Thesis, Cambridge University (1989)

Fig. 2 Initial State for Simulation

Fig. 4 Change of Behavior because of Frustration(Robot 2)

Fig. 3 Cooperative Behavior to Collect Empty
Cans by Two Robots

Fig. 5 Accomplishment of Task

- 88 -

Nonlinear Numerical Optimization with Use of a Hybrid Genetic
Algorithm Incorporated the Modified Powell method
1,fasahiro Okamoto*, Taisuke Nonaka, Shuichiro Ochiai, Daisuke Tominaga
Department of Biochemical Engineering & Science, Kyushu Institute of Technology,
Iizuka-city 820, Japan*E-mail : okahon@bse.kyutech.ac.jp
overcome this, Powell[2] has revised the algorithm byintroducing new criteria for formation of linearly inde­
pendent direction vectors; this revised method is called"The Modified Powell method".

Abstract
We proposed the hybrid numerical optimization
technique which is incorporated the Genetic Algo­rithm (GA) into the Modified Powell method. Escapedfrom the trapping in local minimum, this procedure
can seek out the global minimum with considerably
fast convergence. The effectiveness was shown espe­
cially for the minimum search problem of a variable­
separable multi-peak(bottom) function having overtwenty thousands local minima within the parametersearching region.

1

Introduction

Among numerical optimization techniques, thedirect-search method which need not to evaluate thegradient is much suitable for the analysis of dynam­
ics of complex nonlinear control systems. The Pow­
ell method[l] is well known to have an ultimate fast
convergence among direct-search methods, especiallywhere the cost function is approximated well by aquadratic form of parameters to be estimated. It is
proved for the Powell method that at most m iter­
ations, where m is the number of parameters to be
estimated, yield the optimal solution to the problems
with cost function J of quadratic form, if the directionsof m-dimensional vectors are linearly independent at
every iteration step. Some problems, however, are notalways assured of optimal solutions because the direc­
tion vectors are not always linearly independent. To
Keyword : hybrid numerical optimization, function opti­
mization, global minimum, genetic algorithm, modified PowellmethodHowever even if this method is applied, the mini­
mum points obtained may not be the global minimum;
most of the conventional numerical optimization caneasily be trapped in local optima or by constraints ina region of the parameter space far from the optimal
solution. The Genetic Algorithm (GA)[3] is known as
one of the algorithms which can seek out the global
minimum. The GA is based on the heuristic assump­
tions that the "best" solutions will be found in regionsof the parameter space containing a relatively highproportion of "good" solutions and that these regi­
nos can be explored by the genetic operators of selec­tion, crossover, and mutation. The GA offers a num­
ber of advantages; GA searches from a set of designs(not from a single design), and it is not derivative­
based, and it can explore and exploit the parameter
space without trapping in local optima[4]. However
this method has major disadvantage that the compu­
tational cost (computational time) of the large num­bers of runs of the design code is considerably large.

In the present study, we shall propose the hybrid
numerical optimization technique which is incorpo­
rated the GA into the Modified Powell method. It
is highly expected that a unique combination of the
GA and the Modified Powell offers all the advantagesof both optimization techniques while offsetting theirdisadvantages; the proposed procedure can not only
help the algorithm to escape from the trapping in lo­
cla minima but also seek out the global minimum withconsiderably fast convergence.

- 89 -

2

Procedures

method and back to step 1-1 by trans­
lating the genetic code ( 32-bit strings)
into real number.

The procedures of proposed hybrid numerical opti­
mization can be summarized as follows:(step 1-1) : Set an initial or a re-initial points of pa­
rameter space.

3(step 1-2) : Seaffh optimal, near optimal, or better
solutions with use of the Modified Pow­
ell method.

In order to examine the effectiveness of our pro­
posed procedures, they were applied to the minimum
search problem of the following function :(step 1-3) : Evaluate the estimated values (solu­
tions) of parameters and calculate thecost function f.

Function optimizationf(step 1-4) : If t.he value off is large and does not re­markably decrease during searching pro­cedures in the Modified Powell method,
switch to the GA and go to step 2-1.

J0.01 X ((u - 25)2 + (v - 40)2) + 10
-10 x sin(0.51ru) cos(0.751rv)(1)

This function has 20480 local minimum points in
the region where u and v are within [-256, 256]. The
graph is plotted in Figure 1 in a region with u and v in[O, 50]. The global minimum value is one when u = 25
and v = 40.

(step 1-5) : If the absolute value, lf-fop d, is smaller
than the threshold value E; If-fop d < E,
where fop t represents the desired value( optimal value) of f, output the results.
Otherwise, continue to search (back tostep 1-2)(step 2-1) : Transform the real value, obtained value
by the Modified Powell method ( = ob­tained value trapping in local minimum)
into 32-bit standard floating point for­mat; the sign is represented by a single
bit, the biased exponent and the frac­tion will have 8 bits and 23bits, respec­tively;I I s

1(sign)

23
Figure 1:(step 2-2) : Generate 32-bit m initial genes involv­
ing the genetic code obtained in step 2�1f(Eq.(1)).

(step 2-3) : Generate crossover gene; randomly se­lect two genes and crossover each otherat one point (ont-point crossover).

Illustration of the shape of function

At this optimal point, the value of function f takes
0. In computer simutation, we searched in a region
with u and v in 1-256, 256]. In the Modified Powellmethod, the Rosenbrock method (Success and Faults
method)[5] was used as one-directioal search, and thejudgement of that the obtained value is trapped into
local minimum was as follows:(step 2-4) : Generate mutation gene; randomly se­lect genes and randomly select the mu­tation point and mutate at that point.
(step 2-5) : Evaluate the genes and proceed to
the next generation (introduce the eli­tist strategy; where the best evaluatedmember (gene) of each generation iscopied into the succeeding generation).

If - foptl
lfi-fi-11(step 2-6) : During steps 2-3 to 2-5, if the value bet­
ter than one in step 2-1 is obtained,
switch back to the Modified Powell

>

E

<

0

and

(2)

, where f; and fi-1 show the values of cost function
f at i-th and (i-l)th iteration, respectively and fopt
represents the desired value off. In the present study,- 90 -

Table 1: Summary of convergence frequency in the function optimization (Eq.(1)).
Optimization method Mutation rate CPU-time (sec) Convergence frequency (%)
Simple GA
0.05
4.31
50
0.10
3.21
69
0.15
3.50
57
74
3.10
0.05
Hybrid-method
79
2.88
0.10
0.15
2.85
81
Table 2: Summary of convergence frequency by using the hybrid method with additional procedures.
Optimization method Additional procedures CPU-time (sec) Convergence frequency (%)
none
2.85
Hybrid-method
81
+(1)
2.57
82
+(2)
2.74
86
+(1)+(2)
2.59
89
none
GA
3.50
57
5.76
+(2)
66

the threshold values of E and 0 are fixed at 10- 3 and10-.s respectively.
In the GA procedures, the total numbers of genes
at each generation is fixed at 60 and the crossover rate
is 0.5 (crossover occurs among half of the genes at eachgeneration). The rate of bit-mutation in each gene isset to 0.05
0.15. All the computations have been
performed in UNIX workstation Titan2-300LX/HX(Kubota Computer Inc., Japan, 75 SPECfp92). We
terminated the simulation when the total number ofgenerations in the GA is over 500. Table 1 shows theconvengence frequency (percentage of getting opimal
solutions) in the cases of simple GA method and the
hybrid method (our proposed method). These values
represent the convergence frequency among 100 runs
by simple GA or hybrid method.
The results indicate that the hybrid optimization
technique outperforms the simple GA technique. In
order to improve the convergence frequency, the fol­lowing two additional procedures were added to theprocedures at section 2 :~(additional procedures)(1) At step 2-6, the best three values ob­
tained are back to step 1-1, that is, best
three candidates obtained in the GA
procedures are taken back to the Modi­
fied Powell method.
(2) At step 2-4, if the value of cost func­tion f becomes considerably lower than
the initial value, change the mutation

rate at each bit; the mutation rate at
the biased exponent changes to small,while the mutation rate at the frac­tion changes to large. This procedure
will appreciably narrow the searchingregion.
Table 2 shows the effects of the introduction of theadditional procedures above on the convergence fre­quency. The initial mutation rate was fixed at 0.15.
In this simulation, if the value of cost function f be­comes to be lower than 1.0, the following function,
which calculate the mutation rate at each bit, was ap­
plied as the additional procedure (2):?

Y(i) = M_rate x [_:_{tan-I.:...=_}+ 1]7r2(i=5,6,...,31)
.

8

(3)

, where Y(i) represents the mutation rate at the i­th bit from most significant bit (MSB), and M_rate
shows the initial mutation rate. The other conditions
were the same as in Table 1. As shows in Table 2, bycombining the additional procedures (1) and (2), thehybrid optimization method can seek out the global
minimum at considerably high frequency (89%); in thecase of simple GA method, the convergence frequency
is 57%.

4

Conclusions(1) We proposed the hybrid numerical optimization- 91 -

technique which is incorporated the GA into the
Modified Powell method.
(2) The effectiveness was shown especially for theminimum search problem _of a variable-separable
multi-peak function; without trapping and stay­ing in local minimum, the proposed method can
explore and exploit the global minimum ( optimal
solution) with considerably fast convergence.
(3) This technique will be highly applicable to non­linear complex numerical optimization in the fieldof process control and robotics.

References[1] M.J.D.Powell "An efficient method for finding
the minimum of a function of several variables
without. calculating derivatives", Computer ]., 7,
ppl55-162, 1964.
[2] M.J.D.Powell "On the calculations of orthogonal
vectors'·, Computer ]., 11, pp302-304, 1968.
[3] J.H.Holland "Genetic algorithms",
American, pp44-50, 1992.

Scientfic[4] L.Davis "Handbook of genetic algorithms", Van
Nostrand Reinhold, New York, 1991.
[5] H.H.Rosenbrork "An automatic method for find­ing the greatest or least value of a function", Com­
puter ]., ppl75-184, 1960

- 92 -

A Language for Describing Motions of Mechatronic Systems:
A Nonlinear Position-dependent Circuit Theory
Suguru Arimoto
Faculty of Engineering, The University of Tokyo
Bunkyo-ku, Tokyo, 113 Japan
Abstract
Dynamics and control of nonlinear mechanical andadvanced mechatronics systems can be investigatedmore efficiently by using corresponding nonlinear
position-dependent circuits that describe Lagrange's
equations of motions and interactions with objects
or/and task environments. Such expressions of La­
grange's equations via nonlinear circuits are indebted
to lumped-parameter discretization of mechanical sys­
tems as a set of rigid bodies through equations of mo­tion due to Newton's second law. This observation isquite analogous to validity of electric circuits that canbe derived as lumped-parameter versions of Maxwell'sequations of electromagnetic waves. Couplings of me­
chanical circuits with electric circuits through actuator
dynamics are also discussed. In such electromechan­ical circuits the passivity should be a generalizationof impedance concept in order to cope with nonlinear
properties of circuits and play a crucial role in motion
control problems. In particular, it is shown that thepassivity as an input-output property gives rise to anecessary and sufficient characterization of H00 -tuningfor disturbance attenuation, which can give anothersystems-theoretic interpretation of the energy conser­
vation law.
Key words: Nonlinear Circuit, Mechanical Systems,
Robot Motion, Passivity.

1

Introduction

It is widely known as explained in many textbooks
that there is an analogy between dynamics of a sim­
ple linear mechanical system of mass-damper-spring(MCK) and that of a simple lumped-parameter linear
electric circuit of inductor-resistor-capacitor (LRC)(see Fig.1). Nevertheless, dynamics of actual me­
chanical and mechatronics systems such as robot arms
and mechanical hands have never been analyzed inten­
sively based on their corresponding circuit-theoretic
expressions. Indeed, there is a dearth of papers thathave discussed directly any possibility of expressing
Lagrange's equation of motion for nonlinear mechan­ical systems via lumped-parameter circuits like elec­
tric circuits. Conversely, there is a well-known ap­
proach called "Euler-Lagrange formalism" for analyz­
ing electric circuits (see Meisel [1]) and "Lagrange­
Formalism" for discrete electromechanical systems(see Maisser and Steigenberger [2]). More classically,
there was a proposal for description of mechanical sys­
tems via a network called "bond graph" (see Paynter[3], Karnopp and Rosenberg [4]). However, all thesewere proposed before opening a new "robot era" andtherefore have not yet been extended to cope withdynamics of nonlinear mechanical and mechatronics
systems. Indeed, dynamics of actual mechatronics
systems such as anthropoid robot arms and hands
are nonlinear and have strong couplings between joint
variables. As a matter of course in electric circuit the­ory, generalization of electric circuits towards nonlin­
ear circuits has been attempted in 1960's and 1970's(see Brayton and Moser [5] and Chua and Green [6]),but only a limited class of circuits was investigatedby introducing individually a few of single nonlinear
elements like a nonlinear resistor, an inductor withnonlinear flux, and a nonlinear capacitor.
This paper shows :first that a mechanical systemgoverned by Lagrange's equation of motion implies theexistence of a nonlinear position-dependent circuit andits dynamics corresponds to that of the circuit. In sucha nonlinear circuit the velocity vector can be regarded
as the current vector that passes along a bundle ofbranches and flows into and out of such blocks (seeFig.2) with n input and n output terminals as kinetic
inductor, resistor, virtual resistor, and gravity capac­
itor. These linear or nonlinear blocks are considered
to be operators that operate on a domain of veloc­
ity vectors and generate voltage drops. A nonlinear
kinetic inductor can be regarded as a block with n in­puts and n outputs being dependent on the position
and retains a kinetic energy. A gravity capacitor isalso dependent on the position and retains a poten-- 93 -

tial energy. Both Kirchho:ff's current and voltage laws
are valid to this class of nonlinear position-dependentLM

circuits, correspondingly to d' Alambert's principle. In
addition, a new concept called "displacement capaci­XItor" is introduced in order to cope with set-point con­trol problems of nonlinear mechanical systems.

The

displacement capacitor can be interpreted as an ex­
tension of a mechanical spring. Thus, it is shown thata similar method to the direct-current analysis for elec­
tric circuits can be applied for the stability analysis of
set-point control without compensation for the grav­
ity force. However, the proposed approach is not a
simple extension of this analogy because mechanicalcircuits as a language for describing dynamics shouldhave more flexible syntax and general context to copewith various nonlinearities in electromechanical sys­
tems and three underlying physical variables "veloc­
ity", "torque", and M'position" di:fferently from elec­
tric circuits that have only two kinds of underlying
physical variables "current" (corresponding to "veloc­ity") and "voltage" (corresponding to 'Mtorque"). A
physical variable "electric charge" appears as an inte­gral of "current" in time in electric circuits like 'Mposi­tion" as an integral of "velocity" in time, but a vectorof position variables signifies a real configuration of
the system in space. Hence, a kinetic inductor is agenuinely new concept of inductor that is dependent
on not only "velocity" but also 'Mposition", that is,
the inductance operator that induces "voltage drop"
depends on the present configuration of the systems.
Secondly we show an example of nonlinear circuitthat expresses coupling of the motion of a robot arm
driven by a set of voltage-controlled DC motors asCRK,-.___, VC

Fig .1 Analogy between a Mass-Damper-Springmechanical system and a linear
Induetor-Resistor-Capacitor electric
circuit.
voltage drop

-

.. .

-

current

Elementary
Block

...

-

--'

current

Fig.2 Elementary block of nonlinear
position-dependent circuits.

joint actuators with a set of linear armature electric
circuits. Thirdly we present a necessary and sufficientcharacterization of H00 -tuning of PD control with o:ff­
line gravity compensation in the sense of disturbanceattenuation. This in turn means that the H00 -tuning
for mechatronics systems can be interpreted as anothercharacterization of "energy conservation law".

2

Coupling of Mechanicalwith Electric Circuits

Circuits

The Lagrange equation of motion of a robot arm(see Fig.3) is described by: { : L(q,q)} - : L(q,q) = F
t q
q

( 1)

where L(q,q)
I<(q,q) - P(q),I<(q,q) denotes the
kinetic energy, P(q) the potential energy, and F the- 94 -

Fig.3 Anthropoid manipulator withall revolute-type joints

generalized force vector whose i-th component stands
for a torque generated by an actuator at joint i. In

Kinetic inductancedetail, (1) can be written as

H(q)q + H(q)q - :/<(q, q) + : P(q) = F.
q

(2)

q

VirtualresistanceS(q,q)

Further, note that ( 1) can be expressed as

(3)

F �

Voltagesource

g(q)

where

It is well known that S (q, q) becomes skew-symmetric(see Arimoto and Miyazaki [7]).
Now, the Lagrange equation of motion for the robotof Fig.3 is expressed by the nonlinear circuit depicted

Gravity
torqueFig.4 A nonlinear circuit that
expresses Lagrange's
equation of motion.

in Fig.4. Conversely, applying Kirchhoff's voltage law
for the circuit of Fig.4 leads to equation (3).
Next consider coupling of a mechanical motion ofa robot arm with a set of electric circuits of joint ac­
tuators. The dynamic equation of the armature cir­
cuit corresponding to the i-th joint actuator can be
expressed asi = 1,·· • ,n

(5)

NK

T

q

where L;,R,,Ia;.l{b,, and v; denote the armature in­ductance, armature resistance, armature current, con­stant of motor back electro-motive force, and armature
voltage of the i-th motor armature circuit.

We de­

note the transmission gear ratio n; : 1 from the motor

Fig.5

shaft angle to the joint angle by a number r,(= 1/n;)

and assume that the arm link is sufficientl y stiff tobe considered as rigid and hence r,0; = <J; where q,
denotes the i-th component of position coordinates qof the arm. On the other hand, the torque gener­
ated from the i-th joint motor is equivalent to Kr,Ia,,
where Kr, denotes the torque constant. According to

the d'Alambert principle, the torque Kr,Ia, is equal to

the load due to the inertia moment and damping fromthe motor side itself and the i-th contribution r;F; ofthe motion of the arm, where F; denotes the i-th com­ponent of F in equation (3). This can be expressedasNKrla = F+Hori+Boiz{Ho+ H(q)}q + {½H(q) +Bo+ S(q, q)}q
(6)
+g(q)
1

1
N
diag(r1 , • ••, r; ), Kr
where
2
2
diag(Kr1 ,··· ,Krn), Ho= diag(Jm 1 r1 ,··•,JmnT; ),Bo =The transformer connects the motion ofa nonlinear mechanical system with a setof linear electrical circuits. The input­
satisfies theoutput pair {v,I}apassivity.
2

diag(bm 1 r1 , ••·, bmnr; 2 ) provided that all
joints are driven by such voltage-controlled DC servo­
motors,Now, the nonlinear position-dependent circuit gov­

erned by equation (6) can be coupled with the linear
electrical circuit governed by (5). Since r,0, = q, fori = 1, · ··, n and in principle the motor torque con­stant I<r, is equivalent to the constant Kb, of backelectro-motive force, the circuit of the total dynam­
ics can be depicted in Fig.5, where coupling between
the mechanical nonlinear circuit and electrical arma­
ture circuit is realized by a "transformer" at the ve­locity level. The voltage drop (torque) induced by
the transformer at the side of mechanical motion isequivalent to � NKr la (proportional to the current
vector Ia ) and the actual voltage drop induced by thetransformer at the armature circuit is considered to be- 95 -

equivalent to the voltage NKr q induced by the backelectro-motive force. According to equation (5) andKr ; = Kb;, the set of armature circuits can be de­scribed in the following form:(7)
where /4 = (Ia. 1, .. •,lan )T , v = (v1,.. •,vn )T ,La. =diag(L1,···,Ln ), and Ro= diag(R1,···,Rn)· At thisstage it is natural to expect that the input and output
pair {v,14 } for the total circuit of Fig.5 may satisfy
the passivity. In fact, it is possible to show the valid­
ity of this property by taking inner products between
equation (7) and /4 and between equation (6) and q
and summing up these two resulting quantities, which
results in the form

{Ho + H(q)}q + { ½H(q) + Bi+ S(q,q)}q + g(q) = F(11)
where B 1 =Bo+ B 0• We implicitly assume the exact
knowledge on N, Kr and Ra and thereby the input
F can be composed of the input voltage v throughF = NKr R; 1 v, Further, we must bear in mind that
there are other nonlinear frictional forces that can not
be described and hence can not be included in (11).
Therefore, if we consider a PD feedback with gravity
compensation defined bywhere Qd is a given desired target position and Aq =q - Qd and the closed-loop system ( obtained by substi­
tuting (12) into (11)){Ho+ H (q)} q + {½H(q) + S(q,q) + B}q+g(q)- g(qd) + AAq = u = h(Aq,q) + Au (13)(8)
Since min q P(q) = 0, it follows from (8) thatt

1 I[(r)v(r)dr � -E(0) = --y5(9)where E(0) denotes a nonnegative quantity at t = 0of the total energy defined by the content inside thesquare bracket [ ] in (8). A setpoint control problem
for the total system can be solved by taking advan­
tage of this passivity property though the total sys­
tem seems more complicated with 3n state variables(q, q, 14 ). This has been treated in Arimoto [8].

3Disturbance Attenuation m a Sense
of H Control
00

In this section we treat a special case when the
inductance matrix L a. is small enough in comparison
with Ra. so that the :first term of (7) can be ignoredin the armature circuit dynamics. Then, (7) can be
rewritten in the form
(10)
where F = NKr R';; 1 v and Bo= NKr Ro 1 NKr . Sub­
stituting (10) into (6) yields

where B = Bi+B, then extra input u must be a sum of
unknown but structured frictional forces h(Aq, q) andanother genuine disturbance Au as described in (13).
It should be also remarked that a position feedbackof time delay AAq(t - At) induces a positive velocity
feedback as shown by- AAq(t - At)� -AAq(t) + AtAAq(t).

(14)

Since such a positive velocity feedback must be taken
into account in the structured disturbance h(Aq, q),
we suppose a class of structured disturbances satisfy­
ing

for a :fixed -y 2 and any t > 0.
Definition 1 If the input-output pair {Au, q} con­cerning the system of (13) satisfies the passivity forany structured disturbance h(Aq, q) satisfying (15),then it is said that it satisfies the passivity with margin-y 2 .
At the same time, we are concerned with the per­formance of disturbance attenuation in general in thesense L2 norm of the output q versus that of the total
disturbance u in (13) over any time interval [O, t].
Definition 2 Assume that q(0) = Qd and q(O) = 0.
If for a :fixed -y 2 > 0 and any time interval [0, t] it holdsthat- 96 -bott

1 II q(r) 11 dr � -y 1 JI u(r) 11 dr,
2

2

2

(16)

then we say that the system described by (13) es­
tablishes the H00 -tuning with level 'Y-2•
Here in(16) b 0 is defined as b 0 =minimum eigenvalue of B,(bo = 'Ym (B)).
The most important result concerning these twoconcepts can be stated in the following:
Theorem 1

A necessary and sufficient condition

for the system of (13) to satisfy the passivity with
margin'Y-2 and to establish the H00 -tuning with level'Y-2 simultaneously is that
'Y-2�bo(='Ym (B)),

>

(17)

0 is chosen to satisfy the following
provided that A
0 simultane­
two inequalities with a given small aously:>

�.:lq A.:lq + P(q) - P(qd) - .:lq g(qd) � a 11.:lq 11 ,
2
T

T

T

.:;lq {g(q) - g(qd)}2+ .:lq A.:lq � a 11.::lq 11 .
T

2

(18)

We omit the proof (see Arimoto(9]).

Theorem 1 presents a physical interpretation that

if the energy dissipation rate q Bq has a margin be­yond 'Y-2 II q 112 for any q then the H -tuning isT00

established. This is valid for the nonlinear dynamicsof (13) in a global sense. Another noteworthy featureof Theorem 1 is that the sufficient condition for es­
tablishment of the H00 -tuning is expressed in a very
simple form, which need neither solve any of Riccati's
matrix equations (see Doyle et al. (10]) nor use so­lutions to a Hamilton-Jacobi-Issacs equation (van derShaft (11)(12], lsidori and Astolfi (13], and Astolfi and
Lanari [14]). Readily from the proof of the necessity
part of Theorem 1 the following result follows ( theproof will be given in Arimoto(9]):
Theorem 2[1] J. Meisel, Principle of Electromechanical Energy
Conversion, McGram-Hill, 1966.
[2] P. Maisser and J. Steinberger, "Lagrange­

Formalismus fur diskrete electromechanishe System,"

Z. Angem. Math. Mech., Vol.59, pp.717-730, 1979.
[3] H. M. Paynter, Analysis and Design of Engineering

Systems, The MIT Press, 1960.
[4] D. Karnopp and R.C. Rosenberg, Analysis andSimulation of Multiport Systems, The MIT Press,
1968.
[5] R.K. Brayton and J. K. Moser, "A theory of nonlin­

ear networks Prt I and Part 11", Quart. Appl. Math.,
Vol22, pp.1-33 and pp.81-104, 1964.
[6] L.O. Chua and D.N. Green, "A qualitative analysisof the behavior of dynamic nonlinear networks : Sta­bility of autonomous networks", IEEE Trans. of on
Circuits and Systems, Vol.23, pp.255-279, 1976.
[7] S. Arimoto and F. Miyazaki, "Stability and robust­ness of PID feedback control for robot manipulators of

sensory capabilities", in Robotics Research: First Int.
Symp..

(M.Brady and P.P. Paul eds.), MIT Press,

pp. 783-799, 1984.
[8] S. Arimoto, "Nonlinear position-dependent circuits

for mechanical motion control", Proc. of IFAC Work­
shop Motion Control, Munich, Germany, Oct.9-11,
1995, pp.1-20.
[9] S. Arimoto, "Nonlinear position-dependent cir­
cuits: A language for describing motions of nonlinear
mechanical systems", to be presented at the IUTAM
Symp. on Advanced Mechanical Systems, April 21-26,
1996. Eindhovenm Netherlands.

[10] J.C. Doyle, K. Glover, P.P. Khargonekar, and B.A.

As for the circuit depicted in Fig.6,

the input-output pair {.:lu, q} satisfies the passivity if
and only if 'Y-2 �bo(= 'Y (B)), provided that A> 0mis large enough satisfying (18) and (19) simultaneouslywith a specified a > 0.

4

Reference

Conclusion

In this paper we have introduced a framework ofnonlinear position dependent circuits as another lan­
guage for describing nonlinear dynamics of mechan­ical and advanced mechatronic systems. Instead of
Fourier and Laplace transforms that are used effec­

tively in analyzing linear lumped-parameter electric
circuits, the concept of passivity becomes fundamen­

tal and can be used in effect as a basic tool of char­acterizing input-output properties of such nonlinear
position-dependent circuits.

Francis, "State-space solutions to standard H2 andH00 control problems," IEEE Trans.
Control, Vol.34, pp.831-847, 1989.

on Automatic[11] A.J. van der Schaft, "On a state-space approach
to nonlinear H00 control, Systems and Control Letters,

Vol.16, pp.1-8, 1991.
[12] A.J. van der Schan, "£2-gain analysis of nonlin­
ear systems and nonlinear state feedback H oo control."IEEE Trans. on Automatic Control, Vol.37, pp.770784, 1992.
[13] A. Ishidori and A. Astolifi, "Disturbance atten­

uation and H00 control via measurement feedback innonlinear system, IEEE Trans. on Automatic Control,
Vol.37, pp.1283-1293, 1992.
[14] A. Astolifi and L. Lanari, "H00 control of rind
robot", Proc. of IFA C Symp. on Robot Control, Sept.

19-21, 1994, Capri, Italy, pp.199-204.

- 97 -

Reinforcement Learning of Dynamic Behavior
by Using Recurrent Neural Networks
Ahmet ONAT
Hajime KITA and Yoshikazu NISHIKAWA
Department of Electrical Engineering
Graduate School of Kyoto University
Yoshida-Honmachi, Sakyo,
Kyoto 606-01 Japan
Abstract
Reinforcement learning is a learning scheme forfinding an optimal policy to control a system, basedon a scalar signal such as a reward or a punishment. If
the observation of the system by the controller is suf­
ficiently rich to represent the internal state of the sys­tem, the controller can achieve the optimal policy only
by learning reactive behavior. However, if the state of
the controlled system cannot be assessed completely
using current sensory system outputs, the controller
must learn a dynamic behavior pattern to achieve theoptimal policy.
We propose a dynamic controller scheme, which uti­
lizes memory to uncover the hidden states by using
information about the past output, and make control
decisions using the memory. This scheme integrates
Q-learning proposed by Watkins, and recurrent neu­
ral networks of several types. It performs favorably insimulations which involve a task with hidden states.

1

Introduction

Reinforcement learning is an attractive method forautonomous learning agents because a teacher thatshows the correct policy is not necessary. It uses only
the reinforcement signal, a scalar performance indica­
tor. It works as follows: The controller observes theoutput of the system, and decides on an action. The
system changes its state according to the taken action,and yields a new output, and also a scalar reinforce­
ment signal to the controller. The aim is to find an
optimal policy, i.e., a mapping of the output of thesystem to the action, which maximizes a value func­tion, usually the cumulative discounted sum of the re­inforcement signals. There have been several learningtechniques proposed, to realize reinforcement learn­ing, e.g., TD-learning by Sutton[8], and Q-learning by
Watkins. [9].

Studies on reinforcement learning usually focus oncases where the state of the controlled system is fully
available to the controller as the output of the sys­tem, and try to obtain an optimal mapping of system
outputs to controller actions. However, in realistic sit­uations, it is usually impossible to access the entire
state variables of the system. Some of the states may
be hidden from the controller, because of sensory limi­
tations. In other words, multiple system states may beperceived as the same by the controller. This is called
perceptual aliasing[lO]. While perceptual aliasing may
be advantageous when the aliased states are irrelevantto the task at hand, states requiring different actions
might also be aliased. This impairs the performanceof the controller, and is clearly disadvantageous.
Various methods have been proposed for learning inpresence of perceptual aliasing. Whitehead et. al.[10]have proposed a reconfigurable sensory system to uti­
lize the available sensors more efficiently. Their "lion"
algorithm discovers and avoids passing through stateswith perceptual aliasing. Chrisman[2] and McCallum[3] propose methods which split the modeled state
space of the controlled system, based on the gatheredstatistics. Our approach to cope with perceptual alias­
ing, is to incorporate memory into reinforcement learn­ing by using recurrent neural networks. It is trainedby Q-learning [9] modified for neural networks using
the error backpropagation technique.

2

Q-Learning for Recurrent Neural
Networks

In this paper, we propose a dynamic controller
which reconstructs the missing state information basedon the past outputs of the controlled system. The con­trolled system is assumed to be finite-state, discrete­time, and it is characterized by the transition proba-- 98 -

bilities;

represents its Q value. Its output is then fed to thestochastic action selector (SAS), which selects one ac­tion according to the Boltzmann distribution[!];where, xi, Xj E X are the states; Xt , denotes thestates of the system at time t; At = ak E A is theaction performed at time t. The system returns thereinforcement signal rt +i(Xt , At) to the controller foraction At from state Xt . The observed output of thesystem Y = C X is calculated from the state vector
X. We assume that at some points in time, Xt can
not be determined by considering only Yt, but the sys­
tem is observable, i.e., the state Xt can be estimatedfrom the past output {Yt-i, Yt- 2, •.• }. The aim of thecontroller is to maximize the cumulative discounted re­inforcement (or, the utility) given by;

Q(X,,ak)exp( T, )Prob(ak) = �n
Q(X,,ak)
L..l=O exp( T, )(3)

where, Tt is a parameter called the temperature which
controls the randomness of the action selection rou­tine.

CONTROLLER

ACTIONSAS

00½ = I:·lr(t +k)
k=O

(2)

ELMAN
NETWORK

where, 0 < 'Y :'.S 1 is a constant called the discountfactor.

The proposed architecture of the controller is shownin Fig. 1. It integrates a recurrent network of the El­
man type[4] and Q-learning. The main idea is to use
Q-learning to tune the connection weights of the recur­
rent neural network for a dynamic behavior pattern.
Q-learning is a reinforcement learning scheme which
estimates the optimal utility (Q(xi,ak)) to be obtainedby taking action ak at state Xi- Lin[5] has modified it
for usage with feedforward neural networks and er­ror backpropagation technique. In this modification,
the network has one output unit for each action ak,
which represent the predicted utility Q of doing ac­tion ak for the current input. The actions are selected
randomly, with a bias toward stronger outputs. This
helps avoid local optimal policies. Initially, actions
are chosen with little respect to their Q values, be­cause these do not yet describe the controlled system.
This allows for experimentation as the controller as­
sesses the outcome of each action in all contexts. As
learning proceeds, and the internal model of the con­trolled system is improved, actions with high Q values
are chosen most of the time.
The Elman neural network is similar to multilayer
feedforward networks, except that there is feedback
from the hidden layer to the input, through a one steptime delay. The layer of time delay units is called thecontext register.
We use an Elman network to learn Q values. It
has input units corresponding 1) to the output of thesystem, 2) the last action, and 3) to the context reg­ister. Each output unit corresponds to an action, and

ACTIONOUTPUTSCONTROLLED SYSTEM
Figure 1: The proposed Q-learning recurrent networkmodel. 6 represents a one step delay.

- 99 -The training algorithm is as follows:
1. Get the current output of the system Yt- Concate­
nate Yt, the past action At -I, and the outputs ofthe context units as vector Zt . Feed Zt into the
network, and obtain Q(Zt , ak) values for each ac­tion ak E A.
2. Using the stochastic action selector, choose an ac­tion At = a1 and apply it to the system.
3. Get the new outputs Yt +i , and the reinforcement
rt +l from the system.
4. Obtain the network output; Q(Zt + 1 ,ak) for eachaction ak E A.
5. Calculate the error term AQ ='Y maxak Q(Zt +I, ak)} - Q(Zt , At)-

{rt+i +

6. Construct the error vector e(t) by using t::.Q for
the output unit corresponding to the taken action,and 0 for the rest.
7. Update the weights using the error backpropaga­tion technique, and go to 1.
The above technique can be used with different
types of recurrent neural networks. We have also used
the information loss minimization method, proposedby Noda[6] as an improvement over the Elman net­
work. This method improves network performanceby retaining relevant information longer. It uses the
X-model architecture shown in Fig. 2, which consistsof an Elman network with two additional output unitgroups called the context reconstruction layer, and the
input reconstruction layer. These groups are trained
to minimize the error function

where Yo,do,x1,xc,YRI and YRC are activation pat­tern vectors of the output, desired output, input, con­text, reconst-input, and reconst-context layers, respec­tively. Since the output of the network is a trans­
formation of the output of the hidden layer, the two
additional groups make sure that enough information
about the input and the previous state(context layer)
is retained in the context layer. This results in related
information being retained for a long time.
In our model,the sigmoid function is used as theactivation function of the neurons in the hidden layer,
and a linear function for the output units..

•

: co

---__.., -----

in Fig. 3. Each room is accessible only from the previ­ous one, and is connected to the next one. The door to
the next room is always closed when the controller(anagent, shown as the man in Fig. 3) enters into theroom. Also, the doors have locks, and certain doors
are locked. The controller receives two types of inputs;
the state of the door (open or closed), and the num­
ber of the room the controller is in. In some rooms,
the controller cannot detect the room number. This
causes perceptual aliasing. The task is to go throughthe rooms, and exit the building as quickly as possible.
The controller has three actions; "open", "toggle
lock", and "go aheacf'. Reinforcement signals are given
when the controller gets into the next room, or exits
the building (reward + 1), or if it bumps into a closed
door (punishment -1). No reinforcement is given oth­erwise. The state of the lock is not observable. Eachtrial starts from a room selected randomly out of thosehaving a detectable room number. To achieve the op­
timal performance, the controller must identify therooms and remember the states of the locks.

Observable system output:

Room=#ldoor=open

1

3.2
Figure 2: X-model network architecture

3.1

P4

Figure 3: A graphical sketch of the simulation en­
vironment. "L" represents a locked door, "P" indi­cates rooms which cannot be distinguished from one
another.

RECONSTRUCT :,_...,.,,,,.,.,.,,.,....,.,.,.,..,,,,, ..__

3

P5

6 L

Simulation
The Learning Task

The environment consists of a building with a num­
ber of rooms connected by doors in a chain as shown

Simulation Results

We have carried out computer simulations using theenvironment of Fig. 3. The parameters in the simu­
lation are; 'Y = 0.8, r, = 0.l(the learning rate for the
network), and 17 hidden units. Figure 4 is an exampleof the variation of the redundant moves taken by thecontroller. The number of redundant moves taken toreach the goal state, averaged over 100 trials, is plottedagainst the number of moves;
The performance is improved as learning proceeds.
As training proceeds, the controller settles on a

- 100 -

non dynamic policy, which uses an extra action toprobe the controlled system state for a while. Laterit finds the optimal policy for this task. The behav­
ior pattern finally acquired by the controller is almost
optimal.

s

10

w
0.1

0.01 .____..,____.___.......__.___._.._.___......�.................U-LU
0100000 200000 300000 400000 500000 600000 700000 800000
Number of iterations

4

Conclusion

In this study, we construct a dynamic controllerthat acquires an optimal dynamic control policy under
the reinforcement learning framework. For the con­troller and its learning method, we use recurrent neu­
ral networks and Q-learning. Computer simulations
show that the controller succeeds in learning the opti­
mal policy for a task that causes perceptual aliasing.
In the simulation, information loss minimization
technique is shown to improve learning speed com­pared to the simple Elman network. However, we havealso found that, learning gets difficult in a task withsparse reinforcement signals. Further study is needed
to cope with this difficul� ·.

References

Figure 4: Variation of the redundant moves; Elman
network.
We have repeated the simulations using the
X-model network instead of the Elman network. Pa­
rameters used are the same as above. The resultis shown in Fig. 5. Convergence takes place in ashorter time. Also, it can be seen that convergence issmoother, as compared to the simple Elman network.
We have also examined learning with sparse rein­
forcement signals, as a more difficult learning task; thecontroller receives a reinforcement only after it gets to
the goal or if it bumps into doors. In this test, theconvergence has not been achieved correctly. The pol­icy discovered by the controller is suboptimal, i.e., it
includes one extra move to check the state of the lock.

(1] A.G.Barto, S.J.Bradtke, S.P.Singh: "Learning to Act
using Real-Time Dynamic Programming", Technical
Report, Dept. of Computer Science Univ. of Mas­
sachusetts, 1993.

(2] L. Chrisman, "Reinforcement Learning with Percep­
tual Aliasing: The Perceptual Distinctions Approach",Proc. AAA/'92, pp. 183-188, 1992

(3] A. McCallum, "Instance Based Utile Distinctions for

Reinforcement Learning", Proc. of 12th International
Machine Learning Conference, 1995

(4] J.L. Elman, "Finding Structure in Time", Cognitive
Science, Vol. 14, pp. 179-211, 1990

(5] L.Lin: "Self-Improving Reactive Agents Based On Re­
inforcement Learning, Planning and Teaching", Ma­chine Learning, Vol. 8, pp. 293-321, 1992

(6] I Noda "Neural Networks that Learn Symbolic andStructured Representation of Information", Researchesof the Electrotechnical Laboratory, No 968, Agency ofIndustrial Science and Technology, Japan, 1994

100

(7] A. Onat, H.Kita, and Y. Nishikawa, "Reinforcement
Learning of Dynamic Behavior by Using Recurrent
Neural Networks", Proc. World Conference on Neural
Networks, 1995

10w

(8] R.S.Sutton, "Learning to Predict by the Methods of

0.1

Temporal Differences", Machine Learning, Vol. 3, 1988

(9] C. Watkins and P. Dayan, "Q-learning",

0.01 '----...IIIL-"----'JL...1----'----L.J'----'-----''---'----'--�
0
100000 200000 300000 400000 500000 600000 700000 800000
Number of iterations

MachineLearning, Vol. 8, pp. 279-292, 1992

Figure 5: Variation of the redundant moves; theX-model (information loss minimization method).

(10] S.D. Whitehead, and D.H. Ballard, "Learning to Per­

- 101 -

ceive and Act by Trial and Error", Machine Learning,
Vol. 7, pp. 45-83, 1991

Building Ears for Robot:
Sound Localization and Separation
Jie Huangt, Noboru Ohnishitf and Noboru Sugie1ttBio-Mimetic Control Research Center, The Institute of Physical and Chemical Research (RIKEN)
3-8-31, Rokuban, Atsuta-ku, Nagoya, 456 Japan, E-mail: huang@nagoya.riken.go.jp
http://www.bmc.riken.go.jp/BM C/SENS/Huang/huang.html#Department of Information Engineering, Faculty of Engineering, Nagoya University
Furo-cho, Chikusa-ku, Nagoya, 464-01 Japan, E-mail: ohnishi@nuie.nagoya-u.ac.jp
fftFaculty of Science and Technology, Meijo University
1-501, Shiogamaguchi, Tenpaku-ku, Nagoya, 468 Japan, E-mail: sugie@nuie.nagoya-u.ac.jp

Abstract
This paper describes our research on robot audi­
tion, and introduces sound localization and separation
methods which mimic some effects in the human au­
ditory system such as the precedence effect and the
cocktail party effect. We implemented a DSP-based
real system and conducted experiments in guiding acamera, approaching a minimum path, localizing andseparating multiple sound sources in a real reverber­
ant environment. We demonstrated the system's abil­
ity and potential applications, as well as the need of
improvement and further studies.

1

INTRODUCTION

Audition is one of the most important senses usedby humans and animals to recognize their environ­ment. Sound localization and separation ability isparticularly important. Biological researches have re­vealed that the evolution of mammalian hearing is re­lated to the need to localize sound, and the evolution of
the localization acuity appears to be related to vision[3]. Thus sound localization acuity enables a mam­mal to direct its field of best vision to a sound source.
This ability is important for humans, as well. A robot,
designed to move around our living space and commu­
nicate with human beings, also must be equipped with
an auditory system capable of sound localization andseparation.
Compared to the vision system, audition has sev-

eral unique properties. It is all-directional. W hen asound source emits energe the sound fills the air, and asound receiver (microphone) receives the sound energy
from all directions. Some specialized cameras can also
receive an image from all directions, but still have toscan the total area to locate a specific object [8]. Au­
dition mixes the signals into a one-dimensional timeseries, making it easier to locate any urgent or emer­
gency situation. Audition requires no illumination,
enabling a robot to work in darkness or low light. Au­
dition also is not effected by obstacles, so a robot canperceive audition information from sources behind ob­
stacles. One example of this is localization of a soundsource outside of a room or around a corner. The
robot first localizes the sound source in the area of thedoor/ corner, and then travels to that point and listens
again, finally locating the sound source.
Few researches have been done untill very recently
in the field of robot audition. Brooks stressed the im­portance of a sound processing system for a " thinking"robot [1]. The authors of this paper have done baseresearches on sound localization and separation [5] [6][7], and found that a sound source can be localized in areverberant environment by mimicking the precedence
effect of the human auditory system. Concurrent mul­
tiple sound sources can also be localized correctly inthat environment by using a histogram method. We
also found that the perceptual sound segment group­
ing method, which mimicking the cocktail party effect,
has potential for sound separation in a reverberant en­
vironment.
In this paper, we briefly introduce the methods we- 102 -

have developed and discribe a robot system equippedwith audition, as well as vision and sonar system; wealso discribe some related experiments to demonstratepotential applications of the auditory system.

where O:J e and tie denote the amplitude and delay
time of the first echo, and r is time constant. If thesound (including echoes) is given as s(t), the maximum
possible echo at time to is estimated as:e e ch(to) = Max{ sp (t)ie ch(t)}, 0 < t < oo2

MODELING AND MIMICKING THE HUMAN
AUDITORY SYSTEM

Modeling and mimicking a biological system is anuseful way to implement a flexible and environment
adaptable system. The signal processing method usedby the auditory systems and brains of animals or hu­
mans is usually heuristic in the result of evolution.
A model or a mimicking method should not be con­structed using complex mathematical methods which
would not normally exist in a neural system, therefore
the methods we propose are simple, practical, and eas­
ily achieved.

2.1

Modeling the Precedence Effect

In a reverberant environment, sound mixed withechoes is perceived by our ears (microphones). The
spatial information in continuous portions of a soundis corrupted by echoes. The "precedence effect" sug­gests that the human auditory system can detect the
beginning of a sound and mask the subsequent por­tion of that sound [9], reflecting the adaptation of thehuman auditory system to reverberant environments.
A model of the precedence effect has been proposedby Zurek [11] . According to the model, the soundportion after an onset will be inhibited for a period
of about lOms. There is, however, no quantitive cri­
terion for onset detection. A model which can detect
an 'echo-free' onset has been proposed by the authors[7]. This model is based on the amplitude patternof a typical impulse response in a normal room. We
assume that reflections of an impulse sound will de­lay more than a certain time, which depends on the
distance from the sound source to walls, comparing
to the direct sound. It is because a sound source isusually distanced from walls; a nearby wall in almost
cases does not change the direction of the first reflec­tion sound very much. We also assume the strengthof reflections decreases exponentially over time.
The amplitude pattern of the reflections of an im­
pulse sound is described as:ie ch(t) ={0

O:Jee _ •-•1er,05,t<tJ e,t:2'.:t1e

(1)

(2)

where sp (t) = s(to - t). If the ratio s(t0)/e e ch(t0) is
small enough, signal s(to) will be masked by echoes ofthe preceding signal sp (t). Thus, the masking time ofthe precedence effect can be modeled as:s(t)- < Tthr
ee ch(t)(3)

where Tthr is the threshold ratio of the precedence. ef­
fe('t. A masking pattern for echo-free onset detectionis then obtained by ____s__{_tl_t > Tthr•
�Note that s(t) itself includes echoes and that ee ch(t)is therefore the estimation of echoes and their higher
order reflections. In onsets, however, there is no echo
and the ratio of s(t) and ee ch(t) will be the exact ratioof sound to echo.

2.2

Mimicking the Cocktail Party EffectA human's ability to separate a specific sound froma noisy environment is known as the "cocktail party
effect" [2]. It is also known as a binaural effect.
To mimic the cocktail party effect, sound is repre­
sented by segments of enhanced continuous local peaksin the time-frequency space. The onset of a segment is
its starting point and the offset is its endpoint. Can­didates of arrival te,mporal disparity (ATD) are cal­culated at echo-free onsets by a zero-crossi�g method(atcn = at+n')../C, where atcn are ATD candidates.
at is the real ATD, n is an integer, ').. is wavelength,
and C is sound velocity.) The number of candidates N
is calculated from the wavelength of each band and the
distance between the microphones D, i.e. N = Dj')...
Each candidate contributes to the ATD histogram by avalue of one Nth, and is also weighted by its amplitudelevel and sound to echo ratio. The ATD histograms ofdifferent microphone pairs are then integrated into an
azimuth histogram by a histogram mapping method
to resolve the anterior-posterior ambiguity.
The azimuth histogram is divided into blocks, eachhaving a peak and a dominant region. Because the di­rections of segments belonging to each block are close,
as the psychological proximity principle states, all ofthem are integrated into a single group. Sound seg­ments, which have no sufficient echo-free onset, are- 103 -

regrouped by the harmonic histograms which are cal­culated from grouped segments. Sound segments ofeach group are then synthesized together by the wave­
form overlapping method to form a sound from eachsource.

3

SYSTEM AND EXPERIMENTS

3.1

System Implementation

m.4•·············••

■ound
input

DSP unit

Figure 2: Sound Processing Units.

The sound processing portion of the robot is imple­
mented with three microphones as shown in Figure 1.
Each microphone is set at a vertex of an equilateral tri­
angle with a side length of 13.5cm, about as far apartas a human's ears. Since the microphones of current
system are not equipped with artificial pinna, ampli­
tude disparities are not sufficient as localization cue.
A second version of the robot is now being developed
and will have four microphones, each of which will be
attached with an artificial pinna to enhance amplitudedisparities.

Mobile Robot
ver.1

ramoteho•t•t, (the sampling time) mean operation of time delay,and d = i e ch(tn +i)f i ech(tn ) = e-t./r.
-----.---------.icomp.
s (t)subbandenvelope

resultsof onsetdetection

Figure 3: A real time DSP implementation for echo-free
onset detection.
We used a single subband (lkHz) system and per­formed this test in a normal room. It was difficult to
detect speech correctly by only a single subband, how­ever, sounds such as hands clapping, knocking on thedoor, or an transient voice could be detected well. The
localization was accurate within a few degrees and ro­bust against background noise and non-transient sound.
This system is very simple and can be used to alert hu­
mans to unexpected events or intrusions.

Mobile Robotver.2•·•··········•••

Figure 1: A robot equipped with ears (microphones).
The system of sound processing is shown in Figure
2. The DSP is used for real time processing and theremote computers for non-real time processing. The
robot is also equipped with sonar system for obstacle
avoidance, and camera system for visual processing.
3.2

Some Experiments and Results

3.2.1

Guiding a Camera to a Sounding Object

Real time processing is required to guide a camera toa sounding object. Figure 3 gives a fast algorithm forecho-free onset detection. The sound signal is first fil­tered and the envelope s(t) is calculated. The symbol'*' means operation of multiplying by a value, t Je and3.2.2

Minimum Path Approaching

We tested the previous system to localize a source
which was invisible to the robot. The robot was posi­tioned outside of a room, with the sound source (aspeaker emitting transient sound) positioned inside(Figure 4).
In this situation, the direction of the sound onset
depends on its minimum path in theoretically. In fact,
the system did tend to localize sound at the turn­ing point of the minimum path, the edge of the door.
However, due to the narrow corridor, there were manyechoes with directions differing slightly from the min­imum path sound. The localization accµracy in thecorridor was poorer than in the tests conducted in-- 104 -

!loom
!loom

e

voi

Figure 4: Robot navigation by sound localization.
side a room. After some approach and correction, therobot was able to find the turning point and then fi­
nally located the sound source inside the room [10].
3.2.3Localizing and Separating Multiple Sound
Sources

Localication experiments with multiple sound source
were conducted in a normal room and in an anechoic
chamber. The sound sources were set in two differ­ent positions. Sound source 1 was a recording of aradio weather forecast presented by a male announcer(speaker 1), and sound source 2 was a recording of aradio talk show with one male and one female host(speaker 2). Each sound lasted about 20 seconds. The
azimuth between sound source 1 and sound source 2
was about 38 degrees. The distance from the soundsources to the center of the microphone set was about
2.9m. For simplifing, sound sources are assumed to
be located in front side of the microphone set and two
microphones were used. The processing of this test
was non-real time and off-line, in contrast to the two
provious tests.
Figure 5 shows the results as ATD histograms ofpronunciations of "anoseikatsuo" (male) and "tyusin"(male). The left one was in an anechoic chamber, theright one in a normal room.. Both figures show twolarge peaks, each corresponding to the direction of asound source. While the localization errors in the ane­choic chamber were within a few degrees, the errors in
the normal room decayed into about double.
Figure 6 shows the waveforms before and after sepa­
ration, with the left one being in an anechoic chamberand right one in a normal room. The top plots in the
figures show the mixed sound before separation, andthe middle and bottom plots are of sound 1 and sound2 after separation.
The figures show that the sound sources are clearly
separated in the anechoic chamber. The results ob­
tained in the normal room are similar to the results
obtained in the anechoic chamber, despite the exis­
tence of strong reverberant sound. However, we canfind some distortions in the separated waveforms. The
distortions were partially caused by the reverberant
sound, which colors tembre and does not effect the per­ception very much. The distortions were also causedby the misgrouping of sound segments, which should
to be removed. Due to the misgrouping of sound seg­ments, the separation quality remained low. We needto develop a method for decreasing the misgrouping ofsound segments to improve the quality.

4

SUMMARY

We have developed a robot auditory system anddemonstrated its ability to localize and separate soundin a reverberant environment.. Our experiments in­volved guiding a canera to a sounding object, min­imum path approaching, and multiple sound source
localization and separation. We also have indicatedthat the quality of sound separation in a reverberant
environment remains low and requires improvement.
The potential applications are not limited to theabove-mentioned situations. Further studies, includ­ing sound understanding or recognition are also re­quired for building an auditory system in a robot. We
believe that the development of robot audition willcontribute to the human-robot communication and theenvironment recognition capability of a robot.
REFERENCES- 105 -

[1] Brooks R. A. and Stein L. A. (1993); Building
Brains for Bodies, MIT A.I. Memo No.1439.
[2] Cherry E. C. (1953), Some experiments on therecognition of speech with one and with two ears,
J. Acoust. Soc. Am., vol.25, pp.975-979.
[3] Heffner R. S. and Heffner B. E. (1992), Evolutionof Sound Localization in Mammals, in Webster
and Popper (Eds.), The Evolutionary Biology ofHearing, New York: Springer-Verlag, pp.691-715.
[4] Huang J., Ohnishi N. and Sugie N. (1988), Re­
search on a Sound Source Localization System

ATO hi•togram (/anoselka.tauo+tyuain/)g�-�-�--.....c.----'�-----oc--�-----.,

ATO histogram (/ano.aikatauo+tyuainl)

2.6,-,-----�------�-�---....-----...,

5

0

-0.4

-0.3

-0.2

-0.3

-0.1
0
0.1
Arrival Temporal Oispariti- (ms)

-0.2

-0.1
0
0.1
Arrival Temporal Diaparities (ma)Figure 5: ATD histograms (left in an anechoic chamber and right in a normal room). The two large peaks are contributedby sound source 1 sound source 2 respectively.

l 1 }--� -..�,• l•• i i]•-'.·�· ► .:.Jt} •· •(I �• � ;t;'... �j· I,:1•,-�,,< ...,

1

4
x 10

mixed eound (/anoselkatsuo+tyusln/)

4

x 10
2

mixed sound (/anoselk■tauo+tyusln/),�.:;;·�,�-:,•��-,��i�

-2
0

�

U

W1�•:

0.1

4
x 10

0

lime(s)

U

M

0

�

�

0.2

,11.
0.3
lime{•)

0.4

0.3
ame{s)

0.4

o.s

0.2

U

lme(s)

4

0.8

0

0.1

separated sound VanoHlkatsuol)

0.1

�

0.2

0.3

•me<•):

U

separated sound (/lyullnJ)

separated eound VtYuslnl)

4

X 10

0

U

0.4

M

�

0.5

0.8

M

�

separated aound (tanONllatauw)

o.s

0

0.8

�

�ei..

�

M

•me<•)

I

U

Figure 6: Waveforms before (first plot) and after (second and third plots) the separation (left one in an anechoic chamberand right one in a normal room).
Suggested by Living Orgamisms - Localizationof a Single Sound Source in an Echoic Envi­ronment -(Japanese), IEICE Trans. vol.J71-A,

rectional Image Sensor COPIS J. Robotics Soc.

Japan, vol.11, no.6, pp.868-874.
[9] Wallach H., Newman E. B. and Rosenzweig M.
R. (1949), The Precedence Effect in Sound Local­
ization, J. PsychoL Am., vol.62-3, pp.315-336.

no.10, pp.1780-1789.
[5] Huang J., Ohnishi N. and Sugie N. (1991), A

Multiple Sound Source Localization System Us­
ing Temporal Disparity Histograms(Japanese), J.

[10] Wang F., Ohnishi N. and Sugie N. (1994), Au­
tonomous Mobile Robot Localizing and Approach­ing Sound Sources, Proc. 12th Annual Conf.

Robotics Soc. Japan, vol.9, no.4, pp.409-414.
[6] Huang J., Ohnishi N. and Sugie N. (1995), A

Biomimetic System for Localization and Sepa­
ration of Multiple Sound Sources, IEEE Trans.

Instrumentation and Measurement, vol.44, no.3,
pp.733-738.

Robotics Soc. Japan, pp.1095-1096.
[11] Zurek P. M. (1993), The Precedence Effect, in
Yost and Gourevitch (Eds.), Directional Hearing,
New York: Springer-Verlag, pp.85-105.

[7] Huang J., Ohnishi N. and Sugie N. (1995),
Sound Separation Based on Perceptual Groupingof Sound Segments, Submitted for NOLTA'95.

[8] Nishizawa Y., Yagi Y., Yachida M. (1993), Gen­eration of Environmental Map and Estimation ofFree Space for a Mobile Robot Using Omnidi-- 106 -

Evaluation of Robust Control by Universal Learning Network
Masanao OHBAYASHI*,Kotaro HIRASAWA*, and Junichi MURATA** Dep. of Electrical Eng., Kyushu University, Higashiku, Fukuoka 812, JAPAN
Tel: +81-92-641-ll0l(ext.5330); Fax: +81-92-631-2790; E-ma.il:oba@scel.ee.kyushu-u.ac.jp
Abstract: Characteristics of control system design using Universal Learning Network(U.L.N.) are that a system
to be controlled and a controller are both constructed by U.L.N. and that the controller is best tuned throughlearning. U.L.N. has the same generalization ability as N.N.. So the controller constructed by U.L.N. is able tocontrol the system in a favorable way under the condition different from the condition of the control system atlearning stage. But stability can not be realized sufficiently.
In this paper, we propose a robust control method using U.L.N. and second order derivatives of U.L.N.. The
proposed method can realize better performance and robustness than the commonly used Neural Network. Robust
control considered here is defined as follows. Even though initial values of node outputs change from those atlearning,the control system is able to reduce its influence to other node outputs and can control the system ina preferable way as in the case of no variation. In order to realize such robust control, a new term concerning
the variation is added to a usual criterion function. And parameter variables are adjusted so as to minimize theabove mentioned criterion function using the second order derivatives of criterion function with respect to theparameters. Finally it is shown that the controller constructed by the proposed method works in an effective way
through a simulation study of a nonlinear crane system.
Keywords Robust control , Neural networks, Second order derivative

1. Introduction
Universal Learning Network(U.L.N.) and a comput­
ing method for its higher order derivatives have been
proposed,(1lwhich can be used as a fundametal tool in mod­elling and control of large-scale complicated systems suchas economic, social and living systems as well as industrial
plants.
Each nodes in U.L.N. is allowed to have any nonlin­
ear functions. In case of designing a control system us­
ing U.L.N., the system to be controlled and the controller
are both constructed by U.L.N., and the controller is best
tuned through learning to minimize a criterion function
which is assumed to be function of the target value of sys­
tem node output, actual value of system node output andoutput of the controller. U.L.N. has the same generaliza­tion ability as Neural Network(N.N.). So the controller
constructed by U.L.N. is able to control the system in afavorable way under the conditions different from those of
the control system at learning stage. But stability can not
be realized sufficiently under the conditions much different
from those at learning stage.
Robust control considered here belongs to the control
which is able to control the system stably under the con­ditions much different from those of the control system
at learning stage. The difference in conditions consideredhere is the difference between initial values of node out­
puts at control stage and those of node outputs at learning
stage. A robust control design method is proposed where
the parameters of the control system are tuned to mini­
mize a criterion function, using second order derivatives of

the criterion function with respect to parameters, which
consists of two terms,one is a usual criterion function andthe other is a new term which evaluates the influence of
the above differences for all node outouts of the system.
Finally it is shown that the controller constructed by the
proposed method works in an effective way through a sim­
ulaton study of a nonlinear crane system.

2. Structure of Universal Learning
Network
Basic structure of U.L.N. which consists of nonlinearly
operated nodes and branches that may have arbitrary timedelays is shown in Fig .1.

Fig.1 Structure of Universal Learning Network
Basic equation of U.L.N. is represented by Eq.(l):h(T;, t) = O;( {h' (T;, t)}, {rn(t)}, {u1(t)}, Pm(t)}),- 107 -

(1)

h(T;, t )h'(T;, t)rn ( t )
where
u,(t)
-X m (t)O;

Output of T; node at time t,
Input of T; node at time t,External input variable at time t,
Control variable at time t,
Parameter variable at time t,
Nonlinea.r function of T; node,

If T; node is one of the nodes connected with input side

whereofT; node,then,h'(T;, t) via a branch with time delay D;;

is represented by Eq.(2):h'(T;, t ) = h(T'i, t -D;;)

E = E( {h(T1,, s )}, {u,(s)}, {.Xm(s)})

Z

Ko

e Lo

Lo

Number of nodes of the system,
Number of sampling times.

R

T

Since h(T;, to - 1) does not depend on .\1(to), initialvalue of Eq.(6),Let a criterion function be written in Eq.(3):k E Ko(6)k = 1,2, .....,R,t = 1,2, .....,T,

Pi(T;, to -1,A1 (t0)) = 0, j = 1, 2 , ..... , R.

(3)

Set of nodes related with evaluation,(7)

4. Computation of Second Order
Derivative

Second order derivative of E with respect to parametervariables .\1(to), A2 (to) can be obtained by differentiating
Eq.{4) with respect to .\2 (to) ,Set of control variables relatedwith evaluation,

Set of sampling times related

JlE
dA1(to)d.\2 (to)with evaluation.

In the following chapters,a comput4lg method for deriva­
tives of criterion function E with respect to parameter vari­able Am (to) is presented,which is essential to design a ro­
1bust control system using U.L.N..1 1

3. Computation of First Order
Derivative
First order derivative of E with respect to parameter.\1(to) can be written in the form of Eq.(4),assuming t o to
be designated sampling time,='°' '°' [ d (�)LI LI

dA2(to)8E

Jlh(T1:,s)

IIEKo •ESodh(T1,, s )d.\1(to)+8h(T1:,s) dA1 (to)dA2 (to) ]+d.\2(to)d(a.\�fio>)(B)d.\:(t�(J�i�•t10 5 in Eq.(8) can be transformed into Eq.(9)by differentiating Eq.(5) with respect to A2 (to),8h( T1,,d(aX(TJ, t - �J•> ) dh(T;, t -D;1:)
Jlh(T1c, t )=dA1(to)dA2 (to)dA1( t o)d.\2 (to)
e
; J

L[

8h(T1,, t )J h(T;, t -D;1:)
8h(T;, t -D;1:) dA1(to)dA2(to) ]l+,n.f.f.

As••> and sx�fto) can be calculated easily from
Eq.(3),it is matter of importance to calculate��\c��j>can be transformed into Eq.(5).

"J'i1n>
1

0

:

(9)

di\ 2 (to)Putting Pi{T1c, t ,A1( to )) ="J\\i�Y,and P2(T1,, t , A1(to),A2 (to)) = ax:ct��I�� lto) , a8 in the case offirst order deriva­
tives, iterative equation of P2 by forward propagation canbe obtained from Eq.(9),
t

=

where
J

d(�)

+

P2(T1,, t ,A1(to),A2 (to)). 8h(T1,,d(
B j)al.CT; , t -i•
P1(T;, t -D;1:, .\ 1(to))
LI
d;\2 ( to )

'°' [
jEJ

Set of nodes connected with input side of T1, node.

"��g�•)>,

Putting Pi (T1,, t ,Al (to)) =iterative equationof Pi by forward propagation can e obtained from Eq{5).

- 108 -++

8h(T1,, t )8h(T;, t _ D;1,)d(�)dA2(to),.
P2(T;, t -D;1:, .\1( t o),;\2(to))]
.
.
{lO)

�f!

k = 1,2, .....,R,t = 1,2, ....., T.

P2(Ti, to - 1, A 1 (to), A2(to)) = 0,

'1)

{11)

j = 1,2, .....,R.

t
Sh!T; ·•-b;1,) ) d
1 '0• Eq.{10) can be calcu,m
JX 2 (to)JX 2 (to)lated by the computation of first order derivatives putting
Dh(T& , t )- �\<T V respectively
E- 8h(T;,t-D;1,)' E 1t
to.
Tt
d2 h T ,tdh,
Subst·t
1 ut·mg dA 1 t o ' .>. 1 t o A 2 to obt amed from Eq. (6),2(7) and Eq.{10),(11) respectively into Eq.{8), dA i (t� jf2(to)can be calculated.
d(11i.(T1,, t)( ��(T

can be carried out
Now,computation of d�� andby making use of the first and the second order derivatives
in chapter.3,.4.

-II;; >
Putting A (to) = A , -II;; is able to be computed using< Computation of1

Eq.(4),(6)m< Computation of ��! >First order derivative of EH with respect to A m can beobtained by differentiating Eq.(13) with respect to A m ,dEH
dAm

5. Robust Control Method

5.1 criterion for suppressing changes of nodes
outputs

E is a usual criterion function, EH is a new term whichtakes charge of suppressing changes of node outputs of thesystem caused by the changes of particular node outputs
at time t1.Then a new criterion function L is defined asfollows:{12)

L= E+EH,

=

H
C

SN

I: I: (I:•=S1 rER,•ER,,

dh(Tr,s)b.h(T;, t1))dh(T;, t )

2

1

Now,

compute the,,,t i 7r:7?fl-'::-'<';,t---�f!

are needed in order to(1] Computation of ;; ::::.�:)Putting E=h(Tr,s), A 1 (to)=h(T;, ti) and making use of
the first order derivative in chapter.3, Eq.(16),(17) can beobtained,dh(T,, s)= Pi (Tr,s, h(T;, t ))dh(T;, t i )

(13)

1

{16)

: Set of nodes related with perturbation,
Rp: Set of nodes related with suppression,
R.
S;(i=l,··•,N) : Set of sampling times relatedwith suppression,: coefficient.

,
Tr f' b.h(T;, t 1 ) means the change of Tr node out,, 1
put in the case of a change of '.11 node output at t , namelyb.h(T;, t ). Eq.(13) is the sum of those squared.
dh

1

1

5.2.Learning Algorithm,, 1

Putting E=h(Tr,s), A1(to)=h(T;, t 1 ), A2(to) = Am andmaking use of the second order derivative in chapter.4,
Eq.{18),(19) can be obtained,

The aim of the optimization learning in U.L.N. usedhere is to search for the parameters which make the above
criterion function L minimal. (From now on the parametervariables are considered to be time invariant.)
The parameter variables in order to minimize Eq.{12)should be calculated by a gradient method.
dL(14)
A m -7 Am
dAm
where

.
d 2 h r,•12] Computation Of dh . tT d.>. m

dL
dE
dEH
= dAm
+
dAm
dAm ,

'Y > 0 : coefficient.

- 109 -

d2h(Tr,s)dh('.11, t1)dAm

d( 6/i{ ,t-b·•).
�.\m 1, canof p1 m E q. (19) ',The coeffic1ent
be calculated by computing the first order derivative of.
8h(T&, t )usmg Eq. (4) ' (6)·E = Bh(T;,t-D;•)
8h(T&,t))Nonlinear) cranesystem

6. Numerical Example
6.1 Controled System

The controled system is a nonlinear crane system. A
position of the crane stand, an angle between the rope andvertical line and a position of the load are represented asx, 8, l respectively. Then the nonlinear crane system is
described as follows:
d 2x
dt2

d 20
dt2
2

d£

dt2

=

=

!i.._
_ mg _ D +Gdx
8
ud
+
M ,
M dt
M

M+m
D+Gdx
G
+
Ud
�gB
� dt
lM

h(T2, t) = x(t),h(Ts, t) = £(t),h(T3, t) = 0(t),h(T6, t) = l(t),Eq.(20) can be transformed into discrete type equations
as follows:h(T1, t)=a 11h(T1, t) + a 2 1h(T2, t),h(T2 , t)=

a 22 h(T2, i) + a32 h(T3, i) + b1ud(t),h(T4, t)=h(Ta, i)h(T2, i)a 24--- + a34--h(Ts , i)h(T5, i)h(Ta, t)h(Ts, t)h(Ti;, t)===a3ah(Ta, i) + a 4ah(T4, i),

1 : 1 sampling delay time

Fig .2 Recurrent type control model of a nonlinear cranesystem using Universal Learning Network
M 40[kg], D=300[kg/sec], G=700[N/V], m 2[kg],g 9.8(m/sec 2 ],G m 0.98[N/VJ, C=0.42[ kg/sec] are used,
and reference of moving the crane stand(xre /) is l(m], ref­erence of rolling up the load(l re /) is 0.5[m].
When the parameter variables are tuned through learn­ing, initial values of node outputs of the system are set up
as follows:===

where, Ud, Um are input voltage to a motor for moving
the crane stand and to a motor for rolling up the load
respectively,and C, G, G m , D, M, m are appropriate systemparameters.
Putting as follows,h(T1, t) = x(t),h(T4, t) = 0(t),����r�ller

6.2 Criterion Function(20)

Gm
c+G m di
Um
+
m
dt
m

=

=:;:;::===:+;::===:;::: )

(21)

=

h(Ts, 0)=1.0, h(T;, 0)==l, 2, 3, 4, 6).

(22)

In numerical example, a change of the initial position of
the load ,namely 6h(Ts, 0) is assumed.
The criterion E to achieve the desired dynamics ofthe system and EH to achieve the suppressing the per­turbation of the system caused by the change of the initial
position of the load are defined respectively as follows,

E

=

2
2
½[:Z:{Q11(xr•J-h(T1, s )) }+Q12(h(T2, t1 ))
•ESo

+

b
+a 44h(T4, t)- i � ud(i),h(Ts, t,L

•ESo

{Q13(h(T3,s)) +Q14(h(T4,s)) }
2

•ES,o

a 55h(Ts, i) + a 6sh(T6, i),+�6h(T6, i) + b2um (t).

L

•ESo

Where i = t - 1.

A recurrent type control model of the nonlinear cranesystem using U.L.N. is shown in Fig .2. Each control in­
put Ud, U m is constructed by two control nodes respectively,one is the node with linear function,the other is the nodewith tanh function. (All branches have one sampling timedelay.)

0.0 (i={R 1(h(T1,s))2

2+ R2(h(T9,s))2 }]2� dh(Tr, s )6h(Ts, O ))CH LJ ( d T
h( s , 0)
rER,

(23)
(24)

where, So : Set of all sampling times, t i : final time,
1.0, R1
R2 = 0.001 , r : 1, 2, 3, 4, 5, 6.
Qi6
1 s = 1.0 [sec] in Eq.(24). A
CH = 100, 6h(Ts, 0)
total number of learning is five thousand.
Using these values, two cases have been learned, one,
named caseE, is the case of using the criterion function E,
the other, named caseL, is the case of using the criterion
function L.

Qu

~

=

~

=

6.3 Simulation Results- 110 -

Relation between number of learning and values of thecriterion functions E and L at learning is shown in
Fig.3. Using the control parameter variables obtained atlearning number equals to five thousand, control simula­tions have been carried out for i(0) = 1, 2, 3, 4, • • • andire/ = 0.05, 1, 2, · · ·. In case of i(0) = 1,i re/ = 0.5, that is,the conditions at control stage are the same as those atlearning stage, control results are shown in Fig.4. ( i(0)means the initial position of the load at control stage.)
Both result of caseE and caseL are almost the same. In
Fig.5, system stable operation area is shown as the trian­gle , for example, in case of Fig.5(a), such an area is shownas shadow one. That area means that the system operates
stably, when initial value of i changes between 0.1 and 7.0,and reference value ofi is fixed to 0.5, and other conditions
at control stage are the same as those at learning stage.
Fig.5 shows that in case of perturbation of i(0), the areain Fig.5(b) is larger than one in Fig.5(a). Conversely incase of perturbation of ire / , the area in Fig.5(b) is nar­rower than one in Fig.5(a). It is thought this result has
been caused by the suppression of perturbation of i( 0). In
Fig.6, control results of Fig.5(b)(i( 0) = 20.0,i re / = 0.5)
show that the system operates stably and gets its goal.
Thus in the event of a large change of node outputs,theeffect of EH to stability of the system appears remarkably.

:E
:L

e

6

Fig.4 Results of control in case of i( 0) = 1,i re / = 0.5
27.0
20.0

6.0

6.0

�

if

1

�

'o

1

3.5

1.0
0.5
0.35
0.1
0.05
- Path of Learning ( 1.0 -, 0.5)

0.5
'0.05

!l

�

i
�

3

]

g,

1.0
0.1
0

f
i;'

�

t

'o

i;'

7. Conclusion
A robust control method using forward propagation
U.L.N. is proposed,and through simulation study,it has
been shown that the proposed method is very useful. In
numerical example, the controller obtained makes the sys­
tem operate stably in the event of i(0) = 20. (i( 0) = 20
is twenty times as large as h(Ts, 0) = 1 in Eq.( 22)). Thismeans that the proposed method is the method for prac­
tical use.
In this paper we proposed the robust control method
for a change of nodes outputs. The robust control method
for perturbation of reference, system parameter variables
and external inputs can be derived in the same way as in
the cas·e of a change of node outputs.

time(scc)

g,

~

(b)
(a)
Fig.5 System stable areas (a): caseE, (b): caseL,
when i(0) and ire/ change.

i..,20 _,_I

-

I

I

\

j 10 --- xe----- e

i

I

\

\i

Q)

�
�
0

800

I

-,i:t
0

2

-

I

4

6

Fig.6 Control results in case of caseL(i( 0)20,ire/ = 0.5)--l.

'1160()=

References
200
4000
Leaming Number (N)Fig.3 Values of criterion function E, L

(lj K.Hiraswa,M,Ohbayashi,M,Koga:Forward
Net­
Universal
Learning
Propagation
work ,Memoirs of the Faculty of Engineering,Kyushu
University,Vol.55,No3,pp225-234,1995(2) M.Ohbayashi,K.Hirasawa,J.Murata:Robust Control by
Universal Learning Network,Proc. of 1995 Korea Au­
tomatic Control Conference, pp123-126,1995- 111 -

Chaos Control
on Multi-Branch Universal Learning Network
Masaru KOGA, Kotaro HIRASAWA, Junichi MURATA and Masanao OHBAYASHI
Department of Electrical Engineering, Kyushu University
6-10-1 Hakozaki, Higashi-kn, Fukuoka 812, Japan
Tel : +81-92-641-1101 (Ext 5329), Fax : +81-92-631-2790
E-mail : masa@scel.ee.kyushu-u.ac.jp
ABSTRACTIn.this paper, the chaos control already proposed is extended to multi-branch Universal Learning
Network (U.L.N.) for the purpose of investigating the influence of increasing the number of branches
and changing delay times. Controlling chaotic phenomena on U.L.N. can be realized by changing
Lyapunov Number of U.L.N., which is accomplished by adjusting U.L.N. parameters so as tominimize a criterion function that is the difference between the desired Lyapunov Number and its
actual value. Simulations are done for the single and multi-branch U.L.N.. Based on the simulation
results of the single, so-called the ordinary recurrent neural network, and multi-branch U.L.N.,
suitable U.L.N. parameters and structures have been made clear that can realize generation anddie-out of chaotic phenomena more efficiently.

1. Introduction
The authors already proposed a Chaos Control
Method on Universal Learning Network(U.L.N.)[1].
The method can control chaotic phenomena on such
U.L.N. that have bounded functional nodes.
The method can control generation and die-out ofchaotic phenomena on U.L.N. by changing the Lya­
punov Number of U.L.N. to positive and negativevalues respectively.
From the simulation, it has been certified that thecontrolling chaotic phenomena can be easily realizedon the single-branch U.L.N., that is, the ordinary re­current neural network; no special terms are needed,
whereas chaotic neural network model proposed by
Aihara has a few special terms[3]. And the mecha­
nism of controlling chaotic phenomena has been in­vestigated; some insight has been obtained into thecondition that is necessary for controlling chaotic
phenomena on U.L.N ..
However, a recurrent neural network is only a par­ticular type of U.L.N.. U.L.N. can represent moregeneral and more realistic large-scale complicatedsystems which may have multiple interconnections
between their nodes and various delay times.
In this paper, the chaos control is extended tomulti-branch U.L.N. for the purpose of investigating
the influence of increasing the number of branches
and changing delay times.

Simulations are done for the single and multi­
branch U.L.N.. The results show that, in the case ofgenerating chaotic phenomena, the Lyapunov Num­ber can be more quickly changed to the positive de­sired value on the multi-branch U.L.N. than on the
single-branch U.L.N., so-called the ordinary recur­
rent neural network, that is, generating chaotic phe­nomena is easier on the multi-branch U.L.N.. The
results also show that the larger the time delay is,
the more difficult changing the Lyapunov Number
to desired positive Lyapunov Number is, in otherwords, generating chaotic phenomena becomes moredifficult.
Based on the simulation results of the single andthe multi-branch U.L.N., suitable U.L.N. parame­ters and structures e.g., the number of branches anddelay times, have been made clear that can realize
generation and die-out of chaotic phenomena more
efficiently.

2. Multi-Branch
Network

Universal

Learning

U.L.N.[2] is briefly described in this section.
U.L.N. is a network which consists of arbitrary
nonlinear functional nodes and arbitrary delay ele­ments between the nodes, and is aimed to model andcontrol various large-scale complicated systems likeindustrial plants as well as economical, social and- 112 -

life phenomena. Structure of a multi-branch U.L.N.
is shown in Fig.1, and basic equation of multi-branch
U.L.N. is represented as follows,=h(Ti,t)O;({h'(T;,t)}, Pm (t)}, {rm (t)},{umu(t)}1(j E R) (t ET),(1)

T; node's output atsampling time t(j E R),T; node's input at sampling time t,m parameter at sampling time th(Tj,t)h'(Tj,t)A m (t)(m EM),

m' external input
at sampling time t(m' EM'),m" control variable
at sampling time t(m" E M"),
T; node's nonlinear function,
set of nodes,
set of sampling times,
set of parameters,
set of external inputs,
set of control variables.

O;
R
T
M

M'

M"

3. Chaos Control Method
The method already proposed can control chaotic
phenomena on such U.L.N. that have bounded func­tion nodes[l]. The method is that generation of
chaotic phenomena is realized by changing Lyapunov
Number of U.L.N. to positive value, and die-out of
the chaotic phenomena is realized by changing the
Lyapunov Number to negative one. The gradientmethod utilizes second order derivatives of U.L.N.
because Lyapunov Number of U.L.N. is calculated
by first order derivatives.
3.1. Lyapunov Number and its Derivatives
Lyapunov Number ofU.L.N. is defined as follows (seeFig.2),
Li

=

l.im

1

L{

N-+oo NS

rE R

s

2

dh(Tr, S(n+l))} ],dh(Ti,t)(4)N

S(n+l) - S(n),number of S,

R

number of whole network nodes.

Network Dynamics
���
Fig.I Structure of Multi-Branch Universal Learning
NetworkDi;(p) (p E IJ) means the delay time from Ti to
T; nodes. The input to T; node h'(T;,t) is expressed
as follows,h(T;,t - D;;(p)),
h'(Tj,t)(2)
IJ
set of branches from T; to Ti .

=Criterion function of a system is represented asfollows,(3)
E({h(Tr,s)},{ur1 (s')}),E
r E R 0 , r' E R�, s, s' E S 0 ,Ro, R�, So:
set of nodes, set of control variables,and set of sampling instances related
to the criterion function.

Fig.2 Explanation of Lyapunov Number
The criterion function EL is defined as follows,
EL

=

L�

= L(L� - Lr)2 ,rER
desired values of L r .

(5)

Parameter Am is adjusted to minimize criterion
function by the gradient method,dEL,
Am +- Am - 'Y
{6)�m

Learning of U.L.N. is to search for the parameters,
Am(t) and u,(t), which minimize the criterion func­tion by the gradient method.

- 113 -dEL
o dL r= 2 "(L
L., r _ L r ) dA .
dAm
m
rER

(7)

If dL;/d>.. m can be calculated, parameter >.. m canbe learned. From equation {4), dL;jd>.. m is obtained
as follows,dL;d>.. m1.
1= N:TooNSN-1�S (n+1)-ln=Ot=s(n)

�x� [ �

(n+i)) d h(Tr, +l))
I: ER dh(Tr,dh(T,,t)dh(T,,t r
d>.m
2

S

r

S

2
dh(Tr, S(n+l))
"""
{
}
� rER
dh(T,,t)ld{8h(T1,, t +D;1,(p))}{lh(T; ,t)d>.. m� [ � { dh(Td, ') {/(T.d, ')}] + aE' '
=�
�
a>..
d>..
t

t 'ET

•

m

t

D(>.. m ) : Set of nodes including >.. m ,If dh(Tr,s)/dh(T;, t), d2 h(Tr,s)/dh(T;, t)d>.. m arecalculated, >.. m is able to be learned.
dh(Tr,s)/dh(T;, t) is calculated as follows,dh(Tr ,s)
= 81 (T;, t) 'dh(T;, t)dED(>- m )8'(Tj , t) =

m

{13)

EE

kEK pEJKah(Tk, t +Dik(P)) ,8 (T.k, t +D·{
ik(p))}ah(Tj , t)

{8)

aE'
ah(Ti , t)

+-� .

{14)

3.2. Random Search Method

d2 h(Tr ,s)/dh(T;, t)d>.. m is calculated as follows,d2 h(Tr,s)dh(T·,, t)>.. m= 81 (Ti, ),t(10)

It is difficult to optimize the parameters successfully
by using only the gradient method because Lyapunov
Number is very sensitive to changes in the parameters
and the criterion function is not convex. Therefore, arandom search method is combined with the gradientmethod.
During iteration of the gradient based learning, if
Lyapunov Number is not improved over its previ­
ous value, the previous parameters are randomly anduniformly changed within a fixed range. And new
Lyapunov Number is calculated with new parame­
ters. The Lyapunov Number has to be calculated
repeatedly by changing parameters randomly until itgets better than the previous Lyapunov Number.
If the Lyapunov Number is better than the previ­ous one, learning is resumed by the gradient method
again. If not, the parameters should be chosen whichgive the best Lyapunov Number among those triedin the random search.

4. Simulations

Fig.3 Multi-Branch U.L.N.

{12)
In equation {11), by putting
E' ah(Tk, t + Dj k)/ah{Tj , t),d{ ah(Tk, t +Di k)/ah(1J, t)} /d>.. m is calculated as fol­
lows,

=

Simulations are carried out to generate chaotic
phenomena on multi-branch U.L.N. shown in Fig.3.
Sigmoid functions are adopted as U.L.N. node func­
tions. The equation describing the network is asfollows,- 114 -

0.8

I
. . 7,,___.____ _.____, __j .

. ·'

0.2
0
1

-5 f-+--+----+------1'---l--l.l--4-�-l

0

400

200

I

Leaming Iteration(a)h(T_2, t)00h(T_ 1 , t)(b)

Fig.4 Generation of chaos on Single-Branch U.L.N.

0.8-:.o.s
(')

1

f- 0.40.2
01-5 t-+--+----+----1,---..,__-+-l
0

0.6

400

200

h(T_2, t)

Learning Iteration
(a)

0

0

h(T_ 1 , t)(b)

Fig.5 Generation of chaos on Multi-Branch U.L.N.

(15)

h(Ti , t)Desired Lyapunov Number is set to positive num­
ber three.

(16)
4.1. One-sampling delay time

iEI pEIJ/

I

IJ

sigmoid functions,
set of nodes connected to
input side of Tj node,
set of branches,weight, thresholdand slope of sigmoid functions.

There are three branches between the nodes (seeFig.3), and weights, thresholds and slopes of thesigmoid functions are optimized.

At first, every delay time is set to one-sampling time.
Learning curve of Lyapunov Number and network
dynamics after 500 epochs of both single-branch
U.L.N. and multi-branch U.L.N. are respectively
shown in Fig.4(a), (b) and Fig.5(a), (b ). Fig.4(a)shows that it takes 200 epochs to change the Lya­
punov Number to positive value in the single-branch
U.L.N.. However, from Fig.5(a), in the multi-branch
U.L.N., it takes less than 50 epochs to exceed zero
value of Lyapunov Number.
Furthermore, in the single-branch U.L.N., the Lya-

- 115 -

0.8
�0.6
Ml
l-0.4

."E

0.2
0
1

,51--1--+-----'l----+--+-+---+--+-----t--i
1000
0 0h(T_2, t)h(T_1 , t)
Learning Iteration(b)(a)
Fig.6 Results of D1 network (one sampling delay time is dominant)

0

!

Ii

,.._
0.8

Qt-••

>

�0.6

r

(')I
l-0.4

0.2
0
1

-5
0

1000
Leaming Iteration

h(T_2, t)00h(T_1 , t)

(b)

(a)

Fig. 7 Results of D2 network (two sampling delay time is dominant)-5 t--t--t--+---+-+---+--+----,f--.-1

0

1000
Leaming Iteration

h(T_2, t)h(T_1 , t)

(b)

(a)

Fig.8 Results of Da network (three sampling delay time is dominant)- 116 -

punov Number cannot be reached to the desired value
within 500 epochs. On the other hand, the Lyapunov
Number can be rearched near the desired value atabout over 150 epochs.
Accordingly, in case that every time delay is set
to one-sampling time, the Lyapunov Number can be
more quickly changed to the positive desired value on
the multi-branch U.L.N. than on the single-branch
U.L.N., that means generating chaotic phenomena iseasier.
The reason can be explained as follows; changing
the Lyapunov Number to positive value is caused bymaking 81 in equation (9) a large value. From theequation (9), it is clear that the larger the number ofbranches connecting to output side of T; node is, theeasier changing 81 to a large value is. Thus generating
chaotic phenomena becomes easier.

4.2. Various delay times
Consider the case where the value of delay time isvaried in the multi-branch U.L.N..
The multi-branch U.L.N. has 27 branches. Threecases are selected for assigning delay times to thesebranches. In the first case, delay time D;;
1 isdominant, where 21 out of the 27 delay times areequally set to one and each of the other 6 is randomly
set to either one, two or three. In other cases, delay
time D;; = 2 and 3 are dominant, respectively. The
multi-branch U.L.N. obtained in these three cases arereferred to as D1, D2 and D3 networks, respectively.
Learning curves of Lyapunov Number and the net­
work dynamics after 1500 learning epochs of the
D1, D2 and D3 networks are respectively shown in
Fig.6{ a), {b }, Fig. 7{ a), (b) and Fig.8( a), (b ).
The largest Lyapunov Number of the Di, D2 andD3 networks are respectively 3.0, 2.0 and 1.5. The re­sults show that the larger the time delay is, the moredifficult changing the Lyapunov Number to desired
positive Lyapunov Number is, in other words, gen­erating chaotic phenomena becomes more difficult.
The reason is that changing Lyapunov Number tolarge value is more difficult because the influence ofmagnification of the dynamics by the perturbation
cannot appear in the range S in the case of the large
value of delay time.

=

The following insights are obtained:
1. in the case of generating chaotic phenomena, theLyapunov Number can be more quickly changed
to the positive desired value on the multi-branch
U.L.N. than on the single-branch U.L.N., so­called ordinary recurrent neural network, thatis, generating chaotic phenomena is easier,2. the larger the time delay is, the more difficultchanging the Lyapunov Number to desired posi­tive Lyapunov Number is, in other words, gener­
ating chaotic phenomena becomes more difficult.
Based on the simulation results of the single andmulti-branch U.L.N., suitable U.L.N. parameters andstructures, e.g., the number of branches and delaytime, have been made clear that can realize gener­
ation and die-out of chaotic phenomena more effi­
ciently.

References[1] Masaru.KOGA, Kotaro IDRASAWA, Junichi
MURATA and Masanao OHBAYASID, "Chaos
Control using Second Order Derivatives of Uni­
versal Learning Network", ICNN'95, Vol.3, 12871292. 1995.
[2] Kotaro IDRASAWA, Junichi MURATA and
Masanao OHBAYASHI, "Universal Learning
Network and Computation of its Higher Order
Derivatives", ICNN'95, Vol.3, 1273--1277. 1995.
[3] K. Aihara, T. takabe and M. Toyoda, "Chaotic
Neural Networks", Phys.Lett.A, 144, 6, 7, pp.
333-340. 1990.

5. Conclusions
In this paper, the chaos control is extended to multi­
branch U.L.N. for the purpose of investigating theinfluence of increasing the number of branches andchanging delay times.
Simulations are carried out to generate chaotic
phenomena on the single and multi-branch U.L.N..

- 117 -

A Step Toward Human - Robot Cooperative System

Hironori Saita
Fukuoka Inst. of Tech.,
Fukuoka, Japan

Masaru Ishii,
Fukuoka Inst. of Tech.,
Fukuoka, Japan
Abstract
In near future, we will have to live withmany mach i n e s such as home robots,
hospital robots and so on. Us er friendly
c ommu nication between man and t h o s emachines is very important for cooperative
system utilizing their features and abilities.
As an example of human - robot cooperative
system, we propose an intuitive approach to
robot teac hing method with multi-media
tools.

Keywords: Robot Teaching, Multimedia,
Interface, Human - Robot Cooperation

Macintosh

1. Int roduction

Fig.I. A robot teaching system

In the conventional research on an intelligent
robot system, many autonomous robots weremade for carrying out all tasks automatically bythemselves. These robot systems do not consider
a role of a man as an operator except for master­slave type robot systems.
In near future, we will have to live with many
working machines such as home robots andhospital robots and so on. User friendly
communication between man and those machines
is very important for human - robot cooperative
system that utilizes their features and abilities inorder to accomplish their common operationaltasks.
Realization of such cooperative system is
eagerly required for future robot systems. In thissystem, various communicating functions
between man and robot system are necessary forexchanging their command s, advises andcomments about their common operational goals
with user friendly interfaces.
As a first step for constructing above system,

Mouserobot teaching system with multi-media tools is
described in this paper. Clicking buttons on thedisplay by a mouse device mean controlling arobot system. After we make many primitives ofrobot's operations using this method, we cancreate any necessary tasks easily by a macro
operation that combined those primitives. Also,
we can select a wanted task from robot macro
operation's database. One of our aims is to makesuch a simple command for executing
complicated robot tasks.
This robot teaching method using multimedia
tools provides a foundation for realizing user
friendly human - robot cooperative system.

2. A new robot teaching meto dTeaching is essential in order for robots to carryout various tasks with several kinds of moving
sequences. Robot teaching by showing or guiding
is called playback control. Playback control- 118 -

method is widely used in many industries and agreat deal of operator time and labor is required.
Teaching box, however, is not so flexible toexpand for various teaching commands easily andmust be improved in aspects of user interfaces.
We have developed a structual robot teaching
method using multi-media tools in a Macintosh
computer as showed in Fig.I.
In our system, we use a robot system(Mitsubishi - movemaser EX) with five degrees
of freedom. The robot has six movable parts:
waist, shou Ider, elbow, !ist-pitch, !ist-roll andhand (open or close). The robot has twenty-four
position's control commands, four hand's control
commands and six reading commands using a
RS232C cable.
Hyper card system of a Macintosh computer is amulti-media information exchangeable basesoftware. It has fields for texts, pictures, button
for managing cards and sounds. Hyper-talk is anobject oriented type programming language ofhyper card system for stack control. Its scripts
are defined as an attribute of a button. It is
designed conveniently to transform a commandfrom users. For example, an objective of scripts
written in hyper talk is carried out by selecting abutton with a mouse device.
It is necessary for robot control to send a serial
signal of control commands of a robot systemfrom hyper card system. Hyper-talk has not sucha sending signal command and thus XCMD:
extended command of hyper talk, is used. XCMD
is a machine language image compiled by C
program and added to hyper card for high speed
processing such as signal sending. We use a send­
signal XCMD for sending serial data.
Ideally, operator gives an outline of task to arobot system and then robot system must carryout the delicate and complicated works. In
practical situation, man and a robot system haveto cooperate each other with their features andabilities.
At first, we must teach a primitive operation of arobot system. We select a registration card stackshown in Fig.2. For moving five movable joints
such as waist, shoulder, elbow, list's pitch andlist's roll of a robot system, we slide each lever ofsmall windows on the display of a Macintosh
computer. These slide operations are carried out
by a mouse device. In this registration card, we
can teach robot's primitive operations such as
point to point positioning.

,····-·-·

Shoulder·············-····-·--·-·····+90

saYe operation
0

O Grip Close
npf"fo"7
Pressuret...:..,:;:._J
Rngle-coordinates
Waist
0
0

0
0
0
0

ShoulderElbow

List Pitch 1
List Rollerase all
nest

589
300
0
0

Waist

I

90
-

0

!�waiat20ierase position

go to main card
Quit

+90

0

I

mo11e

List Roll'iitJ I

-90

ristRoll

0

+90

0

--

I mo11eFig.2. Robot control by sliding levers on the display of a Macintosh computer- 119 -

3. Robot macro operation

system to receive these data from a drive unit by :signal handling external command.
In this way, � robot operation's sequence i:created as one card stack by this teaching method
A registered task is performed automatically b:clicking a button of above operation on the mai1
menu card.

Actually, a complicated task of a robot system
consists of several primitive robot operations. So
we can replace a robot complex work with asequence of these primitive operations. In robot
macro card in Fig.3, we can give a name to a
macro operation. Left lower part of this stackshows macro operation's names such as put-into­
video-deck, put- back-video- tape for example.
Operation put-into-video-deck means that an
operator teaches 3D positions and orientation of arobot system for putting a tape into video-deck by
our teaching method. As soon as we push a put­
into-video-deck button, robot system plays backthe same operation quickly.
As this system has a hierarchy structure, we canmake a higher macro operation such as handling a
Video Tape operation by selecting and collecting
optimal operations from lower macro ones. This
operation has manipulating sequences as follows:
grasping a video tape, inserting it and so on. So
we can make a simple command for such acomplicated task.
Also, robot moving speed can be set and standard
pose button means all zero state of all movableparts. In right upper side of this stack, current
positions and angles of each joint are displayed.
We can get 3D positions and orientation of a robot

Robot Macr o Card
IOperation
Name: Video Tape--- - -grasp-video-tapeinsert-video-tapeput-into-video-deckpull-out-of-video-deck
put-back-video-tapejd

I l Traa,rer!Readout

Clear

l San

I

Current Position

4. Sensor and voice control of a robot
In the human - robot cooperative system, man ';commands are expected more simple and a robo
have to carry out complicated tasks utilizing theifeatures and abilities in order to accomplish theicommon operational tasks. Our system make use ooptical sensors attached at a robot hand to find ou
and grasp an object automatically (Fig.4). Afte
we move a robot 's hand near an obeject , robo
system can execute fine operations using senso
feedback.
In an intelligent cooperative system, variou;
functions communicating between a man and robo
systems are necessary for exchanging theicommands, advises and comments about thei
operational goals with user friendly interfaces.
In this system, we use voice control utilities fo
realizing robot operations by voice command:
instead of selecting buttons of the hyper-car<system. Commands' name must be registered b�

0I

.�§i§
Angle

Coord.

b

i-11

L--

Optical Sensor,Trour.r �

Macro Operations
put-into-video-deck
put-back-video-tapegrasp-video-tapeinsert-video-tapepull-out-of-video-deckgrasp-juice-biny0---ila,!

B

I I I I 1,

I I I Il

(Standard pose)

(

.,

Speed
[I]
Tr■Hfer

Wait�
Tr■■sfer

Optical Sensor){iFig.3. Layout of a robot macro operation card

- 120 -

Fig.4. Optical sensors attached at a hand

operator's voice in advance.
Besides this operator's voice command, several
voice responses from a robot system and a teachingsystem are necessary as follows:(I) Guidance about teaching sequence,(2) Report of moving action's results,(3) Reports of current robot's states(4) Help requirement(5) Questions and so on.

5. Experimental results

Fig.6. Display layout of our system

In Fig.5, we show a complete view of anexperimental robot teaching system composing of arobot system, a Macintosh computer, a voicenavigator and a monitoring camera set.
Multimedia display of our system is shown in
Fig.6. In left side of this display, hyper card stack
is seen for controlling and teaching a robot system.
Monitoring scene of this operation is in right up
side. As an experiment example, we make macro
operations about handling a video ta pe fromteaching stacks mentioned above. An experimental
task is to grasp a video tape, insert and push it intoa video deck. Next, these systems pull a video tapefrom a deck and put it back to the shelf. As shownin Fig.7, this system can insert a video tape into a
deck successfully.

6. Conclusion
This paper describes an intuitive robot teaching
method with multimedia tools for user friendly

Fig.7. Inserting operationinterfaces. Simple example of robot teaching iscarried out 1/3 teaching time in comparison withthe conventional method because of using only amouse device for all robot teaching.
The goal of our system is making human - robot
cooperative system for overcoming common
operational tasks with mutual conversation andcollaboration. Sharing and learning ability ofrobot system by operator's advice are expected tosolve common problems in our system.

Acknowledgment
The authors would like to thank to the Language andInformation Laboratory of Fukuoka Inst. of Tech. forsupporting the research.
References[l] M.,lshii,"A robot teaching method using Hyper
Card system," Proc. of IEEE Workshop on Robotand Human Communication, pp.410-412, 1992.

Fig.5. Overview of an experimental system[2] S.,Hirai and T.,Sato,"Motion Understanding forWorld Model Management of Telerobot," Proc. ofIEEE/RSJ Int. Conf. on Intelligent Robots andSystem (IROS), pp.124-131, 1989.

- 121 -

Proposal of Decentralized Consensus-making Mechanisms
Based on Immune System"'Application to a Behavior Arbitration of an Autonomous Mobile Robot"'

Akio ISHIGURO, Yuji WATANABE, Toshiyuki KONDO and Yoshiki UCHIKAWA
Dept. of Information Electronics, School of Eng., Nagoya University
Furo-cho, Chikusa-ku, Nagoya, 464-01, Japan
Phone: +81-52-789-3167, Fax: +81-52-789-3166
E-mail: { ishiguro/ yuji/ kon/ uchikawa }@bioele.nuee.nagoya-u.ac.jp

Abstract
In the field of recent artificial intelligent (Al)technology, much attention has been focused onreactive planning systems (e.g. behavior-based Al,
emergent computation, and so on). However, thearbitration between competence modules (simple
behavior/action) in these approaches encounters itsdifficulty. On the other hand, according to the recent
studies in the biological fields, it has been clarified thatthe immune system works as self-preservation systemagainst dynamically changing environments through theinteraction among lymphocytes/ antibodies. Therefore,
the immune system would be expected to provide a new
methodology suitable for dynamic problems, forexample, robotics.
In this paper, we propose a new, decentralired
consensus-making system inspiredfrom the biologicalimmune system. And we apply our proposed method toaction arbitration for an autonomous mobile robot as apractical example. We furthe rmore try to evolve theproposed artificial immune system by adjusting
affinities among antibodies through genetic process. To
confirm the validity of our method, we carry out somesimulations.
Key words: immune network, decentralired consensus­
making network, autonomous mobile robot, action
arbitration, bottom-up manner

1. Introduction
In the field of artificial intellige nt (Al) technology,
conve ntional approaches based on a functional
decomposition, leading to a so-called "sense-model­
plan-act' cycle have been criticized on many drawbacks
over the last decade . Typical criticisms are that thesesystems show brittleness under environmental changes,
and required much computation tilll"' for mapping
obtained sensory inputs onto complex internal models
before action can be taken. The refore, in recent year
much attention has been focused on reactive planningsystems (e.g. behavior-based AI , new AI , eme rgent

computation, animat approach, and so on), which havealready been demonstrated robustness and flexibility
against dynamically changing world. In theseapproaches, intelligence is regarded to emerge through adynamic and/or instantane ous process resulting frommutual interactions among compelence modules(simple behavior/action) [4][12][ 13]. However, thearbitration between compelence modules in theseapproaches encounters its difficulty.
On the other hand, immune system has various
interesting features such as immunological memory,
immunological tolerance, patter n recognition, and so onviewe d from the engineering standpoint. In addition to
the above , recent studies on immunology have clarified
that the immune system does not just detect andeliminate the not-self materials called antigen such as
virus, cancer cells and so on, rather plays important roles
to maintain its own system against dynamicallychanging environments through the interaction among
lymphocytes/antibodies. Therefore, immune system
would be expected to provide a new methodology
suitable fordynamic problem dealing withunknown/hostile environments rather than static
problem.
Based on the above facts, we have been trying toengineer methods of the immune system, and apply torobotics and so on[l]-[3]. We expect that there would be
an interesting AI technique suitable for dynamicallychanging environment by imitating the immune system
in living organisms. In this pape r, we propose a new,decentralized
inference/conse nsus-making
systeminspired from the biological immune system. To confirm
the feasibility our proposed m ethod, we apply to action
arbitration for an autonomous mobile robots as apractical example. Moreover, we try to evolve theproposed artificial immune system equipped with themobile robot by adjusting affinities among antibodies
through gene tic process. Finally, the validity of our
proposed methods are confirmed by carrying out somesimulations.

- 122 -

2. Overview of biological immune system
The basic components of the immune system arelymphocytes that are mainly classified into two types,
namely B-lymphocytes and T-lymphocy tes. B­
lymphocytes are the cells produc ed from bone marrow.
Roughly 107 distinct types of B-lymphocytes arecontained in a human body, each of which has distinct
chemical structure and produces "Y" shaped antibodies
from its surfaces. The antibody recognizes specificantigen, which are the foreign material that invade
l iving creature. In other words, the response can be
re garded just as a key and lock relationship (see Fig. I).
To cope with continuously changing enviromrent,
living systems possess enormous repertoire ofantibodies in advance. On the other hand, T­lymphocytes are the cells produced from thymus, andthey generally perform to regulate the production of
antibodies from B-lymphocytes as outside circuits of
B-lymphocyte network (idiotypic network) discussed
later.
/r

attaches the antibody 2 to its surface, through theparatope P2. Viewed from the standpoint of antibody 2,
the idiotope Idl of antibody 1 works simultaneously as
an antigen, As a result, the B-lymphocytes 1 with
antibody 1 are suppressed by antibody 2. On the other
hand, antibody 3 stimulates antibody 1 since theidiotope Id3 of antibody 3 works as an antigen viewed
from antibody 1. In this way, these stimulation andsuppression chains among antibodies form a large-scaled
chain loop and works as a self and not-self recognizer.
Again, the heart of Jerne's idea is that the self-nonself
recognition in the immune system is carried out at
system level Therefore, the immune system would
provide a new parallel distributed processing.
� Epitope�.

i/paratope

�, �i,tParatope

JI

•�011

(, Anhoody 2
If

P2A,( Antibody 1.,•···

Id3

antibody)!�>epitope:•·

P3
Antibody 3____,. Stimulation........ .r · Suppression

Fig.2: Jerne's idiotypic network hypothesis.

Fig. I: Structure of antigen and antibody.
For the sake of convenience in the following
explanation, we furthermore introduce several
terminology in immunology. The portion on the antigen
recognized by the antibody is called epitope (antigen
determinant), and the one on the antibody that recognizes
the corresponding antigen determinant is called
paratope. Recent studies on immunology have clarified
that each type of antibody has also its specific antigen
determinant called idiotope (see Fig. I).
Based on this fact, NKJerne, a Danish
immunologist, proposed a remarkable hypothesis which
he has called the "idiotypic network hypothesis",
sometimes called "immune network hwothe sis"[5]-[9].
This network hypothesis is the concept that
antibodiesnyrnphocytes are not just isolated, namely
they are communicating to each other among different
species of antibodies/lymphocytes. This idea of Jerne's
is schematically shown in Fig.2. The idiotope Idl ofantibody 1 stimulate s the B-lymphocyte 2, which

3. Action selection and Immune system
Recently, fatal limits of conventional artificial
intelligence approaches based on the functional
decomposition have been pointed out. Therefore, muchattention has foc used on the behavioral decomposition
approaches. These behavior-based AI techniquesdemonstrate their robustness and flexibility against
hostile environments, however, arbitration among
competence modules (simple behavior/action) arises
difficult problems. Namely, it is difficult to select acompetence module suitable for the current situationand goals. To overcome this problem, several methods
have been proposed. Among them, Maes proposed aninteresting method; behavior network system, underwhich an action suitable for the current situation and thegiven goals emerges as the result of the interaction
among competence modules[ 1O] [ 11].
Against the above-mentioned stream of works, we- 123 -

approach to this problem from the immunologicalstandpoint, namely using immme network. Fig.3
schematically shows the action selection system forautonomous mobile robot and the immune network
system.
As shown in this figure, current situations, forexample, dis1ance and direction to the obstacle detectedby ins�lled sensors work as antigens, and a prepared
competence module can be regarded as an antibody (orB-lymphocyte), while interaction between modules isrepresented by stimulation and suppression between
antibodies.
The basic concept of our method is that the immune
system equipped with the autonomous mobile robot
selects the competence module (antibody) which issuitable for the current situation (antigen).

-- .....

Dynamically Changing
Environment

I(;.

Mobile Robot

Obs1acle

Destination

Competence Module(Simple Behavior)(a) An autonomous mobile robot with an action selection
mechanism.
Antibody

shown in Fig.4.
In this simulated environment, there are following
three kinds of objects: l) predators, 2) obstacles and 3)
foods. And we assume that prespecified quantity of
initial energy is given to the immunobot at the beginningof each simulation, and energy level zero means thedeath of immunobot. For quantitative evaluation we use
the following assumption:(1) if the immunobot moves, it consumes energy E,,,.
(2) if the immmobot is captured by a predator, it
consumes energy E.,.
(3) if the immunobot collides with an obstacle, it loses
energy E,,.
(4) if the immunobot picks up a food, it obtains energyEr-{5) Obtaining food behavior does not emerge if theenergy level of the immunobot is high.(this is forpreventing over-charging)
The predators attack the immunobot if they detect theimmunobot within the prespecified detectable range.
Fig.4 also indicates the structure of the immunobot usedin the simulated environment. We equipped thisimmunobot with external and internal detectors.
External detectors are ins1alled in eight direction, andeach can detect the existence of predators, obstacles andfoods. And we assume that each detector can also detectthe dis1ance to the objects in three degrees, so to speak:near, mid and far. On the other hand, internal sensor
detects the energy level. For simplicity, we assume thatthe immunobot can move toward the above eight
directions. The aim of immmobot is to survive as longas possible. To realize this aim, the immunobot mustselect an action (antibody) sui1able for the current
detected situation (antigen).
front

@¼ c

(b) Immune network.
Fig.3: Basic concept of our proposed method.
�

4. Proposed consensus-making network
based on biological immune system

.. r�
��;;
Lr

obstacle\fqth. . \,.

}lf

left back '--...}_� •..

_, , (};)· · ;_/

detectorright back--.bac;� detectable range

4.1 Problems.
For convenience, we dub the autonomous mobile
robot with the artificial immune system "immunobot".
To verify the ability of our proposed immune systemagainst hostile environments quantitatively, in this study
we use simulated environment for the immunobot as

Fig.4: Simulated environment.
4.2 Definition of antibodies- 124 -As described earlier, the detected current situation

and prepared competence module work as antigen andantibody, respectively. To make the immunobot select asuitable antibody against the current antigen, we must
look carefully into the definition of the antibodies.
Moreover, we should notice that our immunologicalarbitration mechanism selects an antibody in a bottom­
up manner by communicating among antibodies. To
realize the above requiremen1S, we defined thedescription of antibodies as follows. As mentioned in
section 2, the identity of a specific antibody is generally
detennined by the structure (e.g. shapes) of its paratope
and idiotope . Fig.5 depic1S our proposed definition of
antibodies.
As in the figure, we assign desirable condition to theparatope, and disallowed conditions to the idiotope,
respectively. In addition, we divide the structure ofparatope and idiotope into three potions: objects,
direction and distance.

--- Definition of Antibody
Parato e

Action_.::::;::.::::.i:::..._

Ciilldirl(in�� . ,,__..;......;......;.......1· ·. ·

Desirable

4.3 Dynamics
For adequate selection of antibodies, we assign one
state variable called concentration to each antibody. In
this study, selection ofantibodies is simply carried out ina winner-take-all fashion. Narrely, only one antibody isallowed to activate and act its corresponding behavior to
the world if its concentration surpasses the prespecifie dthreshold. The concentration of i-th antibody, which isdenoted by ai, is calculated as follows:(1)
Where N denotes the number of antibodies, and mu andmi denote matching ratios between antibody j andantibody i and between antibody i and antigen,respectively. The first and second terms ofthe righthandside denote the stimulation and suppression from otherantibodies, respectively. The third term represents thestimulation from the antigen, and the forlh term thenatural death (see Fig.7).

.. : ... -�l�o� �f_P�i_-a!o� �n� !dlo!o� .....

' I : ; : tffiiillib' 4> �00 ](]j1si\andttl :-: :- Piedatoi-< ·: ·: ·: Friflir :- : : ·: ·: · i-ileai-< ·: · :-: ::: : ot=1�:::::::: :�:::::::: :�l'�:::::. . Energy.Le-.el .
,· ·.·.•.·.·'R"n1u·. . . ·.•.•. . "N. . . . .
le,n.�l<;.·• . 01le,•.•.•,.·.•.·.

antibodyj

Fig.5: Definition of antibodies.

4'::mul�tion

For the ease of understanding, an example of the
prepared antibody is listed in Fig.6. This antibody isactivated if the immunobot detects the food in the front
direction and mid range, and makes the immunobot
move forward to pick it up. However, ifa predator exists
in front and near/mid range, or if a food exists in nearrange, this antibody might hesitate to be activated.
Another antibodies are designed in the same way.
Note that typical inference systems (e.g. fuzzy
inference) adopt a condition-action description
framework. Against these conventional systems, ourproposed system uses condition-ac tion-condition
fashion. This description method provides decentralized
dynamic inference in a bottom-up manner.
Idiotope
ActionParatope
jFoodlFront IMid.m-�:���--:1 Predator Front Near, Mid.
Near
#
Food

Fig.6: An example of the prepared antibody.

ai : concentration of antibody i
mi,i : matching ratio between antibody i andj
Fig.7: Dynamics.

4A Basic mechanism of theconsensus-making networks

proposed

Ne xt, we explain the fundamental mechanism of ourimmunological consensus-making networks in detail.
For the ease of understanding, we assume that theimmunobot is placed in the situation shown in Fig.8 as
an example. In this situation, four antigens listed in thesame figure invade into the immunobot's interior, andthe listed five antibodies mainly participate in theconsensus-making. For example, antibody 1 means that
if the immunobot detects food within far range in front
direction, it is permitted to move forward; However, ifit
recognizes other food within nearer range/ predator in
front direction/ current energy level is high, this antibody
would give wa y to other antibodies, whose paratopes- 125 -

represent such situations.
Suppose that the current energy leve l is high. In thissituation, antibodies 1, 2, 3 and 5 are stimulated by the
antigen. Note that each of the above antibodies increases
its conc entration just according to its related antigen.
Howe ver, to select an appropriate antibody,· communication among the antibodies is indispensable.
In the figure, the interactions between the antibodies areindicated by arrows. And the solid arrow represents
stronger interaction than the shade one. Through theseinteractions, concentration of each antibody varies.
Consequently, antibody 5 will have the highest
concentration, and then permitted to be activated.
In the case the robot's current energy level is low,antibody 3 tends to be selected in the same way.
Front

Example 1: energy level High__... stirmhtDl---t suppression

antibody 3

disallowance is a significant parameter, since thisparameter determines the affinity (i.e. strength) betwe enantibodies (se e Fig.9). In spire of this, it is very difficult
to determine the magnitude of this parameter a priori
The refore, in this study, we tried to adjust the affinitie s
among antibodies using genetic algorithms. In theevolution process, we used the sum of resultant life timeand remaining energy leve l of the immunobot as fitness
function:

fitness = life time + remaining energy level(2)

Simulation conditions are listed in Table 1. Fig.l 0
represents the simulation results when we applied theimmune networks before and after evolution to 10
different randomly-arranged unknown environments
and recorded survival ratio. From the figure, it isunderslood that the immunobot obtained more
appropriate consensus-making system through thegenetic process. Fig.11 shows the anothe r comparisonof before and after evolution. Obviously, the resuk
obtained after evolution is superior to that obtained
before evolution in the area of the number of times
captured by predators, the number of times obtaining
food, and the number of collisions.
-----Affinity between antibodies :J•---­M(iantibody j,K

ant ibodv i
M(iJ)I F.F.l'ar I move.F t#,W:M '<"�:iii

F.R.Near tum R90

�a_nt_ ibod_,:...y_k___�

1-M(i,k,. P.F.Near tum 180

Mi#!# 1.....

Example2: energy level Low

M(i,\._ !��:::,:�e.# ! P.#.Near

Fig.9: Affinities among antibodies.
Table 1: Simulation conditions.

move.# P.#.Nmr
utibcxly 5

population

uniform crossover

mutation rate

20%

crossover rate

Fig.8: Examples of consensus-making networks bycommunicating among antibodies.

As described before, idiolopes denote disallowed
conditions. However, we should notice that there should
be various disallowance among idiolopes. For theappropriate selection of antibodies, degree of- 126 -

5-50%

maximum life time1000 steps# of antibodies

91

magnitude of affinity

5. Evolution of affinities among antibodies

20

crossover# of predators# of obstacles# of foods

0-1.0
5

10
10

!

B Before Evolution

engineering point of view.

1111 After Evolution

Acknowledgments

This research was supported in part by a grant fromthe Japanese Ministry of Education (No. 07243208).

References

2

3

4

5

6

7

Environment [No.]

8

9

10

Fig.10: Simulation results (survival ratio).

20

i
i

10

'cl

40
3

o�

.,

1

0

B

--"'"' " ' "' :, J
zI
5

0

1

0

0
oOS
�o
{ btaillll'o o�\lfl\""1

Fig.11: Quantitative comparison between before andafter evolutionary processes.

6. Conclusions and Further WorkIn this paper, we proposed a new decentralized
inference/consensus-making network based on the
biological immune system and confirmed the usefulnessof our proposed system by applying to a action
arbitration for autonomous mobile robots under hostileenvironments. And we determined pseudo-optimalvalues of the affinities among prepared antibodies byusing genetic algorithms.
Since this study is still in a rudimentary stage ofinvestigation, we designed antibodies a priori in a top­
down manner. We are now investigating to construct theimmune system of the immunobot in a bottom-up
manner by introducing gene recombination and meta­
dynamics function. Moreover, we are also trying toendow the immunobot with an on-line learning ability
by introducing T-lymphocytes and cytokines from the[1] A. Ishiguro, Y. Watanabe and Y. Uchikawa, "Fault
Diagnosis of Plant Systems Using Immune
Networks", in Proc. of 1994 IEEE International
Conference on Multisensor Fusion and Integration
for Intelligent Systems, pp. 34-42 (1994).[2] A. Ishiguro, S. Ichikawa and Y. Uchikawa, "A Gait
Acquisition of 6-Legged Walking Robot Using
Immune Networks", Journal ofRobotics Society ofJapan, Vol. 13, No. 3, pp. 125-128 (1995) (inJapanese), also in Proc. of /ROS '94, Vol. 2, pp.
1034-1041 (1994)[3] A. Ishiguro, Y. Watanabe and Y. Uchikawa, "An
Immunological Approach to Dynamic Behavior
Control for Autonomous Mobile Robots", in Proc.
of/ROS '95, Vol. 1, pp. 495-500 (1995)[4] D.Cliff, et al.(eds), From Animal to Animat3, MIT
Press (1994)[5] H.Fujita and K.Aihara, "A distributed surveillanceand protection system in living organisms", Tmns.
on IEE Japan, Vol. 107-C, No. 11, pp. 1042-1048(1987) (in Japanese)[6] J.DFarmer, N.HPackard and A.S.Perelson, ''The
immune system, adaptation, and machine learning",
Physica 22D, pp. 187-204 (1986)[7] N.KJeme, "The immune system", Scientific
American, Vol. 229, No. 1, pp. 52-60 (1973)[8] N.KJeme, ''The generative grammar of theimmme system", EMBO Journal, Vol. 4, No. 4(1985)[9] N.KJeme, "Idiotypic networks and otherpreconceived ideas", Immunological Rev., Vol. 79,
pp. 5-24 (1984)[10] P.Maes, ''The dynamic action selection", Proc. of
IJCAI-89, pp. 991-997 (1989)[11] P.Maes, "Situated agent can have goals",
Designing Autonomous Agents, pp. 49-70, MIT
Press (1991)[ 12] R.Brooks, "A Robust Layered Control System for a
Mobile Robot'', IEEE Journal of R&A, Vol. 2, No.
1, pp. 14-23 (1986)[13] R.Brooks, "Intelligence without reason", Proc. of/JCAI-91, pp. 569-595 (1991)- 127 -A Method of Gait Coordination of Hexapod Robots
Using Immune Networks
A.lshiguro, S.Kuboshiki, S.lchikawa and Y.Uchikawa.
Dept. of Information electronics, School of Eng., Nagoya University
Furo-cho, Chikusa-ku, Nagoya 464-01
Abstract
Biological information processing systems can be
regarred as one of the ultimate recentralized systems,
and have been expected to provide various fruitful ideas
to engineering fields. Among these systems, theimmune system plays an important role to cope withdynamically changing environments by constructing
self-nonself recognition networks among different
species of antibodies, and has a lot of interesting
features such as learning, self-organizing abilities andso on, viewed from engineering standpoints. However,it has not yet been applied to engineering fields so far.
Therefore in this study, we pay close attention to theimmune system. We propose a new interpretation ofroles of antibodies in terms of self-assertion andsubordination and apply this idea to a gait coordination
problem of a hexapod robot as a practical example.

above facts, we have been trying to construct
engineering morels of the immune system, and apply
them to robotics and so forth [l]-[3].
In this paper, we propose a rescription method of
antibodies based on the interpretation in terms of self­
assertion and subordination and apply this idea to a gait
coordination of hexapod walking robots as a practicalexample. We take a quite different way fromconventional methods by paying close attention toreactions between antibodies to realize a successful
walking patterns. To confirm the validity of our
proposed method, some simulations are carried out.

Key words: idiotypic network, self-assertion,
subordination, genetic algorithms, hexapod robot

The main task of the immune system is to detect an:leliminate the non-self materials called the "antigens"such as virus and cancer cells and so on which comefrom insire and outsire of the living organisms. The
basic components of the immune system arelymphocytes and these are mainly classified into two
types, called B-lymphocytes and T-lymphocytes(Fig. I).
B-lymphocytes are the cells produced from the bonemarrow. Each type of lymphocyte produces "Y" shaped
protein called the "antibodies" from· its surfaces if
specific antigens invade.
On the other hand, T-lymphocytes are the cells
produced from the thymus, and are classified into threedifferent types; suppresser T-lyrnphocytes, helper T­
iyrnphocytes and killer T-lyrnphocytes. Among theseT-lyrnphocytes, suppresser and helper T-lyrnphocytes
perform to regulate the prodiction of antibodies from
B-lyrnphocytes.
The antibody produced from the B-lymphocyte has a
portion called the "paratope" which recognizes thecorresponding antigen determinant called the "epitope".
The paratope reacts to the epitope in a way similar to alock and key relationship, as a result of the interaction,
the antigen is neutralized.

1 Introduction
Recently, systems have become increasingly
complex and large scale. As a result, conventional,
centralized control methochlogy has reached its limit.
To overcome this problem, it has become necessary toturn to developing decentralized processing systems.
In a sense, biological systems in living organisms
can be regarred as the ultimate in distributed
information processing systems, and are expected toprovire various feasible ireas to engineering fields.
These systems can be mainly classified into thefollowing four subsystems: 1) brain-nervous system,2) genetic system, 3) endocrine system, and4) immune
system. Among these systems, brain-nervous andgenetic systems have already been applied toengineering fields by moreling as neural networks andgenetic algorithms, respectively, and they have been
wirely used in various fields. The other two systems,
however, have not been applied to engineering fields
notwithstanding their important roles, viewed fromengineering standpoints.
Among these two systems, the immune system
works to protect living organisms and cope withdynamically changing environments against countless
unknown materials such as virus and so on. This ability
is just what we need for autonomous mobile robots.
Therefore, we expect that the immune system would
provide a novel information processing methodology toengineering fields, especially robotics. Based on the

2 Immune networks
2.1 Structure of the immune system

2.2 Idiotypic network hypothesis
Recent studies on immunology have clarified thateach type of antibody also has its own specific antigen
determinant called the"idiotope" (Fig. I). Based on thisfact, N.K.Jerne, who is an immunologist, proposed aremarkable hypothesis called the "idiotypic network
hypothesis"[4]-[5].This network hypothesis, which isfundamentally based on clonal selection theory[6],- 128 -

"'

increased by own paratope and decreased by own
i<iotope. Regar<ing the concentration as the regree ofself-assertion, an antibody asserts itself by own
paratope and suborcinates to others by own idiotope.
That is, an antibody has two complementary, in otherwords Janus-faced, parts.
In this method, we attempt to apply this irea to a
gait control problem.

paratope

idiotope

epitopegen¢tlil/�i�{��iiji¢ie•�z::H-•

a

3. Gait control for a multi-legged
walking robot

Fig. I: Structure of immune system.

employs the concept that antibo<ies are not simply
isolated, rather, they communicate with each otherthrough their paratopes and i<iotopes. This irea ofJerne's is schematically shown in Fig.2. The idiotope
Idi of antibody i (Abi) stimulates the B-lymphocyte i ­i, which attaches the antibody i-1 (Abi-1) to its surface,through the paratope Pi- I. Viewed from the standpointof Abi-1, the idiotope Idi of Abi works simultaneously
as an antigen. As a result, the B-lymphocytes i with
Abi are suppressed by Abi-1. On the other hand,antibody i+ l (Abi+1) stimulates Abi since the idiotope
Idi+ l of Abi+ 1 works as an antigen for Abi. In this way,these stimulation and suppression chains among
antibodies form the large closed chain loop whichworks as a self and non-self recognizer. The heart ofJerne's irea is that the self-nonself recognition in the
immune system is carried out at system level.

�-�
�'
- u

·�

ui�0{,(fri+l,......

P.

g
•..v

II.,

d

.1 .••
�

l<l.1

ldi

Ab·
'

Abi+t

Recently, multi-legged walking robots have been
expected to expand working fieldc; of robots since they
can walk on rough terrain such as another planet and soon. Therefore, much attention has been paid torevelopment of their control methodc;. In unstructured
environments, we must consider the situation that legsof the walking robot are sometimes cisturbed. As aresult, the robot often can not walk forward and/or fall
chwn. To avoid such situations, techniques that
automatically maintain adeq.Jate leg coordination arehighly recpired. Note that this problem also provires
one of good examples of self-organizing problems,since each leg must coordinate its movement adequately
to maintain reliable locomotion using information
about the states of other legs.
So far various control methodc; for gait control ofmulti-legged walking robots have been proposed [7]­[ 13]. In contrast to the aforementioned stream of works,
we utilize the dynamics of immune networks to obtainreliable gait patterns. As a rucimentary stage ofinvestigation, we take a hexapod walking robot as apractical example (see Fig.3), and assume that thisrobot walks maintaining static stability (i.e. duty ratio> 0.5).

1,

Ab: antibody
P: paratope
Id: idiotope
stimulation·········► suppression-+

Fig.3: Hexapod walking robot.

Fig.2: Idiotypic network hypothesis.

2.3 Interpretation of roles of paratope andidiotope
As rescribed above, antibodies interact with eachother through their paratopes and idotopes. Now,
suppose that the idiotope of antibody i and the paratopeof antibody j are interacting. In this case, theconcentration of antibody i will be decreased we tosuppression from the paratope of antibody j, whereas
the concentration of antibody j will be increased as aconseq.ience of stimulation from the idiotope ofantibody i. Looking at the interaction in a different way,
we can say that the concentration of an antibody is

4 Im:munological . approach to a gait
coordination problem
4.1 Description of antibodies
Based on the interpretation described in section 2. 3,
we suppose that self-assertion correspondc; to thereqiests of phase change to other legs, while
subordination correspondc; to the obedence to recpests
from other legs. Based on these ireas, we consirer the
structure of antibodies as follows. We assign the stateof "own leg" to idiotope and the states of "other legs",
which the own leg wants to eliminate, to paratope- 129 -

(Fig.4). In this study, we assume that the state of a leg
can be categorized as either 'support' or 'transfer'.
Support means that a leg is on the ground whiletransfer signifies off the ground Explaining our
proposed description of antibodies more concretely, theantibody shown in Fig.4 means that RI support
requests L2 transfer and R2 transfer to chang theirphases. Each leg prodlces antibodies, and they form
networks through paratopes and idiotopes. We expectthat this immune system works to maintain suitablegaits by eliminating the states of legs which disturb
successful walking.

, ,ff : •�jl;/:�����m��,u.-•••�•-••••.. •••�•,.�••••••••••••-•-ao•oouu�-�-••�•n•-•H••/., antibody

pexample of antibody

I

..,,,

Id

(2)

5 Simulation results

LL2.tra11$!¢r}l{i transf�t] Ri support j

4.2 Algorithms
In this study, we assume that B-cells, which exist ineach leg, generate antibodies when the robot falls down.
The concentration of the generated antibodies decreases
as the concerned leg approaches its limit position
because we suppose that the leg in such a state cannot
make a request to others so strong! y. The concentration
changes through the stimulation and suppression. The
leg state changes on the spot if the concentration of theantibody concerned with the leg becomes lower than the
pre-specified threshold level. In ad.titian, other legs
move a certain distance in the walking direction.
The concentration of antibodies is retermined by adifferential equation as follow:dB·-1 =( "'i.,o,-f(B· )- I,a .. f(B· )- k )f(B·) + supply• (I)
. 7]
]
. JI
I
. t
J
J

For simplicity, we use the following assumptions
for GA-search:( I )All the affinities between antibodies are the same.
(2)Requests from legs are symmetric about thelongitudinal body axis.
(3 )No leg makes complementary requests to a same leg.
From these assumptions, we represent the requests
by a one-dimensional chromosome of length 30, andassign one of the following values into each locus:
1 (request to a stance leg), -1 (request to a swing leg),O(no request). For the evaluation of chromosomes, we
use the falling ratio as fitness.
fitness = duration offalling down
pre-specified test duration

Fig.4: Description of antibody.

dt

4.3 Decision of antibodies using GA

J

where B; renotes the concentration of antibody i. G;Jrepresents the affinity between the paratope of antibody
i and the idiotope of antibody j. k andsupp/.v; determine
the natural reath and supply from the bone marrow,respectively. f means a squashing function such as
sigmoid function. In this e<J.Iation, the first term
renotes re<J.Iests to others and second term is req.iests
from others. The third and fourth terms renote thenatural death and the supply from B-cells, respectively.
Summarizing the main point of our idea, each leg
has a B-cell and each B-cell generates antibodies when
the robot falls oown. Antibodies are interacting witheach other through networks and the phase change iscarried out using the concentration of antibodies. From
this, we can see that each B-cell must generate adequate
antibodies for successful gait patterns. So, in this study,
we retermine the rescription of the paratopes of eachantibody using Genetic Algorithms.

Fig.5 shows the transition of average and best
fitness under the condition of duty ratio 0.6. From thisgraph, we can see that the chromosome at the 20th
generation realizes an adxpate gait pattern. Fig.6
shows a set of antibodies retermined by recoding theobtained best chromosome. This set of antibodies is an
example, and we have confirmed that there are several
sets of antibodies to produce a successful gait.
Next, in orrer to verify the robustness of theobtained immune network, we forcibly fixed the
movement of each leg for a certain dlration when the
robot was walking successfully. Fig.7 illustrates thetransition of falling ratio in the case of each leg. In thisfigure, the disturbance was inputted at the 20th time
step. From these results, we can see that the robot
accp.1ired a suitable gait again after the input of
disturbance.
Fig.8 illustrates the convergence rate to suitablegaits when varying the duty ratio. Note that thisimmune system was obtained unrer the condition ofdlty ratio 0.6. From these results, we can see that the
robot can walk successfully even if the duty ratio drops
to about 0.55.

- 130 -

0.6
0.6

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0.0

0.0

20

Fig.5: Transition of fitness.

RI

problem and continued that the robot can acquire a
suitable gait and maintain it in spite of the input of
disturbance and change of the duty ratio.
In the future, we will construct method, to achieve
suitable gaits based on on-line learning, without using
genetic algorithms, and attempt when one leg breaks
oown. In adlition, we will experimentally verify thefeasibility of our method using a real walking robot.

Id!ff

References[ l ]S. Ichikawa, A. Ishiguro and Y. Uchikawa, "A Gait
Accpisition of 6-Legged Walking Robot Using
Immune Networks", Journal of Robotics Society ofJapan, Vol. 13, No. 3, pp. 125-128 (1995) (inJapanese), also in Proc. of IROS '94, Vol. 2, pp.
1034-1041 (1994)[2]A. Ishiguro, Y. Watanabe and Y. Uchikawa, "Fault
Diagnosis ofPlant Systems Using Immune Networks",
in Proc. of 1994 IEEE International Conference on
Multisensor Fusion and Integration for Intelligent
Systems, pp. 34-42 (1994)[3]A. Ishiguro, Y. Watanabe and Y. Uchikawa, "An
Immunological Approach to Dynamic Behavior
Control for Autonomous Mobile Robots", to appear in
Proc. IROS '95 (1995)[4]N. K. Jeme, "The immune system", Scientific
American, Vol. 229, No. 1, pp. 52-60 (1973)[5]N. K. Jeme, "Idiotypic networks and otherpreconceived ideas", Immunological Rev., Vol. 79, pp.
5-24 (1984)[6]F. M. Burnet, "The clonal selection theory ofacquired immunity", Cambridge University Press(l 959)[7]R. D. Beer, "Intelligence as adaptive behavior: An
experiment in computational neuropathology",
Academic press (1990)[8]R. A. Brooks, "A robot that walks: emergent
behavior from a carefully evolved network", NeuralComputation, Vol. 1, pp. 253-262 (1989)[9]H. Yuasa and M. Ito, "Coordination of many
oscillators and generation of locomotory patterns",
Biological Cybernetics, Vol. 63, pp. 177-184(1990)[l0]P. Maes and R. Brooks, 0Leaming to Coordinate
Behaviors", in Proc. of AAAI, pp. 796-802 (1990)[ 11 ]S. Hirose, 0. Kunieda, The International Journal of
Robotics Research 1991, pp. 3-12 (1991)[12]M. A. Lewis, A. H. Fagg and G. A. Bekey,"Genetic Algorithms for Gait Synthesis in a Hexapod
Robot", in Advanced Mobile Robots - Theory andApplications: International Prospectives, Chapter 11,
pp. 317-331, World Publishing (1994)[13]K. Berns, St. Coroos, W. Ilg, "Adaptive, NeuralControl Architecture for the Walking Machine
LAURON", in Proc. ofIROS '94, Vol. 2, pp. 11721177 (1994)

P: paratope S: support
Id: idiotope T: transfer

Fig.6: An example of obtained antibo dies.

1::-l
�o.

.i

�:

O. 2

0.0

.g :,

m

m

w

time step

m

�

L2
.

e

0.6

�

0.4

�

�

�'

o

o.

�
::'!:

�

_

,

O2

0.0

o

:

':

m

�

w

�

�

w

time step.g :e. o.6

0.2
0.

■

!::ml

�

R2

.4
0

'
:

0.2

0

JU

40

60

time step

W

0.

100

'

0

M

�)

00

lime step

W

100

,
. R3
.g : •

�

(),(l

::!

0.4

�

:
::

g :mL
3 100
o
w
40
w
so

0.6

0

,

.4

'

:2
:

: . '
o

time step

m

•o

!

::i

oo

time stepm

100

Fig.7: Robustness against disturbance.
100
90
80

� JO
60
� 50
�
� 30
.$2

4(1

10

lO

�
f--

f--

i

r,.j,l!ll<U.W.J4],.1:ltld:l,jc)41d.!id:W,U4J<IJ.lld:l<IJ4!,Ud:Jdjld:J4l,WJ.t.Jdl4117
0.5

0.55

0.6

0.65

o. 7

0.75

o.s

duty ratio

Fig.8: Robustness against the duty ratio.

6 Conclusions and future work
In this study, we have proposed a new rescription
method of antibodies in tenns of self-assertion andsubordination, based on the fact in immunology. We
have applied the proposed method to a gait control- 131 -

An Immunological Approach to Dynamic Behavior Arbitration
for Autonomous Mobile Robots~ Construction of Immune Networks through Learning ~A. ISHIGURO,

Y. WATANABE and Y. UCHIKAWA

T . KONDO,

Dept. of lnfonnation Electronics, School of Eng., Nagoya University
Furo-cho, Chikusa-ku, Nagoya, 464-01, Japan
Phone: +81-52-789-3167, Fax: +81-52-789-3166
E-mail: { ishiguro/kon/yuji/uchikawa}@bioele.nuee.nagoya-u.ac.jp
Abstract
Conve ntional AI system have bee n criticized for its
brillleness in dynamic changing environments.
Therefore, in recent years much attention has been
focused on the reactive planning systems. In this paper,
we propose a new decentralized consensus-making.\')��tem for the behav ior arbitration of an autonomous
mobile robots inspired from biological immune system.
In addition, we propose a new learning method forreactive planing systems, and try to construct suitable
immune networks using reinforcement signals.

Key Words: Autonomous mobile robots, ldiotipic
networks hJpothesis, Decentralized conse nsus-making,Reinforcement learning

1. Introduction
Conventional AI (artificial intelligence) systems
based on a functional decomposition, leading to a so­called "sense-model- plan-act" cycle have been
cr iticized on many drawbacks over the last decade.
Typical criticisms are that these systems show
brittleness under environmental changes, and required
much computation time for mapping obtained sensory
inputs onto complex internal models before action canbe taken. Therefore, in recent years much attention has
been focused on reactive planning systems (e.g.
belu,v ior-basedAI, new Al, emergent com putation ,and
so on), which have already been demonstrated
robustness and flexibility against dynamically changing
world [1][2].
On the other hand, biological information processing
systems such as human beings have many interesting
functions and are expected to provide various feasible

ideas to engineering fields, especially robotics.
Biological information system in living organisms canbe mainly classified into the following four systems: (1)brain-nervous system, (2) genetic system, (3) endocrine
system, and (4) immune system. Among these systems,
brain-nervous and genetic systems have already been
applied to engineering fields by modeling as neuralnetwork.�[3] and genetic algorithms[4], and they havebeen widely used in various fields. Little attention,howe ver, has been paid to application of the othersystems (i.e. endocrine and immme systems)
notwithstanding their important characteristics.
Immune system, in particular, have various
interesting features such as imnnmological memory,
immunological tolerance, pattern recognition, non­hierarchical distributed structure, and so on viewed fromthe engineering standpoint. In addition to the above,
recent studies on immunology have clarified that theimmune system does not just detect and eliminate thenot-self materials called antigen such as virus, cancer
cells and so on, which come from inside/outside of theliving system, rather plays important roles tomaintainits
own system against dynamically changing environments.
Therefore, immune system would be expected toprovide a new methodology suitable for dynamic
problem dealing with unknown/hostile environments
rather than static problem.
Based on the above consideration, we have been
trying to engineer methods of the immune system, andapply to robotics and so forth[5]-[9]. We expect that
there would be an interesting AI technique suitable fordynamically changing environment by imitating theimmune system in living organisms. In this paper, we
propose a new decentralized consensus-making system
inspired from immune system in living organisms. To
confirm the feasibility of our proposed method, we apply- 132 -

to behavior control of an autonomous mobile robots as apractical example. Moreover, we try to evolve theproposed artific ial immune system equipped with themobile robot by adjusting affinities among antibodies
using reinforcement signals. Finally, the validity of our
proposed methods ar e confirmed by carrying out
simulations.

2. Overview of Immune System
2.1 Structure of Antigen and Antibody
The basic components of the immune system arelymphoc_)tes that ar e mainly classified into two types,
namely B-lymphocytes and T-lymphocytes. B­
lymphocytes are the cells produced from bone marrows.

Roughly I07 distinct types of B-lymphocytes arecontained in a human body, each of which has distinct
chemic al structure and produces "Y" shaped antibodies
from its surfaces (se e Fig. I). The antibody specificallyrecognizes an antigen like a lock and key relationship.To
cope with continuously changing enviromrent, living
systems possess enormous repertoire of antibodies inadvance. On the other hand, T-lymphocytes are the cells
produced from th)mus, and they generally perform toregulate the production of antibodies from B­
lymphocytes as outside circuits of B-lymphocyte
network (idiotypic network) discussed later.

the corresponding antigen determinant is called paratope.
Recent studies on immunology have clarified that eachtype of antibody has also its specific antigen determinantcalled idiotope.

2.2 Jtl"ne's idiotipic netwcrks hypothesis
Based on this fact, N.K.Jerne, who is an immunologist,
proposed a remarkable hypothesis: idiotypic network
hypothesis [I 2]-[16]. This network hypothesis is theconcept that antibodies are not just isolated, namely they
are communicating to each other among different speciesof antibodies. In other word, the immune system isconstructed as a large-scaled system of lymphocytes
through mutual interaction between different species oflymphocytes. This idea of Jerne's is schematicallyshown in Fig.I. The idiotope Idl of antibody I (Ab!)
stimulates the B-lymphocyte 2, which attaches theantibody 2 (A b2) to its surface, through the paratopeP2.
Viewed from the standpoint of Ab2, the idiotope Id! of
Ab I works simultaneously as an antigen. As a result, theB-lymphocytes 1 with Abl are suppresged by Ab2. On
the other hand, antibody 3 (Ab3) stimulates Ab! since
the idiotope Id3 of Ab3 works as an antigen viewed from
Abl. In this way, these stimulation and suppression
chains among antibodies for m a large-scaled chain loop
and works as a self and not-self recognizer. Again, theheart of Jerne's idea is that the self, and not-selfrecognition in the immune system is carried out at
system level.

3. Action selection and Immune system

idiotope-+-- stimulation__.....,,_ suppression

Fig. I Jerne 's idiotype network hypothesis.

For the sake of convenience in the following
explanation, we furthermore introduce several
terminology in immunology. The portion on the antigen
recognized by the antibody is called epitope (antigen
determinant), and the one on the antibody that recognizes

Recently, behavior-based AI techniques demonstrate
their robustness and flexibility against hostileenvironrrents, however, arbitration among competence
modules (simple behavior) aril'Es diffic ult problems.
Narrely, it is difficult to select a competence module
suitable for the current situation. To overcome thisproblem, several methods have been proposed.[10][11]Against the above-mentioned stream of works, weapproach to this problem from the immunologicalstandpoint, namely using immune networks. Fig.2
schematically shows the autonomous mobile robots andthe immune network system.
As shown in this figure, current situation, for example,- 133 -

distance and direction to the detected obstacle work as
antigens, and competence module(simple behavior) andinteraction between modules can be recognized as
antibody and stimulation/suppression among antibodies,
respectively.
The basic concept of our method is that the immune
system equipped with the autonomous mobile robot
selects the competence module (antibody) which is mostsuitable for the curr ent situation (antigen).
Dynamically Changing
En\lironment

01A

Mobile Robot( 1) if the immunobot moves, it consumes energy Em .
(2) if the immunobot is captured by a predator, it
consumes energy E".
(3) if the immunobot collides with an obstacle, it loses
energy E0(4) if the immunobot picks up a food, it obtains energy
Er.
The predators attack the immunobot if they detect theimmunobot within the prespec ified detectable range.
Therefore, to survive as long as possible, the immunobot
must select a competence module (antibody) suitable forthe current detected situation (antigen).
•.._.

Ohstack�Destination(a) An autonomous mobile robot with action selection mechanism.

Antibody

Fig.3: Simulated environment.

Immune network(b) Immune networks.
Fig.2: Ba�ic concept of our proposed method.

4. Proposed consensus-making networks
based on biological immune system
4.1 Problem
For convenience, we dub the autonomous mobile
robot with the artificial immune system "immunobot''.
To verify the ability of our proposed immune systemagainst hostile environments quantitatively, in this study
we use simulated environment for the immunobot asshown in Fig.3.
In this simulated environment, there are following
three kinds of objects: 1) predators, 2) obstacles and 3)
food. And we assume that prespecified quantity of initialenergy is given to the immunobot at the beginning ofeach simulation. For quantitative evaluation we use thefollowing assumption:

Fig.3 also indicates the structure of the immunobot
used in the following simulations. We equipped thisimmunobot with external and internal detectors. External
detectors are installed in eight direction shown in Fig.3,
and each can detect the distance to the objects in threedegrees: near, mid and far. On the other hand, internalsensor detects the current energy leve l. For simplicity,
we assume that the immunobot can move toward theabove eight directions in the following simulations.

4.2 Description of antibodies
As described earlier, in this study, the detected current
situation and prepared competence modules work as
antigens and antibodies, respectively. To make theimmunobot select a suitable antibody against the current
antigen, it is highly important how we describe theantibodies. Moreover, we should notice that ourimmunological arbitration mechanism selects anantibody in a bottor&up manner by communicating- 134 -

among the antibodies. To realize the above requirements,
we defined the description of antibodies as follows. As
mentioned in section 2, the identity of a specific antibody
is generally determined by the structure of its paratope
and idiotope. Fig.4 depicts our proposed description of
antibodies

respectively. The third term represents the stimulationfrom antigen, and the forth term the natural death.

a; : concentration of antibody i
mj,i : matching ratio between antibody i andj
Fig.5: Dynamics.

5. Proposed Learning method
5.1 Learning Procedure
Fig.4: Description of antibodies.

As shown in this figure, we assign a pair of desirable
condition and action to paratope, the number of
disallowed antibodies and the degree of disallowance toidiotope, respectively. In addition, we divide thestructure of paratope into four potions: objects, direction,

distance, and action.

4.3 Dynamics
For adequate selection of antibodies, we assign one
state variable called concentration to each antibody. In
this study, selection of antibodies is simply carried outin
a winner-take-all fashion. Narrely, only one antibody isallowed to activate and act its corresponding behavior to
the world if its concentration surpasses the prespecified
threshold. As shown in Fig.5, concentration of theantibody is influenced by the stimulation andsuppression from other antibodies, the stimulation fromantigen, and the dissipation factor (i.e. natural death).
The concentration of i-th antibody, which is denoted by a;,is calculated as follows:

In tltis section, we attempt to determine the priority
among antibodies on an on-line learningbasis.To realize
this aim, we propose a new learning method using theadvantages of the prementioned description of antibodies
as follows.
For the ease of the following explanation, we assume thatthe immunobot is placed as in Fig.6. In this situation, as
antigen 1 and 2 invade into the immunobot's interior,
antibody 1 (Abl) and 2 (Ab2) simultaneously are
stimulated by each antigen, concentration of eachantibody become higher. Note that the notation [P, F, M]
represents that a predator exists in the front direction andmid range. However, as the priority between Ab 1 and 2
is unknown (because idiotopes are initially tabula rasa,
there are no stimulation/suppression chain), in this case,either of them is selected randomly.

The fitst and second terms of right hand .side denotethe stimulation and suppression from other antibodies,- 135 -

ight

Fig.6: An Example of antibody selection.

Now, assuming that Ab 2 is randomly selected andthe immunobot successfully avoids the predator andobtains food, In this case, the immunobot can receive thepositive reinforcement signals as a reward. To make theimmunobot tend to select Ab 2 in the sameh,imilar
situation, we record the number of Ab 2 (i.e. 2) in theidiotope of Ab l. This procedure works to raise thepriority ofAb 2 againstAb 1.
In the case where the immunobot receives the negative
reinforcement signal, we record the number ofAb 1 (i.e.
1) in the idiotope ofAb 2 in the same way. This works todecrease the priority of Ab 2 againstAb 1.

1000

-�
cu

-�cu

800
600
400
200
0
0

200

400

600

800

1000

learning iteration
Fig.8: Transition of life time.

Fig.9 represents an example of obtained networks.
Detail investigation of the obtained immune network iscurrently undergoing.

Fig.7: learning process.

For simplicity, in this study, we give thereward/penalty signal to the immunobot in the situation
listed in Table 1.
Fig.9: example of obtained networks.

Table 1: Reward/Penalty Criterion.
Obtain foodReward Approach foodAway from predator/obstacle
Captured by predator
Collide with obstacle
Penalty
Approach predator/obstacle

6. Conclusions and Furthtr Work

5.2 Simulation Result
To verify the feasibility of our proposed learning
method, we carried out same simulations. Fig.8 denotes
the transition of life ti� of the immunobot. From thefigure, it is understood that the life ti� gradually
increases.

In this paper, we proposed a new learning method forthe decentralized consensus-making network based on
the biological immune system. And we applied the
proposed method to the behavior arbitration of anautonomous mobile robot in the simulated environ�nt
and confirmed the feasibility by carrying out simulations.
Through the simulations, we showed that the immune
networks can be automatically constructed using
reinforcement signals.
To reveal the validity of our proposed method
quantitatively, detail analysis of the resultant immune
networks is highly important. This is currently underinvestigation.And we are new planning to apply this ides
to a real experimental mobile robot. This project is also

- 136 -

currently undergoing.

Acknowledgments
This research was supported in part by a grant fromthe Japanese Ministry of Education (No. 07243208).

Refo-ences[ l ] R.Brooks, "A Robust Layered Control System for a
Mobile Robot", IEE E Journal of R&A, Vol.2, No.l,
pp.14-23, 1986
[2] R.Brooks, "Intelligence without reason", Proc. of
IJCAI-91, pp.569-595, 1991
[3] D.E.Rumelhart, et al., "Parallel distributed
processing", MITPress, 1986
[4] D. E.Goklberg, "Genetic algorithms in search,
optimization and machine learning", Addison-Wesley,
1989[5] A. Ishiguro, Y. Watanabe and Y. Uchikawa, "Fault
Diagnosis of Plant Systems Using Immune Networks",
in Proc. of 1994 IEE E International C onference on
Multisensor Fusion and Integration for Intelligent
Systems, pp. 34-42, 1994
[6]S. Ichikawa, A. Ishiguro and Y. Uchikawa, "A Gait
Acquisition of 6-Legged Walking Robot Using Immune
Networks", Journal of Robotics Society of Japan, Vol. 13,
No. 3, pp. 125-128, 1995 (in Japanese), also inProc. of
IROS '94, Vol. 2, pp. 1034-1041, 1994
[7] A. Ishiguro, S. Kuboshiki, S. Ichikawa and Y.
Uchikawa, "Gait Coordination of Hexapod Walking
Robots Using Mutual-Coupled Immune Networks", Proc.
of ICEC'95, Vol. 2, pp. 672-677, 1995
[8] A. Ishiguro, Y. Watanabe and Y. Uchikawa, "An
Immunological Approach to Dynamic Behavior Control
for Autonomous Mobile Robots", Proc. of IROS '95,
Vol.l, pp.495-500, 1995
[9] A. Ishiguro, T. Kondo, Y. Watanabe and Y.
Uchikawa, "Dynamic Behavior Arbitration ofAutonomous Mobile Robots Using Immune Networks",
Proc. of ICEC'95, Vol.2, pp. 722-727, 1995
[10] P. Maes, "The dynamic action selection", Proc. of
IJCAI-89, pp.991-997, 1989
[11] P. Maes, "Situated agent can have goals",
Designing Autonomous Agents, pp.49-70, MIT Press,
1991[12] N.K.Jerne, "The immune system", Scientific
American, Vol.229, No.l, pp.52-60, 1973
[13] N.K.Jeme, "The generative grammar of theimmune system, EMBO Journal, Vol.4, No.4, 1985
[ 14] N.K.Jerne, "Idiotypic networks and otherpreconceived ideas", Immunological Rev., Vol.79, pp.524, 1984[15] J.DFarmer, N.HPackard and A.SPerelson: The
immune system, adaptation, and machine learning,
Physica 22D, pp.187-204, 1986[16] HFujita and K.Aihara, "A distributed surveillanceand prote ction system in living organisms", Tmns. on
IEE Japan, Vol.107-C, No.11, pp.1042-1048, 1987 (inJapanese)- 137 -

Artificial Evolution and Real Robots
Inman Harvey
School of Cognitive and Computing Sciences
University of Sussex
Brighton BNl 9QH, UK
Abstract
Artificial evolution as a design methodology allows
the relaxation of many of the constraints that haveheld back conventional methods. It does not require acomplete prior analysis and decomposition of the task
to be tackled, as human designers require. However
this freedom comes at some cost; there are a whole new
set of issues relating to evolution that must be consid­
ered. Standard Genetic Algorithms may not be appro­
priate for incremental evolution of robot controllers.
SAGA, Species Adaptation Genetic Algorithms, hasbeen developed to meet these special needs.
The main cost of an evolutionary approach is thelarge number of trials that are required. Simula­tions - especially those involving vision in complex
environments, or modelling detailed semiconductor
physics - may not be adequate or practical.
Examples of evolved robots will be discussed, in­cluding a specialised piece of equipment allowed forthe testing of a robot using simple vision in real time,
and what is believed to be the first successful example
of an evolved hardware controller for a robot.

1

Why Evolutionary Robotics?

Humans are naturally evolved creatures, and the se­lection criteria under which our ancestors were judged
did not include the ability to design complex systems- in fact, we are not very good at it. A common anduseful trick to overcome our shortcomings is that of di­vide and conquer: a complex problem is decomposed
into separate easier sub-problems.
However, the interactions between such sub­problems must be few in number, so that the humandesigners can temporarily ignore them while solving
one sub-problem at a time. When it comes to design­
ing such complex systems as a cognitive control system
for a robot, there are at least three major problems.
• It is not clear how a robot control system should be
decomposed.

• Interactions between separate sub-systems are notlimited to direct connecting links, but also in­clude interactions mediated via the environment.
• As system complexity grows, the number of potential
interactions between sub-parts grows exponentially.
Classical approaches to robotics have often assumed
a primary decomposition into Perception, Planningand Action modules. Many people now see this asa basic error. Brooks' subsumption approach [1] is ex­
plicitly claimed to be inspired by natural evolution.
Initially simple behaviours are 'wired into' a robot ,and thoroughly debugged, before adding the next be­haviour. This incremental approach echoes the phylo­
genetic history of complex cognitive creatures, some ofwhose behaviours we are trying to emulate in robots.
Nevertheless, each new layer of behaviour is wired inby hand design; despite the heuristics used to min­imise interactions between layers, it seems that un­
predictable interactions may become insuperable when
the number of layers gets much bigger than 10.
So an obvious alternative approach is to explicitly
use evolutionary techniques to incrementally evolve in­creasingly complex robot control systems, rather thanattempt to figure out each evolutionary step by handdesign. Unanticipated and elusive interactions be­
tween sub-systems, though tricky or perhaps impene­trable for human designers, need not directly botheran evolutionary process where the only benchmark isthe behaviour of the whole system.

2

Artificial Evolution for Robots

Genetic Algorithms (GAs) are the most common
form of algorithm which uses evolutionary ideas forsearch, optimisation and machine learning - the fields
covered in [2]. However, recently concerns have been
voiced to the effect that GAs, when originally pro­
posed by Holland [3], were intended as algorithms
for complex adaptive systems, and their use for func­tion optimisation is perhaps not best suited to their- 138 -

strengths. Evolutionary robotics typically needs adap­
tive improvement techniques, rather than optimisation
techniques, and this critical but little-understood dis­tinction needs to be made clear.
Most published GA work, both applications andtheoretical analysis, refers to optimisation problems
which can be seen as search problems in some high­dimensional search space, of known ( usually enor­mous) size. Each dimension typically corresponds tosome parameter which needs to be set, which is coded
for on a small section of the genotype, a 'gene'. Whatsuch optimisation problems share is the well-defined fi­
nite nature of the search space. This allows the choiceof a genotype coding, such that a genotype, often bi­
nary, of fixed length can encode any potential solution
within the space of possibilities. In robotics, a geno­
type specifies the characteristics of a control system.
The GA works with a population of such genotypes,
each of which is evaluated in terms of how good is thepotential solution that it encodes. Genotypes whichhappen to be fitter in the current population (which
initially may be generated at random) are preferen­
tially selected to be parents of the next generation.
Offspring inherit genetic material from their parents;
usually inheriting part of this from each of two parents.
A small number of random mutations are applied to
the genotypes of the offspring. This cycle of selection,
reproduction with inheritance of genetic material, andvariation, is repeated over many generations, with the
population remaining the same size as old members are
replaced by new ones, the offspring of those members
demonstrated to be fitter.
A GA optimisation problem has typically been seen
as starting with a population of random points effec­
tively spanning and crudely sampling the whole search
space. Successive rounds of selection, reproduction
and mutation focus the population of sample points
towards fitter regions of the space, homing in on an
optimum or near-optimal region. One consequence of
this approach has been the primary reliance on re­
combination as the genetic operator, which mixes andmatches information from different samples in order tomove towards regions of expected higher fitness; muta­tion is typically relegated to the role of a background
genetic operator.
However, some domains - including much of evo­lutionary robotics - do not always fall into this con­venient picture of a fixed-dimensional search space.
Standard GA theory does not necessarily then apply.
In evolutionary robotics a genotype will specify thecontrol system (possibly more, see below) of a robot
which is expected to produce appropriate behaviours

when tested in its environment. However the evalu­
ation of fitness is in terms of the robot's behaviour;
for all except toy problems there is unlikely to be any
obvious way to predict in advance the necessary com­
plexity of control system for a given behaviour. Hence
it is often appropriate to choose a genetic encoding
which allows for, and encodes the characteristics of, avariable number of components. This has the added
benefit of making incremental evolution possible: ini­
tially simple robots are evolved under a selection cri­
terion based on simple tasks, and then the same robot
population is allowed to increase in complexity in re­sponse to a gradual and continuing increase in taskcomplexity. Such incremental evolution calls for GAs
as adaptive improvers rather than GAs as optimisers.

3

SAGA

The conceptual framework of SAGA was introduced
to deal with the dynamics of a GA when genotype
lengths are allowed to increase [4]. It was shown, us­ing concepts of epistasis and fitness landscapes drawn
from theoretical biology [5], that progress throughsuch a genotype space will only be feasible throughrelatively gradual increases in genotype length. A gen­eral trend towards increase in length is associated withthe evolution of a species rather than global search the population will be largely genetically converged.
Evolutionary search can be thought of as searchingaround the current focus of a species for neighbour­
ing regions which are fitter (or in the case of neutral
drift, not less fit) while being careful not to lose gains
that were made in achieving the current status quo.
The population can be visualised as moving around ona mountainous fitness landscape, where altitude rep­resents fitness, and movements measured in horizon­
tal directions loosely represent movements in genotype
space. Selection is a force which tends to move a pop­
ulation up hills, and keep them centred around a local
optimum; mutation produces offspring exploring out­
wards from the current population.
To increase exploration mutation rates should be
increased; but if they are too high then the popula­tion disperses completely, losing the current local opti­
mum, and the search becomes random. For any given
selection pressure, there is a maximum rate of mu­
tation which simultaneously allows the population toretain a hold on its current hill-top, while maximising
search along relatively high ridges in the landscape,potentially towards higher peaks [6]. In SAGA, thismeans that rank-based or tournament selection should
be used to maintain a constant selective pressure (the- 139 -

expected number of offspring of any individual should
depend on its current ranking within the population,rather than the ratio of its fitness to the average fit­ness); and mutation rates should be maintained at a
rate of about 1 mutation per genotype [7].

4What building blocks?

We are relying on evolution for the design of a con­trol system, but we must choose appropriate building
blocks for it to work with. There is good reason tobelieve that the primitives manipulated by the evolu­
tionary process should be at the lowest level possible.
Any high level semantic groupings inevitably incorpo­
rate the human designer's prejudices. Primitives thatare equivalent to a programming language give rise toa rugged fitness landscape with steep precipices. A
program taken as a linear string of characters can betreated as a genotype, but typically a single mutation
in a working program is fatal - a 'precipice'. Genetic
programming [8] relies on recombination rather thanmutation, but typically relies both on clever, domain­
specific, choice of primitives, and on enormous popu­
lation sizes which are difficult when evaluating robots.
With Brooks [1], we dismiss the classical Percep­
tion, Planning, Action decomposition of robot control
systems. Instead we see the robot - body, sensors,
motors and control system or 'nervous system' - asa dynamical system coupled (via the sensors and mo­tors) with a dynamic environment [9]. This coupledinteraction generates the robot behaviour which is tobe evaluated. The control system is itself a dynamicalsystem, and hence its genetic specification should beat the level of the primitives of a dynamical system.
One convenient form of dynamical system is an (ar­tificial) neural net (NN). If this takes the form ofa feedforward net, from sensors, perhaps via hidden
nodes, to motors, then such a control system would
have no internal state, and be capable only of gen­erating reactive behaviour. However if a recurrent
net is used, with temporal specifications to determine
the timescales on which internal feedback is propa­
gated, then non-reactive behaviour is also possible.
Dynamic recurrent NNs (DRNNs), with temporal de­lays on links between nodes, are a class of dynamicalsystems capable in principle of replicating to an arbi­
trary degree of accuracy the dynamical behaviour of
any other dynamical system with a finite number ofcomponents. Such DRNNs are equivalent (only triv­
ial transformations are needed) to Brook's subsump­tion architectures using Augmented Finite State Ma­
chine (AFSMs). The temporal properties can derive

either from within each AFSM (Brooks), or throughthe time-delays on links between them (DRNNs).
One significant difference from subsumption archi­
tecture is the deliberate introduction of internal noiseat the nodes of DRNNs, with two effects. First,
it makes possible new types of feedback dynamics,
such as self-bootstrapping feedback loops and oscilla­
tor loops, which would not initiate themselves without
the noise. Second, it helps to make more smooth thefitness landscape on which the GA is operating; a mu­tation which deletes a link or a node is comparable toa lot of noise, and hence the change in behaviour due
to such a mutation is more closely correlated in thepresence of noise than it would otherwise have been.
Thus genotypes need to specify a finite numberof nodes, together with their thresholds or other de­tails of a non-linear activation function transforming
summed node inputs into node outputs; and links andconnections between nodes, specifying weights andtime-delays on the links. This can be generalised to in­clude weight-changing rules. A specified subset of thenodes are designated as input nodes, receiving sen­sory inputs; similarly there is a set of output or mo­
tor nodes. Other nodes ('hidden') can be arbitrary innumber, and genetically specified links are not neces­sarily restricted to feedforward ones.

5

Experiments

Using these ideas, a series of experiments were per­formed at Sussex with the gantry-robot using low
bandwidth vision in a noisy real-world domain. A se­quence of simple navigational tasks of increasing com­
plexity was presented to the robot, and artifical evo­lution used to develop the control system and the vi­
sual morphology appropriate for success at these tasks.
This work is reported elsewhere [10], and is believed tobe the first example of artificial evolution for a robot
using real vision.
The work used a genetically specified dynamicalsystem as the control system, which is conceptualised
as a DRNN, but in practice been implemented on acomputer. There is a related approach of evolving con­trol systems directly onto hardware, which has beentaken within our group by Thompson [11].
This work is intrinsic hardware evolution, in thatfor each genetically specified piece of hardware, theactual hardware is tested in situ; as contrasted withextrinsic hardware evolution, where simulations of the
hardware are evaluated during evolution. The ac­tual low-level physics of the hardware can be utilised,- 140 -

and the realtime dynamics operate at their proper
timescales. A human designer usually constrains andcontrols through clocking such features as switchingtransients; but by using artificial evolution, such de­sign constraints can be relaxed in an unclocked system.
Thompson used artificial evolution to design a real
EHW circuit as an on-board controller for a two­
wheeled autonomous mobile robot required to display
simple wall-avoidance behaviour in a wide corridor.
The robot's only sensors were two sonars mounted on
the left and right sides which fire simultaneously fivetimes a second; the output from each sonar to the con­trol system changes when the echo returns. In a con­ventional system the time of flight would be processed
to estimate the range of obstacles, but in Thompson's
implementation the pulses are fed directly into thehardware control system, termed a Dynamic State Ma­
chine (DSM). This is related to a Finite State Machine,
except that each internal signal may (under genetic
control) be clocked or unclocked. The global clock fre­quency, where used, is also under genetic control.
The DSM accepts pulses directly from the sonars,
and outputs signals directly (no post-processing) to
the motors for the left and right wheels which guide therobot. During evolution a population of DSM specifi­
cations (instantiated one at a time on the real hard­ware) is evaluated at the task of navigating to thecentre of the corridor. Success was achieved within
35 generations; for full details see [11]. One success­ful evolved DSMs was found to be using just 32 bitsof RAM and 3 flip-flops, excluding the clock genera­tion. This minimal hardware produced the appropri­ate sensorimotor coupling between sonar echo signals
and motor pulses, to guide the robot in its task. Thisis believed to be the first ever artificial evolved hard­ware controller for a robot.

6

Conclusions

Evolutionary robotics allows the relaxation of con­ventional design constraints, but new theoretical is­sues need to be studied. The SAGA framework allows
for incremental evolution, and robot control systems
should be treated as a class of dynamical system. Ex­
amples have been given of simple evolved robot be­haviours in noisy real world conditions, including theuse of evolvable hardware.

References[1] Brooks, R. A. "A robust layered control system
for a mobile robot," IEEE J. Rob. Autom., Vol.
2 pp. 14-23, 1986.
[2] Goldberg, D. E. Genetic Algorithms in Search,
Optimization and Machine Learning. Addison­
Wesley, Reading MA, 1989.
[3] Holland, J. Adaptation in Natural and Artificial
Systems. University of Michigan Press, Ann Ar­
bor, USA, 1975.
[4] Harvey, I. "Species adaptation genetic algo­rithms: The basis for a continuing SAGA," In
Varela, F., Bourgine, P. (eds.), Toward a Prac­tice of Autonomous Systems, pp. 346-354. MIT
Press/Bradford Books, Cambridge, MA, 1992.
[5] Kauffman, S. "Adaptation on rugged fitness land­
scapes," In Stein, D. L. (ed.), Lectures in the Sci­
ences of Complexity, pp. 527-618. Addison Wes­ley, 1989.
[6] Eigen, M., McCaskill, J., and Schuster, P.,"Molecular quasi-species," Journal of Physical
Chemistry, 92, pp. 6881-6891, 1988.
[7] Harvey, I. "Evolutionary robotics and SAGA: thecase for hill crawling and tournament selection,"
In Langton, C. (ed.), Artificial Life III, pp. 299326. Addison Wesley, 1993.
[8] Koza, J. R. "Genetic programming," Tech. Re­
port STAN-CS-90-1314, Dept. of Comp. Sc.,
Stanford University, 1990.
[9] Beer, R. D., and Gallagher, J. C. "Evolving dy­
namic neural networks for adaptive behavior,"
Adaptive Behavior, 1(1), pp. 91-122, 1992.
[10] I. Harvey, P. Husbands, and D. Cliff. "Seeing thelight: Artificial evolution, real vision." In D. Cliff,
P. Husbands, J.-A. Meyer, and S. Wilson, (eds.),
From Animals to Animats 3, pp. 392-401. MIT
Press/Bradford Books, Cambridge MA, 1994.
[11] Thompson, A., Harvey, I., and Husbands, P. "Un­
constrained evolution and hard consequences," In
Sanchez, E., and Tomassini, M. (eds.), Towards
Evolvable Hardware. Springer-Verlag, 1996.

Acknowledgements
I acknowledge funding from the E.P.S.R.C., and dis­cussions with Phil Husbands and Adrian Thompson.

- 141 -

Design principles of autonomous agents: a case study of classification
Rolf Pfeifer
AI Lab, Computer Science Department
University of Zurich, Winterthurerstrasse 190
CH-8057 Zurich, Switzerland
pfeifer@ifi.unizh.ch

Abstract
Building artificial life-like autonomous agents is still
considered an art, rather than a science. A generally accepted
precise methodology is missing, and-given the propertiesof the real world-it is. doubtful, whether there will ever beone. Nevertheless, it is possible to define criteria and provide
heuristics for good designs. We have developed a number ofdesign principles which, when applied, lead to what wewould consider good designs from a cognitive science or
ALife perspective. The paper illustrates some of these prin­ciples using a case study of classification.
Keywords: autonomous agents, sensory-motor co­ordination, value-based learning, ecologically balanced
designs.

1

Introduction

One of the main research interests in ALife is building
artificial creatures. We have been interested in particular
types of artificial creatures, namely real-world automous
agents. By building artificial autonomous agents, we are try­
ing to understand naturally intelligent agents. We hope that
this endeavor will lead to insights about principles of intelli­gence that include, but are not restricted to, natural systems.
The field of automous agents is still in its infancy. Thereis no concensus whatsoever on how to build intelligent
agents, and a theory is entirely lacking. While some believethat the theory is to be found in the area of non-linear dy­
namics, others bet on principles from the theory of evolution.
The information processing paradigm that held great promisein the early days of artificial intelligence and cognitive sci­ence, has been shown to be too limited to explain intelli­gence. One of its major problems was to deal with behavior
in the real world. So, it seems, that we are currently lacking
good candidates for a theory of intelligence. But the fact that
there is no generally accepted unified framework or theory
around does not imply that no relevant knowledge exists. It
is only unsystematic and widely distributed in various re­search communities and individual laboratories. The com­munities where pertinent knowledge can be found include
robotics and engineering, neurobiology, biology, artificial
intelligence, psychology, and, of course, ALife (even though

the latter does not yet constitute a coherent research com­munity).
As a first step in our understanding of intelligent artifi­
cial real-world agents, we have started establishing a set ofdesign principles, i.e., a set of principles stating what we
consider to be the important ideas underlying good designs.
They represent an initial approximation and are not compre­
hensive. Moreover, we realize, that some researchers will not
agree with them.

2

Design principles: overview

There are three sets of principles. The first is about thetypes of agents that are of interest. In the field of ALife, re­searchers study virtual as well as real-world agents. We areinterested in the latter kind. The most prominent real-world
autonomous agents are robots. Out of the many types ofrobots, we are particularly interested in autonomous mobile
robots, rather than stationary industrial ones.
The first set of design principles states that those agentsare of relevance to the study of intelligence that are embod­
ied, autonomous, self-sufficient, and situated. They are ofhigh interest, because they display behaviors that we nor­mally associate with intelligence (for detailed arguments,see, for example, [2], [7]).
The second set of principles concerns the quality of thedesigns themselves,i.e. the morphology, the architecture andthe mechanisms. The main principles in this set that we will
discuss are:Intelligence is to be viewed as emergent from a large( 1)
set of loosely coupled processes.
The agent should be designed as a learning system(2)
whose learning processes are based on its own value
system.
The sensory system, the motor system, and the neural(3)
substrate should be "ecologically balanced."
Finally, the third set of principles contains design heuris­
tics, strategies, and metaphors. When employed, they should
lead to good designs. We will discuss a particularly impor­tant principle; it concerns the frame-of-reference problem.
Rather than explaining the principles in abstract terms we
illustrate them with a case study on categorization, which isinspired by a garbage-collecting robot that has to learn anumber of distinctions.

- 142 -

3

Case study: a garbage collecting robot

One of the fundamental abilities of natural living sys­tems, is classification. Food needs to be distinguished fromnon-food, conspecifics from enemies; the nest from the restof the environment, and so on. A garbage-collecting robot
should learn the distinctions between garbage and non­garbage, various locations, like the incinerator, the sites
where garbage is deposited, the charging station, etc. We
designed an agent capable of learning some basic distinc­
tions in a particular environment. Let us now discuss theagent design and subsequently summarize the essential
points in terms of design principles. For more detail, thereader is referred to [8].

3.1

0 °0

b,, /
''

Basic set-up

The basic set-up is shown in figure 1. There is awheeled robot, equipped with eight IR sensors, two individ­
ually driven motors, and a gripper with two degrees of free­dom. There is also a sensor in the "hand" of the gripper for
detecting whether there is an object in the "hand". In the en­
vironment, there are small, large, and very large pegs. Our
agent should pick up the small ones and cary them to a lightsource. The large ones are too heavy to carry, so it should
push them towards the periphery of the arena. The very large
ones are too heavy for anything and should be left alone. We
want the robot to learn these distinctions, because we are
specifically interested in learning. Moreover, it should learn
the distinctions from its own perspective, without instruction
from the experimenter. In this simple case, it might be possi­
ble to actually program the agent with these distinctions, but
first we want to study learning, and second, as the situations
and the types of objects get more complex, pre-programming
distinctions will get more and more difficult.

3.2

some not so recent one), suggests that perception is not somuch an information processing phenomenon, but rather amatter of sensory-motor coordination ([3], [4], [6], [11]).

Perceptual aliasing

If the agent is to learn the distinctions between the vari­
ous types of pegs, it somehow has to solve the problem ofperceptual aliasing. Perceptual aliasing has two forms. On
the one hand, the proximal stimulus, i.e., the particular sen­sory stimulation is ambiguous and may correspond to several
situations of the environment. On the other hand, the sensory
stimulation from one object varies greatly. It strongly de­pends on distance and orientation. A large number of statesin sensory space correspond to one type of object.
The standard approach is to take an array of sensor read­ings, e.g., a pixel array, and to perform some processing on
it, in order to map it onto an internal model. As we know
from computer vision, this has not worked very well. The
deeper reason why it does not work well is that the problem
is treated as one of information processing which is done "onthe input side". A close look at the recent literature (and

'-

\
I /
' \ \ I/ /----�,/

L--"""''----------.U...L-i-------Figure 1: Simplified set-up for the garbage collecting
robot. Details, see text.

3.3

Architecture

The architecture that we have used is shown in figure 2.
The underlying idea is that behavior-and ultimately intelli­gence- is, in essence, an emergent property from a large
number of parallel processes. There is no set of individual
actions from which the agent has to select one for execution,i.e. there is no process of action selection. Rather, there arepotentially large numbers of loosely coupled processes
which all influence th.e motor variables. What the agent does
is determined by the values of these variables. Because theseideas can be traced back to Braitenberg's seminal book [1], itis called an Extended Braitenberg Architecture (EBA) (formore detail, see [8], [10]).

3.4 Value-based learning and sensory-motor coor­dination
A situated agent should learn from its own perspective.
For this purpose, it needs a value system (e.g. [5], [9],).
Figure 3 depicts the process through which the agent learns
to form the categories. Value is associated with three typesof behaviors, grasping and carrying, pushing, and ignoring.
Value is used here to designate what matters to the robot.
Take grasping for example. Grasping is intrinsicallyrewarding which means that whenever the agent manages topick up a peg, a value signal is generated by the value
system. Likewise for pushing. Ignoring is slightly different,
but the idea is that avoiding uninteresting objects also has
value to the agent.

- 143 -

fromsensorsandeffectors

from sensors

from
other
processes!Rs

move-forward

speed
left
motor

wheel
encoders

fromgrippervarious
motor
variables

avoid-obstacle
tum-towards-object

speed
right
motor

Figure 3: Illustration of the tuning of the various pro­
cesses using the grasp process.

Figure 2: EBA architecture of the garbage-collecting
robot. The shaded area is explained in more detail in
figure 3.
As is well-known, reinforcement learning is very slow if
it is based on random movements. In order to speed up
learning, we increase the probability of successful behaviors
like grasping or pushing by introducing a number ofreflexes. For example there is a reflex for turning away fromobjects ("avoid-obstacle") and one for turning towards anobject ("turn-towards-object"). The interaction of the twocauses the agent to move along an object. There is anadditional reflex that makes the agent sense the object("sense-object"). Sensing is done by lowering the gripper
over the object, trying to determine if it can be picked up or
pushed. This reflex is triggered whenever the agent has been
moving along an object for some time. The rationale behind
this reflex is that if there is indeed an object, there is a goodchange that the subsequent behaviors will lead to success. In
sum, the purpose of the reflexes is to bias the general­purpose reinforcement scheme.

The following point is central to our argument. Rather
than statically taking sensor images and applying algorithms
to these images, the agent engages in a sensory-motor coor­dination which is caused by the reflexes. If the agent
encounters an object, it will start moving along it. This inturn causes a correlation of sensor activation in different
sensors, namely the wheel encoders and the IR-sensors. This
correlation is induced by the agent's movement via theinteraction of the agent with its environment. The correlation
between the motor speeds (as measured by the wheel
encoders) and the IR sensors comes about because, as wehave seen, the agent moves around objects. Thus, a lateral IR
sensor will be maximally active (whereas the others would
typically be low), and the ratio of the values of the wheel
encoders is constant (constant angular velocity). In otherwords, while the agent is engaged in a sensory-motor
coordination, the dimensionality of the sensory-motor spaceis reduced. This makes it much easier for the reinforcement
system to learn. In this scheme, learning only takes place, if
a value signal is generated. In order to facilitate learning wehave used an incremental sensory-motor map which we
don't discuss any further, since it is not the main point here.

3.5

Results

Initially the agent has to move along an object for quite
some time (about 60 steps of 150ms each) before the "sense­
object" process takes over. After a few encounters with thevarious types of objects, it learns to perform the appropriate
behavior after only about 10 to 20 steps without having to
make use of the "sense-object" process any longer. We have
performed extensive statistical analyses on the data [8]. The
algorithms we have used are all incremental, meaning thatnew objects can be added in at any one time.

- 144 -

4

Design principles: evaluation5

Let us now review the case study in terms of our designprinciples. Principles concerning the kind of agent: Our
agent it is obviously embodied, and it is situated in that itlearns from its own perspective. The purpose of the casestudy was to illustrate categorization, which is why thecurrent version of the robot is not self-sufficient, and only
has a limited degree of autonomy. However, the EBA­
architecture allows the incorporation of additional processes,
and thus provides the potential for self-sufficiency.
Principles concerning the properties of the agent: Let us
mention the most important ones. The EBA-architecture,
which consists of parallel processes, implements the designprinciple that intelligence is to be viewed as emergent from alarge number of loosely coupled processes. This principle
implies that there must be no central control. A similar point
has been made by Brooks [2] in the context of the subsump­tion architecture.
The distinctions the agent makes over time (as can beseen in the agent's behavior) between different types of ob­
jects are based on a process of sensory-motor coordination.
Note that through this process, correlations between the two
sensory systems are induced. The reduction in dimen­
sionality achieved in this way makes learning tractable. Thisinstantiates the principle that the agent should (a) engage in akind of sensory-motor coordination, and (b) the sensory
system, the motor system, and the neural substrate should be"ecologically balanced". This is in sharp contrast to most ap­
proaches which view perception as a pure information
processing problem. This principle of "ecological balance"
implies that it is not sensible to merely increase thesophistication of one part of the agent, without increasing thecomplexity of the others. If that were done, no interesting
additional sensory-motor coordinations could take place.
The learning in our agent is value-based, meaning thatthe agent provides its own reinforcement. This is achievedby the agent's value system. This kind of learning is a pre­requisite for a situated agent. It works well in the agent be­cause the learning problem has been simplified through thesensory-motor coordination.
Principles concerning the design heuristics: Let us justmention one. It states that the so-called "frame-of-reference"
issue has to be taken carefully into account. For example, we
say that the agent has learned to make distinctions, if-aftersome time-it consistently picks up small pegs and nolonger tries to pick up large ones. Note that thischaracterization is entirely observer-based and does notresort to internal representations whatsoever. In thisperspective, it would be a mistake to define learning in termsof internal representations.

Conclusions

We have illustrated a number of design principles forautonomous agents. The idea is that they should capturesome of our intuitions and some of the vast but highly dis­tributed knowledge that is around in various fields. Thoughnot (yet) very systematic, we hope that they will lead to a lotof discussion, and provide a starting point for further produc­tive theorizing

Acknowledgments
This research was supported in part by grant nb. 2040581.94 of the Swiss National Science Foundation.

References[1]

Braitenberg, V. Vehicles: experiments in synthetic
psychology. Cambridge, Mass.: MIT Press, 1984.

[2]

Brooks, R.A. Intelligence without reason. Proc. IJCAI91, pp. 569-595, 1991.

[3]

Clancey, W.J. Situated cognition. Cambridge:
Cambridge University Press (in preparation).

[4]

Dewey, J. The reflex arc in psychology. Reprinted in
J.J. McDermott (eds.). The Philosophy of John Dewey.
Chicago, IL: The University of Chicago Press, pp.
136-148, 1981. (originally published 1896).

[5]

Edelman, G.E. Neural Darwinism. The theory of neu­
ronal group selection. New York: Basic Books, 1987.

[6]

Milner, A.D., and Goodale, M.A. The visual brain inaction. Oxford: Oxford University Press, 1995.

[7]

Pfeifer, R. Cognition-perspectives from autonomous
agents. Robotics and Autonomous Systems, 15, pp. 4770, 1995.

[8]

Pfeifer, R., and Scheier, C. The metaphor of sensory­
motor coordination in autonomous agent design.
Robotics and Autonomous Systems. (in preparation)[9]

Pfeifer, R., and Verschure, P.F.M.J. Distributed adap­
tive control: a paradigm for designing autonomous
agents. Proceedings of the First European Conferenceon Artificial Life. Cambridge, Mass.: MIT-Press, pp.
21-30, 1992.

[10] Scheier, C. and Pfeifer, R. Classification as sensori­motor coordination. In: Proc. European Conference on
Anificial Life, ECAL-95, pp. 656-667, 1995.
[11] Thelen, E. and Smith, L. A dynamic systems approachto the development of cognition and action.
Cambridge, Mass.: MIT Press, Bradford Books, 1994.

- 145 -

Artificial Passive Vision using High Speed Pixel Matching
H. Hanaizumi, T. Tsumashima
College of Engineering,
Hosei University
Koganei, Tokyo 184, JAPANS. Fujimura
Graduate School of Engineering,
The University of Tokyo
Bunkyo-ku, Tokyo 113, JAPAN

Abstract

and the procedures of the method. Some results of the method
applied to the head of a wooden statue are also shown.

A system and a method for artificial passive vision areproposed to measure shape of small object with high accuracy
and high efficiency.
For camera calibration, a turn tablewith some different length rods are used. A spiral matchingalgorithm is used for high speed matching of stereo pair
images.

1 Introduction
We propose a system and a method for an artificial passive
vision, which measures the shape of small object and to record
it as a digital 3-D image [l]. In the vision, relative orientation,
absolute one and matching of all the pixels in stereo pair
images are essential. In our system, we use a turn table withsome different length rods for both orientations. Stereo pair
images are acquired by two TV cameras at a time. We
observe the object twice; once for observing rods (normal
image) and once for zooming in on the object (zoomed
image). We use the former for determining the orientationand the latter for achieving high spatial resolution in
observation the object. In matching process, we regard
matching as image registration[2].
In this paper, we detail our system and describe the principle

2 Principle and Procedures
In our vision system, 3-D shape of an object is measured
by following procedures : I) data acquisition for orientation,2) observation of the object with high spatial resolution, 3)high speed pixel matching with spiral scanning, and 4)
reconstruction of the 3-D shape.
2.1 Data Acquisition and Shape Reconstruction
Figure 1 shows the geometry of an object and two cameras
for observation. A point A(X, Y, Z) on the object is recorded
as points A 1 (u, v) on a film P, and A/s, t) on P2 , respectively,
with points on the reference plane. The relations among
these points are described by perspective transformations

and

S

t

Reference plane
Fig. I Geometry of object and cameras.

= dl X + d2 Y + d3 Z + d4
d9X + d lO Y + d11Z + 1_ d5 X + d6 Y + d7 Z + d8 ) •(2)

d9X+dl() Y+d11 Z+lParameters ci (i = 1-11) and di (i = I -11) are estimated byusing four or m ore referen ce points having known
coordinates. In our system, the camera calibration is carriedout by the determination of these parameters.
The coordinate� of the object point A is retrieved from twostereo pair images as the solution of equations- 146 -

z = U - C4

to it. The correlation coefficient is used as the index for.
• searching. The addresses of seJected correspond'mg pomts= V - c8
in IMG3 are transformed to those in IMG2.
. (3) The next is matching of all the pixels. Generally speaking,=·· al scannmg·( d1 - sd9 )X + ( d?_ - sd10 ) Y + ( d3 - sd1 1 ) Z s - d4matching all p1xe
· I s takes huge time. we adopt spir=
to achieve fast matching. The spiral scanning starts at one( d5 - td9) X + ( d6 - td10) Y + ( d7 - td11
t - d8of the pairs at the lattice point in both IMG 1 and IMG2. It
proceeds spirally along the edge of the area which have
In order to achieve high spatial resolution, we get zoomed
already matched as shown in Fig. 2. We use a checkered
images from the same position, and same value of thesemask for reducing the processing time for the correlation
parameters estimated above are used for retrieval. In thisprocessing. Scanning masks spirally along the edge iscase, film points A 1 (u*, v*) and A/s*, t*) in the zoomed images
effective in reduction of size of search area. We use 3 x 2
are described byrectangular area as the search area on the assumption that theobject has smooth shape and stereo pair image has lessdisparityin the line direction. There may occur miss-match• pc1 X + pc2 Y + pc3 Z + pc4
u =during the spiral scanning. When miss-match occurs, we
c9X+c10 Y +cu Z + 1
stop the current scanning and restart a new scanning at another)c• = p 5 X + pc6 Y + pc7Z + pc8
pair of lattice point. In order to detect the miss-match, we(4)v
produce a triangle net model to approximate the disparity
c9X + c10 Y + cu Z + 1
between IMG l and IMG2 as shown in Fig. 3. Matching
results for lattice point pairs are used for the production ofandthe rough surface model. We divide IMG 1 into many
triangles whose vertices are lattice points, and assume that
disparity g due to parallax is represented by a linear function
s* = qd1X + qd2 Y + qd3Z + qd4
of position (x, y) as
d9X +d10Y+d11Z + 1)(6)
g= ax+ f3y+ y,' = qd5 X + qd6 Y + qd7Z + qd8(5)t
in a triangle. A miss-match is detected when the difference
d9X + d10 Y + duZ + 1
between disparity obtained by the matching and that by the( cl - UC9 ) X + ( c?_ - UC10 ) f + ( C3 - UC1 I )( C5 - VC9 )X + ( C6 - VC10 )Y + (c7 - VC11)z)z

net model exceeds a threshold. There may. be remaining
unmatched areas after finishing the spiral scanning for all
the pairs of the lattice point. We interpolate these areas byusing surface spline [3].

respectively, where p and q are zooming ratios.
2.2 Pixel Matching
We regard matching of all the pixels in stereo pair images
as image registration. From the registration point of view,
stereo pair images have locally large disparities due to parallax
between them. In order to reduce processing time withkeeping high accuracy, we adopt hierarchical processes
rough registration, pre-matching and matching with spiralscanning.
We refer to one of the stereo pair images as IMG 1 and theother as IMG2. At first, we produce new image IMG3 byroughly registering IMG2 onto IMG 1 so that we can easily
search corresponding point pairs. In the rough registration,
the difference of spatial resolution, parallel shift and rotation
are removed. Next, we generate many lattice points on
IMG 1, and search their corresponding points on IMG3. Thissearching process is based on spatial correlation. We set asquare mask on a lattice point in IMG 1 and scan the same
size mask in a search area in IMG3. We make density scatter
diagram for pixels in both masks and assume a linear relation

3 Experiments and Discussion
3.1 Measurement system- 147 -

Figure 4 shows our measurement system. The system
Correlation Masks

Search Area

+ ...................fl++++ ......,,._..___...

IMG1• Matched Point

IMG2

Fig. 2 Spiral scanning of correlation masks.

Triangulation mesh
Matched pixel'-TV Camera�Dataacquisition
y

LED
Rod

Controller Table �otati�n,Turn Table
LED Lt htenmg ---'---'-------,

IMG1

Fig. 4 Measurement system.

Rough surface modelFig. 3 Triangle net model for miss-match detection.
consists of a tum table (30 cm in diameter) and four kind ofrods (25 cm, 20 cm, 15 cm and 10cm in length) with LED on
the top. These rods are removable but their positions can be
accurately determined. Rotation of the tum table and LED
lightening are controlled by a computer. A TV camera is
fixed at position about 1 m away from the object with look­down-angle of about 30 degrees, and is used for directacquisition of image data with size of 640 x 480 pixels. The
TV camera can observe the object with several zooming
ratios. For each observation, we get rod image without
zooming and object image with high zooming ratio. We
estimate parameters for the perspective transformation fromthe rod image, and compensate them by the zooming ratio
between rod and object images.
3.2 Experimental Results and Discussion
We measured 3-D shape of a wooden statue to confirm thevalidity of our method. Normal image and zoomed images
were acquired for every IO degrees of the rotation of thetum table. Figure 5 shows three examples of acquired images: left view (a) and (d), front view (b) and (e), and right view(c) and (f). We refer to images (d), (e) and (f) as IMG_A,
IMG_B and IMG_C, respectively, in the following
discussion. The zooming ratios were all 3.71. We obtained
lattice point pairs between IMG_A and IMG_B as shown in
Fig. 6, where space of the lattice was 30 pixels. We applied
the spiral scanning algorithm to image pairs (IMG_A,
IMG�B) and (IMG_B, IMG_C), respectively. While aconventional pixel-by-pixel matching process took 306
minutes to register them, each spiral scanning process took
only 20 minutes (we used HP-735). In the pixel-by-pixel
matching process, we used 31 x 31 pixel mask to calculate acorrelation coefficient , and used the search area of 60 x 10
pixels.
Figure 7 (a) shows an example for the registration (pixel
matching) result of IMG_A onto IMG_B. The result for
IMG_B and IMG_C is also obtained. We reconstructed threeshapes of the wooden stature from results for image pairs(IMG_A, IMG_B), (IMG_B, IMG_C) and (IMG_A, IMG_B,
IMG_C). For evaluating the performance, we compareddistances among 6 test points as shown in Fig.7 (b). Table I
indicates the evaluation results. From this evaluation, we
see that root mean square error (RMSE) of the relative shapereconstruction is about 0.5 mm. Locations of the test point
reconstructed from image pair (IMG_A, IMG_B) weredifferent from those from (IMG_B, IMG_C), and thesedifferences were lager than the RMSE above. This is due toinsufficiency in estimating parameters for the perspective
transformation.
4 Conclusions
We propose a system and a new method for matching all
the pixels in stereo pair images are proposed for artificial
passive vision. Three dimensional shape of the head of awooden statue is measured with high spatial resolution. The
spiral scanning algorithm in pixel matching achieves highspeed processing.
Improvement of accuracy in estimating parameters forperspective transformation and measurement of whole 3-D
shape of an object are subjects for a future study.
References[l] H. Hanaizumi, T.Tsumashima and S. Fujimura,"Application of Proximity Stereometry to Archeology - 3-D
Shape Measurement of the Head of a Wooden Stature -",Proc. Close Range Techniques and Machine Vision, Vol. 30,
No. 5, pp.150-154, 1994
[2] H. Hanaizumi, R. Takesaki and S. Fujimura, "An
Automated Method for Estimating Terrain Height from SPOT
Stereo Pair Images", Proc. Int. Geosci. Remote Sensing
Syrup., pp. 2167-2169, 1995
[3] R.L. Harder and R. N. Desmarais, "Interpolation using
Surface Splines", J. AIRCRAFT, Vol. 9, No. 2, pp.189-191,
1972- 148 -

(a)

(c)

(b)

(f)
directions, and zoomed images from
Fig. 5 Original images : normal images acquired from left (a), front (b), and right (c), IMG_B and IMG_C, respectively).
left (d), front (e), and right (f) directions (we call images (d), (e) and (f) as IMG_A
(e)

(d)

Table I Evaluation results of accuracy.
Point
Pair

(a)

Fig. 6 Lattice points on IMG_A (a) and IMG_B (b).

1, 2
1,3
1,4
1,5
1,6
2,3
2,4
2,5
2,6
3,4
3,5
3,6
4,5
4,6
5,6
RMSE

(a)

(b)

Fig.7 Registrtaion (pixel mathcing) result of IMG_A onto IMG_B (a) andtest points (b) for accuracy evaluation.

- 149 -

Point to point distance [mm]Data Pair
True
B&C A&B&C
A&B
Value

27.0
14.0
15.0
28.0
40.0
18.5
30.0
30.0
33.5
13.0
17.0
27.0
16.0
28.0
14.0
-

27.7
13.7
14.9
28.3
40.2
18.7
30.7
30.2
33.6
12.6
17.5
27.8
16.4
28.7
13.9
0.45

27.6
13.9
15.2
28.6
40.7
18.5
30.7
30.3
34.0
12.7
17.7
28.0
16.6
29.0
14.1

0.58

27.7
13.8
15.0
28.4
40.4
18.6
30.8
30.3
33.8
12.7
17.6
27.9
16.5
28.8
14.0
0.51

A Fuzzy Adaptive Model of Emotion and Personality for an Autonomous Robot
Kenneth H. L. Ho
Bio-mimetic Control Research Centre,
RIKEN (The Institute of Physical and Chemical Research),
8-31 Rokuban 3-Chome, Atsuta-Ku, Nagoya, 456 JAPAN,
email: ho@nagoya.bmc.riken.go.jp,
Abstract

circumstances based on our own experience.
In this paper we will try to use a model of emotions
to choose between different behaviours for our mobile
robot under different circumstances. Our robot will
have a set of external sensors and a set of internalsensors. It will use these sensors to evaluate its cur­rent position and its surrounding environment. It will
try to perceive risks based on its own experience andmodify its own mood or feeling, i.e. emotion. If it
perceives dangers to a certain degree, it will shift its
mood toward fear. By accumulating these effects, we
can create different moods for different circumstances
based on past experience. Different sets of behaviours
are attached to these moods. When the robot requires
instant decision on its action due to external or inter­
nal stimulus, e.g. low battery power or approachingobstacles, it will try to choose the set of behaviours
which associate with its present feeling or mood.

This paper proposes a fuzzy adaptive model of emo­
tions for an autonomous mobile robot. We will argue
in this paper that emotion is one of the most impor­tant attribute for an autonomous intelligent machine.
If a robot has to survive in this unpredictable world, it
has to adapt and to learn. We propose a simple fuzzymodel of emotion for an autonomous mobile robot.
The robot will learn its emotional behaviours throughits own experience. Therefore, after a certain time therobot will start to express preferences of its behaviours
based of its emotional model and its own experience.
In our view, if the robot exhibits a preference of emo­
tional behaviours, then it has acquired a personality.

1

Introduction

Today robots are used for mass production of prod­
ucts with similar features. Robots operate in facto­ries where there is no human intervention or involve­
ment. In the future robots have to work with people
in an environment which is designed to be used byhumans. Therefore, robots have to aware of its sur­roundings. They have to operate safely without harm­
ing themselves as well as others surrounding them.
These robots are also likely to work outside of a fac­
tory, therefore they have to work in an uncertain andrapidly changing environment.
Current obstacle avoidance algorithms in robotics
assume a static environment with predefined obsta­
cles. Most solutions provide optimal paths. Unfortu­
nately in reality, we seldom use an optimal solution
for obstacle avoidance. If there is a big object runningtowards us, we will normally try to avoid it by getting
away from it as much as possible instead of finding
an optimal distance. Similarly if there is a child run­ning after us, we will probably let the child run into
us rather than to avoid him or her. Therefore, wechoose different behaviours and strategies in different

2

Emotion - the missin g link in an au­
tonomous intelligent system

The motivation for this approach is based on thefact that emotions only exist on higher living organ­
isms, e.g. humans and animals. They are createdby billion years of evolution. Intelligent animals seemto be able to express and possess more complex emo­
tions, for example humour is only expressed by humanbeings. Although emotions are normally regarded as
irrational behaviours, they help us to survive and to
evolve within our very uncertain and unpredictableworld.
If we accept Darwin's evolution theory [2, 3), i.e.
natural selection or the survival of the fittest, thenwe can see that emotion is not just a by-product ofthe development of intelligent species. If they are justa by-product or irrelevant, emotion would have been
elminated by natural selection a long time ago. Using
the same argument, we can see that if mathematical- 150 -

logic is a basic attribute of an intelligent system, weshould see more logical species within our own world.
Since we cannot find any life forms which utilise math­ematical logic, we can conjecture that mathematical
logic may not be an important attribute in creating
artificial intelligence.

3

solution for our problems. That is why we sometimes
view our emotional decision as irrational.
Characterisitc of emotional control:• no reasoning• short term
• interact between the external events and our own
internal states of a being. One has to aware ofoneself. Therefore, this is another argument for us
to have internal sensors to evaluate our personal
position given a set of external events.

Emotion and Behaviours

Modeling emotions has recently become fashionable
within the AI and robotic research community. Mar­vin Minsky[7) outlined his idea of modeling emotion in
an autonomous system. Mochida et al[5) and Pfeifer[9)tried to model emotion on a mobile robot in order tocreate more than one behaviour for a given environ­ment. Gomi[8) tried to make emotion as an emergent
property within the Patti Maes's [6] Action Selection
Dynamics framework.

3.1

Fuzzy Behaviours and Instincts

The famous psychologist William James[4) sug­gested that instinct is different from Emotions. In­
stinct is similar to reflexes. In our view, behaviour­based robotics proposed by Brooks[l) can be treatedas a model of instincts. Events are linked together withactions. Reflexes and instincts are parallel and they
do not need logical analysis. Instincts can be higher
level reflexes. Therefore, we can be able to think thatemotions are higher level instincts.
In our model, different instincts, reflexes and bee.
haviours are defined as fuzzy membership functionsof the robot sensors. They are represented by fuzzy
IF-THEN rules similar to one used by Tunstel[ll, 12].
Unlike traditional robotic control algorithms which aremostly based on binary logic reasoning, our behaviours
are governed by fuzzy if-then rules. We would like to
use fuzzy reasoning to model emotions. The reason isthat fuzzy logic is designed to model human impreci­
sions as well as vague concepts.

3.2• One also tries to give out signal to the oth­ers within ones surroundings. Therefore, one
can project ones desire through emotions with­out communicating through language. Anger isa good example in which one will try to tell oth­ers that one is dangerous without wasting a lot ofenergy or effor to fight for what he or she wants.
• Emotional control exists in many animals which
are less developed than us.
External Sensors

Internal Sensors

External EVflnts

Internal Statuse.g.

(not real time)

updating

I
I
I
I
I
I
I
I

obstacles app-oac;hingsurrounding light level is dark
a.irrent position - in known co-on:16.
road condition&, etc.

real tine=1r·········I------·-·--.
FflBllngs or Moods
I1
2. happy

3. &ad4.lrustration

inetenti----------------------.

I

I1

•
..•••••••••♦:

1. fearModelling Emotions:

Sets of Behaviours&i!uations depend on thea.irrent mood or feeling of
the robot

Possible Actions
Figure 1: Modelling Emotions
In our model of emotion (see figure 1), we have ahierarchical structure of emotions. If instincts and re­flexes can be grouped togther and can be categorised- 151 -

I

I
I1

:

I

l........1············J
lnstlne1B

5. despair or lost

Emotions can be treated as sets of basic instinctsor behaviours. Emotion is our natural reaction when
we are confronted with a complex environment. It is abuilt-in mechanism for us to react to situations which
will require us to act upon immediately. It is our basic
software which allows us to act on situation without
analysis. It helps us to adapt to our ever changing
environment. However, it does not provide a long term

comlncuslyenergy level - i.e. beltery levelrisk level
pnicelved threata.irrent statedesired state, etc.

e.g.

into different groups, we can treat each of these groups
as an emotional state, then each individual emotion
has a set of behaviours which are linked directly withthe external sensors of the robot. Therefore, emotional
state "happy" will have a set of behaviours and "fear"
will have another set of behaviours. At any one time,
the robot will only associate with one emotional state,i.e. the robot will be either happy or in fear, although
it is quite possible to extend our model and allow suchemotiona,l state to overlap with each others. There-­fore, at any one time, a robot will have only one set of
behaviours to choose from. If an external event trig­
ger a reaction, the robot will then activate a behaviour
from the current emotional state and ignore all otherbehaviours in other emotional states.
The emotional state of the robot can only be af­
fected by the internal sensors. In our view, emotion isa manifestation of our internal sensors. Since internalsensors depends on changes of the external sensors,emotional states are indirectly connected to the exter­
nal sensors (see figure 1).
An emotional state will change from one to theother depending on its internal sensors. It is actually afunction of the status of its internal sensors over time.
When certain sensors remain at the same status overa period of time, e.g. risk level is high and battery
level is low, it will trigger a change in its emotional
state. This is defined by using a set of fuzzy if-thenrules. Therefore, certain external events may cause an
accumulation of effects on these internal sensors whichindirectly will cause a change in its emotional state.
These set of fuzzy if-then rules can then be learned
by the robot through its own experience. If it findsthat the action that it took causes undesirable effect,
it will penalize the previous rule. Therefore, over time
the robot will try to form its own preferences based on
its experience. Different experiences will form differ­ent set of membership functions, hence different robots
will form different personalities depending on their ex­periences.
It is similar to human beings, some people are al­ways happy while other are more pessimistic. There-­fore, one can learn to have different weight on ones
emotional system. Some may put more weight on thefear emotion because one has had very bad experience
while others may be more brave because he or she has
more success on taking gambles. Therefore, one candefine ones personality by modifying its intensity ofones emotional system.
This adaptation of our emotional system can beused in obstacles avoidance. If there is a fairly large
object coming toward two robots. One robot has a op-

timistic personality, it will assume that the object will
also detect the presence of the robot and hence no needto do anything. On the other hand, the robot with apessimestic personality will assume that the large ob­ject may harm itself. Therefore; it will try to plan
an avoidance path. In this case, it will try to be as
far away as possible from the predicted position of the
obstacle. This path planning is quite different fromthe normal optimal path planning procedures of mostmobile robot, since optimal path is not necessarly thesafest path.

4

Conclusion

In this paper we have proposed a fuzzy adaptivemodel of emotion for an autonomous mobile robot.
We outlined that the robot will form different sets ofemotions based on different experiences. We believethat an emergent property of a personality is exhib­
ited by the robot. In the future, we would like to see
more complex expression of emotions emerges from ourrobot, e.g. expression of affection or fear for people,
etc.

References[1] Brooks R.A., A Robust Layered Control System fora Mobile Robot, IEEE Transactions on Robotics
and Automation, RA-2, No.l, p. 14-23, 1986
[2] Darwin C., The Origin of Species, John Murray,
1859[3] Dawkins R., The Selfish Gene, 2nd Edition Oxford
University Press, 1989
[4] James W., The Principles of Psychology, Vol.1,2,
Henry Holt, 1890
[5] Mochida T., Ishiguro A., Aoki T., Uchikawa
Y., Behavior Arbitration for Autonomou; Mobile
Robots Using Emotion Mechanisms, .Proceedingsof the 1995 IEEE/RSJ International Conference onIntelligent Robots and Systems (IROS'95), Pitts­
burgh USA, Vol.3 p.516-521, 1995
[6] Maes P., A Bottom-Up Mechanism for Behavior
Selection in an Artificial Creature, Proceedings ofthe 1st International Conference on Simulation of
Adaptive Behavior, also in: Meyer J.A.,Wilson
S.W., "From Animals to Animats", p.238-246,
MIT Press, 1991

- 152 -

[7] Minsky M., Autonomy and Emotion Machines,
IEEE/RSJ International Conference on Intelli­gent Robotics and Systems (IROS'93), Yokohama
Japan, Plenary Session, 1993
[8] Gomi T., Ide K.1., Emulation of Emotion Using
Vision with Learning, Proceedings of the IEEE In­
ternational Workshop on Robot and Human Com­
munication (RO-MAN'94), p.210-215, 1994
[9] Pfeifer R., Emotion in Robot Design, Proceedingsof the IEEE International Workshop on Robot andHuman Communication (RO-MAN'93), 1993[10] Toda M., The Urge Theory of Emotion and Cog­
nition - English version, School of Computer andCognitive Sciences, Chukyo University, 101 Toko­date, Kaizu-cho, Toyota, 470-03, JAPAN 1994
[11] Tunstel E., Coordination of Distributed Fuzzy Be­
haviors in Mobile Robot Control, IEEE Interna­
tional Conference on Systems, Man and Cybernet­
ics, vol.5, p.4009-4014, 1995
[12] Yanaru T., Hirotja T., Kimura N., An Emotion­
Processing System Based on Fuzzy Inference andits Subjective Objections, International Journal of
Approximate Reasoning, Vol.10, No.l, p.99-122,
1994- 153 -

A Parallel Genetic Algorithm for Matching and Similarity Measure of Graphs
Yong Wang
N aohiro Ishii
Department of Intelligence and Computer Science
Nagoya Institute of Technology
Gokiso-cho, syowa-ku, Nagoya, 466, Japan
Abstract
Graph matching and similarity measure of graphs
have many applications to pattern recognition, ma­chine vision in robotics and similarity-based approx­
imate reasoning in artificial intelligence. This paper
proposes a method of matching and similarity measure
between two directed labeled graphs. We define thedegree of similarity, the similar correspondence andthe similar map which denotes the matching between
two graphs. As a approximate computing method, weapply genetic algorithms(GA) to find the similar map
and compute the DS between graphs. For speed-up,
we make parallel implementations on almost steps of
GA. We have implemented our method in a parallel
C program and made a simulation on APlO00, a highparallel computer by Fujitsu. The simulation resultshows our method is efficiently and usefully.
Key Words: Graph Matching, Similarity Measures,
Genetic Algorithms, Parallel Computing, similarity­
based approximate reasoning.

graphs. We start from defining the correspondenn'
and map from one graph to the other based on thesame concepts in set theory. Then, we define the de­gree of similarity(denoted by DS) of graphs on a cor­
respondence. Based on this, we define the DS, thesimilar correspondence and the similar map which de­notes the matching between graphs.
It is clearly that the problem of computing the DS
and the similar map between graphs is a kind of com­binational optimization problem. As a approximate
computing method, we apply genetic algorithm(GA)to our problem. For speed-up, we make parallel im­
plementations for almost steps of GA.
In section 2, we give all definitions and theorems
used in this paper. In section 3, we propose a sequen­
tial GA to computing the DS .of graphs and find thesimilar map between graphs. Section 4 describes theparallel GA. Section 5 presents the simulation and dis­cusses the results.

2
1

Introduction· Graph matching and similarity measure of graphs
have many applications to pattern recognition, ma­chine vision in robotics and similarity-based approx­
imate reasoning in artificial intelligence. Obviously,graph matching and similarity measure of graphs are
two close related problems. But many methods havestudied only take one as the problem, and therefor
can not fit to the other. For example, if the parts of
two graphs are not isomorphic, then graph matching
methods, many of them are based on subgraph isomor­
phism, will fail to make matching on the parts. But forsimilarity measure of graphs, an appropriate matchingon the parts will be needed. Since these two problems
can not be separated in many cases, we should study
some methods for the both problems.
This paper proposes a method for both matchingand similarity measure between two directed labeled

The DS and the Similar Correspon­
dence between Graphs

In the definitions and theorems showed in this sec­tion, G denotes a directed labeled graph. V is the setof vertexes and E is the set of edges. The numberof vertexes of graph Gare denoted by NN(G). The
weight of a vertex v and an edge e are denoted by
W(v) and W(e), respectively. Let W(<t>) = 0.
Definition 1 The correspondence from graph Gt to
G s is denoted by C : Gt --+ Gs and defined as follows:{1). 'rlv t E ½, Vt correspond to only one tiertex V s inGs (denoted by C(v t ) = V s }, or does not correspondto any vertex in G s (denoted by C(vt) = 4>). And
if V1, V2 E ½, v1 =/. v2, C( V1) =/. q>, C(v2) =/. q> thenC(vi) =/. C(v2).
(2). The correspondence of edges is decided by thecorrespondence of vertexes, that is: Ve t = (V t ,, Vt 2) E
Et, if C( Vt 1) = Vs 1 =/. <P, C( Vt 2) = V 82 =/. (/) and es =- 154 -

(Vs p V s J E Es , then et correspond to e s (denoted byC(et) = es ), else Et does not correspond to any edge in
G s {denoted by C(et) = </J).

The inverse correspondence of the correspondence

C is denoted by c- 1.

Definition 2 Let 1{ be a correspondence from graph
Gt to Gs . We call 1{ the map from Gt to Gs if Vvt E
Vi, H(vt) =f <p, i.e. H(vt) E Vs )Let C be a correspondence from graph Gt to Gs ,and let Vt E ½, V s E ¼, et E Et, es E Es . The
DS of vertex Vt and V s on correspondence C is de­noted by DSc(Vt,V s ), and the DS of edge et andes on correspondence C is denoted by D Sc (et, e s ).
They should be decided beforehand. For example, letDSc(vt,v s )= 1 if Vt= V s , or O otherwise.
Definition 3 Let C be a correspondence from graph
Gt to Gs . The DS of Gt and G s on C is denoted by
DSc(Gt,G s ) and defined as follows: DSc(Gt,G s ) =Fn +Fewhere
Mn + M.
eFnF,= L W(vt) + ;(C(vt)) DSc(vt,C(v1))=W(et) + (C(ei)) DS;c (e1,C(et})= max( L W(vt), L W(v.))
VtEVt

M.

v 8 EV8= max( L W(et), L W(e,))
etEEt

e 8 EE 8

Definition 4 Let C be a correspondence from Gt to
G s and Vt E Vi- The DS of graph Gt and Gs on C at
vertex Vt is denoted by DSc(Gt,Gs )l v , and defined asfollows:

where G(v) is the subgraph of G, which consists ofvertex v and the edges take v as their endpoints.

Definition 5 The DS of graph Gt and Gs is de­noted by DS(Gt,G s ) and defined as DS(Gt,G s ) =maxD Sc(Gt,Gs ), where C is a correspondence fromC

The above theorems can be derived easily from re­lated definitions. It is also not difficult to derive thenext theorem by the above theorems.
Theorem 3 If NN(Gi ) � NN(G s ) the,,,,where 1{ is a map from graph Gt to G s .

Using definitions and theorems described in thissection, we present the method of computing the simi­
lar correspondence and the DS between graphs in next
subsection.

The Sequential GA

From theorem l; we can assume NN(Gt) �
NN ( G 8). By theorem 3, we only need to search maps

•tEEt

Mn

Theorem 2 Let 1{ and C be map and correspondence
from graph Gt to Gs , respectit1ely. For all v where v E
l-t, C(v) =I </>, if C(v) = H(v) then DS'H(Gt,G s ) 2:':DSc(Gt, Gs)-

3

v,EVt

L

= DSc-1(Gs ,Gt) andDS(Gt,G s ) = DS(G s,Gt)- where C is a correspon­
dence from graph Gt to Gs .

Theorem 1 DSc(Gt,Gs )Gt to G s .
If DSc0 (Gt,G s ) = DS(Gt,G s ), then Co is called thesimilar correspondence from Gt to G s , or the similar
map /rQm Gt to G s if it is a map.

to find the similar map and computing the DS between
graphs on the similar map. It is clearly that this prob­
lem is a kind of combinational optimization problem.
We apply GAs to this problem because they can findbetter solution fastly in combinational optimization
problems.
In the following, we propose the genetic algorithm
for our problem.
Let Gt and 'G s denote two directed labeled graphs
and assume NN(Gi ) � NN(G 8 ).
Coding maps from Gt to Gs : For every graph,
we number all vertexes of that for distinguishing them.
Each map from Gt to Gs is encoded by a string of
numbers with length NN(Gt)- Each number in thestring denotes a vertex of Gs , and its ordinal ·number
denotes the corresponding vertex of Gt, For example,
the string 23568 denotes that vertex 1, 2, 3, 4 and 5 ofGt correspond to 2, 3, 5, 6 and 8 of G s , respectively.
Population model: We set the population size as
NN(Gt)+NN (G s ). The fitness of an individual is set
as the DS between graphs on the map which is encodedby the individual (refer to definition 3). The fitness of- 155 -

a gene in an.individual is set as the DS between graphson the map at the vertex of G t correspond to the gene(refer to definition 4). The fitness of a populationis set as the maximum fitness of individuals in the
population.
Stopping condition: If the same fitnesses for2(NN(Gt ) + NN(G s )) populations running are ob­tained, then bring the genetic algorithm to a termina­tion.
Initialization: The start population is initialized
randomly un.der the following conditions:(1). Each individual is made under the following con­dition: if there are the same labels for vertex Vt in Gt
and vertex Vs in G s , then let v 1 correspond to V 8 in
the map encoded by the individual. By this way, it
can be expected to find the similar map fastly.
(2). Do not make the identical individuals: It is forkeeping diversity of individuals in a population. For
doing this, if an individual which is equal to anotherwas made, then mutation operates on it until it is dif­ferent from all other individuals.
Selection: All individuals in the population are
sorted by their fitnesses. Let n = NN(G 1 ) + NN(G s )which is the population size. In our method, all indi­viduals are selected for mating. We perform crossoverto the ith individual and the (i + l)th individual togenerate [(n + 1)/2] individuals of the next popula­tion, where i = 1, 2, ..., [(n + 1)/2], then the individ­
uals with higher fitnesses could be generated because
the fitnesses of parents are higher. Also, we perform
crossover to the jth individual and the (n + 1 - j)th
individual to generate [n/2] individuals of the next
population, where j = 1, 2, ..., [n/2], then the averagefitness of individuals in the next population could be
increased because the fitness of one parent is higher
and the fitness of another is lower.
Crossover:Two selected individuals are calledparent-A and parent-B, and assume the fitness ofparent-A is higher or equal to that of parent-B. Ev­
ery crossover results in one offspring. We present themethod of crossover in following.
(1). Decide genes inherited from parents: In parent­
A, Genes whose fitnesses are equal to or greater thanthat of corresponding genes of parent-B, are copyed
to the offspring. The other genes in the offspring are
inherited from parent-B. By this way, the genes withhigher fitnesses in the individuals with higher fitnesses
can be left to individuals in the next population.

(2). Avoid that there are the same genes in the off­
spring: Maybe there are the same genes in the offs
spring. By the definition of map from Gt to G s , thissituation must be avoided. If a gene inherited fromparent-B is equal to one inherited from patent-A, thenchange this gene into one of genes which do not appear
in the offspring, randomly.
(3). Let the fitness of the offspring be equal to or
greater than the fitness of parent-A: If the offspringwhose fitness is less than that of parent-A was gener­
ated by the way described in the above, then eliminat­
ing it and use parent-A as the offspring to leave to thenext population. By this way, it is certain that thefitness of the next population will be equal or greaterthan that of the previous one, and the average of fit­nesses of individuals in the next population will beequal or greater than that in the previous one.
Mutation: The probability of mutation is 10%. In
our method, mutation means that replace a gene by
a new gene not in the individual, or exchange it withanother in the individual. The gene chosed for muta­tion will pick out from genes whose fitnesses are lower
than that of the individual by the probability of 80%,or from the other genes by the probability of 20%.
Differentiation of the same individuals: If there
are two individuals then mutation will operates on one
until there are not same individuals in the population.
In this case, the gene chosed for mutation will be de­cided randomly.

4

The Parallel GA

Since the offsprings are produced independently ineach generation, the most steps of GAs are easily madeparallel. The parallel implementation of the GA de­scribed in the above section is showed in the following.
It consists of a host program and a cell program which
is run parallel by a number of cells under control of thehost.
------- Host Program --------....

- 156 -

send population size p and graphs to all cells.
while ( stopping condition not met) {receive fitnesses of all individuals from cells.
if (stopping condition not met)
send go-ahead message to cells.
else
send finish message to cells.

}

receive final population from cells.

------- Cell Program ------­

about 99% graph pairs, the similar map and DS ob­tained by the GA is equal or very close to that we de­fined. The average, maximum and min minimum run­ning time( CPU time) are 29.2998 seconds, 154.0580
seconds and 0.6266 seconds. These results shows our
method is efficiently and usefully as a approximate
computing method.

receive p and graphs from host.
initialize the subpopulation whose size is Pi•
while (finish message is not received) {if ( C elll D = 0) send 0 to itself.
while (go-ahead & finish message not received) {receive a message.
case an individual is received:
if ( there is a same previous received one)mutate and sent it to related cell.
else
insert it to previous received individuals
and send its fitness to host.
case an number n is received:
send n + Pi to the next cell.
send individuals as parents to related cells.
case a parent is received:
if (all parents are received)
implement crossover and mutation.
}}
send final subpopulation to host.

5

6

Conclusion

Graph matching, as applied to pattern recognitionand computer vision, is an NP problem. This paper
proposed a method of graph matching with similarity
measure of graphs using GAs. our method is based on
directed labeled graphs which has many applications
such as knowledge representations and flowchart. It
can easily be fitted to other graphs. Our method canalso be applied to analogical reasoning and case-based
reasoning when taking graphs as knowledge represen­tations. The parallel implement described in this pa­
per is only for reducing running time. We will consider
to improve the quality of solutions in the next work.
Using the method to build a real application system
is also left as our future work.

Simulation

We have implemented our method in a C programand made a simulation on APl00O, a high parallel
computer with 32 cells by F�jitsu. For simulation,
we made 1000 pairs of graphs. the number of vertexes
in a graph is bonded from 5 to 50. each pair of graphG t and G s is made by the following:(1). Making Gt randomly.
(2). Copying G t to G s . Now we get a map on which
each vertex of Gt corresponds to the same one of Gs .
we call it model-map.
(3). Revising G s randomly with keeping the model­
map. the revision number is equal or less than halfof sum of numbers of vertexes and edges of Gt anddecided randomly. the revision operators are deletion,
insertion and substitution (includes exchange).
The DS between graphs on model-map is called
model-DS between graphs. obviously, the model-map
is equal or close to the similar map, especially forsmaller number of revision.. In the simulation, we com­pare DSaA which denotes the DS between graphs obs
tained by GA to DSma de l which denotes the model-DS
between graphs. We divide 1000 pairs of graphs into
groups by the difference d = DSma de l - DSaA, andevaluate numbers of groups. The simulation re�ult
shows that, there are 375 graph pairs ford E (-0.1,0),614 graph pairs ford= 0, 6 graph pairs ford E (0, 0.1)
and 5 graph pairs ford E (0.1, 0.3). It means that, for

References[1] Almohamad, H. A. and Duffuaa, S. 0., "Linear
Programming Approach for th Weighted Graph
Matching Problem." IEEE Trans. on PatternAnalysis & Machine Intelligence, Vol. 15, No. 5,
pp. 522-525, 1993.
[2] H. Bunke and B. T. Messmer, "Similarity Mea­sures for Structured Representations." Lecture
Notes in AI 837, Topics in Case-Based Reasoning,
Springer-Verlag, pp. 106-118, 1994.
[3] H. Kitano, Genetic Algorithms(in Japanese).
Sangyou Tosyo, 1993.
[4] Richard E. Blake, "Partitioning Graph Matchingwith Constraints." Pattern Recognition, Vol. 27,
No. 3, pp. 439-446, 1994.
[5] Robert D. B., Gareth J. and. Peter W., "Matching
Two-Dimensional Chemical Graphs Using Genetic
Algorithms." J. Chem. Inf. Comput. Sci., Vol. 34,
No. 1, pp. 63-70, 1994.
[6] Samir W. M. and David E. G., "Parallel Recom­
binative Shnulated Annealing: A Genetic Algo­rithm" Parallel Computing, 21, pp. 1-28, 1995.

- 157 -

Fine Motion Strategy for Skill-Based Manipulation
Akira Nakamura*
Takashi Suehiro**

Tsukasa Ogasawara*
Hideo Tsukune**Electrotechnical Laboratory**Real World Computing Partnership

1-1-4 Umezono, Tsukuba, Ibaraki, 305 Japan

1-6-1 Takezono, Tsukuba, lbaraki, 305 Japan

Abstract
Generally, a manipulator task can be divided into
several motion primitives called "skills". Skill-based
motion planning is an effective way to execute· acomplicated task. W hen planning an assembly process, atechnique of fine motion planning such as the
backprojection method in configuration space is often
used. This paper describes fine motion planning using askill library, which consists of a pattern of trajectories ofskill motions in configuration space.
Key words: manipulation skill, fine motion planning,
backprojection, skill library

1 Introduction
In the future, robots will play an increasing role in ourlives and will perform many tasks using an artificial
brain instead of people. However, it is still difficult for
robots to carry out even simple human tasks such as
cleaning, cooking, dishwashing, handicraft and carpentry.
In order to make robots in the home a reality, it isnecessary to realize these tasks by using special
techniques of manipulating robots. By analyzing humanmotions in tasks such as assembly and disassembly,
movements were found to consist of several significant
motion primitives [1],[2]. We called these "skills" andexplained how most of the tasks of the manipulator canbe composed of sequences of skills. That is, wedemonstrated that robots can perform human tasks byusing the concept of skill.
On the other hand, techniques of fine motion planning
performed in configuration space are often used forplanning assembly processes. In configuration space, anobject is represented as a point to simplify the planning.
Lozano-Perez et al. proposed the concept of pre-image,and carried out planning using generalized damping withuncertainty of the sensor, control and model [3].
Erdmann proposed the backprojection method, in which
the goal region is projected in reverse using an error cone[4], which made it easier to obtain the reachable region
to the goal than using the pre-image. Some methods ofplanning with rotation in configuration space have also
been proposed [5],[6]. However, there is no method
using the concept manipulation skills.
In this paper, we propose a fine motion planning forvarious tasks composed of several manipulation skills,
using a skill library which consists of patterns oftrajectories of skill motion in configuration space. A
skill sequence composing the task and the initial
position and orientation from which the object can reachthe goal can be obtained by this method.
In the next section, we explain the concept ofmanipulation skills. The representation of skills in
configuration space is shown in section 3. Constructionof the skill library represented by trajectories in
configuration space and the composition of a task are
explained in section 4. Two examples, a peg-in-hole task
and unfastening of a bolt using a wrench, show theeffectiveness of our method in section 5.

2 Manipulation Skills
In the execution of a manipulation task, its motion
can be divided into several motion primitives, each· ofwhich has a particular target state. We call theseprimitives "skills", and in each primitive there is atransition from one state to another state [l],[2]. Most
manipulation tasks can be realized by a combination ofskills. A set of skill primitives is regarded as one
component of the control hierarchy of manipulation. The
skill layer is placed between the servo layer and the task

- 158 -

"vertex-to-face" contact to "edge-to-face" contact (Fig.2).
The accomplishment of this skill is detected by an
abrupt change of the instantaneous center position.
(3) "Rotate-to-insert" Skill(a) move-to-touchr skill

In an insertion task, it is generally difficult to achieve
directly the state of Fig.3(d) in the case of a small
clearance. First, the transition from the state of Fig.3(a)to the state of Fig.3(b) is performed by using the move­
to-touchf skill, and the state of Fig.3(c) is achieved byusing the move-to-touchc skill. Then, the state ofFig.3(d) is accomplished by gradually raising the object
while keeping the contact state of Fig.3(c). The rotate­
to-insert skill means this motion of rotating the object
obliquely into the hole in order to carry out theinsertion. The accomplishment of this skill is also
detected by an abrupt change of the instantaneous center
position. In this paper, we assume that the rotate-to­insert skill includes the press motion to the goal of an
insertion task (Fig.3(e)). Therefore, this skill is
continued until the resistance in the direction of pressing
increases sharply.

(b) move-to-touchc skill

Fig.1 Move-to-touch skill

(b)

(a)

(c)

Fig.2 Rotate-to-level skill
program layer, so it is possible to program a task as asequence of skills, independent of the control of theservo layer.
Skills in which the contact states vary at assembly anddisassembly tasks are particularly significant. In thispaper, we consider three skills of "move-to-touch","rotate-to-level" and "rotate-to-insert" which play animportant part in such tasks. We consider skill motions
in two-dimensional environments.

3 Skills in Configuration Space(1) "Move-to-touch" Skill
The move-to-touch skill means the transition from"free" to "vertex-to-face" contact between a grasped
object and another object (Fig.1 (a)). The similartransition of keeping the contact in a different directionof motion is also included in this skill (Fig.1(b)). These
are represented by the "move-to-touchr" skill and the"move-to-touch/ skill, respectively. These skills arecarried out in velocity control mode, and the contact is
detected by the increased resistance in the direction of
motion.
(2) "Rotate-to-level" Skill
The rotate-to-level skill means the transition fromIn this section, we consider the trajectory of skill
motions in configuration space.
(1) "Move-to-touch" Skill
In configuration space, the trajectories of the object by
the move-to-touchf skill and the move-to-touchc skill are
drawn in Fig.4(a). Considering the uncertainty of controlof the manipulator with respect to the move-to-touchf
skill, the trajectory using the control uncertainty cone isdrawn in Fig.4(b).
(2) "Rotate-to-level" Skill
For simplicity, we suppose that the reference point OA
is a vertex in contact with the surface in advance (Fig.2).
Then, the position of OA on the YZ-plane int

8 tl a fi"lII
(a)

(b)

(c)

(d)

(e)

control.
uncertamty

cone�

move-to­touchrCB(a)Fig.4 Move-to-touch skill in C-space

Fig.3 Move-to-touch and rotate-to-insert skills- 159 -

derived region as a subgoal, another backprojection isdrawn. Repeating this procedure, a skill sequence
composing the task and initial position and orientationfrom which the object can reach the goal can beobtained.

5 Example

�yFig.5 Rotate-to-levelskill in C-space(I) Peg-in-Hole Task

Fig.6 Rotate-to-insert
skill in C-space

configuration space is kept constant, even if theorientation 8 of the object varies within a negativevalue by the rotate-to-level skill (Fig.5). On the other
hand, after the orientation 8 = 0 where edge-to-facecontact is just achieved, the transition of the position of
OA occurs. However, in the ideal case, the motion of themanipulator stops and this trajectory in the configuration
space terminates at this boundary 8 = 0, so thisunnecessary transition does not occur. Therefore, we
consider the ideal trajectory without overhang, though
the manipulator goes beyond this in practice because of
the state detection based on the change of theinstantaneous center position.

This example shown in Fig.7 is similar to theinsertion task described in (3) in section 2, and we willshow the initial positions and orientations from which
the object is guaranteed to reach the goal by the
instructed sequence of skill primitives. First, C-obstacles
CB in 8 = [0, rr /2) are derived as shown in Fig.8, andthe back trajectories (Fig.9(b)) from the goal (Fig.9(a))
by a practicable rotate-to-insert skill are obtained.
Secondly, regarding the region of Fig.9(b) as a subgoal,
the backprojection by a practicable move-to-touchc skillof the transition in the y-direction while pressing in the

8 = 1r: /2(3) "Rotate-to-insert" Skill
In configuration space, the trajectory of the object by
the rotate-to-insert skill is shown in Fig.6. The transfer
motion of a vertex from Fig.3(c) to Fig.3(d) is done at
the orientation 8 = 8 1 in Fig.6, where the phase of C­
obstacle CB changes, and passing this point can bedetected by a change of the instantaneous center position.

Fig.7 Peg-in-holeFig.8 C-obstacles

4 Construction of Skill Library
and Composition of Task
First, the skill library is constructed in advance. The
trajectories drawn in configuration space are derived forseveral skill primitives such as the above-mentioned
skills. A skill library which consists of skill primitives
expressed by trajectories in configuration space is thus
constructed.
Next, the skill sequence required to perform the
instructed task is derived. First, some skill which can be
performed in configuration obstacles is selected, and abackprojection is drawn by projecting the trajectory fromthe goal in the reverse direction. Second, regarding the- 160 -

Goal

(a)

(b)

(c)

(d)

Fig.9 Backprojection in each skill

l)�
�t(
touch

insertHowever, even under such unfavorable conditions, thistask composed of two skills succeeds if it starts from theinitial region.

-

6 Conclusion

tilt

Fig. IO Unfastening of a bolt by a wrench

Fig.11 C-obstacles

Fig.12 Backprojection-z-direction can be derived (Fig.9(c)). Next, thebackprojection from this region by a practicable move­
to-touchf skill of the transition in the -z-direction can bederived (Fig.9(d)). Although the area of backprojection
becomes larger as the value of 0 becomes smaller, too
small a value 0 is undesirable for the reason of (3) insection 2. The initial position and orientation are decided
by considering this equilibrium.
(2) Unfastening Task of a Bolt Using a Wrench
Let us consider unfastening a bolt using a wrench(Fig. I 0). In this example, we deal with the stages oftouch and insertion. We suppose that a bolt is a movable
object and a wrench is a fixed obstacle, opposite to thereal world. Assuming that the reference point OA is thecenter position of the bolt, the C-obstacles CB in therange of angle of n: /3 are shown in Fig. I I considering asymmetry of bolt. Fig.12 shows a collection of theboundaries of the backprojections from the goal by therotate-to-insert skill and move-to-touch skill with
control error, and the radial blank region (to be exact, its
inverse transformed region on position and orientation)means the appropriate initial position. Generally, since abolt is small for a range sensor, it is difficult to measure
the angle of the bolt around the axis of rotation, eventhough the position of the axis of rotation is known.

We have shown a fine motion planning of tasks withskills using a skill library which consists of skill
primitives represented as patterned trajectories in
configuration space. The procedure of multi-stepplanning and its computability have been mainly
researched for the backprojection method used as a finemotion strategy in this paper. However, there has been
no research on backprojection for the skill-based
technique. The method proposed here uses not only
straight-line motions but also rotary motions with skill.
Therefore, general and skillful planning can be derived,
and manipulation tasks closer to human tasks can beachieved. In future, we will study the derivation of amore appropriate initial position and orientation forreaching the goal, the precise treatment of errors oncontrol, sensor and model, and reinforcement of the skill
library.

Reference[l] T. Suehiro and K. Takase, "Skill Based Manipulation
System," (in Japanese), J. of Robotics Society ofJapan, Vol.8, No.5, pp.551-562, Oct. 1990.
[2] T. Hasegawa, T.Suehiro and K. Takase, "A Model­
Based Manipulation System with Skill-Based
Execution in Unstructured Environment," '91 Int.
Conf. on Advanced Robotics, pp.970s975, Pisa,
Italy, June 1991.
[3] T. Lozano-Perez, M. T. Mason and R. H. Taylor,"Automatic Synthesis of Fine-Motion Strategies for
Robots," Int. J. Rob<:>tics Research, Vol.3, No.I,
pp.3-24, 1984.
[4] M. Erdmann, "Using Backprojections for Fine
Motion Planning with Uncertainty," Int. J. Robotics
Research, Vol.5, No.I, pp.19-45, 1986.
[5] B. R. Donald, "Planning Multi-Step Error Detection
and Recovery Strategies," '88 Int. Conf. on Robotics
and Automation, pp.892-897, Philadelphia, USA,
April 1988.
[6] R. C. Brost, "Computing Metric and Topological
Properties of Configuration-Space Obstacles," '89
Int. Conf. on Robotics and Automation, pp.170-176,
Scottsdale, USA, May 1989.

- 161 -

An Emergence of Fuu.y Control Rules for Mobile Robots using DNA Coding Method
Tomohiro Yoshikawa, Takeshi Furuhashi, Yoshiki Uchikawa
Department of Information Electronics, Nagoya University
Furo-clw, Chikusa-ku, Nagoya, 464-01, Japan
Tel.+81-52-789-2793 Fax. +81-52-789-3166
E-mail: yoshi@bioele.nuee.nagoya-u.ac.jp

Abstract

II. DNA CODING METHODThis paper presents a new coding method based on biologicalDNA. A mechanism of development from the artificial DNA
is also presented in this paper. This mechanism realizes
flexible representation of fuzzy rules. The artificial DNA is
composed of four kinds of bases. The proposed DNA allows
redundancy and overlaps of genes. Effective fuzzy rules formobile robots are emerged through chasing and avoiding
operations.

Fig. I(a) shows a flow from biological DNA to cells. The
biological DNA consists of nucleotides which have four bases,
Adenine(A), Guanine(G), Cytosine(C), Thymine( T)l4ll5J_
Most of these bases in the top figure in Fig. I(a) are not used
for the synthesis of proteins. A messenger RNA (mRNA),
which has many unused parts, is first synthesized from theDNA. Then the unused parts are cut out. This operation is a
splicing. After this splicing has occurred, the mRNA iscompleted. Three successive bases called codons are allocatedsequentially in the mRNA. These codons are the codes foramino acids. 64 kinds of codons correspond to 20 kinds ofamino acids. This allocation of amino acid makes proteins,and proteins make up cells.
Fig. I(b) shows the proposed DNA coding method and theflow of development to sets of fuzzy rules. This figure
shows a correspondence of the proposed method to the
biological development. A · chromosome consists ofcombinations of four bases, A, G, C, T. The chromosome
has many redundant parts, and after a splicing, the mRNA iscompleted. This process is similar to that shown in Fig. l(a).
The codons in Fig; I(b) correspond to amino acids. Unlike
the biological amino acid, each artificial amino acid has
several meanings, and the meanings of a gene is determinedby the combination of the amino acids. An amino acid canbe translated as an input variable or a form of membershipfunction, and so on. A sequence of amino acids makes a
fuzzy rule. The DNA chromosome makes up sets of fuzzy
rules for controlling a mobile robot.
Fig.2 shows an example of the DNA chromosome and its
translation mechanism. In this figure a gene starts from thestart codon ATG, and ends at the end codon TAG, and codons
in the gene are translated into amino acids: Tyr, Thr, · · · .
Each amino acid has its own role for the problem.
By the proposed mechanism of development from theDNA, the starting point can be shifted from a base to anotherand some genes overlapping on other genes can be translated.

I. INTRODUCTION
Genetic Algorithms (GA)l1U2l have been widely studied andapplied to many problems. This paper proposes a new coding
method based on biological DNA and a mechanism ofdevelopment from the artificial DNA. This paper calls thiscoding method the "DNA coding method". The GA based ona biologically motivated model was proposed by Willfried
WienhoW 3 1. This model took into account gene expression
which involves translation of nucleotide sequences of DNA
into amino acid sequences. This method, however, is devised
to solve parameter optimization problems. The DNA coding
method �nd a mechanism of development from the artificial
DNA proposed in this paper are suitable for knowledge
representation. Complex knowledge are emerged from thesimple coding. The length of the proposed DNA. chromo­
some is variable and it is easy to insert and delete parts of thechromosome. GAs based on this DNA coding method arecapable of selecting appropriate sets of input/output variables
and proper number of rules. The DNA chromosome has aredundancy, and it can be translated flexibly.
This paper studies emergence of effective fuzzy rules using
the proposed DNA coding method. These fuzzy rules are used
to control mobile robots which play chasing and avoiding.
Effective fuzzy rules of a robot are emerged throughinteractions with the other robots.

- 162 -

Each overlapping gene plays an important role. Fig.3 shows
this overlapped representation. In this figure, GENE5 in
addition to GENE3 and GENE4 can be read from the DNA
chromosome. This chromosome has redundancy and also
compresses information by overlaps of genes.
Fig.4 shows examples of crossover and mutation. Fig4(a)is an example of one point crossover. Right hand sides fromthe crossover points are exchanged and new GENE2', 4'. 5'
are generated. By this method, depending on the crossover
points, genes can be drastically changed. There is noconstraint on the crossover points. Fig4(b) shows anexample of mutation. One base indicated in the figure ischanged from T to G. As a result, the GENE I is changed to
GENE I'. By this change, the start codon ATG is newly
generated and new GENE7 is generated.
The DNA coding method has the following features:(a)Flexible representation of knowledge.
(b)The coding is redundant and overlapped.
(c)The length of the chromosome is variable.
(d)No constraint on crossover points.
III. PROBLEM FORMULATION FORKNOWLEDGE ACQUISITION

Two different types of mobile robots play chasing andavoiding in the area of 2.33m wide and 3m long surroundedby walls. There are several foods in the area for the avoiding
robot. The radius of the chasing robot is 150mm, and that of
the avoiding robot is I 00mm. The chasing robot has eightultra-sonic sensors (seven in the front, and one in the rear).
These sensors can measure the distances between obstacles
and themselves in the range of 200mm to 1700mm. The
chasing robot must acquire mies to catch the other robot.
The avoiding robot has twelve infrared sensors to see around.
These sensors can measure limited distance less than
350mm. Therefore the avoiding robot cannot recognize theapproaching robot until the enemy comes near to it. The
avoiding robot must find fuzzy rules not to be caught by theother robot. Each robot has a chromosome containing a setof fuzzy rules. The fuzzy rules steer and accelerate/deccelerete
the robot to chase/avoid the other robot and stay away fromthe walls.
The robot which reaches or avoids the other robot receives
more payoffs from the environment. Considering thesepayoffs as fitness values, the genetic operators are applied tothe chromosomes and the fuzzy rules are evolved.

IV. A PPLICATION OF DNA CODING METHOD

The way of development from DNA chromosome to thefuzzy rules and the way to find effective rules are described inthis section.
4.1 Representation of rules
The candidates for the input variables of each robot are thedetected values of sensors D and the robot's velocity V, andthose for the output variables are the steering angle u, and its
velocity V. The chromosome has sets of fuzzy rules which
are represented by IF~ THEN~ rules. The chromosome
determines combinations of input/output variables andmembership functions of each fuzzy rules. The central
position xc and the width a of the membership functions arealso encoded into the chromosome.
In the biological DNA, a gene starts from the start codon
ATG, and ends at the end codon TAA, TAG or TGA. In thispaper, a gene also starts from the start codon ATG whichcorresponds to IF. The end codon is not definitely determined.
A gene consists of the codons between IF codon and somerelated codons succeeding to THEN codon. Reading from the
top of the DNA chromosome, translation to a fuzzy rule
starts upon finding the start codon ATG. As described in
Section II, overlaps of genes is allowed in the DNA
chromosome. After reading a fuzzy rule, re-reading is
restarted from the second base of the IF-codon and a new IF
codon is sought. Fig.5 shows an example of the DNA
chromosome and genes (fuzzy rules). In this figure, bases areread from the head of the DNA chromosome, and if the startcodon ATG is found, a fuzzy rule starts from this part. In
this example, the next codon GCT is Alanine, and Alanine
here means that the input variable is sensor. The sequence of
Alanine, Serine, Leucine means that this sensor is No.0
sensor (input variable is Do). The next part of Glycine,
Cysteine determines the form of membership functions, thecentral position xc and the width a , for the input Do. The
next codon GCC makes also Alanine, and this Alanine heremeans AND. Like this, each amino acid has several
meanings, and one meanings is selected based on theposition in the gene. In this example, GENE2 in addition to
GENE! can be read from the DNA chromosome.
4.2 Genetic operators
There are seven chasing robots and seven avoiding robots- 163 -

in one generation. Each robot, either chasing or avoiding, istested twice by randomly choosing its opponents from theseven counterparts. Each robot has initial payoffs E1. Eachtest ends when either of the two confronting robots crashes
the wall or the chasing robot catches the other or a certain
amount of time has passed. The payoffs for fitness values of
the chromosomes are given as follows:
If the robot crashes the wall. the robot loses payoffs E..
If the chasing robot can reach the avoiding robot within afixed time, the chasing robot receives payoffs Ee, conversely
the avoiding robot loses payoffs Ea
If the chasing robot cannot reach the avoiding robot in a
fixed time, the chasing robot loses payoffs Em
If the avoiding robot can reach the foods, the avoiding robot
gains payoffs E; per a food.
After these tests are done, the genetic operations are applied
to the chromosomes of the robots by regarding the payoffs ofeach robot as its fitness value. One chromosome of chasing
robot which has the smallest payoffs is deleted. Twochromosomes which are selected from the remaining six
robots are reproduced, and one-point crossover is applied tothem. One of the two new chromosomes is selected
randomly for a chasing robot. The chromosome of avoiding
robot is newly generated in the same way. There is noconstraint on the crossover points as described in section II.
The mutation operator is also applied to the newly generated
chromosomes. The mutation operation can be done simply
by changing the bases. This mutation is done to each base at
a rate of Pm. The payoffs of all robots are reset at E1 again.
After another simulation for the new generation is done, thegenetic operations are applied again. These steps are repeated,
and fuzzy rules which control the robots to chase or avoid theother robot and to avoid the wall are expected to be evolved.
V. SIMULATION
Simulations were done. The length of each initial DNA
chromosome was 500. The payoffs E1, Ew, Ee, Ea, E,,,,, EJ
were 0, 20, 50, 20, 20, 5, respectively. The probability ofthe mutation Pm was 0.05. The chasing robot which could
not reach the avoiding robot within 30 seconds lost payoffs
Em. The genetic operations were applied to the chromosomesof chasing and avoiding robots for I 00 generations
alternately.
The effectiveness of the proposed DNA coding method andthe mechanism of development from the artificial DNA was
examined. The following two cases were compared:(i)redundancy and overlapping, (ii)no redundancy nor
overlapping. In the case of (ii), no start codon is used. The
DNA is translated from its head. Fig.6 shows the average ofall payoffs of 7 chasing robots during the 3000-5000th
generation. The solid line is the case (i) and the dotted line isthe case (ii). Each is the average of 30 trials. In a trial, thesimulation from the initial generation to the 5000th
generation was done. Since the genetic operations wereapplied to the chromosomes of chasing and avoiding robots
for I00 generations alternately, the payoffs fluctuatedperiodically. The case which has redundancy and overlapping
worked well. An example of emergence of effective control
rules of chasing and avoiding robots is shown in Fig.7. This
was the case where the avoiding robot acquired very effective
rules. The avoiding robot was running along the wall. The
chasing robot could measure only the distance using theultra-sonic sensors. Therefore, the avoiding robot had, so to
speak, protective coloring. The avoiding robot could avoid
the chasing robot within 30 seconds.
VI.CONCLUSIONSThis paper proposed a new coding method, DNA coding
method, and the mechanism of development from theartificial DNA. This DNA coding method is suitable forknowledge representation. The DNA chromosome has aredundancy, and allows overlappings of genes. This paper
applied the proposed DNA coding method to emerge ofeffective fuzzy rules based on chasing and avoiding actions ofmobile robots. Further work is the experiments of real
mobile robots using this DNA coding method.
REFERENCES[I]D.E.Goldberg, "Genetic Algorithm in Search
Optimization and Machine Learning, Addison Wesley (1989)[2]L.Davis(Editor), "Handbook of Genetic Algorithm ", Van
Nostrand Reynold (1989)[3]W.Wienholt, "A Refined Genetic Algorithm for Parameter
Optimizat ion Problems ", Proceedings of The Fifth
International Conference on Genetic Algorithms (1993)[4]B.Albers and others, "Molecular Biology of the Cell ",Garland Publishing (1994)[5]A.Kornberg, "DNA Synthesis ", W.H.Freeman andCompany (1974)- 164 -

�=�:�:y=�:

GCCTGTTACCGATCGTTCCGCT
[DNA�
=
GENE
_
;::: .
Mi I :: -· ;;:•:.:::S::-�.. Splicing,I.splicing

mRNA�

...
- QfQ{Qf"';;;;:i-:-:�- • • .

IFR-- •=.::o:;-- •-•TACACCfOTCAA

OA

IF~ntl!N~
IF-ntEH-THEN-(a)Biological Way(b)This Method
Fig.I Flows of development from DNA Chromosome

DNA O!romosome:_ATGCAATTACGGT_

CCCA

AG
�

GENEI

GENE2

Fig.2 Example of a Chromosome and Translation
Mechanism
GENE3

CTGT�CCCA�AGGENEIGTAOTOOCOOTCTCTACCOTTA?
Allj
OENE4'�

TACACCTOTCA
6llj

COG

OENEI'OENE7TAA
TAGTOOCOGTCJ'CTACCGF
Allj°
OENE4'

GENE2""

1� , '1 J"''°

GGCA,j=J...LGroCC • • • CGTGTT,.,.,JTCGmAAT • • •a''1 •i

Sa Lo,Vi"''SEN ORO B G AND · · N
RIGHT
Fig.5 Example of Chromosome and Genes

.J.

ee

TAA

(b)Mutation
Fig.4 Crossover and Mutation

lllfN...

[Sets of Fuzzy Rules)

coo

G NEI

IPVkSmall AND• · ·� @J@J@J ...
CTGToPAYOPP

100.00
9".00!I0.00
M.00
00.00
75.00
70.00
6'.00
00.00
"·00
.50.004.5.0040.00
3,.oo.50.00
25.00
20.00

GENE2

Fig.3 Overlapping of Genes

r1
I

u

i

1,.00

10.00
,.oo
o.oo

3.00

3..50

4.00

ll

I

I

I�

l

4•.50

omlBRA.TIONx ta3,.oo

Fig.6 Paypffs of chasing robots (30 trials)
DNA Chromo:rimne :OA

TACACCTOTCAATIACOOT

Amino Acid : Tyr Thr Cys Gin Leu Arg

OENEI

TAAATCCOTCTTAOO
�

OENES

GENE2

CfOT�ACCCA�AO
OA

GENE3

OENE4

TACACCTOTCAATTACOOFTAAA+C
CTTAOOj�
GENE I

OENE5

Crot!tOVet Point....

OENE 2AO
CTOT�ACCC�
�'rmroocc,OTCTCTACCO
�
.
OENE3
. OENE4OA

6llj

AGTGOCOGTCTCTACCG1TATAATACACCTOTCAATTAa,JO?
F
Allj°"

GENE IOENE5'

GENE 4'OENE 2

CTOT�c.J:TdcAATIACOOTCjtMiACCCA
XTdTACOOTIO�T
AnjCC
�
�
OENE2'(a)Crossover
Fig.4 Crossover and Mutation

basing Robot---·-------......._Fig. 7 The movement of each robots- 165 -

A Fuzzy Classifier System for Evolutionary Learning of Robot Behaviors
Yasushi lwakoshi*, Takeshi Furuhashi**, Yoshiki Uchikawa*: Chubu Hitachi Electric Co. Ltd.
**:Dept.of Information Electronics, Nagoya University
Furo-cho, Chikusa-ku, Nagoya 464-01, Japan
Tel. +81-52-789-2792, Fax. +81-52-789-3166
Email fu ruhashi@nuee.nagoya-u.ac.jp

Abstracts: This pape r pre sents an evolutionary
learning of robot behaviors by Fuzzy Cla ssifier
System(FCS). The FC'..S introduce s a fuzzy rule baseand a fuzzy inferenc e system in place of the rule baseand the produc tion system of the Classifier
System(CS). The FC'..S ha'> a feature in that it isrobust to environmental changes. The FCS is applied
to behavioral learning of a mobile robot for evolving
fuzzy control rule s. Simulations are done under eight
different conditions. Robustne ss of the acquired
fuzzy rules is compared to that of the rule s obtainedby the CS.

1. INTRODUCTION
Our proble m is whethe r a robot, having ashort-sighted sensor s and no maps, can reach a goal
within a limite d number of steps without cra'ihi ng
into walls. Onuma and Hoshino have studie d thisproblem by using the Classifier System(CS)f1 1•
They pointed out that the CS was not robust toenvironmental changes.
Fuzzy
Valenz uela-Rendo npropos ed
Classifier System (FCS )'21 by introducing a fuzzy
rule base and a fuzzy inference system in place ofthe rule base and the production system of the CS,
respec tively. The FCS can handle continuous
variable'>. The FCS wa'>, howeve r, applie d to an
approximation of a single input-single output
fu nction in [2]. Studie s on apportionment of credit s
to fu zzy rules for describing the multi-input
system s have not been done. The author s havestudied application of the FCS to knowledge
acquisition of large scale system s13 · 51• A new
method for apportionment of credits to the fuzzy
rule s were proposed in [4] and [5] . By this method,
fuzzy rule s which describe knowledge in complex
multi-input/output systems were made possible to be

applied to find fuzzy control rules for driving amobileincorporating
robot.
Simulations
expe rimental conditions are done. Robustne ss of thefound fuzzy rule s is compared to that of the
production rules acquired by the CS.

2. MOBILE ROBOTThe robot used for expe riments is shown in
Fig. I. The robot is a micro-mouse on which eight
infrared sensor s are installed. The allocation of thesensor s is shown in Fig. 2. The infrared sensors s 1 - s 8
detects the distance between an obstacle and thesensor itself within the range of 5 - 35 cm. The robot
is movable controlled by its own CPU.

Fig. I Exterior of robot

3. FUZZY CLASSIFIER SYSTEM
Figure 3 shows the configuration of the FCSThe FCS consists of a fuzzy inferenc e system, anappor tionment of credit system, a fuzzy rule base ,and a rule generation mechanism.

found.
This pape r studies an evolutionar y learning ofrobot behaviors by the FCS and exa mine s robustnessof the FCS to changing environment. The FCS is

Fig.2 Allocation of sensors

Environment
Fig.3 Configuration of Fuzzy Classifier System- 166 -

3.1 Fuzzy rule baseThis fuzzy rule base has n fuzzy rules. The
input variables of each rul e are the detected values ofinfrared sensors s 1 - s8 • The antecedent part of eachrule consists of eight loci and each locus has one of
the five labels, small (S), medium small (MS),medium (M) , medium big (MB), big (B), ofmembership func tions shown in Fig. 4 (a). The
output is the steering angle u of the robot. The locus
for this output also has one of the five labels of themembership func tions in Fig. 4 (b).

52035Distance from obstacle(cm)--:rxx»c
-40
Left

(a)

0

40
Right

(b) Si-Ing angle

Fig.4 Me mbership func tions
3.2 Fuzzy inferenc e system
The fuzzy inferenc e system senses theenvironment by the eight infrared sensors s 1 - s8 anddrives the robot. This system generates the commandof the steering angle u for the drive system of therobot using the fuzzy rules in the fuzzy rule base.
The inferenc e method used in this system is produc t­
sum-center of gravity method. The control of therobot is done until the robot reaches the goal or it
collides with the wall. The payoffs ar e given to the
FCS for the result of the control.
3.3 Apportionment of credit system
The apportionment of credit system delivers
credits to the fuzzy rules in the fuzzy rule base. The
credits to each rule is in proportion to the payoffs
from the environment and the degrees ofcontributions of the rules to the results of control.
The.c redit <lof a fuzzy rule is updated as follows:(a) When the robot reaches the goal,"i-1

Ia = a + ;� W.r, x 1000 x "1,-11

(I)

-�

where i is the sampling sequence of the controller, n.
ts the number of steps from the start to the goal, kg is

the sampling sequence at the time of the goal, Wx istruth value of the fuzzy rule at x, and u., is thedefuzzified value of the output of the rule at x, i. e.
the central position of the membership func tion in
the consequent portion.
(b) When the robot reaches one of the sub-goals in
the maze,

I I

a = a + •�' °'1,.-, x l 00 x u1 ,,-,

(2)

where k,g is the sampling sequence at the time toreach the sub-goal and nsg is the number of steps tothe sub-goal from a previous sub-goal.
(c) When the robot collides with the wall,rl
a= a - • W. , x 1000
U1r ,I
(3)
r

ki:

XI

where ly is the sampling sequence at the time ofcollision and n1 is the number of steps to consider thecontributing rules to the failure.
3.4 Rule generation mechanism
Fuzzy rules are selected and reproduc ed using
the genetic algorithm (GA). The FCS controls therobot inn, mazes. Then the GA operation is applied
to the fuzzy rules with the ac c umulated credits.
n_.,1 rules with the least credits <l are changed.
This change to ea ch rule is done by crossover ormutation selec ted randomly.
One point crossover operation is applied to arule in the n,, 1 rules and a reproduced chromosome
from remaining n - n.,1 rules. One of the twogenerated rules is randomly chosen to replace therule in the n_.,1 rules. If the labels of membershipfunc tions in the antecedent are the same as those of
one of the existing rules, this rule will be deleted.
The crossover operation is repeated until a rule witha new antec edent part is generated.
Whe n the mutation operation is selected, thelabels of membership func tions in the ante cedent of
one of the n,, 1 rules is changed. This mutation
operation is also applied to generate a rule with a
new antecedent J>art.
Another mutation operation, which is tochange the labels of the membership func tions in theconsequent portions is applied to the n rules with aprobability of p,,,.
After these genetic operations are applied, thecredits of then rules are set at zero. The rules at thenext generation are now generated and the control of
the robot is resumed.

4. SIMULATIONS
Simulations. were done undern, = 8 different
conditions as shown in Fig. 5. Ea ch maze was 5 m
wide and 5 m long, and divided into 25 sub-areas.
The center of every border of the sub-areas was the- 167 -

sub-goal. The actual experimental conditions
including the robot in Fig. 2 and 3 and theenvironment were incorporated into the simulations.
The default direction of the robot at the start point
was parallel to the wall. We changed the starting
direction randomly within the range of + 30
degrees from the default direction at every trial. The
fuzzy rule base ha5 n = I 00 rules. The numbers of
steps to consider the contributing rules n1 was 2. The
probability of mutation in the consequent portion of
the fuzzy rules p,,, was 0.02. The number of rules tobe screened out by the selection operation n.,1 was I 0.
By the genetic operations, the FCS could findfuz zy rules which were able to control the robot toreach the goal under all the conditions without
colliding with the wall. The tracks of the robot
were shown in Fig. 6.
The robustness of the acquired fuzzy rules
were compared with those obtained by the CS. Table
1 shows the conditions for the comparison.
Production rules were in the rule base of the CS.
Each input space were divided crisply as shown in
the table. When the distance from an obstacle to thesensor was less than 20 cm, it was detected as near.
When the distance was lar ger than 20 cm, it was far.
* meant "don't care of obstacle". The steering angle
u was discretized in 5 different angles. The CS could
also find rules to control the robot to reach the goal.
Fig. 7 shows the mazes used for evaluating theacquired rules by the FCS and the CS. The GA
operations were stopped. Ten start points wereselected as indicated by the numbers in Fig. 7. At
each start point, both the rules of the FCS and the CS
were tested 360 times, respectively. The initial
direction of the robot was rotated 1 degree at every
test. Table 2 shows the success rates at each startpoint. The rules acquired by the FCS were better
than those by the CS. Fig. 8 ·shows an example oftrack of the robot controlled by the rules of the FCS
starting from the number 8 start point. The robot
went along the left side wall.

Fig.6 Tracks of robot
Tablel Conditions of FCS and CS
Fuzzy Rule Base

Production Rule Base

grade

grede

0

•

-·

� -

-- --

..

--

.

.

--

- --

-

I

-

-

..

.-

-

�-

-

•

,_,;_;,,·'° Distances from20
35
obstacle(cm)35Antec:emnt

:fr:

\

I

Antecemnt:fr:\

s1S2S3S4S5S6S7S8 ulll��J:,;1
i i'ffJ
-40
O
40
°-40
0
40

I

-

20

S1$2S3S4S5S6S7S8 U

..

- •.

5

don't care-•*•Distance from obslacle(cm)(l]lllm.
. : �-__ -_ [l-- . - [fiffl·-=I[I] [l]: ffl: m·
-

cs

FCS

...

. �

Fig.5 Mazes for learning- 168 -

C

ff

Left

Rip�

c

Steering angle

Rig�

Steering angle

@

@

i(i

Left

;n

a

. - I- t--

,_

•=

@

-·--

-·--

Fig.7 Mazes for evaluation

'"'

I

II

'
'

iii

�

.. .... .J....IC.
,·-�.

_,,__J

Fig.8 Example of result... @T able2[4] T. Furuhashi, K. Nakaoka, Y. Uchik awa, "A
Study on Knowledge Finding Using Fuzzy Classifier
Systems", J.. of Japan Society of Fuzzy Theory and
Systems, vol.7, no.4, pp.839-848 (1995)[5] K. Nakaoka, T. Furuhashi, Y. Uchikawa, "A
Study on Appor tionment of Credits of Fuzzy
Classif ier System for Knowledge Acquisition of
Large Scale Systems", Proceedings of Third IEEE
International Conference on Fuzzy Systems,
pp.1797-1800 ( 1994)

Success r ate of rules by FCS
and CS in unknown mazes
Success rate
FCS

Start
point

CD

9%

® 0%
® 18%

@

®

®

0

3%

cs

2%

0%
3%
3%

7%

0%

22%

22%

0%

0%

14%

6%

@
@
6o)

31%

19%

11%

11%

Average

11.5%

6.6%

5. CONCLUSIONSThis paper presented an evolutionary learningof robot behaviors by Fuzzy Classifier System(FCS).
The FCS has a feature in that it is robust toenvironmental changes. Simulations were done
under eight different conditions. Robustness of theacquired fuzzy mies was better than that of the rules
obtained by the CS.
The authors are now carrying experiments.
The results will be presented at the symposium. The
authors also study further on the repre:-entation of
fuzzy control rules and the apport ion ment of credit
system for acquiring more robust fuzzy rules.
REFERENCES[1] K. Onuma and T. Hoshino, "Evolutionary
Le aming of Robot Behaviors by Classifier Systetm",
JSAITechnical Report, pp. 9-14 (1994)[2] M. Valenzuel a-Rendon, "The Fuzzy Classifier
System: A Classifier System for Continuously
Varying Variables", Proceedings of the fourthInternational Conference on Genetic Algorithm,
pp.346-353 (1991)[3] T. Furuhashi, K. Nakaoka, K. Morikaw a, Y.
Uchikawa, "Controlling Excessive Fuzziness in a
Fuzzy Classif ier System'', Proceedings of the fifthInternational Conference on Genetic Algorithm,
p.635 (1993)- 169 -An Optimization Method of a Combinatorial Optimization Problem using a
Replicator Equation
Kazuo Tsuchiya

Katsuyoshi Tsujita
Yutaka Inoue
Dept. of Mechanical Engineering for
Computer Controlled Machinery
Faculty of Engineering
Osaka University
Suita, Japan 565

Dept. of Aeronautics and Astronautics
Advanced School of Engineering
Kyoto University
Kyoto, Japan 606-01

Abstract
This paper deals with an optimization method of
a combinatorial optimization problem by using repli­cator equations. The growth rates are designed interms of the performance index and the constraints
posed on the problem. The replicator equations show
the successive bifurcations ; Solutions satisfying theconstraints become stable successively by increasing
the control parameter and solutions with high perfor­
mance become stable earlier than solutions with low
performance. Then, sub optimum solutions are ob­tained if the control parameter is set to be as small as
possible. The characteristics of the bifurcations andthe performances of optimization of the equations are
examined analytically and numerically.

1

Introduction

An optimization problem is a basic problem in theengineering science. A path planning is one exampleof the optimization problems in robotics. Among opti­
mization problems, it is well known that a combinato­rial optimization problem is one of the hard problems[l]. A lot of algorithmic methods have been proposed,
and recently, optimization methods based on dynam­
ical systems have been proposed. The methods basedon dynamical systems have some advantages, one ofwhich is to realize a parallel algorithm easily. The
method proposed by Hopfield et al. [2] is one exam­ple. These methods, in the most case, utilize a class ofdynamical system, i.e., a gradient system; First, a po­
tential function is designed in terms of a performanceindex and constraints and then, a dynamical equations
are derived by the use of the potential function. Sub
optimum solutions of the optimization problem are ob-tained as steady state solutions. These methods cansearch good sub optimum solutions, but the perfor­
mance largely depends on the values of the parame­ters involved in the potential functions and there is no
guiding rule to determine the values of the parameters[3]. One method to overcome the difficulty is a simu­
lated annealing method [4] [5]. This method tunes theconstraints posed on the problem gradually by varying
the parameters in the modal and obtains good sub op­
timum solutions. A simulated annealing method has
been first applied to a stochastic dynamical systemand then applied to a deterministic dynamical system
as a mean field annealing method. A mean field an­
nealing method has been applied to an optimization
problem and obtained good sub optimum solutions.
Here, we will propose an optimization method us­
ing a replicator equation. A replicator system is asystem composed of reproductive elements of which
growth rates are functions of populations of the ele­ments [6]. When all the growth rates are always nega­tive, the replicator equation has a trivial solution that
populations of all the elements are zero. On the other
hand, by tuning the growth rates appropriately, vari­
ous patterns can be formed through bifurcations. An
optimization method can be constructed using repli­cator equations ; The growth rates are designed interms of the performance index and constraints posedon the problem. The equations show successive bi­
furcations ; By increasing a parameter ( control pa­
rameter) in the equations, solutions with high perfor­
mance become stable earlier than solutions with low
performance. Then, sub optimum solutions may beobtained if the control parameter is set to be as small
as possible. The characteristics of bifurcation and theperformances of optimization of this class of dy nam­
ical system are examined by applying to a Travelling
Salesman Problem.

- 170 -

2

Analysis

The Travelling Salesman Problem is a problem
which finds the shortest path starting from a city, trav­
elling all other cities once and coming back to the city.
The problem is formulated as follows ; Define statevariables ui; ( tt;; � 0) where u;; = 1 if city i is vis­
ited at jth visit. The variables u;; obey the following
equations

{1 -

u;;

u;; - (tu;,; tu;;,)
�o

i'¢i

+

Sol. B

Sufficient conditions of stability of Sol. A and B aregiven byj'-;/:jSol. A(1)-n-1 � /;;,u;,;±i} u;;Sol. B
where

where /;;, is a distance between city i and i'. The sec­
ond term in the right hand side of Eq. (1) represents
the constraint that each city must be visited once andonly one city must be visited at a time. On the other
hand, the third term represents the performance of thetravel, that is, the length of the path of the travel. In
the case where the parameter n-1 is equal to zero, Eq.
(1) has the following solutions
Sol. A

Sol. B(i,j=l,··•,N)1
Uo=-----1 + ao(N - 1)u;;=uou;; = 1 ( � = �o,� = �o){ u;; = 0 (z = zo,J =/:- Jo)(i =/:- io,i = io)(2)

While Sol. A is an uniform solution which does notsatisfy the constraint, Sol. B is a solution which satis­
fies the constraint. Sufficient conditions of stability ofSol. A and Sol. B are given as follows ;Sol. A;
Sol. B;(3)

When parameter n-o is increased, the stable solutionof Eq. (1) changes from Sol. A to Sol. B througha bifurcation at the point where n- 0 = 1. When theparameter n-1 is set to be a small positive value, Sol.
A and B are modified as follows ;Sol. A

O'oA
An-oA

(2:::

An-oB

i'¢i

1 + An-oA
n-1
".. ,
N

I: ,
i,i'

1 + An-oB
2a,
N

=

(

D
.,. •)(5)"
L _ _I_
N .. ,

L is a length of the path of the travel corresponding
to the solution. When the parameter n-o is increased,a solution of Sol. A become unstable near the point
where n-o = n-oA while a solution of Sol. B become
stable near the point where n-o = n-oB. Since the pa­
rameter n-oB is a function of L, the length of the pathof the travel corresponding to the solution, a solutionof Sol. B with a shorter path become stable earlier
than a solution with a longer path. Then, sub opti­
mum solutions may be obtained if the parameter n- 0 is
set to be as small as possible.

3

Simulation Results

Numerical studies are carried out to a Travelling
Salesman Problem with 51 cities. The locations of thecities and its optimum path are shown in Fig. 1.
3.1

Characteristics of Bifurcation

Parameter S1 is defined as follows.

where variable Au;; satisfies the following equation
Au;,; +=O'OBu;; = (uo + Au;;)

2Au;; + n-0

0 < O-o � O'0A
n-oB �n-o

L Auii')

i'#i

S1

=

- L; LP;; lnp;;p;;=< _.•_,_ >

u� -

I:ul;
i

- 171 -

(6)

*

where < > means the averaged value of * over a setof initial values. Parameters S1 has a large value in theregion where Sol. A is stable, and has the minimum
value in the region where Sol. B is stable. So, parame­
ter S1 shows the characteristics of a bifurcation of Eq.
(1). Numerical results are shown in Figs. 2 and 3.
Fig. 2 shows parameter S1 as a function of Aao in thecase where a1 = 0.1. Figure 3 shows parameter S1 in
Aa0 and 01 plane. Parameter S1 is shown to change
rapidly near the point Aaon. opt. the point where
the solution with the optimum path become stable.

3.2

Performances of Optimization

Stability condition (5) indicates that sub optimum
solutions may be obtained when parameter Aa0 is set
to be as small as possible. Figure 4 shows the perfor­
mance of optimization as a function of Aao. Case (a)and (b) are the cases where a1
0.01 and 0.1, respec­tively. The performance of optimization is gradually
improved as parameter Aa 0 become small. On theother hand, the values of parameter a1 has little effecton the performance of optimization. Figure 5 shows
numbers of iteration required to obtain a steady statesolution. The numbers of iteration increase rapidly
when the value of Aao become small. Then, a lower
limit of the value of Aa0 is limited by the CPU time
to obtain a steady state solution, and this, in turn, re­stricts the performance of optimization of this method.

=

4

Theory and Beyond, World Scientific, Singapore,
1987.
[2] J.J.Hopfield and D.W.Tank, 'Neural Computationof Decisions in Optimization Problems� Biological
Cybernetics, Vol. 52, pp. 141-152, 1985.

[3] G.V.Wilson and G.S.Pawley,uon the stabilityof the Travelling Salesman Problem Algorithm of
Hopfield and Tank," Biological Cybernetics, Vol.
58, pp. 63-70, 1988.
u[4] S.Kinkpatrick, C.D.Gelatt Jr. and M.P.Vecchi,
Optimization by Simulated Annealing," Science,Vol. 220, pp. 671-680, 1983.

[5] D.E.Van den Bout and T.K.Miller, 1mproving the
Performance of the Hopfield-Tank Neural Network
Through Normalization and Annealing," Biological
Cybernetics, Vol. 62, pp. 129-139, 1989.
[6] A.S.Mikhailov, Foundations
Springer, Berlin, 1990.

of Synergetics

I,

70
60
50

40
30

20

Summary

10

This paper deals with an optimization method of a
combinatorial optimization problem by using a classof dynamical system, replicator equations. The dy­
namical system shows successive bifurcations ; By in­creasing the control parameter, the solutions which
satisfy the constraints posed on the problem become
stable successively according to the values of the per­formance index. Then, sub optimum solutions may
be obtained if the control parameter is set to be as
small as possible. The characteristics of bifurcationand the performances of optimization of the dynam­
ical system are examined by analytical methods andnumerical simulations. Based on the results obtainedin this paper, to establish an optimization algorithmwith parameter tuning will be remained in a future
work.

References[1] M.Mesard, G.Parisi and M.A.Virasoro, Spin Glass- 172 -

0

0

10

20

30

40

50

60

70

Fig. 1 The locations of 51 cities and the optimum
path

80
60
40
20
0 ....__.,____.___.___....___.__�_,-0.6 -0.5 -0.4 -0.3 -0.2 -0.1
0A«oFig. 2 Parameter S1 as a function of Aao% : (Si/S1maz) X 100 (01
0.1)

=

0.16

il

0.14
0.12

/

0.1

....

Llaoopt

0.08

t,

0.06

\it

3

5

0

2.6
2.4
2.2
2
1.8

I 1%

•

1.6
-�

�...,jf,l"' "it.i

1.2
0.001

0.02

0.1

(a)

0

-0.8

-0.6

-0.4/:J,.

-0.2
ao

0.2

0

2.2

--

60

2
1.8
1.6
1.4

120

.....,mean­min •·•····

2.4

160 --�-�--.-�-...---..--..--r--ia.1=0.01 _,__
a.1=0.1
140....

nlQX-+­

2.6

(x40)

100

3
2.8

Fig. 3 Parameter S1 in �ao and a1 plane% : (Si/S1max) X 100

(IJ

!
J

✓ :

1.4

100%

0.04

max-+­mean­min •·•--

2.8

%

1.2

80

(b)
Fig. 4 L/ L op t as a function of �ao(a): a1 0.01, (b): a1 0.l
L opt : length of the optimum path

=

40
20
0

i___.___.,__,_____.__L--....1-___,J'---'---'---'

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.6. <Xo

Fig. 5 Numbers of iteration- 173 -=An Advanced Simulation Scheme for Electric Railway Power Systems
Based on Artificial Life Approach
T.Jinzenji and M.Sasaki
Electrical Engineering Department, Faculty of Engineering
Tohoku University
Aramaki Aza Aoba, Aobaku, Sendai 980-77, Japan
Abstract
This paper describes a simulation scheme for
DC electric railway power systems taking train
dynamic behaviors into consideration. This
simulation is useful to determine the rated
capacities of new DC substation equipment. In
the present simulation, train operating conditions
are assumed. Therefore, simulation results arequite different from the field data when a fleet
behaves like lives under crush load circumstances.
Two factors determine the behaviors. One is thepassengers' dynamic flow at every station, whichcauses unexpected delay. The other is the railroad
signal, which keeps trains apart in an appropriate
interval. These behaviors must be introduced into
the simulation. Typical simulation result of theadvanced simulation scheme shows that thefeeder current variations of the simulations are
similar to field data.
Keywords: Electric train, Train dynamic behavior,
DC system simulation, Life1

The field data of DC substation are often
different from the simulation results, especially inthe rainy morning, as shown in Fig. I.
This paper deals with an advanced simulation
scheme for DC electric railway power systems
considering how trains behave in a fleet. They
behave some types of patterns like a life. We
introduce these behaviors to simulations for DC
electric railway power system.
� E

1

◄

�3

--------------------------------------------_________________________ vote of record_,_ 10/0◄/85------------------------------- ------------

12
l I
l

) 0
9, 00
7, 3a
8, 00
8, 30
U _ , 00
I 0, 00
9, 30
I -------------------------------------------tI
1.. -2 ------------------� !'! -(!:'�::."]----- - - - ---------__________________________ Cate of record__, 10/0◄/85

Introduction

Over the past few decades, DC electricrailway power system simulations consisted fromthree steps, train run curve simulation, diagram
editing and DC power simulation.
The first step had been based on railroad
conditions, train performance, number ofpassengers and scheduled time, assuming that thepantograph voltage will be kept constant. In thesecond step, a diagram had been edited according
to the typical train run curves and the time table.
The diagram assumes that the train runs will be
repeated in a specified time.
In the last few years, three simulation steps
are joined to make sure that the simulation andthe field data provide same results(see [I]).
However, little attention has been given to themotormen and passengers. Even if the trains try
to follow up on a diagram during the crush load
circumstances, the train operations are restricted
physically and controlled by the railway signals.

�5-------------------------- -----------------_________________________ Date of record __ 10/0◄/85

�3--------------

1

◄

1

2

l I
l

J 0

0 0
0
u _ 1 , 0 ____ 1, 30_____e� �� ____ e, ;;0_____e� ��-___ g, 30_____1 , 0
� -2 _________________-�I!'!_(!'��]-_________________

I\ 5

� ◄
� 3

i

2
l I
l
) 0

_________________________ Cale of record_, 10/04/B5_-------------- ------- ------------- -------0 00
U _ 1 • �� ____ Z �3�_____B� �� ____ � '._3�____-�� �� ____ � '._3�_____ ,T Imo (hour)-------------------------------------------Fig. I. Field data of current of substation.

- 174 -

2

Electric Railway Power Systems

Fig.2 shows DC electric railway powersystems. DC substations locate within five to ten
kilometers and are connected in parallel. They
convert AC supply voltage into DC. They supply
DC power to trains in up and down lines at up
and down directions in parallel. Fig.3 shows theequivalent circuit of DC electric railway powersystems on one side ,one section of substations,where source of voltage means substation voltage,
source of current means train current. Solving
this equivalent circuit leads to following
equations. Since there are actually up and downlines on up and down directions, whole current
value of substation is calculated by adding four
current values.
nls1={Vs1-Vs2+ �r(L-Xi)hi}frLnHowever, trains are restricted by signal
conditions and passengers, too.
As for signal condition, a block system is used.
Fig.4 shows a block system. There are someblock sections in a line. To keep trains safe, there
is a block signal before each section. If there is a
train in this section, it indicates red signal andtells following motorman not to enter this section.
If we put too many sections in a line, there can be
many trains in the line, but block signals arealmost red and the velocities of trains get slow as
a whole.
In respect of passengers, they cause late froman expected schedule, especially in the morning
rush hour. The numbers of passengers go on
increasing from about 8:00 to 9:00, (generally
Japanese office opens at 9:00) and decrease
gradually after 9:00.

(1)
(2)

Is2={Vs2-Vs1 + �rxJu}I rL

Fig.4. Block system.

To take these conditions into account, train runs
assume following restrictions.
a) A train will be controlled by signals (signal
condition),b) A train can be late from an expected schedule(train diagram conditions),c) The numbers of passengers are establishedcorresponding to time zone.
The trains will recover time delay as early as
possible while it is late from a diagram.

Fig.2. DC electric railway power system.

0

Xf

X1

Xn ;,/

r [ 0 /km]

Islt __...,._...,____...,__ tls2
Vs2

Iu

Iu

3.2

lLn

Fig.3. Equivalent circuit.

3

Train behavior

3.1

Restrictions

Train operations are restricted physically. So
if it can always get constant voltage, it can moveregularly. Even if many trains run at the sametime and they can't get constant voltage, we canestimate these actions by solving train motion
equations.

Train behavior

Broadly speaking, there are three types of
behaviors of the fleet.
First, for the time zone that there is a great
increase in the number of passengers, stoppage
time at substations gets increasing. Because they
spend a lot of time in getting on and off, which
shorten the distance from the stopping train to
the next running one. Accordingly each distanceof trains will get short (see Fig.5(a)).
Secondly, for the time zone that there is aslight increase in the number of passengers, thedistances of trains are fluctuating. The reason isthat the forward train stops long time because
there are many passengers in the station, while- 175 -

To make sure the behaviors of trains in a fleet
under these circumstances, we established asimple model for train running.

the following one.· stops schedule time because
there aren't so many passengers that cause timeloss. In this way, these behaviors will be repeated(see Fig.5(b)).
Thirdly, for the time zone that there are lessthan the number of passengers that cause no timeloss. If trains are late from the expected schedule,
they all together tend to recover time as early as
possible without coasting so far as signals permit.
The distances of trains will keep short(seeFig.5(c)).

J";J �Jf
'®'
---

2km

1�IcDD�D•1 ? ? 1
M

train

<

'®�stations�ation20km

Fig.6. Simple model for train running.

>Fig.6 shows a typical location about stations andtrains. The stations locate as intervals of two
kilometers in parallel. Six trains start all together
for the same direction from each station. At thissimulation, we change the number of persons
which is related to delay at the station.

stationstation(a) Narrowing distances between trains.

JJJJ

ij ij

®��0-L t 6
J®U-1
station ---

Blue

LL

J ®U JI L
station(b) Trains in clusters.

J J

J ; ij
�

JJJJJ

6

6

Fig. 7. Flow chart of train operating mode.

Fig. 7 shows the flow chart of train operating
mode by signals from a station to the next one.
The results are presented in Fig.8. We can findbehaviors of trains like a life.
Fig.8(a) shows the scheduled running patternwhich is off peak. There is a small number of
persons and trains which stop scheduled time
every station. The trains almost keep sameintervals. Fig.8(b) shows the train runnings
which is on peak. Fig.8(c) shows that there is along stop at the first train and recovers again.

JJIJJJ(c) Recovery operation

Fig.5. Behaviors of trains in a fleet.

- 176 -

20

T- {T : Delay time
P : Number of person at station"'-i:::i�0(P - 200 )/10

Table I Simulating conditions

10

contents

conditions

500
Time[sec]

1000(a) Off peak.

20

railroad conditions

model:Toei Asakusa line(l8.3km)
number of substations: 5
number of stations:20
line resistance:0.03O/km

train performance

5000series
resistor control
maximum speed:70km/h

operating conditions train fonnation:6M
train weight:290.4t(crush load 200% )diagram:5minute headways
computer conditions sampling time: I sec
simulating time: 3hour

"'

-� 10

(P s 200 )(P > 200 )

,

i:::i

Table II
Establishment of number of persons

500
Time [sec](b) On peak.

time zone
station 7:00---,7:30~8:00---,8:3(}--,..,9:00---,9:30~10:00
ordinary 6
7
8
4
7
3
stationjunction 8
4
9
10
5
9
stationunit [person/sec]

1000

r,

500

1000

T:im e[sec]

1500

7:30

8:00

7:30

8:00

8:30

9:00

9:30

10:00

8:30
9:00
TIME [hour]

9:30

10:00

TIM E [hour](c)Long stop and recovery.
Fig.8. Time-distance curve of trains.

4

Practical simulation

4 .1

Conditions and results

Table I shows the simulating conditions undersignal conditions. Table II shows increase rate ofpersons per a second corresponding to time zone.
Until a train comes persons increase up to 600.
The delay time is define as:- 177 -

7:00

<

::::.
Q

!:"

u
"'
0..

8
7

6

5

4

3

2

1
0
7:00

7:30

8:00

8:30

9:00

TIME [hour]

9:30

10:00

persons are roughly established corresponding totime zone. And the equation which is defined asdelay time due to persons in getting on and off
may cause an error in calculation.
Secondly, when a train accelerates, muchcurrent flows in a train. So when trains accelerate
crowed together at the same time, there is a steep
increase of the current of substations. Though adriving interval is estimated, the load current ispractically dispersed because of the other factors
except stoppage time.

5
0

7:00

7:30

8:00

8:30

9:00

TIME [hour]

9:30

10:00

Fig.9. Current of substation.

The results are shown in Fig.9.
4.2

Discussion

Compared with the field data, we can say thatthe simulation results have a characteristic
similar to the field data. For example, there arepeak current after 9:00. It means that trains run
as fast as possible to recover time in this timezone. But there are some different points between
two figures.
First, the simulation current is higher than thefield data on the whole, especially peak current
after 9:00. It is for this reason that the numbers ofConclusion

The behaviors of trains in a fleet are restrictedby signal and the passengers' flow. Under thesecircumstances, trains behave like a life. They
seem to have a defined will, but their behaviors
are restricted by some factors. By recognizing
their behaviors we can calculate current valuesof substation precisely.
Most important of course is the determinationof the human factor which is fed back from thefield data in our simulation. The human factor
should be given according to the artificial life
practice in the future.

References[l]R. Takagi and S.Sone, "Precisely Fixed Start­
to-Stop Time Simulation of DC Railway Power
Feeding Systems, "Transaction of IEE Japan,
Vol.115-C, No.8, pp.975-983,1995.

� 178 -

Adaptive Block Dividing and Qunatization for Image
Compression using Artificial Breedings

tH.Hiramatsu,ttY.Kurematsu and ttS.Kitamura
tDepartment of Computer and Systems Engineering Kobe University, Kobe, 657
ttCollege of Industrial Technology,Amagasaki,667
Abstract

This paper describes a method of adaptive block­ing and quantization of a digital image compression
based on the genetic algorithms. In conventional com­pression techniques, an original image is divided into
small blocks and the discrete cosine transform(DCT)
is applied on each block. The resulting coefficient isthen quantized by the quantization table and coded
for transmission. Subblocks have same size and thequantization table is generated by experience or ex­
perimentally, so that the feature of image is not re­flected. vVe have applied the genetic algorithm andartificial breeding techniques for both problems. In
this method, elements of the quantization table andthe size and position are regarded as a genotype. Fit­
ness of each genotype is examined by subjective evalu­
ation. It is shown that better quantization tables andblock dividing methods matching to original images
can be obtained.

1

Introduction

Global computer networks have become widespreadin recent years, and researches of multimedia, which
integrates images, sounds, text and so on, have become
an important subject. Especially, digital image data
is most important media, and the high performanceof image compression techniques have been pursued
for fast transmission and low cost storage. For thispurpose, many techniques have been proposed, for in­
stance, the discrete cosine transform(DCT) and thevector quantization.
The international standard for digital video and still­
image compression and decompression was developedby the .Joint Photographic Experts Group(.JPEG)[l].
This algorithm uses the OCT of 8 x 8 block size,
a quantization table and Huffman coding. But thistransform coding technique has a problem of deterio­ration of image quality such as blocking and mosquito
noises in reconstructed images.[2] There are two meth­ods for improving this situation. The first is an adap-

tive quantization taking into account human's visualcharacteristics.[2] The another is an adaptive blocking(i.e. variable block size) which emphasizes various fea­tures of a given image.
In this paper, we propose a method for generating aquantization table and dividing of images by the ge­
netic algorithm. In this procedure, the artificial breed­
ing technique was used by taking into account human's
subjectivity (or sensibility) for reserving the featuresof original images in the reconstructed images.
It is shown that better quantization tables and block
dividing method matching to the features of original
image can be obtained. Noises of reconstructed im­ages were decreased, and the quality of reconstructed
image was better than that of the JPEG algorithms,
with high compression rate.

2

Techniques of Digital Image Com­
pression

A diagram of image compression systems is shownin Figure 1. An image is divided into subblocks ofsize N x N and 2-Dirn DCT is applied for each block
f(x, y) to produce the DCT coefficient block F(u, v)
of same size [1][2].

F(u, v)=2

N

C(·u)C(v)cos [N-1 N-1

L L f(x,y) •z=O y=O

1rn(2x + 1} .
1rv(2y + l)
] cos [
Jl}
2N
2N

where
C(i) = {"f2 (i(i =# 0)0)

The resulting coefficient array F( u, v) is then quan­
tized by a quantization table and coded for transmis­
sion.

- 179 -

Initial Table

Quantization Table,�--�

�----------------!OCT-----------------1

Dequantize

DecodeI�-- ------ -- --�-------------------------

Crossover
Mutation

Figure 1: DCT block diagram

Figure 2: Flow of generating the quantiza­tion table3

Generation of Quantization Table by
Artificial Breedings

Normally, the performance of image compression
techniques is evaluated by the quality of reconstructed
image and compression rate, where quality of images
has been evaluated by objective (or numerical) values.
However, the goodness of images depends much on hu­
man's subjectivity. This problem has been left open,
since we did not have an appropriate method to take
into account the human's subjectivity.
The quality of reconstructed images is dependent on
the quantization table. In most researches in this field,
they have been generated by heuristic or experimen­
tal method. In the following, we propose a method
for generation of a quantization table by taking into
account human 's subjective evaluation as well as thenumerical evaluation.

3.1

Population Number of individuals was 6 for easinessin artificial selection.
Genotype A qunatization table is transformed from
2-Dim array to 1-Dim array in the binary form.
The array obtained is a genotype (Figure 3).
Genetic operations As reproduction operations,the single-point crossover and the mutation wereused.
Fitness This is not numerical but operator's subjec­
tivity. The operator selects better 3 images fromreconstructed 6 images.

Generating the quantization table

I I IaI I aI I I
1·we define first an individual with a chromosome, asa genotype in the GA, consisting of elements of a quan­tization table. We apply the JPEG compression pro­cedure with this quantization table for a given image.
This process was repeated for all individuals in the
population. Next, we select better individuals from re­constructed images (i.e.phenotype) by the subjective
evaluation as well as numerical values of the compres­sion rate and the number of subblocks. Finally, thepopulation in the next generation is generated fromthe selected individuals by the genetic algorithm op­
erations. The flow of t.his algorithm is shown in Figure
2. The followings are important parameters:

O

I

1

I

Geno Type (7bilS coding)
Pheno Type (8x8 QTable)

Figure 3: Coding of quantization table

3.2

Results of Simulations

The algorithm was repeated for 100 generations.
Figure 4 shows a reconstructed image at the first gen­eration, and Figure 5 at 100th generation. The meansquare error(MSE) and the compression rate of both
images are shown in Table 1.

- 180 -

4

Adaptive blocking

Generally in the transform encoding method, anoriginal image is divided by N x N arrays of pixels
with a constant N. This method is simple, however,neglects various features in the image. The situation
is improved by variable block size methods.[4],[5] In
the following, we propose a method. for optimal block
sizing by applying the GA.

4.1

Adaptive block dividing

In this method, the size and position of each sub­block are defined as a genotype. An original image isdivided into subblocks according to this genotype. The
image coding techniques are applied for each block.
The fitness function was defined in terms of MSE,
bitlength(i.e. compression rate) and the number ofsubblocks for a reconstructed image(i.e.pheno type).
The followings are important parameters:

Figure 4: Image at Initial Generation

Genotype The number and position of subblocks arecoded as shown in Figure 6.

! sizex y size

Figure 5: Image at 100th Generation

Chromosome
Table 1: MSE and Compression rate
MSE Compression rate
Image at first gen.
140.4
9.1
Image at 100th gen. 47.7
8.9

(a)

size
(b)

Figure 6: Definition of genotype

The blocking and mosquito noises were obviously
reduced in Figure 5. Also the values of MSE were
improved with almost same compression rate. These
results show an extensibility of the proposed method
for more general images.

Block Each subblock is a square with 4 x 4, 8 x 8, 16 x
16, 24 x 24, 32 x 32 or 64 x 64 pixels.
Genetic Operations We used two-points crossoverand the mutation.
Fitness The fitness function for nth individual is de­fined bywhere M, B and S denote the scores of MSE,
bitlength and the number of subblocks, andWm, Wb and W8 are weighting constants.

- 181 -

4.2

Results of Simulations

The distribution of subblocks corresponding to Fig­
ures 7 and 8 are shown in Figures 9 and 10. In Figure
10, we can observe that small subblocks are allocatednear the edges in the image and large subblocks at thehomogeneous areas. T his shows the effectiveness ofthe proposed method.

Figure 7 shows an image at the first generation, andFigure 8 that at 500th generation.

5

Conclusion

Generating a quantization table and an adaptive
blocking method by the genetic algorithm were pro­
posed. It showed better and reasonable results as aneffective data compression method. As shown here,
the algorithm ba5ed on the evolutionary computation
is rather simple, while it can enable us to take into
account human's subjectivity (or sensibility) for eval­uation of the features of images.

References

Figure 7: Image at first generation(1] H.Yasuda: International standard of multimedia
encoding, pp.1-47, Maruzen 1991
(2] K.R.Rao and P.Yip: Discrete Cosine Transform,
Academic Press 1990.
(3] Rafael C.Gonzalez and Paul Wintz: Digital Image
Processing, Addison-Wesley 1977
(4] Arie Heiman,Variable Block-Size Transform Im­
age Coder, IEEE Transaction on Communica­
tions,Vol.38,N o.11,November 1990
(5] S.Okada: A Study on Block Dividing Method
for Variable Blocksize DCT Coding, The Trans­
action of the Institute of Electronics, Information
and Communication Engineers, Vol.J75-D-II, No.8
pp.1386-1396, August 1992.

Figure 8: Image at 500th generation

Figure 9: Subblocks atfirst generation

Figure 10: Subblocks at-500th generation- 182 -

Evolution of Locally Defined Learning Rules andTheir Coordination in Feedforward Neural Networks

Shinzo Kitamurat
Hajime Murao*
Computer and Systems Engineering,
Kobe University,
Kobe, 657
Abstract
This paper shows how an evolutionary process cancoordinate locally defined learning rules in a neuralnetwork. Each of these rules is defined as a functionof only local information for the corresponding neu­ron. Learning rules are exposed to the selective pres­sure based on the fitness values which are calculated
from errors between teaching signals and output sig­
nals of the network. That is, if the fitness value of alearning rule becomes less than that of an appropri­ate number of learning rules, this rule is replaced by
new one. As a result, the efficient learning rules sur­
vive in the network. The locally defined learning rules
coordinate each other and generate the macroscopic
adaptive behavior of a neural network.

1

Introduction

Learning is one of the most fundamental processesof organisms' adaptation. To utilize this adaptive
behavior, various artificial neural networks (ANNs)
and associated learning methods have been proposed.
These methods are based on physiological aspects;
changes of the connective strength between neurons
play an important role during the learning of organ­
isms. The methods calculate the changes of connective
strength in an ANN by using errors between teach­ing signals and output signals of the ANN. Currently,
physiology can not explain how the changes of con­nective strength coordinate each other and generatemacroscopic adaptive behavior of the brain. Practicallearning methods for huge and complex ANNs are notcurrently available, either.
Evolution is another process of organisms' adapta­
tion. The structure and the functions of the brain
are both gained as a result of evolution. Recently,
to cope with huge and complex ANNs, a relatively

large number of studies on combinations of evolution­
ary computing and ANNs have been reported[!]. In
most case, genetic algorithms have been used to adapt
the connective strength in a neural network, or to findan architecture for a neural network. These methods
have yielded interesting results on the optimization of
network design. Much interesting work has been doneon the evolution of learning[2][3]. However, only pa­
rameters in the learning function of ANNs have been
optimized.
In this paper, we apply genetic programming (GP)to evolve learning rules in an ANN. First, we outline
a general framework to describe a learning rule which
only use local information of the corresponding neu­ron. Then, GP is employed to evolve learning rules
and their coordination suitable for supervised learn­ing of layered ANNs. Learning rules are encoded into
genomes and exposed to the GP operation.

2

Framework for Evolution of Learning
in ANNs

2.1

Neuron Model for Our Simulations

In this section, we specify the neuron model which
is used in our ANNs. Figure 1 shows the signal flow ofneuron i. An output signal y i (t) of neuron i at time t
is given byu.i(t)=L wj(t)x)(t)jf(u\t))(1)(2)

where 11.i ( t) is an internal variable of the neuron, which
is given as a weighted sum of input signal xi.(t) and,J
connective strength wj(t). We use a sigmoid function
as f defined by* murao@kobe-u.ac.jp
tkitamura@kobe-u.ac.jp

1f(x) =-1 + e-X- 183 -

(3)

[ �wJ

Learning Rule:=xJ + (wj + (yi xr2 ))Genome:lFigure 1: Neuron i calculates its output signal y i (t)at time t from the internal variable 11.i(t), which isa weighted sum of input signals :rj(t) and connec­
tive strength wj (t). 6.wf (t) is a change of connective
strength between neuron i and a downstream neuronk.

Framework for Learning Rule2.2

We define a learning rule for neuron i as a functionof only local variables for the neuron,

where b;(t) is defined byb i (t) =L ::.wf (t)k(-5)

6.wf (t) is a change of connective strength between
neuron i and following neuron /.'.. \i\Then neuron i is an
output neuron, we put a teaching neuron k(= i) next
to the neuron i. The teaching neuron k is employed forgenerating 6.wf(t) = t i (t) or 6.wf (t) = t i (t) - y i (t).

Genetic Coding of the Learning Rule2.3

Now, we specify how the GP is employed to evolve
learning rules that have been represented in Section
2.2.
A learning rule of a neuron is encoded into a genome
which is constituted by binary trees. Figure 2 showsa genome and corresponding learning rule. Internal
nodes of the trees correspond to functions (i.e. opera­
tions) and external nodes correspond to terminals (i.e.
input signal). We define the set of functions F and theset of terminals T as follows:FT={+,-,/,*}= {·y i ,IJ.. i ,.'tj,Wj,. . i bi ?/-3
?1-:l
•,--

(6)

i

S + 1, S + 2, S + 3}

(l = 0, 1, ..., 4) (7)We include connecitve terminals S + 1, S + 2 and S + 3
in the terminal set. When a connective terminal in agenome is evaluated, it evaluates another tree whichconstitutes the same genome and returns the result.

Figure 2: A genome and corresponding learning rule.
Internal nodes of the tree corespond to functions, ex­
ternal nodes correspond to terminals.

2.4

Fitness Value of the Genome

To evolve coordination among the learning rules,
we have to specify fitness values for the learning rules
based on the success of an ANN in producing correct
output signals. The fitness values are given byfitness= 100 x(1- !)(8)

where E is a total number of errors of an ANN for
training data sets, N is the number of training data
sets. As each output signal of the ANN ranges from 0
to 1, Eq.(8) yields a fitness value between 0 and 100.
A fitness value is calculated for an ANN and is given
to all learning rules in the ANN. In our simulation, thefitness value is given to all neurons which constitutethe ANN.

2.5

Genetic Operations

We use three genetic operations to modify genomes:
reproduction, recombination and mutation. The re­production operation removes a genome, for which thefitness value is less than that of appropriate numberof neurons, and generate new one by recombination.
A number of neurons which determine a threshold forthe fitness values is given by multiplying a populationof genomes and a reproduction rate Pr.
The recombination operation creats new offspring
genomes by exchanging sub-trees between randomly
chosen two parental genomes.
The mutation operation creats new genome by re­placing a node with one of the functions and the ter­minals, which is randomly chosen from the set F and- 184 -

T. The mutation operation has a probability of Pm
for each node.

3

Evolution of Learning m
Layered ANNs

3.1

to yield output signal for each training data set.
Then, the coimective strength in the ANN are
adjusted according to the learning rule af output
neuron.

Single­4. Calculate a fitness value from errors between
teaching signals and output signals according to
Eq. (8).

The ANN Architecture

We use single-layered ANNs for our simulation. The
ANN consists of 5 input neurons and 1 output neuron,
which is described in Figure 3.
Input layer
Output neuron
Teach�neuron
�

3.3

Results of Simulations

We trained a population of 100 single-layered ANNs
through 5000 generation, where reproduction rate
Pr = 0.1 and mutation rate Pm = 0.001. Figure 4
shows resulting fitness values and fitness values fortesting data sets.

·�
� vv
y
'

\.

--0-- Maxirrum Fitness-0- Test Fitness,......

,','

Teaching data

Figure 3: Fully-connected single-layered feedforward
ANN. Whole network is trained by the learning ruleof the output neuron.
luput neurons make no changes on input signals
but transmit them to an output neuron. The neuronmodel described in Section 2.1 is used for the out­
put neuron. A teaching neuron transmits the teach­ing signals as 8 for output neuron. The learning ruleof the output neuron calculates changes of connective
strength between input neurons and the output neu­ron. ,v110le network is trained by the learning rule ofthe output neuron.

3.2

Evaluation of Fitness

Fitness values of each ANN is evaluated as follows:
1. Create training data sets. Each data consists of
5 input signals and randomly chosen 1 teaching
signals. As our ANN has 5 input neurons, we canprepare N training data sets out of 32.
2. Create a single0 layered ANN, for which each con­nective strength is initialized randomly between-1 and 1.
3. For a number of epochs ( typically 10), the ANN
is learned trough the training data sets. We prop­
agate input signals trough the ANN and employ

40

10

15

20

Number of Training Data

25

Figure 4: Resulting fitness value for various numberof training data sets. "Maximum fitness" shows thegreatest fitness value for the training data sets. "Testfitness" shows a fitness value for testing data sets,
which is given by au ANN that gives the greatest fit­
ness value for training.
Testing data sets consists of 32 testing data set, forwhich we determine randomly chosen teaching signal
for each input signals. As a number of training data
sets increasing, fitness values for testing data sets also
increased. This means that the diversity of the train­
ing environment is important to acquire a learning rulethat is efficient for different environments.
As a result of the simulation for N = 19, we
observed a version of the well-known delta rule ( orWidrow-Hoff rule),
�w;=y·(t-y)·x;(9)

We could also find versions of the delta rule for otherN, but they yielded less fitness values than the mostefficient rules. Figure 5 shows evolution of fitness val­ues for N = 19.

- 185 -

=

0.1 and muta­
neurons is 5, reproduction rate Pr
0.001. Fitness values of each neurontion rate Pm
which consists ANNs is calculated by Eq. (8). Figure
7 shows evolution of fitness values for N = 19. Maxi­
mum fitness values fluctuate but are fairly good over
55 points.

=

60fl) 50-- Maximu Fitness- - - - · Average Fitness--------· Test Fil.t::

U.. 40
30
0

1000

2000

3000

Generation

4000

70

5000

:!! 60Figure 5: Evolution of fitness values for single-layered
ANNs (N = 19).

4
4.1

Further Experiments with Multi­
Layered ANNs

Furthermore, we experiment with three-layered
ANNs. The ANN consists of 5 input neurons and 1
output neuron, which is described in Figure 6.
Changes in connective strength

rr.

Input layer

Error between output·-._ and teaching signal\ _.. --L..
I

,,

"'�

Output neuron
Teaching neuron

Hidden layer

Figure 6: Fully-connected 3-layered feedforward ANN.
Input neurons work as ••fan out neuron" that trans­
mit input signals to hidden neurons only. The neuronmodel described in Section 2.1 is used for the hid­
den neurons and the output neuron. A teaching neu­
ron transmits errors between teaching signals and out­
put signals as delta. for output neuron. Connective
strength between input layer and hidden layer is up­
dated by learning rules of hidden neurons. Connective
strength between hidden layer and output neuron 1s
also updated by learning rules of output neurons.

4.2''.1·,,,1\,1 ,, :,, ... ,.. - ... ... '' ,, --,\�,.
'.'' ,,
,,

0

�:

1000

2000

3000

4000

500(

Generation

Figure 7: Evolution of fitness values for three-layered
ANNs (N 19).

=

The Architecutre__ _\___'

50

Results of Simulations

We trained a population of 100 multi-layered ANNs
through 5000 generation, where a number of hidden

5Discussion and Future Direction\Ve have proposed a framework of the evolution oflearning rules in ANNs. Through the numerical sim­
ulations on single-layered ANNs, we have shown anability of our method to evolve efficient learning in­cluding the delta rule.
Furthermore, we have shown fairly good adaptive
behaviors by three-layered ANNs, though they aretrained only by locally defined leai"ning rules. It seems
that the coordination of the learning rules is evolved.
We should analyze what kind of learning rules are
evolved in three-layered ANNs.
We think the evolution of learning rules and theircoordination are depend on how to distribute fitness
values on neurons. Future improvements concentrateon distributing a fitness value calculated from ANN'sbehavior to each neuron which consists a multi-layered
ANN.

References(1] .John R. Koza and James P. Rice, Genetic Generaionof Both The Weights and Architecture for A Neural
Network, I.JCNN-91-Seattle, Vol.II, pp.397-404, 1991
[2] David .J. Chalmers, The Evolution of Learning: An
Experiment in Genetic Connectionism, Proc. of the1990 Connectionist Models Summer School, San Ma­
teo[3] Takashi Sumikawa, Studies on Acquiring of Learning
Rules of Neural Networks, Master Thesis of Graduate
Instrumental Engineering, Kobe University, 1994

- 186 -

A Self-Organization Learning Algorithm
for Visuo-Motor Coordination in Unstructured Environments

H.B. Zha

T. Onitsuka

T. Nagata

Department of Computer Science and Communication Engineering
Faculty of Engineering, Kyushu University
6-10-1 Hakozaki, Higashi-ku, Fukuoka 812 JAPAN
Abstract
We present a new method for coordinating a robot
system where a manipulator controller is directly con­nected to a stereo camera system. We try to avoid thecalibration process between the manipulator and thevision system by dividing the whole nonlinear trans­
formation into local linear transformations. The lin­earization is achieved by a learning process based ona Kohonen-style self-organization network. We givesome modification to the learning process in two as­pects. Experimental results are reported to show theproperties of our method and its applicability.
Keywords : self-organization, visuo-motor coordina­tion

1

Introduction

Many researches on robots equipped with some kindof sensors have shown the effectivenesses of use of sen­sory data. It may be still poor in accuracy and relia­bility if we have no appropriate methods to have themmake quick reactions to the sensor signals. It is, there­
fore, very important to develop efficient techniques for
coordinating robot motions according to sensor signals.
A manipulator coordination based on a camera sys­
tems is usually referred to as visuo-motor coordina­tion. The visuo-motor coordination problem is definedin a more specific form how to make an end effectorof the manipulator move to a target pose, which isdesignated through a camera system. Essentially, thecontrol should be carried out by performing a nonlinear
transformation from an arbitrary pose in its image co­ordinates to the joint-angle space of the manipulator.
One of conventional methods to perform such trans­
formation is to depend on a precise calibration of thecamera system and the manipulator. Even calibrated
by time-consuming procedures, it may be still poor in
accuracy and reliability according to utilized instru­
ments. Moreover, the system based on precise calibra­tion is also very vulnerable to changes in the working
environment.
There has existed some researches for the visuo­
motor coordination with a large degree of adaptation

capability on the basis of neural network techniques.
In the coordination of the end effector with stationary
cameras, Kuperstein (3] reported a method for map­ping stereo disparities to robot joint angles directly bya nonlinear network. However, for initializing the fixed
map it is necessary to give more a priori knowledges
about input information and the learned map is hence
not able to deal with homogeneous inputs. Walter et
al. [4] evolved the idea into the dynamic mapping reso­
lution based on a self-organizing feature map proposedby Kohonen [1] (2]. However, to achieve an accurate
mapping, a large number of learning steps should be
used. As a result, it will take large computational costs.
In this paper, we present a new method for coordi­
nating a robot system where a manipulator controller isdirectly connected to a stereo camera system. Clearly,
the relationships between the set of visual data andjoint angles relating pose of the end effector are non­linear. In our method, this whole nonlinear transfor­
mation is decomposed into local linear transformations
and the linearization map is recorded by the manipula­
tor controller. The linearization is achieved by a learn­ing process based on a Kohonen-style self-organization
network.
We give some modification to the learning process inthe following two aspects: The first is the reduction ofhuge computational costs required in implementation
practically; The second is the investigations about the
distributions of the neuron cells in more complex situ­
ations. Experimental results are reported to show theproperties of our method and its applicability.

2

Visuo-motor coordination

A typical make-up of a visuo-motor coordination
system is shown in Fig. 1. In general, learning of thecoordination is carried out by iterating the following
steps: 1) For one designated target in random, joint
angles are set so that the end effector of the manipu­
lator moves to a nearby pose of the target; 2) Visual
information about the end effector is extracted by pro­cessing two images obtained by the stereo camera sys­tem; 3) The visuo-motor coordination is modified inorder to decrease the error between the resultant pose

- 187 -

visual infonnation
nonlinear transfonnation
joint angles

Fig. 1: Visuo-motor coordinationand the target. Essentially, the joint angles have to be
associated directly with the visual information on theend effector poses.
The input and output signals we choose a.re de­scribed as follows. First, input is given as ea.ch two­
dimensional image coordinates of the target which is
identified in the two camera images. Next, parame­ters a.re essential to control the manipulator as output,
which correspond to ea.ch of the joint angles. Conse­
quently, the visuo-motor coordination becomes a prob­
lem of how to identify the nonlinear transformationfrom an input vector Ut to joint angles 8, which is rep­resented by eq. (1).

locally and linearly transformed control signals defined
inside a topological neighborhood around the target
vector. For any cell µnder consideration, its neighbor­hood is defined by a neuron ordering procedure. Given
the reference vector of the cell, the other neurons areordered by the Euclidean norm between this reference
vector and those computed by eq. (2) for all of them.
The learning rule has. following characteristics owing to
the introduction of the topological neighborhood. The
neighboring units charge analogous patterns, that is,
the analogy of input patterns is reflected to the close­ness on the position of responding output units.
Fig. 2 illustrates the correction of each element byapplying the following learning rules. The length of
the arrow on the solid line denotes the correction value
depending on the neuron order.

(1)

3

Self-organization network for visuo­
motor coordination

Input spacereference vector

The nonlinear transformation, shown in section 2,
is divided into local linear transformations- on the self­organization network. At first, input space (that is,
vector space formed by the two camera image coor­dinates) is quantized into several cells, each of themassociated with a neuron:. Then, each of the neurons ismodeled with para.meters that have been described inprevious section, and stores three elements as follows:
1) Reference vector w which corresponds to input space
and indicates the. center coordinates of a cell; 2) Joint­
angle vector 9 which makes the end effector move tothe pose as represented by the reference vector; 3) Ja­
cobian matrix A which indicates the first derivativesof the joint-angle vector with respect to the reference
vector. As the result of the self-organization based on
the following learning rule, these. relationships a.re de­scribed as

Fig. 2: Neuron activity changes in the self-organizat­
ion network(2)

where Ii is certain positive para.meter, and the sub­scripts in and out indicate para.meters belonging to in­
put and output spaces, respectively. g( k) is the Gauss­type function of the k th neuron and is used to weight

where target vector u 1 is derived by a random motionof a manipulator in each learning step and 0(1.1. t ) is the

The three elements given above should be modified
in order to decrease the error between the resultant vi­
sual feedback signals and the target vector. Here, we
use a self-organization network with N neurons with
the number determined ori the basis ofthe system com­
plexity.
The lea.ming rule ( as described in detail in [5]) for
the k th neuron (k = 0, • • •, N-1) is as- 188 -

Wk +-Wk+ £in •'gin(k) · (ui - Wk)(3)

jjk +- jjk +£out· gout(k) · "19k(4)

Ak +- Ak +£out· gout(k) · ..!l.Ak,(5)each neuron on the basis of the topological :µeighbor­hood around the target vector. Here, e and the neigh­borhood size decay as the learning step increases.

4

The 1·epulsive force Fk is defined on the basis of the
distance rk between the reference vector oH:th neuron
and the obstacle. It is given as(6)

Modification to learning process5

In this section, we will describe some modification
to the learning process.

4.1

In the following, we present experimental results ob­
tained by applying the self-organization networks.

Network initialization based on simu­
lation5.1

In practical implementation of the learning algo­rithm, the number of the learning steps should be re­duced in consideration of time-consuming manipulator
motion and image processing.
To achieve a more effective implementation, thewhole learning process is made up of a simulation stage
and then a real manipulation stage. The simulation
stage, which is carried out on a computer on the ba­
sis of some rough measured data on the coordination
system, aims to provide a better initialization for thereal network. This strategy decreases the number of re­quired manipulator motion and image processing .con­siderably:

4.2

Local linear maps

The state of local linear maps is illustrated by the
distribution of neurons. We perform the visualizationby projecting the reference vectors which correspondto input space onto one camera image plane.
Fig. 3 shows changes of the local linear maps withrespect to learning steps. Through the learning process
in the simulation stage (a) and then (b), the final ho­
mogeneous distributions in the real manipulation stage( c) enable us to fulfill the linear transformation in thedivided space, as described in eq. (2).

5.2

Introduction of repulsive forces

Positioning performance and adapta­
tion capability

Fig. 4 presents positioning accuracy versus the num­
ber of the learning steps. The positioning accuracy is
calculated by the Euclidean norm in 3-D space between
the target position and the resultant position for theend effector.
As shown in this figure, the learning process with ini­tial setting given by a simulation stage, approximates
the nonlinear transformation accurately, which is rep­resented by the solid curve. Moreover, in the result
illust.rated by the dashed curve, a structural change bypulling the manipulator away from the camera by 100
mm after 60 learning steps is occured. In spite of such

In a case of unstructured environment where obsta­
cles may exist, we suppose that the representation ofinternal models differs from those in the environment.
This is caused by constraint that the degree of freedomof the manipulator decays. For the reason, we intro­
duce the following two forces into the original learningalgorithm. One is the attractive force which acts sothat the neurons are closer to target. The other is therepulsive force which acts so that the neurons are far­ther to obstacles.

(a)Experiments(b)Fig. 3: Changes of neuron's distributions along with the learning steps- 189 -(c)

environmental change, the system can be adapted
quickly to it and almost recover to the previous ac­
curacy.

....

Fostt1on1ne Error <->100 ,----,--,--,--..----,---.---,------,,----,---,-----,

•

a

• •

70

"'

....

50

20
10

• L____,__....__...____.___;_,;_.__...=._.....:.._._;__..._;;_.=
0

10

20

l»

<O
50
00
!Wier of loaming Steps

70

80

80

Fig. 6: Distribution of neurons according to the work­
ing space

100

Fig. 4: Positioning performance

5.3

Effectiveness of the Simulated Net­works

We initialize the whole network with a simulated
network to reduce the implementation time. Hence,
we give a comparison of it with a network which is
initialized by random values.
Positioning Error <->"" ........---.----T----.-----.----,-----.

..
..
..

__ __ __ __• ____.

,..._

50

6

Conclusions

By applying a two-stages learning algorithm, thesystem can approximate the joint angles of a manip­
ulator by local linear maps in faster implementation
time. The adaptation capability with respect to a sud­den change in the environment is also proven. More­over, by applying the algorithm into which the repul­
sive forces are introduced, we confirm that the distri­bution of the neuron cells is closely connected with theworking space.
Our future task will be oriented toward a visuo­
motor coordination in real situations where some ob­
stacles exist.

1-.,...,\v,J-V·Vv·- \/.;Vv-.f\,.1-.,,j'...v.,..,,
____.

100

_._

....._,.__. DI LNrnins Steps

200

References...._"--__,
250

300

Fig. 5: Performance for different initializations
Fig. 5 shows the error changes for the position learn­
ing corresponding to the two different networks. The
results of the learning with a simulation stage are rep­resented by the solid curve, while those without thesimulation stage by the dashed curve. It is shown thatthe convergence rate is remarkably improved by thesimulation stage.

5.4

Fig. 6 shows local linear maps similar to the resultin § 5.1. Non-uniformity of the distribution indicatesthat a neighborhood differs inside which each neuron
acquired a local linear transformation.

Property based on the repulsive forces

We present a result of simulation in more complex
situation where a circular obstacle exist.

[1] T. Kohonen, "Self-organized formation of topolog­
ically correct feature maps", Biol. Cybern., 43, pp.
59-69, 1982.
[2] T. Kohonen, "Analysis of a simple self-organizing
process", Biol. Cybern., 44, pp. 135-140, 1982.
[3] M. Kuperstein, "INFANT neural controller foradaptive sensory-motor coordination", Neural Net­w01·ks, Vol. 9, pp. 131-145, 1991.
[4] J. A. Walter and K. J. Schulten, "Implementationof self-organizing neural networks for visuo-motor
control of an industrial robot", IEEE Trans. Neural
Networks, Vol. 4, No. 1, pp. 86-95, 1993.
[5] H. B. Zha, T. Onitsuka, and T. Nagata, "Self­
organization based visuo-motor coordination for areal camera and manipulator system", in Proc.
IEEE Conl Syst., Man, and Cybern., pp. 33223327, 1995.

- 190 -

Hardware Evolution - a real "life on the silicon"
Katunori Shimohara
Evolutinary Systems Department
ATR Human Information Processing
Research Laboratories
Kyoto 619-02, Japan

Hitoshi Hemmi
Evolutinary Systems Department
ATR Human Information Processing
Research Laboratories
Kyoto 619-02, Japan
Abstruct
A hardware evolutionary system that automaticallychanges HDL (Hardware Description Language) speci­
fications of digital circuits and adapts them to applied
specific problems is proposed. Outputs from the sys­
tem can be converted to real electronic circuits insteadof the conceptual data structure in computer memory.
Such individual circuits can be seen as an artificial lifethat behaves at electoronics speed: true "life on thesilicon".

1

Introduction

Hardware evolution and evolvable hardware will be­come the most important components for complex
adaptive systems. Despite long tenacious research ef­
forts, creating flexible autonomous agents is still atough problem for engineers. Thinking about theamazing creativity of nature, however, evolvable hard­ware that mimicks natural evolution can very likely tocreate such an agent. Furthermore, such agents are
expected to consist of a huge number of components;a software only technique solely is not feasible. Mas­
sive parallelism of hardware are indispensable to copewith such hugeness.
We have been constructing a hardware evolution­
ary system [1, 2] named AdAM (Adaptive Architec­
ture Methodology). This system uses a special pur­pose programing language, HDL (Hardware Descrip­tion Language), and evolves HDL-programs. Section 2
gives an overview of this system.
The main technique used in this system is Produc­tion Genetic Algorithms (PGAs)[l], which skillfullychange the HDL programs and makes their evolvement
possible. Section 3 briefly reviews this technique.
Section 4 is devoted to experiments and results ofapplying the AdAM system to some application prob­
lems.
Section 5 explain the future work.

In work related to the AdAM project, Higuchi etal.(3] uses a FPGA (Field Programmable Gate Array),looks at the architecture bits of the FPGAs as chro­mosomes and evolves them using Genetic Algorithms.
The main difference between Higuchi's method and
AdAM is that the former aims to evolve circuit struc­tures while the latter aims to evolve circuit behaviors
expressed as HDL programs.

2

The AdAM system

The AdAM system uses LSI (Large Scale Integrated
circuit)-CAD (Computer Aided Design) system andadd automatic-al evolutionary mechanism �. the sys­tem. Figure 1 shows an overview of the AdAM system.
This system provides a program development pro­cess. In the process, production rules translated from
HDL grammar and structured chromosomes are used
to automatically generate HDL source programs. The
generated HDL programs are simulated along withsome application problems by using a behavior sim­
ulation tool. The results are then evaluated as to how
well each HDL-program (i.e., digital circuit) fits the
problem. After the evaluation results are obtained,some transformations are performed on the develop­
ment process to improve circuit performance. The
generated HDL-programs are converted into circuit
schematics, FPGA (Field Programmable Gate Array)configuration data, or LSI mask patterns using a net­
list synthesizer and other CAD tools. In this sense,
the HDL programs can be considered to correspondto hardware.

3

Production Genetic Algorithms

This section briefly reviews the Production Genetic
Algorithms (PGAs) [l].

- 191 -

AdAM system ovsrview

Figure 1: The AdAM system

Figure 3: Chromosomes

One significant drawback of HDL as a hardware
evolution tool is that HDL is too vulnerable to thechange caused by evolutionary operations. The main
objective of PGAs is to apply evolutionary operations
safely to the HDL programs. We will explain PGAs
along with Fig. 1.
3.1

Production rules- K_MOD name list_comp list_pin list_action
module- K_NAME
naine- complist_comp- list_comp comp
list_comp- emptylist__pinlist_pin- list_pin pin
list_action - action
list_action - par_action
list_action - cond_action(r7. O)(r7.1)(r7.2)(r8. OJ

actionactionactionactionl(r9. OJ

action2(r20. 0) COlllp(r20.1) comp(r20.2) comp(r20.3} comp

Genetic operations

The AdAM system has five genetic operations.
Here, we explain three of those.
CrossoverFigure 2 shows production rules corresponding toa HDL grammar. In Fig. 1, HDL programs are de­veloped by a rewriting process using these production
rules.
(rO.0)(rl. 0)(r2.0)(r2 .1)(r3.O)(r3.1){r4. O}{r4 .1}(r4 .2)

3.3

In tree-structured chromosomes, crossover can be op­
erated between subtrees for root nodes having identi­
cal category rules; that is, two rules are applied to thesame symbol.
Figure 4 shows a crossover example.

- actionl- action actionl- action action2- register- memory- K_INPUT input_name- K_OUTPUT outputJ).8J[Le- K_BIDIREC� bus_name- K_INSTRI� inst_name

Figure 2: Production Rules (HDL grammar)

Parent

The numbers following to 'r' are production numbers. For ex­
ample, 'r4.1' indicates a rule having category number 4 andsub-number 1

3.2

Chromosomes

Offspring

Figure 4: Crossover

MutationIn the development process in Fig. 1, chromosomes
in Fig. 3 are used to determine the production rules
applied to each symbol.

Mutation acts on a node of a chromosome and changes
the label to index another rule with the same category
as the previous rule.

- 192 -

Gene duplication

This operation is related to recursive rules. There isa rule that generates its left hand symbol among its
right hand symbol sequence. In a chromosome, sucha rule can appear repeatedly in a line. Gene duplica­tion is an operation that copies a node and its subtree
block, and arranges them in a line. This is a neutral
operation; that is, the HDL program does not change
its functionality.
This operation is useful when combined with muta­tion. When mutation occurs in either subtree block,the HDL program gets a new functionality, but its pastfunctionalities are preserved.

two outputs are decoded and used to determine four
actions of the ant: go straight, turn left, turn right, or
do nothing.

Figure 6: John Muir Trail
We set three different situations for this problem
and did experiments to each case. The result of basic
one was reported in previous paper ([l]). Here, we
explain two others.
Experimental conditions were same in all cases:
population size was 200, crossover rate was 50% perindividual, mutation rate was 0.5%, gene duplication
rate was 2%, and deletion rate was 1 % (deletion is a
operation roughly the opposite of gene duplication).
In all cases, the fitness evaluation waw given by thefollowing formula:performance= score + ( timeJimit - time_steps)
Score is the number of pieces of food (maximum 89)
the ant ate within the limited time (timeJimit = 350).
Time..step is the number of steps that the ant took. If
the time limit was reached before the ant ate all thefood, then time_step equals 350.

Figure 5: Gene duplication
A copy of blockl is inserted between blockl and block2

4

Experiments

Case AThis section describes some experiments and theirresults, applying the AdAM system to behavior con­trol tasks that concern artificial ant problems.
The artificial ant problems are tasks to make anartificial ant on toroidal lattice space eat food ade­quately arranged on the space as much as possible.
Jefferson et al. [4] succeeded in making an ant follow
a trail called the "John Muir Trail" (Fig. 6) by us­
ing a Finite State Automaton and an Artificial Neural
Network. In Fig. 6, the black cells indicate the food's
existence. Similarly, Koza did this on the "Santa Fe
Trail" with GP [5].
Our target is to generate a digital circuit capableof controlling an ant on the John Muir Trail. One
input and two outputs are sent to the circuit. The
ant only gets one sensory input from the cell in frontof it, whether or not the food exists on the cell. The

In this experiment, each circuit consists of two mod­ules. Each module has one output, and these outputs
are combined to make two outputs of whole circuit.
One input to the whole circuit is distributed to both
modules of the circuit. Each module is treated as be­longing to different species and evolving in different
populations. This is an experiment for a coevolution
model.
Case BThe HDL we use has a lot of built-in operators that
express logical and arithmetic operations. In this ex­
periment, the production rules are modified to in­hibit some operators form appearing in the HDL pro­grams. This experiment tests the independentness of
this technique to the particular HDL.

- 193 -

Figure 7 graphically show the results for case A
and case B. Each graph shows average fitness of the
population and fitness of best individual.
In the left graph of Fig. 7, at the 144th generation,a circuit with a perfect score appears. This circuit
traverses the entire trail in 314 time steps. At the270th generation, a circuit that takes 266 time steps
to traverse the trail appears. The two modules of thiscircuit both have 4 control states. Here, two circuit
modules of whole circuit belonging different species
were co-evolved and cooperatively control one artificial
ant. Figure 8 shows part of schematic diagram of thebest circuit.
In the right graph of Fig. 7, at the 103rd generation,a circuit with a perfect score, that takes 300 time steps
to traverse the trail appears. At the 381th generation,a circuit which takes 282 time steps appears.
This result implies that the success of the AdAM
method is not sensitivity dependent on a particular
programing language.
ArtlnclaIAnt

ooc

'"

ArUficlalAnt

Perfonnance_Lr- L,-+l,J

00

-

..

[1] Jun'ichi Mizoguchi, Hitoshi Hemmi, and Katsunori
Shimohara. Production genetic algorithms for au­
tomated hardware design through an evolution­
ary process. In IEEE Conference on Evolutionary
Computation, 1994 .

.00

�

r

w
.00

00

w
.00

IN

·- "' "

In biological nature, the development processes dif­fers from species to species. We plan to modify thesystem so that some set of rewriting rules that areused in generation of HDL-specifications are obtained
as the result of evolution from a simple set of rewrit­
ing rules; each set of rewriting rules corresponds toone "species".
We also plan to implement this evolutionary
sysytem on hardware. This will increase strikingly
the system's operation speed. The result will be areal time evolutionary system. We will no longer haveto wait to see the progress of evolution.
The system mentioned here can be used to develop
controllers of robots and animats. The system itself
may be used as controllers of some kind of interactive
Artificial Life (A-Life) art works, such as an automaticmusic composers or a general simulator of A-Life phe­nomena.

References

Avo:rage,.00

Future work

.00

0,

..

5

,..

00

A

� r
Generation·,r,,c.,,,

"j l

.,.

IL,, 11>..Ai/
Generaiioo[2] Hitoshi Hemmi, Jun'ichi Mizoguchi, and Katsunori
Shimohara. Development and evolution of hard­ware behaviors. In Rodney Brooks and Pattie
Maes, editors, Artificial Life IV. MIT Press, 1994.

Figure 7: Artificial Ants

[3] Higuchi Tetsuya, Iwata Masaya, Kajitani Isamu,
Iba Hitoshi, Hirao Yuji, Furuya Tatsumi, andBernard Manderick.
Evolvable hardware andits applications to pattern recognition and fault­
tolerant systems. In Towards Evolvable Hardware,
Lecture Notes in Computer Science. Springer­
Verlag, 1996.
[4] David Jefferson et al. Evolution as a theme in arti­
ficial life; the genesys/tracker system. In Artificial
Life II, pages 549-576. Addison-Wesley, 1992.
[5] John R. Koza. Genetic Programming: On the pro­gramming of computers by means of natural selec­tion. The MIT Press, 1992.
Figure 8: Part of schematic diagram of circuit ob­
tained in case A- 194 -

Experimental Study on Autonomous Mobile Robot Acquiring Optimal Action to Avoid Moving Obstacles**Takeshi AOKI,"'Toshiaki OKA,"'Soichiro HAYAKAWA,*School of Electric Eflh>ineering, Nagoya University
Furo-cho, Chikusa-ku, Nagoya, 464-01, JAPANTEL: +81-52-789-2778, FAX: +81-52-789-3140
Email: oka,suzuki,okuma@okuma.nuee.nagoya-u.ac.jp

Abstract

The principal aim of this study is to show how
an autonomous mobile robot can acquire the optimal
action to avoid moving multiobstacles through theinteraction with the real world. In this paper, we propose a
new architecture using the hierarchical fuzzy rules, fuzzy
evaluation system and learning automaton. By using our
proposed method, the robot autonomously acquires thefine behavior how to move to the goal avoiding moving
obstacles using the steering and velocity control inputs,
simultaneously. Also we show the experimental results toconfirm the feasibility of our method.

1. Introduction
In this paper, we propose a new action selection
technique based on the framework of learning automata[ l]so that an mobile robot can autonomously acquire theoptimal action using the steering and velocity control
inputs, simultaneously. Our proposed system consists of 6
modules explained in the following sections. Basicsteering and velocity control inputs are decided based on
the fuzzy logic in the hierarchical fuzzy module[2] andfine adjustment or combination of these control inputs are
acquired by the reinforcement learning algorithm in theaction learning module. This is because the combination
rules between two control inputs can hardly designed inadvance. In the learning module, each rule describing how
to combine or adjust the control inputs is regarded as anaction, and several actions are prepared in advance. Then
using this learning method, the robot can autonomously
acquire the optimal action suitable for each situation the
robot faces through the interaction with the real world.
The reinforcement learning method on theframework of the learning automata used in this study has
the property called "absolute expediency", which meansthat action selection probabilities are guaranteed
mathematically to converge under the unknown static
random environment. Moreover, This learning algorithm
is executable in real time because of its simplicity.

"'Tatsuya SUZUKI,*Shigeru OKUMA**Nagoya Municipal Industrial Research Institute3-4-41, 6-ban, Atsuta-ku, Nagoya, 456, JAPANTEL&F AX: +8 l -52-654-9932
Email: aoki@okuma.nuee.nagoya-u.ac.jp

In case of using learning methods like thereinforce learning ones, we also have to consider how toevaluate the performance. The world always changes andthe performance index is inclined to be complicated. In
our proposed system, we construct the evaluating function
using the fuzzy logic. This is because the fuzzy logic canreflect the designer's experiences or intuitions explicitly
and evaluate the performance easily.
Using our proposed system, the robot performs"sense-select-act-evaluate­
thelearning
cycle
learn(update)" in the real world. Finally, the robot
autonomously acquires the optimal action selection
suitable for the real world. In the following section, thedetails of our system are explained.

2. Proposed System
2.1. System for Single Obstalce
2.1.1 Architecture
In this subsection, we state the outline of our
proposed system for single obstacle. The . proposedarchitecture is shown in Figure 1. The system consists of 5
modules. The role of each module is as follows.
l )Sensing Module: This module senses the environment
around the robot and calculate the relative positions andvelocity vectors between the robot and obstacle.
2)Hierarchical Fuzzy Rule Module: This module decides
the steering and velocity control inputs for each obstacle
avoidance, respectively. The advantage of thishierarchical algorithm is that the number of fuzzy rules
can be reduced without degrading control performance.
Figure 2 shows the structure of hierarchical fuzzy rules,
the definition of the fuzzy parameters and the relations
between inputs and outputs. The details of this structureis shown in [2].
3)Action Generating SubModule: This module selects
one action (rule) among actions by the roulette method
based on the action probabilities explained later in order- 195 -

to combine the steering control input with velocity
control input based on the selected action for eachobstacle. In this module, there are four action selections
Action in the World
Updatin2

Gates

Probability Tabl�

Sub(e.g.:Combine 50% of the steering control input for
obstacle avoidance with 50% of it for goal, and use thevelocity control input for obstacle avoidance)4)Fuzzy Performance Evaluation Module: This module
evaluates how good or bad the selected action (rule) is
by using the fuzzy logic. The fuzzy evaluation rules for
single obstacle are shown in Table l. Using this table,
the fuzzy evaluation module evaluates the robot action.
5"),4ction Learning Module: This module updates andmemorizes the action probabilities in the element of thetable based on the evaluation value from the evaluation
module by the reinforcement learning method in
Learning Automata.
2.1.2 Probability Table

Relative An2le RA

100

World
Figure 1. Proposed System for Single Obstacle
Table 1 E va uation Fuzz Ru es tior s·me:Ie 0bstacle
Parameter@

ParameterCDI �n) I - I �n-1) I

Steering Control
Input for 2oal

Negative
Big
Negative
Small
Positive
Small
Positive
Big

Change of Distance
Between and Goal
Robot

Negative

Zero

Positive

Positive
Big
Positive
Small
Negative
Small
Negative
MediumPositive
MediumPositive
Small
Negative
Small
Negative
Big

Positive
MediumZerot

Relati've Angle
to �o alRT

Figure 2.

Negative
MediumNegativeBig

The probability table for single obstacle isdesigned as the two-dimensional finite space. The
environment around the robot is mapped into this table. In
other words, each element of the table reveals thesituation that the robot faces. The dimensions of the
SubModule Table represent the dynamic danger J3[3] andthe relative angle RA shown in Figure l [2]. For example,
the element X in Figure l means that the obstacle isdangerous in the sense of dynamic danger and approaches
the robot from the front.
2.1.3 Learning Scheme

It is difficult for a designer to specify theselection rule of the action in advance because thedesigner can not get all the information about the real
wodd in advance. One of the most promising method to
overcome this problem is adopting the algorithm based on
the framework of Learning automata[l]. Using thisreinforcement method, the robot can autonomously
acquire the good action inany situation through theinteraction with the realworld. As the reinforceVelocity Control Input
ment scheme, we use thefor Obstacle AvoidancePLr-i algorithm[l]. The
PLr-i method updates the
TRO
action probabilities basedon the binary value of
Preevaluation. When thePrediction
robot action is good, thatis, the output from theperformancefuzzy
moduleevaluation
before1s
described
positive,theactionprobabilities are updated
as follows.

Hierarchical Fuzzy Rules- 196 -

PLr - i Scheme
pi(n +I) = pi(n) + c[ I-pi(n)]{Action in the Worldif a(n) = ai and b(n) = I*ipj(n +I)= { 1-c)pj(n)for jpi(n +I)= pi(n)for all i, if b(n) = 0

Probability Table

where c is learning parameter, ai and a(n) denote the ith
action and selected action at the nth stage, respectively.
Pi(n) denotes the probability of action i at the nth stage
and b(n) is the output of the fuzzy evaluation module atthe nth stage. When the robot action is bad, that is, theoutput from the evaluator is not positive, the action
probabilities are not changed.
Each element of the probability table has four
probabilities which correspond to four actions. The initialvalue of each probability is 1/4. After choosing an action
according to the probabilities of the element which reveals
the situation of the robot, the robot updates theprobabilities of the element with the evaluation value
using PLr-i method. The main idea of the PLr-i algorithms
is that the absolute expediency in the unknown random
world is guaranteed[ I]. This is due to the operation of"inaction" in case that the evaluation is not good.
2.2 Extended System for Multiple Obstalces
2.2.1 Architecture
In this subsection, we state the outline of theextended system for multiple obstacles. The extended
architecture is shown in Figure 3. The system consists of 6
modules. Sensing Module, Hierarchical Fuzzy Rule
Module and Action Generating SubModule work thesame as those of the system for single obstacle. The roleof other modules is explained in the following subsection.
2.2.2 Role of Modules in Meta Level
Action Generating MetaModule:This module also selects
one action (rule) among actions based on the action
probabilities in order to combine the steering control
inputs for each obstacle with weights based on theselected action. This module has 4 actions which means
the number of obstacles to be paid attention to . Note that
the Action Generating SubModule is assumed to belearned enough in advance.
Fuzzy Performance Evaluation Module: This
moduleevaluates how good or bad the selected action (rule) of themetamodule is by using the fuzzy logic. The fuzzy
evaluation rules are like ones in Table I.
Action learning Module:This module updates andmemorizes the action probabilities in the element of thetable for the action generating metamodule based on theevaluation value from the evaluation module by the

Varinace ofDegree of t-t--+--+----11---i
Dy_namicDangerfl
200·

Variance of Relative Angle RA

World
Figure 3. Extended System for Multiple Obstacles
reinforcement learning method in Learning Automata. The
probability table for multiple obstacles is also designed as
the two-dimensional finite space. The environment aroundthe robot is mapped into this table. In other words, eachelement of the table reveals the situation that the robot
faces. This table for the metamodule shown in Figure 3
represent the variance of the dynamic danger 13 and thevariance of the relative angle RA. The variance of theparameter 13 indicates a kind of the distribution of thedangerous moving obstacles. For example, the element Yof the table in Figure 3 means that all the obstacle are
equally dangerous in the sense of dynamic danger and thatthe obstacles are located at the same direction.
3. Experimental StudyIn this section, the experimental results areshown to confirm the feasibility of our proposed system.
The assumptions made in the experiments are as follows.
ObstacleI )The number of obstacles is three.
2)The start points are set at random in the front side of
the robot at every run.
3)The moving directions are set so as to approach the robot4)The Obstacles move straight with constant velocity.
The velocities are set at random up to the initial
velocity of the robot at every run.
Robot.
5) The start point, moving direction and initial velocity
is same at every run.
6)The action probabilities are updated at each time when

- 197 -

[3]Y.Maeda et.al,"Collision Avoidance Control among Moving
Obstacles for a Mobile Robot on the Fuzzy Reasoning," J.ofthe
Robotics Society ofJapan, Vol6, No.6, pp.518-522, 1988.

the robot performs one action.
7)PLr-i algorithm is used as the learning method.
The mobile robot and three obstacles used inthis experiment are Khepera made by EPFL. The robot is
controlled by a NEC PC980 IBA Computer ( CPU :Intel
486DX2 40MHz) via a RS232C line by means of a cable.
This cable is also used to supply electrical power. As forthe sensor, CCD camera is used to get the information of
the environment around the robot. The camera is set at theheight of 2m from the ground in order to watch 2.5x3m
area. The camera is also connected with the computer. Allprocesses such as picture reading and analyzing, decision
making and motor control are performed with on-line
process by the computer. The experimental setup is
depicted in Figure 4. Leaming of the probability tables are
performed by simulation in advance. Then using thosetables, the fesibility of our method is tested with realrobots .Before learning, the action probabilities in the
Action Generating MetaModule are not well-tuned. In thiscase, the robot often chooses irrelevant actions and bumps
into the obstacles. On the contrary, after learning, therobot has acquired the intelligence of action selection
based on the action probabilities. In Figure 5, the robot
chooses fine actions and inhibits unnecessary actions andthe robot can move to the goal avoiding moving obstacles.
Figure6 shows the probability history of the element Y of
the action probability table in Figure 3. This figure
indicates that the robot has acquired action rule that robot
regards obstacles as one group and pays attention to justthe most dangerous obstacle in the situation corresponding
to element Y.

Figure 4 Experimental Setup\Obstacle I

Figure 5 Experimental Result after Leaming

4.Conclusion'-­
Action 1

We propose a new action selection techniquefor moving obstacles avoidance using hierarchical fuzzy
rules, fuzzy evaluation system and learning automata
through the interaction with the real world. By using our
proposed system, an mobile robot autonomously acquires
the fine behaviors how to move to the goal avoiding
moving obstacles using the steering and velocity control
inputs, simultaneously. We show the feasibility of our
proposed system with experimental results.
References[l]KNarendra et.al.,"LEARNING AUTOMATA", Prentice-Hall.
Inc., 1989
[2]TAoki et.al,"Motion planning for Multiple Obstacles Avoidanceof Autonomous Mobile Robot Using Hierarchical Fuzzy Rules,"1994 IEEE International Conference on Multisensor Fusion andIntegration for Intelligent Systems, pp.265-271.

Action 0
0

'-

--'

L....l...o�--1��2-oo�--'-�-4 o-o�......._,
Learning Time

Figure 6 Action Probability History of Element Y of Action
Probability Table in Figure 3- 198 -

Auto Navigation on the Wheel Chair
Hongbo WANG*
Chul-Ung KANG*
Takakazu !SHIMATSU* Tsumoru OCHIAI*** Department of Mechanical Systems Engineering, Nagasaki University , Bunkyo-machi, Nagasaki 852, JAPAN**Ube Technical College, Nishihon-machi, Ube, Yamaguchi, JAPAN

Abstract
The paper presents a wheel chair system with thecapability of self-localization and obstacle avoidance. Firstly,
the approaches of landmark recognition and the self­
localization of the wheel chair are described. Then, theprinciple of the obstacle avoidance using laser range finder
is described. Finally, the total system of the wheel chair is
introduced and a navigation experiment is given.
Experimental results indicate the effectiveness of our system.

and the ceiling surface and do not require special installation.
Dulimarta [2] used the ceiling light as landmark. However,the ceiling light was used only for light counter. We use theceiling light for landmark to perform the self-localization of
the wheel chair.

Keywords: Auto-navigation, Wheel.chair, Landmark,
Image Processing, Laser range finder, Obstacle avoidance

I Introduction
The commercial electric wheel chair has been widelyused by handicapped persons suffering from lower extremity
amputations. However, it is difficult for handicapped
persons, having both upper and lower extremity amputations,to drive a wheel chair. For this reason, an automatic wheel
chair with the capability of self-localization and obstacle
avoidance has been developed in our laboratory so thatdaily tasks can be coped with easily.
Our goal is that the handicapped persons can use thewheel chair to move from one position to another positionin an indoor environment avoiding both static and dynamic
obstacles. This paper describes an auto-navigational wheel
chair system with the capability of self-localization andobstacle avoidance.
The capability of self-localization is critical to reliable
performance of the wheel chair. There are many methods
that can determine the self-localization of the wheel chair.
One simple and reliable method is by providing landmark
features of the workspace as external reference sources.
Several researchers have approached the self-localization
problem of mobile robots by employing "standard marks"[6, 7]. The key idea is to use special marks tliat include a
wealth of geometric information under perspective projection
such that the camera location can be easily computed fromthe image of the guide-marks. In our research, we havechosen "ceiling lights" as landmarks because they can beeasily detected due to the high contrast between the light::·. ,:,:,:,:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;: :;:;: : : :;e: : : : :: :;:'c;: :':,: ,:,:=(f{': :.:Fig. 1. Wheel chair with self-localization and obstacle
avoidance
Our desire is to enable the wheel chair to navigate fromone position to the desired position without any human assist.
It is assumed that this wheel chair is used only inside of one
building whose geometry are known in advance. In order torealize the auto-navigation in the building, our wheel chair
has a geomagnetic azimuth sensor and a vision sensor todetect ceiling lights. While the geomagnetic azimuth sensor
gives data about the azimutl1 angle, the output data are often
affected by the environment. Therefore, our system employed
the ceiling lights as landmarks to guide the wheel chair withenough reliability. Of course, the position and the postureof every ceiling light are assumed to be known in advance.
In the case that the ceiling lights are not detected, the wheel
chair navigates with the information of the geomagnetic
azimuth sensor and rotating angle of both wheels.

- 199 -

When the wheel chair navigates automatically. it isunavoidable to encounter the static and dynamic obstacles.
The obstacle avoidance is one of the fundamental
requirements for an auto wheel chair to execute its tasks.
Several studies have been done for the obstacle avoidance
problem of mobile robot by the use of ultrasonic (3, 4), andvisual sensors (5, 8). Among the sensors used for mobile
robots, visual sensors probably provide the richest source of
useful information about the surrounding. However, visualsensors are also the ones that are the most computational
expensive. One possible solution forthe problems of vision,is to employ specialized hardware for navigation, such as
laser range finder, sonars, and inertial navigation system. In
our system described here, we employ the original laser
range finder to realize the obstacle avoidance of a wheel
chair.
Our wheel chair is equipped with two CCD cameras.
The camera facing up is used for detecting landmarks andother camera facing in front in combination with a slit laser
sensor is used for detecting the obstacles as shown in Fig 1.
In section 2, the approach of landmark detection andrecognition and the self-localization of the wheel chair aredescribed. In section 3, the laser range finder is described.
The total system of the wheel chair and experimental resultsof auto navigation are described in section 4. Finally, we
give the conclusion of the paper.

method is shown in Fig. 2. Using the contour points of thelight, the central line of the light can be obtained. From thecross points of the central line and image contour, theraster coordinates P; (i=l,2,3,4) of the light ends can beobtained [1].
uVFig. 2. Image of ceiling LightOn the basis of the raster coordinate of the ceiling light,
we have the vectors Pi=(Uj vi t) (i=l,2,3,4) in camera
coordinate system. The position vectors q (i=l,2,3,4) oflight ends relative to camera coordinate system can be
determined using the three-dimensional image processing
method. From perspective transformation, the relation
between q and Pi can be expressed as

2 Landmark Recognition and Self-Localization
In our experimental environment, the "ceiling lights" aretwo fluorescent lights that are installed parallel. When thewheel chair navigates, the camera may acquires someunnecessary images as a result of wall light, the sunlightfrom window and another light sources. From all images,
the useful image as landmark can be retrieved by checking
the area, boundary, distance and position of every image.
Considering the shape of fluorescent light, the procedure is
described as follows.
(1) Image signals from the CCD TV camera are converted
into binary image signals.
(2) Profiles of white cluster in the image (Run-LengthData) are extracted from the binary signals by a logic
circuit constructed by FPGA (Field Programmable
Gate Array: XC4010).
(3) By the labeling program, we obtain the geometrical
data (area, length, width and the position) of every
white cluster.
(4) Considering the area, length , width and the distancefrom the neighboring white cluster, we can select thewhite clusters corresponding to the images offluorescent lights from the other clusters.
The contour of two lights extracted· using· the above

Fig. 3. Relation between camera and light system(1)
where q=(Xj Yi zi) indicates the position vector of light end
in camera coordinate system, k; is the proportion coefficient- 200

between two vectors. Since the two light are parallel andthe lengths of the two light are equal, following vector
equation can be obtained(2)

Therefore, the logic circuit has two output signal lines. One
output signal is used to inform the controller.whether theway is safe or not. The other output signal is used to informthe controller where the obstacles exist via serial signals.

From above equation, the proportion coefficient k1 can bederived as follows [1](3)IPath Way
ofI

where C21 =p3.(p4xp1) / p3.{p4xpi) and 42-q1 is equal to
the length of the light. Similar to k1, the k2, k3 and k4 canbe obtained. From the equation (1), the CJi can be calculated.
After the CJi is obtained, the relation between the coordinate
systems of the light and the camera can be established.
Considering that the light is installed on the ceiling of thebuilding, we take the light coordinate system as the basecoordinate system. The base system is set as shown in Fig.
3. The origin O of the base system is chosen at the middle
point of points 41 and 43, that is, q0 (43+q1)/l. X-axis isalong with the direction from 41 to q2; Z-axis is along thenormal line of the plane that is determined by three points
4 1 , qz and 43. The Y-axis can be determined by right hand
rule.
After the base system is determined, the transformation
matrix [A] of the base coordinate system relative to camera
system can be obtained. Therefore, the position andorientation of the wheel chair in base coordinate system canbe determined easily [1].

Fig. 4. Principle of laser range finder using slit-ray- -➔ : : �

Safetyne=

3 Laser Range Finder
In order to detect the obstacles on the way, a compact
laser range finder is developed. The principle is based on
the slit-ray projection method. As you see in Fig. 4, a slit-ray
is projected from a laser emitter. The slit-ray image is detectedby a CCD TV camera and the image signal is processed bya logic circuit constructed with FPGA. Two examples ofslit-ray images are shown in Fig. S(a) and Fig. S(b). If someobstacle or strong undulation is on the way, the slit-ray
deviates from straight line. Otherwise, the slit-ray shoulddraw a straight line on the image. Based on this idea, thelogic circuit detects the most bright point in every horizontal
scanning which corresponds to the slit-ray image..
Furthermore, the logic circuit examine if the slit-ray is inside
the allowable range or not. If the slit-ray is not inside theallowable region, the vertical coordinates where allowableregion is violated are transmitted to main computer.

Obstacle

Wheel Chair

Slit-ray----00----00(a) without Obstacle(b) with Obstacle

Fig. 5. Slit-ray image
Due to the processing by a logic circuit, all the processingof data are executed without any time. delay, Therefore, thelaser range finder tests the way 60 times per a second. One
problem of the above sensing system occurs in the case thatexternal light is stronger than the laser light. In order tocope with this problem, our laser range finder has anotherfunction. Under the condition that the slit-ray is often affectedby the external lights, the laser light is turned on and off
synchronous with the video signal. The frequency of theswitching is 30 Hz, and the logic circuit executes subtraction
between the sequential two images. Due to this subtraction,
the effects of external lights are reduced remarkably. Using
the subtracted image, the existence of the obstacles on theway is examined. When we use this function. The laser
range finder tests the way 30 times per a s�nd.

4 Total System and Experimental Results
The total paviga�on system of the wheel chair is shownin Fig. 6. One of the two CCD cameras and a slit laser- 201 -

sensor are employed to detect obstacles and another camera
to recognize landmarks. Two rotary encoders are installedat two drive wheels respectively to measure the distancewhich the wheel chair has traveled. The resolution of counteris 40 degrees. To know the direction of the wheel chair, thegeomagnetic azimuth instrument is employed. The sensor
can measure two vertical geomagnetic components on
horizontal. From the output analogue volts of the geomagnetic
azimuth instrument, the direction angle can be calculated
after AID transformation.

5 Conclusion
In this paper, we present an automatic wheel chairsystem with the capability of self-localization and obstacle
avoidance. The ceiling light is chosen as landmark to guidethe navigation of the wheel chair. The laser range finder isused to realize the obstacle avoidance. Due to the employment
of FPGA and RISC-type CPU (SH7032), the control systemand the laser range finder could be constructed in compact
body. Furthermore, the laser range finder could examine thesafety of the path at video rate (60Hz). We had experimentsof the navigation of the wheel chair in a room. The
experimental results indicate the effectiveness of theapproach we presented here.

References[1] H. B. Wang, S. Tanaka, C. Kang and T. Ishimatsu,"Computer control of wheel chair by using landmarks",

in Proc. of 10th Korea Automatic Control Conference- international program, pp 388-341, Seoul, 1995
[2] H. S. Dulimarta and A. K. Jain, "Mobile robot
localization in indoor environment," in Proc. of third

int. Conj. Automat, Robotics, and Computer Vision,

Fig. 6. Navigation system of wheel chair[3]

We had an auto-navigation test in a room where ten
ceiling lights are installed. The navigation map of the wheel
chair is shown in Fig. 7. We have tested the navigation of
the wheel chair 20 times along the navigation course specified
with the solid line in Fig. 7. Our wheel chair is able tolocate its final destination with a maximum position error of
0.35 meters and a maximum orientation error of 17 degrees.
During this experiment, we put one obstacle on the way.
Our wheel chair succeeded to avoid the obstacle with 70 cm
apart from the static and dynamic obstacle.

[4]

[5]

[6]

t

E
E

End

�

[7]

[8]
Fig. 7. Navigation map of wheel chair- 202 -

vol. 3, pp. 2204-2208, Singapore, 1994.
J. Borenstein and Y. Koren, 'The vector field histogram- fast obstacle avoidance for mobil robots," IEEE Trans.
on Robotics and Automation, vol. 7, no. 3, pp. 278-288,
1991.
J. L. Crowley,"World modeling and position estimation
for a mobile robot using ultrasonic Ranging," in Proc.
IEEE int. Conj. Robotics Automat, Scottsdale, AZ,
pp. 674-680, 1989.
K.weher, S. Venkatesh and D. Kieronska,"Insect basednavigation and its application to the autonomous control
of mobile robots," in Proc. of third int. Conj. Automat,
Robotics, and Computer Vision, vol. 2, pp. 1228-1232,
Singapore, 1994.
M. R. Kahoka and A. E. Arenas," Position vertificationof a mobile robot using standard pattern," IEEE Trans.
on Robotics and Automation, vol. 3, no. 6, pp. 129-142,
1989.
S . Y. Chen and W. H. Tsai, "Determination of a robot
locations by common object shapes," IEEE Trans. on
Robotics and Automation, vol. 7, no. 1, pp. 149-156,
1991.
Y. Yagi, S. Kawato, and S. Tsuji, "Real-timeominidirectional image sensor(COPIS) for vision­guided navigation," IEEE Trans. on Robotics andAutomation, vol. 10, no. 1, pp. 11-22, 1994.

Vision Chip Architecture with Light Adaptation Mechanism
H. Kobayashi t
T. Matsumoto tt
T. Yagi t t t
T. Tanaka tt
t Teratec Corp. Tokyo 180 Japan tt Waseda University, Tokyo 169 Japant t t Kyushu Institute of Technology, Iizuka 820 Japan
Abstract

Light adaptive algorithms/architectures are proposed
for regularization vision chips. The adaptation mech­anisms allow the regularization parameters to change
in an adaptive manner in accordance with the lightintensity of given images. This is achieved by adap­
tively changing the conductance values associatedwith massively parallel resistive networks. The al­
gorithms/architectures are inspired by the adaptation
mechanisms of the horizontal cells in the lower verte­
brate retina.

1

Introduction

Vision chips generally refer to massively paral­
lel arrays of simple analog circuits together withparallel array sensors. Many of the chip architec­tures/algorithms are motivated by visual informa­tion processing mechanisms found in vertebrate retina
while some of them are derived from the Tikhonov
regularization theory. This paper proposes regular­
ization vision chip architectures which incorporate
light adaptation and enable one to change the filter
width of a 'v 2 G-like filter in accordance with the in­put light intensity. Two adaptation architectures areproposed: global and local. In the former, global lightintensity information controls a particular parameterin the information processing cells (pixels) altogether,while in the latter, local light intensity information
regulates a parameter in each individual pixel. CMOS
circuits are also proposed to implement the adaptation
mechanisms. The problem is formulated in terms of
the regularization theory, while the adaptation algo­rithm has been inspired by the adaptation mechanismof the horizontal cells in the lower vertebrate retina.

2

Early Vision Problems via Regulariza­
tion
When a solution to an operator equation,
Av=d,

vEX,

dEY,

(1)

loses existence or uniqueness or continuity in d, equa­tion (1) is called ill-posed. The Tikhonov regulariza­tion [1] converts (1) into a family of minimization prob­
lems:
where 11 • II denotes a norm, n : X -+ R is contin­uous and strictly convex. If Av• = d*, then underreasonable conditions, (2) regularizes (1) in the sensethat for any c-neighborhood N t (v*)of v•, there is a 6neighborhood N0(d*) of d* such that if d E N0(d*),
and if A(6) > 0 is appropriate, then there is a uniquev(d, .A(6))E N0(v*) which minimizes (2). It is arguedin [2] that many of the early vision problems {edge de­tection, stereo, optical flow, etc.) can be formulated
as Tikhonov regularization problems.
Let v := {V1, V2, .... , vn ) E Rn and replace thederivative operations by the difference operations, e.g.,(1;)(x)-+ Vk - Vk-1, c1:�)(x)-+ Vk+l - 2vk + Vk-1
which can be put in a vector form as ( 1; )(-) -+Dv, ( � )(-) -+ Lv respectively, where D and L
are appropriate matrices. Therefore, the regulariza­tion problem on Rn corresponding to (2) calls for theminimization of
2A
G(v, d) = IIAv-dll 2+ � { rllLivll
ArllDL •;' vll2
�

r: evenr: odd(3)

where A is now a map between finite dimensional
spaces and A r := ACr are called the regularization
parameters. Differentiating (3) with respect to v andsetting it to zero, one hasp�2 aa a =A {Av - d)+ E<-1r.A rv=O.
T

V

r:1

r

{4)

If Ar -# 0, the solution ( 4) contains the Lr v term,
which corresponds to the presence of the d2 r v/dx2r
term in the infinite dimensional case. In this paper
A=1, the identity map.
Let us explain how ( 4) can be naturally mapped
into parallel resistive networks. Consider the case P =- 203 -

1, and (4) reads v k - d k - A 1(vk-1 + v k +1 - 2v k ) =0, which is naturally mapped into a parallel resistive
network where each node k is induced by a current
source U k , and connected with go to ground and nodes
k+l and k-1 with 91 with A1 = gif go, dk = uk /9o [3].

3Light Adaptive Architectures

A. Double-Layer Network :In all the vision chiparchitectures that we know of, Ar 's are fixed. Our
proposed architectures make A r variable so that adap­
tation can be incorporated. We will consider the min­imization of (3) where A r is A r (d), and among many
possible adaptive networks, the smoothing-contrast
enhancement filter network [4] has probably one of themost interesting structures suited for this adaptation.
Let us first state the following fact proved in [5].
Fact Consider the double-layer network in Fig.I.
(i) The second layer voltage distribution vi solves
the second order regularization problem with A 1 =g,i/9ml + 9,2/9m2, A2 = 9,19s2/gml9m2, dk =Ti/9ml9m2U k . (ii) The first layer voltage distribution
vl solves the first order regularization problem withA� = g,if 9 ml, d� = u k /9ml· (iii) The difference
Xk := vl-vi enhances contrast of U k after smoothing.
Fig.2 shows the responses Xk to a narrow "slit" lo­cated at the center with two different g, 2 's, whereas
other parameters are fixed. We see that the network
response naturally approximates the "v 2 G filter, andwith different g,2-values, one can have different "filter
width". There is one feature associated with our net­
work; a larger g,2 results in a higher filter gain.
This is very natural if g,2 increases when the envi­
ronment is darker, which is exactly what happens invertebrate retina.
B. Global Adaptation Network :Consider the
double-layer network in Fig.I, where g,2 has an adap­
tation mechanism described byg,2(u) := 1/(GL u k ),
kG > 0.

(5)

Then (i) the second layer voltage distribution
vi solves the second order regularization problem
with A1(u) = g,i/9 ml + g, 2 (u)/9m2,
A 2 (u) =A2(u)/ A1 (u)g, 1 g,2(u)/(gm1 9m2),
1/(gmi/9,1 + 9 m2G(2_::: k U k )), dk = T1u k /(9ml9 m2).
Statements (ii) and (iii) of Fact are still valid.
When the total input current L k u k gets larger (theenvironment is light), g,2 and the ratio A2(u)/A1(u)decrease. This means that when L k U k is large, theemphasis of the network on the second order derivative
decreases, which has rather interesting implications.

- 204

Suppose that U k = u� + e k ' where u� is the noiselessimage while ek stands for noise with zero mean. L k U klarge means that effect of noise is less significant thanwhen L k U k is smaller. Thus when L k U k is smaller,
noise is more significant and the network putsmore em­phasis on the second order derivative penalty. Fig.3
show the network responses for a rectangular input
image with the Gaussian white noise. Fig.3 ( a) shows
the network response Xk without adaptation. A dra­
matic effect is discernible when the g,radaptation (5)is incorporated. Observe that while Fig.3 (a) gives
no information about the edges of the original object,
Fig.3 (b) which is the network response with the g,2adaptation given by (5), correctly identifies the edgeof the original image by its zero crossings. Fig.4 gives
the responses of the networks for a slit input.
C. Local Adaptation Network :If

9,2( k , k+1) := 1/L(vk + Vk+1), L > 0,

( 6)

the second-layer horizontal conductance 9 s 2(k,k+l) be­
tween nodes k and k + 1 is inversely proportional to
the sum of the first-layer voltages vl and Vk+i. Fig.5(a) is a rectangular input while Fig.5 (b) compares the
response incorporating the local adaptation (6) withthose responses without adaptations. One can see that
where the input intensity is high, the response with (6)is closer to that with 1/g,2 = 5Mr2, and where the in­tensity is low, the adapted response behaves similarly
to the one with l/g,2 = 500kr2. Therefore with (6)contrast is even more enhanced where interesting dif­ference exists.

4

Physiological Findings on Retina

This section describes a neural adaptation systemfound in the lower vertebrate retina. In the eye, lightpasses through the transparent retina to reach the pho­toreceptor which transduces light into an electrical sig­
nal. The electrical signal is transmitted to the sec­
ond order neurons, which are the horizontal cell andthe bipolar cell. These three types of neurons inter­act via chemical synapses in the outerplexiform layer.
Previous physiological studies revealed that neighbor­ing horizontal cells are coupled electrically and possessa large receptive field which sometimes covers almost
the entire retina [6, 7]. It was also revealed that neigh­boring photoreceptors are also coupled electrically but
the size of receptive field is much smaller than that ofhorizontal cell [8]. In the present study, the networkof photoreceptor and horizontal cell is decsribed by
the equivalent electrical circuit shown in Fig.I. Each

photoreceptor is represented by 9 m l and each hori­zontal cell by 9m2• Ual and g 6 2 represent the cou­pling resistance of photoreceptor and horizontal cell,respectively. The significance of electrical coupling isthought to be relevant to the noise reduction. Whencells are electrically coupled, the current generated ina single cell spreads into neighboring cells, and thesignal to noise ratio can be improved when the image
has an appropriate size [9, 10]. However, the electricalcoupling blurs the image in return.
The bipolar cell is the first neuron which exhibits
a v' 2 G-like receptive field in the vertebrate visual sys­tem. The response to a stimulus placed in the centerregion antagonizes the one placed in the surround re­gion. The center response is mediated by the direct
input from photoreceptor and antagonistic surroundresponse by the horizontal cell. The v' 2 G-like recep­
tive field is able to perform a smoothing and contrast­
enhancement of the input image simultaneously [11].
There is another type of neuron which modulates
the signal processing in the outer plexiform layer. Theinterplexiform cell (IP cell) is a centrifugal neuron in­
nervating to the horizontal cell [12]. It has been de­
mostrated that IP cell reduces the receptive field sizeof horizontal cell by increasing the resistance U a 2 [10].
The function of IP cell is not yet well understood.
Considering previous physiological observations [13],however, we hypothesize that the IP cell adaptively
controls the receptive field size of the horizontal cellaccording to the signal to noise ratio of the image. If
the intrinsic noise is constant regardless of the adapta­
tion level of the retina, the relative magnitude of noise
to signal is small in the daytime since the light inten­
sity of signal image is high. In that situation, the sizeof the bipolar cell receptive field is to be reduced togain the spatial resolution. This adaptation is likely
to be carried out by the IP cell in the retinal neuralcircuit and the architecture of the vision chip shown
here was inspired by these physiological studies.

5

Analog CMOS Circuits for Adaptation

An analog CMOS implementation of the network
in Fig.l without adaptation has been reported in [4](Fig.6). The photosensor is realized as a phototransis­
tor and its current is converted to a voltage which is
fed into the first layer network, and· the node voltageof the first layer network is applied to the node of thesecond layer network. This implements the Thevenin
equivalent of the current sources in Fig.1. The node
voltage of the first layer network is subtracted fromthat of the second layer network, and it is read out
through analog switches.

Fig.7 shows a circuit which realizes the global adap­
tation network. The voltage vf is first converted intocurrent 1-,,, by the V-1 converter so that I-,,, is propor­
tional to vf, and the summed current I is given by
I := -,,, I-,,, ex -,,, vf. I is fed into the bias voltage gen­erator which produces a bias voltage Ve so that the 9 a 2
value is inversely proportional to I. Also Fig.8 showsa block diagram of the local adaptation network.

I:

I:

References[l] A. Tikhonov, "Solution of Incorrectly Formulated
Problems and the Regularization Method," Soviet
Math. Dokl. 1035-1038 (1963).
[2] T. Poggio, et.al., "Computational Vision and Regular­
ization Theory," Nature, 314-319 (1985).
[3] C. Mead, Analog VLSI and Neural Systems, Addison­
Wesley (1989).
[4] T. Matsumoto, et.al., "A Second Order Regularization
Vision Chip for Smoothing-Contrast Enhancement,"
IJCNN, Beijing (1992).
[5] H. Kobayashi, et.al., "Image Processing Regularization
Filters on Layered Architecture," Neural Networks,327-350 (1993).
[6] K-1. Naka, et.al., "S-Potential from Luminosity Unitsin the Retina of the Fish ( Cyprinidae)," J. Physiol.,
437-461 (1967).
[7] T. Yagi, "Interaction Between the Soma and the Axon
Terminal of Retinal Horizontal Cells in Cyprenus Car­pio," J. Physiol., 121-135 (1986).
[8] D. Baylor, et.al., "Receptive Fields of Cones in the
Retina of the Turtle," J. Physiol., 256-294 (1971).
[9] T. Lamb, et.al., "The Relation Between Intracellular
Coupling and Electrical Noise in Turtle Photorecep­
tors," J. Physiol., 256-286 (1976).
[10] T. Teranishi, et.al., "Dopamine Modulates S­
Potential Amplitude and Dye-Coupling between Exter­
nal Horizontal Cells in Carp Retina," Nature, 234-246(1983).
[11] D.Marr, et.al., "Theory of edge detection," Proc . ofRoyal Society London, 187-217 (1980).
[12] J. Dowling, et.al., "Synaptic Organization of the
Dopaminergic Neurons in the Rabbit Retina," J. Com­
parative Neurology, 203-220 (1978).
[13] T. Shigematsu, et.al., "Effects of Dopamine on Spa­tial Properties of Horizontal Cells in the Retina of theGoldfish," Neurosci .. Res. Supple., s69-s80 (1988).

- 205 -

V
'·

I
I

I
I

(a)
node

Fig. l : The dou hie layer network realizing a V 2 G-like filter.

V

1-----t-----;M1i\_·c-----+,I ---i

Fig.5 : Response of the locally adaptive network.
(a) A rectangular input image.
(b) Responses with l/g,2 = 5.M!1, l/g,2 = 500k!1(no ;1claptation) and L = 2 x 10 7 (local adaptation).

/I!\l'.I.\--+I----;1m·�\ -,-----··>--"_-.
:/ / I \\ 1 / gs2 � 5 M ohm -

\
..o
===
-�-.:
-r' .1--�=:;::''pl=== ::-:::::::::-7,'
�, 1/ gs2 = 500 k ohm,
/

j

i I l

node

Fig.2 : Responses of the double layer network with
lfg,2 = 5M!1 and 1/g,2 = 500k!1. for a slit input.

V
Fig.Ci : Unit cell circuit of a double layered network [4].

(a)

(b)

Fig.:3 : Responses of the network in Fig. l(a) with l/g,2 = 5M!1 (no adaptation)(b) with G = 1.0 x 10 13 (adaptation).·node
Vcar(ll+ll+-•I•)

Fig. 7: Dlock diagram of the global adaptive network.

V

1.w.-,\\ 1 / gs2 ";' 5 M ohm_I

global adaptation.

.

' nodeFig.4 : Responses of the networks in Fig.I with
lfg.2 = 5M!1 (no adaptation) and G = 2.0 x 10 1 5(adaptation) for a slit input.

Fig.8 : . Block diagram of the- 206

local adaptive network.

Aspects of Non-Cartesian Robotics
Takashi Gomi
Applied AI Systems, Inc.
340 March Road, Suite 600
Kanata, Ontario, Canada K2K 2E4
Tel. +I 613 592 3030; Fax. +I 613 592 2333
email: gomi@AppliedAl.com
Abstract
A new trend in robotics that began with theintroduction of behavior-based artificial intelligence
open up possibilities of creating very fast, flexible,
robust, efficient, animate, and economic robots using
methods much simpler than conventional robot
development. These robots are.fundamentally "open"
in their design, and hence are amenable toincremental or evolutionary review of theirfunctionality. Robots with such characteristics may
be called non-Cartesian robots because of the strong
underlying differences in philosophical backgroundfrom conventional robotics [I]. Most efforts todescribe and formalize the cause and theoreticalaspects of those desirable characteristics are still
speculative, and at best tentative. However, afteralmost a decade of R&D in the field, the effort for theapplication of a new form of robots is progressing
steadfastly, some almost reaching the realm of apractical real world application. Using examples, thispaper examines each of the desirable characteristicsof robots implemented on behavior-based or non­Cartesian principles.

1. Introduction
The behavior-based approach to create robots androbot-like automata marks the debut of what is now
referred to as non-Cartesian robotics. This approach,which began in the mid-1980's, produces robot
motions which are faster and more flexible than those
produced using traditional robot control methods andachieves similar functions with code which is up to
one-thousandth the size of that using conventional
methods. The most important aspect of behavior­
based control is not in computer hardware or software
but in the architecture. The crucial requirement is aframework which allows execution in parallel ofshort, simple processes developed in hardware,
software, or their mixture, called competence. The
competence structure implemented on a specific robot
is non-hierarchical and there is no "main routine" or"control box". Self-organization among these mini­processes results in an emergent global behavior of
the system as a whole. Execution time of eachcompetence is very short, allowing pseudo-parallel

execution of competences using even the simplest of
conventional von Neumann processors. Because theglobal behavior of the robot is dependent on emergent
properties of many mini-processes the robots
demonstrate graceful degradation in the event of
failure, rather than abrupt halting of the entire system
as seen in conventional robots and computer systems.
Subsumption Architecture (SA) begun by R. Brooksof MIT [2] [3] [4] [5] is the most successful
realization of behavior-based AI principles in robotics
to date. An SA robot typically has several to several
dozen hand-coded or automatically generated
competences. Each competence takes in sensory
inputs from one or more sensors and generates output
to one or more actuators through which actions are
generated in response to a specific situation.
D. Floreano and F. Mondada successfully achieved
an evolutionary system using the Khepera robot, asmall robot 5.5 cm in diameter and 3 cm tall [6]. It isconnected to a workstation through an RS232 cablewhich uplinks sensor inputs and downlinks motor
signals generated at the workstation to drive Khepera's
two differential motors. The input layer of a two­layer feed-forward neural network is connected to
Khepera's 6 forward and 2 backward-facing active
infrared proximity sensors. One hundred individual
control programs (neural networks) per generation aresequentially executed. Each network learns a set ofcompetences in the form of input pattern/output signal
pairs governed by weights in the network. A standard
variant genetic algorithm with fitness scaling, roulette
wheel selection, one point crossover, and high degrees
of mutation is used. Individuals who evolved
successfully navigated around a circular track with asmooth path and constant direction. Other similarachievements in recent years in this new field are also
seen in [9] [10].

2. Characteristics of Non-Cartesian Robots
Figure 1 shows the serial nature of processing andthe hierarchical decomposition of the procedural
structure typical for a Cartesian intelligent robot. The
insistence on a hierarchical module structure andhence the propagation of a fault which may originate
at any module at any level upward through the
hierarchy, and the lack of uniformity in module- 207 -

obvious from the two diagrams, there is also a large
difference in the size of code required (often 1,000 to
I 'difference) when implemented in software and an
estimated 100 to I difference in the number of circuitcomponents when implemented in hardware.

structure are all characteristic of the Cartesian control
system evidenced here.

Sensors¢ag.

reason about behav1ar of object
plan changes to the wol1d
Identify objects
monitor changes

Sensors¢

bulldmaps¢Actuators

explorewander--ovoid objects

Figure 1 Structure of Cartesian intelligent robot
systemS : Sensing

This is contrasted by Figure 2 which depicts themodulestructure for a typical non-Cartesianbehavior-based robot. Here there is relative uniformityof the modules (competences), even distribution of
the modules, and no static master-slave relationships,
no explicit functional or signal connections among themodules, and no center of control. Although it is not

A : Action

Figure 2 Non-Cartesian intelligent robot system
Some of the differences in characteristics ofCartesian robots and non-Cartesian robots are
summarized in Table 1.

Table 1 Comparison of Cartesian and non-Cartesian intelligent robots
Aspects

Cartesian intelligent robots

Non-Cartesian intelligent robots

Nature of operation

Control

Autonomy

Identification ofentities

Definitions - Defining action, position,
movement, task, work, and operational
environment

No definitions - Position, motion, even
effective spheres in operational
environment are emergent

Representation ofentities in robot

Models to document and visualize thedefined entities

No models - instead, a robot acquire� orgenerates a collection of competences tocope with reality

Monitoring of robot's
actions

Measurements - positions, velocity,· momentum, mass, force, all need to bemeasured continuously

No measurements. Humans as anintelligent agent, for example, do notmeasure parameters in their execution ofactions

Nature of processing

Continuous computation to verify
compliance to the definitions andmodels and to carry out measurements

No computation. It is not computationthat mobilizes actuators of highly
intelligent agents but reactive signals- 208 -

Generation ofaction
sequence

Planning - tasks and a detailed
sequence ofactions in each task need to
be planned in advance and in explicit
terms, and monitored throughmeasurements during execution

Emergent plans. Most serious real world
tasks are too complex to be explicitly
planned. Only simple task sequences(agenda) would be given. Action
sequences emerge as a result ofinteractions among goals, situations in theoperational environment, and robot's
internal conditions

Method of obtaining"intelligence"

Algorithm prepared in accordance witha plan is executed and "intelligence" isNo algorithm -intelligence emerges as aresult of self-organization among
competence processes running in parallel

Mode of execution

Centralized control or implied central
control

No centralization nor control.

Distribution of action
generation mechanism

Overt and/or covert singularity

No singularity - competences are fully
distributed throughout system

Method of managing
levels of abstraction

Explicit static functional hierarchy
designed and implemented in control
structure

No predefined hierarchy - hierarchicalstructures may emerge dynamically duringexecution but none permanent

assumed to be generated as a result

Spontaneous and asynchronous invocationof independent competence modules• COG [7] Generation of human-like consciousness
by artificial means through a fully situated andembodied robot.
• Emergent Computation [8] A relatively new theory
on computation which takes a number of
independent, simple task-achieving computational
modules and executes them in parallel in a fully
distributed manner to observe the emergence ofintelligence.
• Selectionism [9] Each competence is decomposed
into behavior systems which in tum have adynamic relationship between a set of sensory andaction pairs and a set of internal quantities.
• Fungus Eaters [10] An autonomous agent cannot
simply go after its predisposed tasks but also must
look after other functions such as eating.
• . Enaction [11] There are common principles
between sensory (perception) and motor (action)systems such that how the perceiver is embodied
affects its actions.
• PerAc [12] Emphasizes the importance ofperception-action pairs sans processing' as afoundation for intelligent actions.
• Collectionism (C. Langton) High level structure
emerges to govern the ·· operation of the systemthrough self organization of low level reactivefunctions.
The following list points out some of the examplesof non�Cartesian robot's behavioral characteristics
which open up their application in heretofore difficult
fields such as service applications.
1) Swift movement at very high efficiency

3. Significant Behaviors Generated by Non­
Cartesian Robots
The observed unique features of non-Cartesian robots
are:• Speed and agility: as a mobile and a stationary
robot, this new approach achieves unprecedented
speed of action;• Flexibility: the robots have the ability to rapidly
adjust their response to changing constraints;• Robustness: the robots persistently resist external
and internal factors attacking their wellness,allowing a true graceful degradation;• Efficiency: the robots exhibit considerable code,material, energy, and size/weight efficiency;• Evolvability: it is drastically easier to apply
evolutionary computational methods to non­Cartesian robots than to conventional counterparts;• Naturalness: highly animate motions withnaturalistic accelerations and non-uniform reactions
canbe easily generated;• Economy: smaller hardware, software development,
and testing costs for similar functions; and,• Affordability: the robots can be implemented bythose with less formal training and experience inrobotics.
The theories and mechanisms that are responsible forthese features are currently being studied by a range
ofresearchers. Existing theoretical attempts, some inrobotics and others in Computer Science, AI and theirneighboring fields include the ones shown below:

0

- 209

Since non-Cartesian robots do not execute
predefined algorithms to generate their motions, it
can be made highly reactive. This ability toactivate their actuators much more rapidly andspontaneously allow them to respond to changing
situations detected through sensors more
appropriately.
2) Robust recovery from confinement
Confinement is a situation in which a robot's
movements towards achieving its goal isundesirably restricted. By providing a robot with
the necessary set of competences to break loose
from such a situation, allowing them be invokedautonomously through sensor signals, the robot
can dislodge itself from confinement.
3) Flexibility of emergent trajectory
Since actuators are under the direct influence ofsensor signals, and there is no explicit path
planned for their motion nor algorithm to generateit, the trajectory of the robot's actuator is emergent,
as in humans. It is up to sensor inputs and a
specific competence associated with them to adjust
the trajectory in accordance with a given situation.
4) Persistent behaviors
Robots can be built to automatically repeat
behaviors in difficulties towards achievement of agoal.
5) Flexible adaptation to changing goals
The goal of a robot can also be changed andadapted to respond to changing situations.
6) Emotive factor
Robot's movement can be easily modified byemotion-like internal state changes.

4. Discussion
As described above, non-Cartesian robots have anumber of significant performance characteristics
which are unique and appear useful in a new range ofrobot applications. These robots are capable ofproducing movements which have not been achievedby most conventional robot control methods. This
inspired us to believe non-Cartesian robotics will behighly suitable for building robots which conduct
tasks that are commonly found in service applications
such as cleaning, transportation, dynamic interaction
with humans and other robots, explorations,
underwater activities, and tasks which are generally
dangerous to humans. The most significant aspect of
non-Cartesian robots which have these promising aswell as interesting characteristics is that these task­
achieving behaviors are realizable with a very limited
amount of resources. This simplicity is due to thedependence of non-Cartesian robots on emergent
functions generated from executions of competences
either implemented in computer software or hardware,rather than direct and explicit execution of

instructions. This is a stark contrast to the way
motions of robots are generated in conventional
robotics where the movement of the robot or robot
parts is the direct result of the execution of explicit
algorithms implemented in either computer program
or electronic circuity.
References[I] T. Gomi, "Non-Cartesian Robotics", Speciallssue
on Biorobotics in Robotics and Autonomous Systems,

Elsevier Science, to be published in January 1996.
[2] R.A. Brooks, "A Robust Layered Control System
for a Mobile Robot", IEEE Journal of Robotics andAutomation, RA-2, March 1986.
[3] R.A. Brooks, "New Approaches to Robotics", inScience, Volume 253, 13 September 1991.
[4] R.A. Brooks, "Intelligence Without Reason",Proceedings, International Joint Coriference on
Artificial Intelligence (JJCAI'9I), Sydney, Australia,
August 1991.
[5] R.A. Brooks, "Intelligence Without Reason",revised version in The Artificial Life Route to
Artificial Intelligence, edited by Luc Steels andRodney Brooks, Lawrence Erlbaum Associates,
Publishers, 1995.
[6] D. Floreano and F. Mondada, "Automatic Creationof an Autonomous Agent: Genetic Evolution of a
Neural-Network Driven Robot", Proceedings, Third
Coriference on Simulation of Adaptive Behavior(SAB'94), edited by D. Cliff, P. Husbands, J. Meyer,
S.W. Wilson, MIT Press, Bradford Books, 1994.
[7] R.A. Brooks and L.A. Stein, "Building Brains for
Bodies" MIT A.I. Memo No. 1439, August 1993.
[8] S. Forrest, "Emergent Computation" in Emergent
Computation, The MIT Press, Cambridge, MA, 1991.
[9] L. Steels, "A Case Study in the Behavior-Oriented
Design of an Autonomous Agent", Proceedings, Third
Coriference on Simulation of Adaptive Behavior(SAB'94), edited by D. Cliff, P. Husbands, J. Meyer,
S.W. Wilson, MIT Press, Bradford Books, 1994.
[10] R. Pfeifer, B.Goetz, C. Scheier, "Building
Fungus Eaters: Accounting for Self-Sufficiency",
Submitted to Artificial Life IV, Fourth International
Workshop on the Synthesis and Simulation ofLiving
Systems, Cambridge, MA, July 1994.
[11] F.J. Varela, "The Re-Enchantment of theConcrete", in The Artificial Life Route to Artificial
Intelligence, Building Embodied, Situated Agents,
edited by Luc Steels and Rodney Brooks, Lawrence
Erlbaum Associates, Publishers, 1995.
[12] J.D. Nicoud, Introduction at From Perception to
Action, Lausanne, Switzerland, September 1994.

- 210 -

Artificial Retina Chips
Kazuo Kyuma and Jun Ohta
Advanced Technology R&D Center, Mitsubishi Electric Corporation
8-1-1 Tsukaguchi-Honmachi, Amagasaki-city, Hyogo, 661 Japan(TEL:81-6-497-7078, FAX:81-6-497-7288, e-mail:kyuma@qua.crl.melco.co.jp)
Abstract

We have developed a novel type of image .sensor called
an artificial retina which can simultaneously sense andprocess images. Device structure, fundamental
performance, operation principle of image processing,processing functions, and applications are described.

1. Introduction

Recognition and understanding of real world images is
one of the most important information processing
technologies for the multimedia systems of 21st century.
Such images contain a very large amount of information
which present image processing systems cannot analyze inreal time. This is because the present systems separate
image sensing and image processing ; images are sensed
by camera and processed by computer. Consequently
performance is limited by slow camera frame rate and lowtransmission rate between camera and computer. By
contrast, human beings can execute very difficult tasks inreal time. For example, we can instantly recall the namesof the people by glancing at their faces, and we cananalyze the motion information by selectively detecting
the moving object. This is due to the inherent parallelismof our visual system, particularly of our retina, which
combines image sensing and processing functions.
The artificial retinas defined in this paper are devices
that can simultaneously sense and process images. To
date, several kinds of related devices have beendeveloped, such as vision chips, focal plane processors andsilicon retinas. However, the specialized functions of thedevices reported so far precluded them from use in large­
scale industrial or commercial applications. Therefore,
devices with more on-chip functionality are required. The
most profound distinction of our artificial retinas comparedto other devices is its on-chip functional flexibility.

circuit, which functions are to set the detection sensitivity of
the VSPC to one of the three values, (1,0,-1), by applying
the control voltage to the random scanner, and toaccumulate the photo-carriers to increase the sensitivity.
The performance of the fabricated artificial retina issummarized in Table 1. The size of each VSPC is 35 X 26
µ m2 and the total size of the chip is 12 X 12mm2 • The
profound feature compared to other image sensors is its
functional flexibility, that is, on-chip image processing
capability, wide dynamic range (38dB), and variable frame
rate (1-lO00Hz). Figure 2 shows a microphotograph of theartificial retina.
nput Image
Input Circuits(Random
Scanner).

Pixel Core Circuits

II

Function Selectio/
SignalOutput• Circuits(Multiplexer)

Output Image(Edge Extraction)

Fig. I: Image processing in the artificial retina as a matrix­
vector multiplication. The product (vector J) of the
input image (matrix W) with the sensitivity control
voltage (vector S) is obtained through multiplexer
in semi-parallel operation.

2. Structure, Performance

Figure 1 shows our artificial retina which consists of a
2-D (256 X 256) variable sensitivity photodetection cell
(VSPC) array, a random scanner for sensitivity control, anda multiplexer for output. These circuits are integrated on a
Si substrate using 1- µ m standard MOS technology. The
VSPC's contains a pn photodiode (PD) and a transistor- 211 -

Table 1: Characteristics of the artificial retina.
Pixel Number
Pixel Size
Chip Size
Sensitivity
Dynamic Range
Frame Rate
Imager Type
Functions

256x 25635um x 26um
12mm x 12mm
0.20 u Allx • msec
More than 40dB
variable: !Hz~ IOOOHzMOS Tvne
Image Sensing, Edge Extraction,
Noise Elimination, Pattern Matching,
Attention Focus, Random Access,
2D-+ ID Projection

lli f
�

-IO I

,0--

�

(a) Image Sensing

�
,0--

,0--

00

(b) Edge Extraction

-IO I

r

■
�
,0--

,0--

�

(c) Smoothing(12mmX 12mm)

Fig.2: Microphotograph of the artificial retina with
256 X 256 variable sensitivity photodetection
cell (VSPC) array.

3. Operation Principle, Processing Functions

The on-chip image processing is based on optical matrix­
vector multiplication. In Fig.1, the input image is directly
projected onto the chip as the weight matrix W. All
VSPC's have one electrode (detection sensitivity
electrode) connected along rows, yielding the sensitivity
control vector S. Thus the VSPC sensitivities are set toone of the three values (1,0,-1) at each row. On the other
hand, the output electrode is connected along columns,
yielding an output photocurrent vector product J=WS. The
remaining electrode (reset electrode) is used to reset theaccumulated photo-carriers.
The significant feature of our artificial retina over otherdevices is the variety of processing that can be achieved
through simply changing the control voltage pattern S.
Several examples of processing are summarized in Fig.3,
including TV-camera like image sensing, edge extraction,imagesmoothing, p a t t e r nmatching, image
compression/recognition. Let us describe three examples.
The first example is the edge extraction operation in
Fig.3(b ). The sensitivities of two adjacent VSPC rows are
set to+l and -1, respectively, while all othersensitivities are set to 0. In this case, the output. current is
proportional to the difference in light intensities of the two
By shifting the control voltage patternactive rows.
cyclically (0, +1, -1,0,0,...0), the horizontal edges of the
input image are extracted. Thus the system operates in atime-sequential and semi-parallel operation mode.
The
processing time, which· is equal to the frame rate, is 1 ms
at fastest. The experimental results of the edge extraction
is depicted in Fig.4(b), together with the image taken by
the TV-camera like image sensing mode (Fig.4(a)). Theseexperiments are made by the camera system with theartificial retina (see Fig.5), under the conditions of 8msec
frame rate and 15mW/cm2 illumination power.
The second example is a random access function (see(d) Random Access(e) Pattern Matching

N, :ll � tc fl l;t ;I:'(!> I� ?>"'1:v ih � 'It:i< </? IQ. t;� '6 I,,":i It -tt-r n" /J)!J, fa� -'t I:: er, ll t, J: �(f) ProJectJon

Fig.3: Examples of image processing options in the artificial
retina.
(a)TV-camera like image sensing mode: all VSPC
rows are successively scanned with a single polarity
voltage, one at a time.
(b )Edge extraction mode: two adjacent rows are
addressed with opposite polarity.
(c)Smoothing mode: voltage sequence such as (1,1,1)
is used to suppress noise and smooth out edges.
(d)Random access mode: a part of the rows is
selectively scanned to take out a partial image
projected onto the retina chip.
(e)Pattern matching mode: the target item is detectedby using it as the scanned control signal.
(f)Projection mode: all rows are simultaneously
addressed with the same polarity, and the retina
output (projection) is then fed into a neural network
for classification.

(a) TV mode(b) Edge extraction mode

Fig.4: Experimental photographs of edge extraction mode(b), together with the image by the TV-camera like
image sensing mode (a).

- 212 -

lOK.............. ! ...........

··•····!········•········!················································ Measuremen of :: p
Artificial' Re
lK Present1

�
� 100

·Autom'
&
Aire

1 2
10

Fig.5: Photograph of the artificial retina camera system.

;:,,.,'CCD

Industrial
Factory Automat
tobotics:

10

Defense i,
Artificial Retina(Goal):··············+··••· ·····················

10

10

Resolution (Pixels)
Fig.7: Performance of the artificial retina andpotential application areas.

4. Applications
The application fields of artificial retinas include, as shownin Fig. 7, man-machine interface for multimedia systems,
automotive and avionics sensors, factory automation androbotics, video still camera, and defense applications.
Particularly interesting use is a man-machine interface inmultimedia systems where it can replace the present
keyboard with gestures and/or body actions.
Anotherinteresting use is an artificial eye in moving vehicles where
it can assist the driver by recognizing road markers. With its
particularly fast response, the artificial retina could prevent
collisions by sensing obstacles and approaching vehicles.

Fig.6: Experimental photograph of light spot tracking.

Fig.3 (d)) where only a part of the image projected onto
the VSPC array (for example, a partial image of attention
projected onto the 16X 16 cells) is scanned by the random
access scanner, and output by the multiplexer. The
experimental result of the tracking of the moving object(light spot), which uses the random access function
described above, is shown in Fig.6.
The third example is an image recognition system using
a neural network as a post-processor of the artificial retina
output. As shown in Fig.3(f), all VSPC's are set to a
unifonn sensitivity of one. The photocurrents (proportional
to light intensity) in each VSPD column are summed upsuch that the output current vector, J, constitutes a verticalprojection of the input image. The output signal J is acompressed representation of the input image W and fed
into the neural network for recognition. We have applied
this system to the optical character recognition, andsucceeded in the recognition of the full set of 1945
Japanese Joyo Kanji characters with almost 100%
recognition rate.

5. Conclusions (Towards New Generation
Artificial Retinas)

Artificial retinas execute multiple, easily programmableforms of image processing with inherently high speed andparallelism. A new generation of high resolution retinas thatincorporate memory devices and/or light emitting devices
will further processing capabilities.

(References)
1) E.Lange et al.,IEEE MICRO, 14,6, p.29,1994.
2) K.Kyuma et al. nature,372,6502,p.197, 1994.

- 213 -

Distributed and Autonomous Sensing based on Immune Network
Yoshiteru Ishida.
Graduate School of Information Science 'Nara. Institute of Science and Technology,
Ikoma., Na.ra., 630-01 J a.pan
Abstract
A distributed and autonomous sensor network is
proposed based on the informational feature of im­
mune network; recognition of non-self by distributedand dynamically interacting units, recognition bysimple comparison with units themselves, dynamical
propagation of activation that would lead to systemlevel recognition, and memory embedded as stableequilibrium states in the dynamical network.
The network is explained by an illustrative examleof eight coin puzzle: a balance must be used only threetimes to identify one coin with weight different fromseven other coins. Our network also uses dynamical
structure network rather than fixed structures usedin neural networks. Simulations show that non-self(different coin in the eight coin puzzle, sensor/process
fault in the monitoring example) will be identified bydynamically propagating activation through the net­
work.

1

Introduction

Theory of immune network proposed by Jerne [1]provides the network view that B-cells are mutuallyand dynamically connected by antigen antibody inter­action. Not only antigen but also antibody generatedby B-cells will act as an angigen against the other B­
cells, thus presupposing internal image of the antigen.
Both antigen and its internal image activate the same
specific type of B-cell. This network view would im­
plicitly suggest that the immune memory is somehow
embedded in the network; when disturbed by antigen,the network in an equilibrium will move to other equi­
librium point, thus memorized the encounter with theantigen.
After the proposal of immune network (idiotypic
network) theory by Jerne 1 , systems approach have
1 Theory of immune network by Jerne is not much mentioned
recently in immunology after more practical view of interleukin

been made extensively to characterize immune re­sponse by dynamical models (2]. These dynamical
models essentially describe population dynamics ofantigen, antibody, and specific immune related cells
in a similar manner to the models describing popula­tion dynamics of species in mathematical ecology.
Although the systems approach may explain someimmunological phenomena by associating character­istics of immune response with interactions among
antigen, antibody and B-cells, fundamental problems
such as mechanism of immune memory still remained;
whether the memory can be attributed to local cells(long-lived cell) or to the network of cells.
Several models based on informational feature of
immune systems have been also studied [3, 4]. Ourmodels have been developed [5] in a similar line tothese models; rather than modeling immune systems,
extracting important informational character for thepresent information systems such as robots, computersand computer network. Ultimate goal of our researchis to develop a relation science that can deal with dy­
namical networks where the population of units, topol­ogy of network structure, and the mode/function ofinteractions may change.
Features of immune systems we have tried to use
may be the following: recognition of non-self by dis­tributed and dynamically interacting units, recogni­tion by simple comparison with units themselves, dy­
namical propagation of activation that would lead tosystem level recognition, and memory embedded asstable equilibrium states in the dynamical network.
The immune network architecture we adopted canbe understood with the following weighing metaphor.
There are roughly two methods of weighing objects:
one is using scale that require a sophisticated central
network (which is complex network involving not only B-cells
but T-cells, MHC molecules; not only antibody but interleukinthat do not correspond to specific antigen) becomes dominant.
Nevertheless, it may give an important insight for information
processing to attain a specific pattern recognition: self/non-self
discrimination.

- 214 -

processing mechanism that maps weight to the infor­
mation of number, another is using balance and many
types of balance weights. The former is often accu­
rate and efficient, once the mechanism of mapping isdevised. However, the latter method of object-against­
object offers a distributed and robust way of weighing.
The network view of immune systems can be under­stood with this metaphor; regarding many types ofbalance weights as recognizing agent (immune related
cells such as B-cell and T-cell that a receptor withspecific antigen) and action of weighing by balanceas recognition by paratope and epitope with spatialcomplementarity. The result of recognition is used
to activate other recognizing agent, similarly to thefact that the result of balance is used to determine
more appropriate balance weight against the target
object. This object-against-object weighing is more ro­bust, since the weighing mechanism is simple compar­
ison and that the information is distributed into manybalance weight. Although this metaphor is rough2 ,it seems to reflect many important informational fea­tures of immune network such as specificity, diversity
and control by activation-propagation.
Section ?? discusses

2

Figure 1: An example of evaluation chain
A possible association corresponding to the evalua­tion chain would be the following dynamical system:

dr1 (t)/dt
dr2(t)/dt
dr3 (t)/dt
dr4(t)/dt
drs(t)/dt

Sensor Network Based on the Con­
cept of Immune Network

Based on the three features of immune network
above, we proposed the dyanamical network which
would solve self/nonself discrimination.

dr,(t)/dt
R,(t)=f( {R;(t)}, {r,(t)})
1
1 + exp(-r,(t))

In this network, not only state of units (i.e. r i (t) or
R i (t)) but also the connection between units are sub­ject to change as presented in the next section. We
have proposed several variations of this model, andanalyzed their properties. To show that the network
would have an emergent property, consider the prob­
lem of identifying nonself in the network shown in Fig.
1. Each unit decides the identity of other units; out­
puts+/- depending on the target unit is self/non-self,
2This metaphor would be closer to immune network if recog­
nizing unit (balance weight equiped with balance) can reproduce
its clone with mutation to enhance elimination -capability andaffinity. In this paper, however, we do not persue the adaptive
behavior by adaptive change of the population of specific agent
mentioned reproduction and mutation above. These features
are discussed in the immune algorithm we proposed elsewehre[6].

respectively. However, non-self units may not follow
this rule. State variable (r i and its normalization R;,)
indicating the identity of units are assigned to each
unit; R i = 1 for self R i = 0 for non-self. Dynamicalsystem is constructed by associating the time deriva­
tive of the state variable with state variables of otherunits connected by the evaluation chain.

=
=
=
=

-4�(t) - r1(t)-4�(t) - 4R5 (t) - r2(t)-4R4(t) - 4Rs(t) - r3(t)-4R1(t) - 4R2(t) - 4R3(t) - r4(t)-4R2(t) - 4R3(t) - rs(t)When the units 4 and 5 are nonself, we have thetest result as shown in Fig. 1. Simple voting at each
unit does not work, since three units 2, 3 and 5 areall evaluated as nonself by two other units and hencecannot be ranked in terms of the possibility of nonself.
However, the above dynamical model would solve theproblem (7].

3The Eight Coin Problem

In this section, we elaborate the model so that a setof units can test other set of units and that interactions
are also dynamically activated.
The eight coin problem is the famous puzzle:
how many times a balance must be used to iden­
tify the one coin whose weight is different fromother seven coins. To solve the puzzle, let eight
be labeled by 1, 2, 3, 4, 5, 6, 7, 8 and grouped into:{1, 2, 3}, {4, 5, 6}, {1, 4}, {2, 5}. Then the coin with dif­ferent weight will be identified using balance only three- 215 -

R(7)1 r-= _-.,. .

R(2)/. ,,··=

: " R(3)
1. .t............................. uac ......................................... -�'"-/
,

R

1.

\
1

]

\

\

0.999999 :\

\R-(

"\
\

',

\

I

R(l) :\\ R(l4)123>
0. 999999
l,I1'-'"-----.......:�= ���----I,_. �.
0.999999
5
10
15
20 25
30
35

l

t

Figure 2: Network Expressing the Eight Coin Problem
when the Coin 1 is different

times among these groups and coins. Let U(ij ... k)indicate the total weight of all the coin in the set{ i,j, .., k }. First compare the weight between U(123)and U(456). When their weights are not equal thenthe coin with different weight is included in these sets,
subsequently narrowing the set by comparing U(14)and U(25). Otherwise, U(7) and U(8) will be com­pared. The third comparison will be performed basedon the results of these two comparisons. Thus, controlof the sequence of comparisons allow the gradual nar­rowing of the subset including the coin with different
weight and small number of comparison. This puz­zle motivates our network that activates comparison
based on the state of other units.
dri(t)/dt = E, 'UljiR; + Ei Wi;R; - ½ Ei(wi; + 1)
where Wij = ai, j (t)Tij and Tij is comparison result
between U(i) and U(j): +1 for equal and -1 for not
equal, thus Tij = Tj i for this example. Oij(t) (alsoassumed to be symmetrical in this example) is activa­tion level dependent upon the state of the units that
activate or inactivate the link from the unit i to j. For
the simulation, the following is used.

0!123,456(t)
0!14,25(t)a1,s(t)a1,7(t)a2,7(t)0!3,7(t)a7,s(t)ae,7(t)0!4,7(t)as,7(t)= 1= 1- (Rm(t) + Rm(t))/2= 1- (R7(t) + Rs(t))/2= 1- (R7(t) + Rs(t) + R14(t) +R2s(t))/4= 1- (R14(t) + R2s(t))/2= (R14(t) + R2s(t))/2= (Rm(t) + Rm(t))/2= (R14(t) + R2s(t))/2= 1- (R14(t) + R2s(t))/2= 1- (R14(t) + R2s(t))/2

Figure 3: Simulation for the Eight Coin Problem when
the Coin 1 is different

In this modeling, we use the binary information
whether two weights are equal or not, hence the in­teraction between units is symmetric; O'.ij ( t )Tij =O'.ji(t)Tj i Fig. 2 shows the network solving the eightcoin puzzle when the coin 1 is different. Comparison
associated with +(-) indicates that these weights areequal(different) and the number beside the sign is theorder when of the test activation. Comparisons with­out +/- are those not activated. Fig. 3 shows thesimulation result for this case. It is known that thesubset including the coin with different weight is sub­sequently narrowed down until the coin 1 is identified
finally. This would capture not only structurally dy­
namical feature of immune net, but also its cooperativeinteraction for identifying nonself.

4

Application
Problemto

Flow

Monitoring

Consider the example of flows shown in Fig. 4. The
constraints of this flow follows:

F(2) + F(4)F(l) + F(2) +F(4)

=

F(3)
F(5)

= F(4) + F(6)F(l) + F(3) = F(5)F{5) = F(6) + F(4)F(l) + F(3)Let U(i) indicate flow lost after the sensor i. Checking
the balance of each equation can identify .whether theflow lost occurs after the sensors of the left handsindeof the equation. When the balance of the equation- 216 -

sensorr--�--�--�--�---,---�---r-

1. 4 tt
1 3
R(3) R(l ;3) R(4,6) R(5). 1 · ... ... ... ... ... ... ...... ... ... ... -----..,,,.- ... ... ... ... ... ...... ... ...... ... ..

1

!r

·-,

I
RO. 8 I
0.6

I

.,\

\ R(l ,2,4)\\R(2,4)0,40.2

Figure 4: An Example of Flow Network

2.5

5

7.5

10

12.5

15

17.5

20

t
F(l) + F(3) = F(4) + F(6) hold, then the balance of
F(l)+F(2)-t--F(4) = F(5) and F(2)+F(4) = F(3) will
be checked. Otherwise, the balance of F(l) + F(3) =F(5) and F(5) = F(6) + F(4) will be checked. Fig. 5
shows the test activation order and their results when
fl.ow after sensror 4 is lost. Fig. 6 shows the simulation
result for the case.

Figure 6: Simulation for the Flow Monitoring Problem

pendently, as explained in the examples of the eightcoin puzzle and the fl.ow monitoring problem.
Application to the sensor integration ofrobot would
lead to more life-like system. Enormous recognition
capability ofhuman may require not only conventional
learning technology, but also self-organizing and evolv­
ing network of sensing found in immune systems.
For further research, aunotnomy of unit should beenhanced so that they can form groups and activate
other links and units based on the environment.

References[1] N. K. Jerne, "The Immune System" Sci. Am. Vol.
229, No.l, pp. 52-60, 1973.
[2] R. R. Mohler, C. Bruni, and A. Gandolfi, "A
Systems Approach to Immunology", Proc. of theIEEE, Vol. 68, No. 8, pp. 964-990, August, 1980.
Figure 5: Network for the Flow Monitoring Problem[3] J. D. Farmer, N. H. Packard, and A. S. Perelson,"The Immune Systems, Adaptation, and Machine
Learning" Physica, 22D, 187, 1986.

a45,13(t) == 1, as,124(t) = Ra(t), •aa,24(t) = R1a(t)[4] H. Atlan and I. R. Cohen, Eds, Theories of Im­
mune Networks, Springer-Verlag, 1989.

as,1a(t) = I - R1a(t}, a4s,s(t) = 1 - R13(t)5[5] Y. Ishida, " Fully Distributed Diagnosis by PDP
Learning Algorithm: Towards Immune Network
PDP Model",Proc. of IJCNN 90,San Diego, 1990.

Conclusion

The distributed and autonomous sensor network
we proposed is the first step towards self-organizing,learning, and evolving network found in immune net­
works. The advantage of this network is that it can
synthesize the system level interpretation by cooper­
ation of sensors, rather than sensing each value inde-[6] Y. Ishida and N. Adachi, "An Immunological Al­
gorithm and Its Application to Disturbance Rejec­tion," NAIST-IS-95030, 1995.
[7] Y. Ishida and F. Myzessyn, "An Immune Network
Approach for Sensor-Based Diagnosis with Sensor
Faults," NAIST-IS-95027,1995.

- 217 -

Artificial Active Vision Using Spatial Coding by M-Sequence
Sadao Fujimura and Senya Kiyasu
Department of Mathematical Engineering and Information Physics,
Graduate School of Engineering, University of Tokyo,
7-3-1 Hongo, Bunkyo-ku,
Tokyo 113, JAPAN
Abstract
Methods for measuring the 3-D shape of objects us­
ing spatial coding by M-sequence are described. Usinga micro property of an M-sequence, the pattern of apart of M-sequence observed is placed on the originalM-sequence. This enables us to correspond a point in
an image of an object to a point in the other imageof the object. The correspondence is used to measure
the 3-D shape of objects with diffuse surfaces as wellas those with specular surfaces.
For diffuse objects, a grid pattern coded by an M­
sequence is projected onto an object and observed bya TV camera. The position of each point on the object
is determined by triangulation.
For specular objects, the plane light source is codedby an M-array which is a two-dimensional array madeby an M-sequence. The array pattern reflected by aspecular object is observed by a TV camera. The posi­tion of each point on the object is determined by using
geometry of reflection.
Key Words: computer vision, triangulation,
M-array, diffuse objects, specular objects

1

Introduction

methods for measuring the 3-D shape of objects us­
ing spatial coding by M-sequence (maximum-length
sequence), one objects with diffuse surfaces and theother for those with specular surfaces.

2

Spatial Coding by M-Sequence

An M-sequence [3) ai of degree n, generated by arule is a binary sequence of length N = 2 n -1 specified
by a primitive polynomial.
Although M-sequence is usually used as a random
sequence (macro property), we use a micro propertyof M-sequence "window property": that is, if a win­
dow of size n is slid along the sequence, each of the
2 n -1 nonzero binary n-tuples is seen through the win­
dow once and only once when the end of the sequenceis connected to the other end of the sequence. We
project a grid pattern onto a diffuse surface object.
The pattern is encoded by black and gray stripes ac­
cording to the M-sequence.
The width of an observed stripe changes according
to the orientation of surfaces, on which the stripe is
projected. Therefore the number of stripes is not de­termined if the same level of stripes are neighboring.
We put a white stripe "return-to-zero code" between
every black and/or gray stripe as shown in Fig. 1.

Recognition of objects using v1s1on is one of themost important task for intelligent robots. Binocularstereo is a typical technique of passive measurement.
However, there are many difficulties in determining
correspondence between two images and it requires a
lot of time for processing. Slit light or spot light pro­jection methods [1) are often used for active measure­ment. However, they require special equipments andtake rather long time for scanning.
Spatial coding techniques [2], which also belong to
the active methods, do not require scanning. Mostof them, however, require to process many images.
This also takes a lot of time. Here we present two- 218 -nnnn
r
1 uuuuu--u
1

,.

o.s
0

7

1

nn_n

1

1

1

1

0

0

1

leroCode
M-Seque.-

Fig. I. M-sequence coded grid patternThus three level coding is a key in this method. Each
stripe is recognized correctly by quantizing the imageof TV camera into three levels.
The M�array is a two-dimensional artificial random
pattern constructed from an M-sequence. We take anumber N=2 k 1 k 2 -l such that N1 (= 2k 1 -l) and N2(=N/ N1) are prime to each other and greater than 1.
Then the M-array b(j, k) of size N1 xN2 is obtained by(i = 1, · · ·, N) (1)where ai is an M-sequence. This means that the M­
sequence fills an N1 x N2 array by writing ai down themain diagonal and continuing from the opposite sidewhenever an edge is reached.
The M-array has also the "window property": thatis, if a prescribed size (k1 x k2) of window is slid over
the array, each of the 2 k 1 k 2 -l possible nonzero arraysis seen through the window once and only once whenthe end of the array is considered to be connected tothe other side of the array.
We encode the plane light source by circles on thewhite background assigned by gray and black accord­
ing to the M-array. This is also three level coding.
Light source of size (N1+k1 -1) x (N2+k2 -1) is con­structed from the M-array of size N1 x N2 . In the
18x18 binary pattern shown in Fig. 2, every 4x2 bi­nary pattern appears just once.
----17 +1---•o••••00o00•••• 0 ••-2--••ooe0000000eo0••� -·----.·••0•0••0••0•0�!
•••• 0 0 0 • 0 • 0 0 0 .,.-.,. •,oe00••0•0•0••�0•00

10 •e0000eoeoeo•o�oo�•
O O
0••0•••oo o •••0!ee!0o
e O 0eeeee O ... ee .�.....-••••o•ooeoooeooeo••• --------·--·
15+30eoeo•••o•••o•o•00
I Ioo•••ooeoeooeeeooo
eoeooeeoooeeooeo••0•••0•00000•0•••00
1 1v::::::::::::::::::::::::::::::::::::•••o•o••o••o•o••••3

Measurement of the 3-D Shape ofDiffuse Objects
Principle of Measurement

3.1

For diffuse objects, a grid pattern coded by an M­
sequence is projected and observed by a TV camera as
shown in Fig. 3. The space is coded by an M-sequence,and divided into stripe areas, where (Xe, Ye, Ze) isthe center of the camera lens, (Xp, Zp) is the center ofthe projector lens, le and lp are the distance of image
and projector plane from each lens, respectively. The
angles of each lens axis are denoted by 00 and 4>o.

z

Projector

CameraFig. 3. System configuration for grid
pattern projection

t

4

l

When the position of a point P(xp) on the projec­
tor's plane and that of a point Q(xe , Ye) on the image
plane corresponding to the object point Oare known,
the coordinates of O(X, Y, Z) are given by,

0 1
0 0
1 1

X

= X e - Z' tan(O +0o),

Y

=

Z

Fig. 2. An M-array coded light source
and the window property=

Ye - Z' tan{,(2)

Ze - Z',

where

Z'
This property is used to match the pattern of apart of M-sequence (M-array) with a part of the same
sequence (array). Thus, each point on the pattern is
uniquely positioned by using the regularity of the M­
sequence or M-array. This enables us to correspond apoint on an image of an object to a point on the otherimage of the object. The correspondence is used tomeasure the 3-D shape of objects with diffuse surfaces
as well as those with specular surfaces.

=

Xp - Xe+ (Ze - Zp) tan(cf> + 4>o)tan(c/> + 4>o) - tan(O + Oo)(3)

tan0 = xc/le, tan{= Ye/le, tanc/> = xp/lp,
and y axis of projector's plan is parallel to the Y axis.
To calculate the coordinates of 0, we have to know
the correspondence between P and Q. They are deter­
mined by the property of M-sequence. First, the im­age is normalized by using the image of uniform lightsource. Next, the image is quantized into three lev­
els. Then a part of the sequence observed is matched- 219 -

SP-9cular Surface

with a part of the M-sequence. By using the result ofmatching, the correspondence of the points P and Q
are determined and the 3-D shape of the object is re­constructed by calculating the coordinates of the point

Image

0.

3.2

n

Xo
NormalVector no
ident rayExperimental Results

Three objects (box, cy linder, bowl) are used for ex­
periment. The distance from the object to a camera
was about 100cm, and the distance from the camera
to the projector was about 30cm. The angle between
the axis of the camera and the axis of the projector
is about 17 degrees. The effective size of the image
acquired by the CCD camera was 127 x 256 pixels.
The result of reconstruction is shown in Fig. 4. Sur­face of three objects were reconstructed correctly. The
accuracy of measurement was estimated to about 1cm
for the objects set about 100cm apart from the cam­era.

Fig. 5. Geometry of observation
We have determined the direction of the reflected
ray and the corresponding point of the light source.
However, they are not enough to determine the three­dimensional shape of the object,. We assume that theobject is a polyhedron so that the local surface of theobject is a plane.
Figure 5 illustrates the geometry of vectors we de­fine, where o is the center of the camera, x0 the re­flecting point on the specular surface, r 0 and s0 thecorresponding point on the image plane and the lightsource plane, respectively, and i 0 a unit vector of theincident ray. The vector r 0 indicates the direction of
the reflected ray. As the normal vector n 0 at point x 0
lies on the plane spanned by r0 and s0, an equation(roxso,no) = 0

holds, where ( , ) and x mean inner and outer productsof vectors, respectively. If we take another point x 1 on
the surface close to x0 , we obtain another equation

Fig. 4. Result of 3-D shape reconstruction ofdiffuse objects (box, cy linder, bowl)

4
4.1

(4)

Measurement of the 3-D Shape ofSpecular Polyhedrons
Principle of Measurement

For specular object, an image of a light source re­flected on a specular surface is acquired by a TV cam­era.
Specular objects in an acquired image reflect thepatterns on the light source plane. Using the window
property, we can determine the. position on the lightsource plane where the detected pattern comes from.
First, the image of coded light source is quantized
into three levels after the brightness is normalized us­
ing the image of uniform light source. The black levelmeans '1', and the gray level '0', respectively. The
white level shows the background.
The point on the light source plane is found where
the pattern matches the "window" pattern.

(5)
If the points Xo and x 1 lie on the same plane, thenormal vectors n0 and n 1 have the same direction.
Thus no is given by(roxso) x (r1xs1)
no = --:-:-----:-----,,-----,-,..
l(roxso) x (r1Xs1)I(6)

In practice we determine the normal vector which
minimizes the value of
821)rixsi,no)
i=O

(7)

to reduce the effect of measurement error, where Si (i =1,· · ·, 8) are the 8 nearest points around s0 on the lightsource plane and ri (i = 1,· · ·, 8) the corresponding
points on the image plane.
After the direction of n0 is determined, the posi­tion on the specular surface x0 can be determined by- 220 -

using the condition that the normal vector n0 bisect
the angle between the incident and the reflected rays.
The vector of incident ray i 0 is expressed by
.

10

=

2 (ro,no)no - ro
lro I
·

(8)

The point on the specular surface is written in two
ways:
(9)
Xo = tro
andx 0 = ui 0 + s0,(10)

Fig. 6. Result of 3-D shape reconstructionof a specular polyhedron

where t and u are real-valued parameters. From eqs.
(9) and (10), we obtain the equation
troxio = soxio.

(11)

Thus, t is written ast=(soxio,roXio)(r0 xi0 ,r0 xi0)

(12)

From eqs. (8), (9) and (12), we can determine point
x0 on the specular surface.
4.2

References

Experimental Results

A specular polyhedron composed of three specularplanes was used for the experiment. Each side of theobject is about 10cm long, and the distance from theobject to the camera was about 50cm, and that to thelight source plane about 15cm. Images were acquiredby a high-resolution CCD camera which provides adigitized image of 1024 x 1024 pixels.
The result of reconstruction is shown in Fig. 6. Cal­
culated points on a surface are connected by lines. The
dashed lines show the regression planes obtained fromthe calculated points. It is seen that each plane of thepolyhedron is reconstructed correctly.
In order to estimate the precision of this measure­ment, one specular plane was measured 6 times withvarying orientation. The mean residual is less than
1mm.

5

objects about 100cm apart from a camera, and 1mm
for specular objects about 50cm apart from a cam­era. Improvement of accuracy for the measurement ofdiffuse objects by using a camera with much higher
resolution, and extension of the algorithm for specular
objects to curved specular surfaces are the subjects fora future study.

[1] Y.Shirai, "Recognition of Polyhedrons with a Range
Finder", Pattern Recognition, Vol.4, pp.243-250, 1972.
[2] S.Yonezawa and Y.Tamamura, "Coded Grating
Method for Measuring Three Dimensional Objects",
Trans. [EGE Japan, Vol.J61-D, pp.411-418, 1978.
[3] F.J.MacWilliams and N.J.A.Sloane, "Pseudo-Random
Sequences and Arrays", Proc. IEEE, Vol.64, pp.17151729, 1976.
[4] X.Ye and S.Fujimura, "Measurement of 3-D Object byUsing M-Sequence Grid Projection", Trans. Soc. ln­
strum. Control Eng., Vol.25, pp.644-650, 1989.
[5] S.Kiyasu, H.Hoshino, K.Yano and S.Fujimura, "Mea­surement of the 3-D Shape of Specular Polyhedrons
Using an M-Array Coded Light Source", IEEE Trans.
Instrumentation and Measurement, Vol.44, pp. 775-778,
1995.

Conclusions

We have described two methods for measuring thethree-dimensional shape of diffuse objects and specu­
lar objects as well using spatial coding by M-sequence.
The shape of diffuse objects and specular polyhedron
were reconstructed correctly. The precision of mea­
surement was estimated to be about 1cm for diffuse- 221 -

Cooperative Behavior of Interacting Robots
Ken Sugawara and Masaki Sano
Research Institute of Electrical Communication
Tohoku University
Sendai,980-77 Japan
Abstract
This paper studies the relation between the numberof robots and the efficiency of group through the task
to gather pucks in the field. In this study, we assumeda robot which has a very simple structure to clarify
the behavior and the efficiency of group. The robot
had a driving system, a light and some sensors. They
could have interactions by using their lights and sen­sors. The effectiveness of group behavior was studied
under various {homogeneous, localized) puck distribu­
tions. To evaluate the efficiency of group behavior, we
examined the relation between the job time and thenumber of robots, and the relation between the inter­action period and the efficiency of group. We foundthat a simple interaction between robots improves effi­
ciency of group considerably compared with indepen­
dent individuals.

1

Introduction

Insects such as ants and bees establish flexible andwell-organized societies. It seems that each individual
does simple work, but their society is very complex
and has subtle functionality [1] [2] . This is because
they have an interaction with each other and functions
are self-organized in their society through cooperative
behavior. It suggests that we may be able to syn­thesize complex systems just like the ant societies bycoordinating interactions between elements even if thefunction of each element is simple. From this point ofview, some experimental researches have been madeby using a group of autonomous robots [3] [4] [5] . It
seems that the robot systems which were used in thesestudies were on a large scale. The purpose of our study
is to investigate the behavior and the efficiency of ac­tive elements which have the simple architecture when
they act as a group. So we studied the behavior of theelements with the simplest structure at first. In thisstudy, we assume a mobile robot as an active element.
The robot has a driving system, a light and some sen-sors and operates autonomously [6] . To evaluate theefficiency of these active elements, we chose a task tocollect pucks that were dispersed in a limited field.

2

Experiment

We made some robots with simple architecture andmade experiments to collect pucks.

2.1

Materials and Methods

The shape of robot that was used in this experi­
ment is circular and its diameter is 12cm. It is driven
by a pair of DC motors. This robot has two fixed arms
{Figure 1). There is a mechanical switch at the tip ofeach arm. When the switch is turned on, the motor on
the opposite side rotates reversely: If the left switch
touches something, the right motor rotates reversely.
As a result, robot can avoid colliding boundary walls
and other robots. There is a micro switch between
the arms, which detects the puck. The robot has apair of light sensors. These sensors are used to leadthe robot to the center of the field. Magnet switches
for detecting the center of the field are placed at thebottom. A light switch placed on the top is a powerswitch for main circuit of robot. We can initiate therobots maneuvering simultaneously by turning on/off
the room light. The battery for the robot is attachedby magnets. The battery can be changed easily with­out disturbing its direction.
The robot has two modes: search mode and returnmode. Search mode is to move straight until it meetsboundary walls or other robots. When the puck sensor
catches a puck, the robot is in return mode. Return
mode is to move toward the light at the center of the
field using their light sensors. When it arrives at thecenter, the robot goes back and turns and changes its
mode to search mode.
The field for this experiment was 190x190cm.
Boundary has a wall. Light and magnet are placed- 222 -

Touch Sensor

3

Touch Sensor
Puck Sensor

Simulation

To confirm the behavior of robots in various situa­
tions, we simulated their movements on computer. We
assumed three types of distributions of pucks: homo­
geneous, localized in 25% area of the field and localized
in 1% area of the field (Figure 3).

Light Sensor(/<::>

Figure 1: Schematic drawing of the robot. It has touch
sensors on its arms to avoid obstacles, a puck sensor
between arms and a magnet sensor at the bottom.

Puck
100
0

near the center for robots to find out the center ofthe field. We will call the center of the field "home."
Pucks were placed homogeneously in the field. Total
number of pucks was 32. The size of puck was 2.5cm
in diameter and 5cm in height. Each robot could goback to home by the light, so the experiments weremade in the dark room.

2.2

-100

. . . . .
. . . . . .
field 1

200

. • .

0

. . . .
. . . . . . . .
-100

0field 3

200
100

•

0

Result

100

Home

-�

.. .. .. .. .. .. .. ..
.. .. .. .. .. .. .. ..
.. .. ... ... ... ... ... ...
• ......

field 2

200

-100

100

..

200 -�00

100

200

-100

0

100

-100

Figure 2 shows the time evolution of collected
pucks. Each plots are the average of three trials. From
this figure, we know that the completion times of thetask by one, two and three robots were 874 sec, 502
sec and 353 sec, respectively.

100

l
�
a.

••

O
2 robots
0 O
0 1 robal
0

l

80
60
40

20
0

0

200

400

600

time(sec)

800

1000

1200

Figure 2: Time evolution of collected pucks for exper­
iments involving one to three robots.

- 223

-�

-100

0

Figure 3: The fields which were used in computer sim­
ulation. field 1: homogeneous, field 2: localized in
25% area of the field, field 3: localized in 1% area ofthe field.
We also assumed simple interaction between eachrobot. In this study, interaction is expressed as follows.
The robot which meets a puck stays there and turns
on its light for a fixed period. We call the period as"interaction period." Other robots which have no puckreact to this light and change their direction to thelight. In this case, we can control the strength of theinteraction by changing its interaction period.
Figure 4 shows the relation between the number ofrobots and completion time of tasks. In this figure,time means calculated step and 1 step corresponds to
0.1 sec. Each data point represents the average of 20
trials. In this study we defined the completion timesuch that 90% of all pucks were collected to home.
Figure 4(a) shows the results for robots without inter­action and Figure 4(b) shows the case with interaction.

200

10000·-..

··-·-······· ·· ····-···-

0 rTTm'l'lll'""T'Tlmnll'-r,nmor-...,.,.,,..,..""T"Tnm.--.-rn,m11

(a)

··•..

·· .. .
--. ..·. ........···-..•::·--••••····s.
fie1d·a·•·15_,g···•
0 ·--.. _
•
· _ •
_

I-

...
..
·-...
.. . .
'·•...

+afield 2"'e·•· ·•field 1 t,,.,,.,_

1000

2

I No interaction I------,
3

4

5

6

Number of machines

0- .. ..

7

.....

-e

ca.
'E

....'·•·•'1>.-a._a>o field 1
e field2
a field3

-0.5

a

0

0

0

0

• ••

---8"---------------•--------a••-1

w -1.5

···

0

a

a

a

a

a

-2

8 9 10

0.1

N

10000

10

100

1000

Interaction Period

10000

Figure 5: The relation between interaction period andexponent of the power law. In this figure, exponent-1 means that the completion time and the numberof machines are inversely proportional. The case thatthe exponent is less than -1 implies the group is more
effective for the task.

I-

+a

1000

2

3

4 ·

5

6

7

8 9 10

Number of machines

N

Figure 4: Relation between the number of machines
and the completion time. (a):The results for robots
without interaction. (b):The results for the case withinteraction.

Note that the completion time T and the numberof robots N have power law relation in Figure 4.

T~ N/3
The relation between interaction period and exponent/3 is shown in Figure 5. /3 -l(dotted .line in figure 5)means that the completion time and number of robots
are inversely proportional which is expected if eachrobot work independently. If the exponent /3 is lessthan -1, it implies that the group is more effective forthe task.

4

Discussion and Conclusion

We showed the relation between the number ofrobots and efficiency of group through the task togather pucks in the field. If there is no interaction be­
tween each robot, their working ability is in proportion
to a number of robots. But if there are interactions,- 224

their efficiency as a group depends on the situationof field. In case of the field which is extremely lo­
calized, robots can improve their working ability by
because of their interactions. We evaluated the effi­
ciency of group quantitatively. Power law exponents
for job time as a function of number of robots changes
from -1 to -1.8 when the optimal interaction period ischosen. It implies that a simple interaction can im­prove the efficiency of the whole society drastically as
the number of elements is increased.
In this paper, we mainly studied relation between
the number of robots and efficiency of group for thetask to gather pucks in the field. The interaction pe­
riod and the efficiency of group is shown in figure 5. If
there is no interaction, each robot collect pucks soli­
tarily. As a result, the group is slightly effective infieldl (homogeneous) compared with in field3 (local­ized). And its completion time is inversely propor­
tional to the number of robots. In other words, ( com­pletion time)*(number of robots) = constant. On theother hand, if there is an interaction, the efficiency ofgroup depends on both the situation ·of field and inter­action time. In case of homogeneous field, the longer
the interaction period is, the worse the efficiency ofgroup is. When pucks are localized, interaction period
is more effective. However too long interaction period
lowers the effectiveness. It is because the interaction
period prevents robots to carry pucks. Therefore there
exists optimal values for interaction period. Making
balance between interacting and independent period

is essential to synthesize efficient cooperative behav10r.

Acknowledgements
We would like to thank Y. Sawada, Y. Hayakawa
and T. Mizuguchi for valuable comments and discus­
sions.

References[l) R.A. Brooks, "A Robust Layered Control System
For A Mobile Robot", IEEE Journal of Robotics
and Automation, Vol. RA-2, No. 1, pp. 14-23, 1986.
[2) R.A. Brooks, "Intelligence Without Representa­tion", Artificial Intelligence, Vol. 47, pp. 139-160,
1991.
[3) R. Beckers, O.E. Holland and J.L. Deneubourg,"From Local Actions to Global Tasks: Stigmergy
and Collective Robotics", Artificial Life IV, editedby R.A. Brooks and P. Maes, pp. 181-189, The
MIT Press, 1994.
[4) M.J. Mataric, Ph.D Thesis, MIT, 1994.
[5) Z.D. Wang, E. Nakano and T. Matsukawa, "Co­operating Multiple Bhavior-Based Robots for Ob­
ject Manipulation", Proc. 1994 Int. Conj. on Dis­
tributed Autonomous Robotics Systems(DARS'94},
pp. 371-382, 1994.
[6) V. Braitenberg, Vehicles: Experiments in synthetic
psychology, Friedr, 1987.

- 225 -

Classification of Landsat TM Data Using Neural Network Approaches
Yoshiyuki Kawata
Environmental Information Research Lab.
Kanazawa Institute of Technology
Ogiga-oka 7-1, Nonoichi-machi
Ishikawa, 921 Japan

Yumi Yashima
Pasco Corporation
Akasaka 7-10-20
Minato-Ku, Tokyo,
03 Japan

Abstract

ground sites.These ground truth data were used for choosingclass samples as the training data, as well as for theverification data of the classification accuracy. We selected
32 ground control points within the study site image tocompute the geometric transformation coefficients by using
a least square method and the registration error was thusachieved to be less than 1 pixel. As for the resampling
process of the geometrically corrected image, the nearest
neibouring method was adopted.

Although statistical classification algorithms are the mostcommonly used in remote sensing, the neaural network
approaches to classify multispectral remote sensing images
has been investigated recently (Benediktsson et al, 1990).
In this study we have examined several types of input
signatures in a 3 layer neural network model, using Landsat
TM (Thematic Mapper) data with six spectral channels.
We found that the classification accuracy of LandsatTM
image by neural network using spectral data as input
signatures is as good as that by the statistical maximum
likelihood method. Furthermore, a significant improvement
by about 10 % in classification accuracy was obtained by
the neural network using coocurrence matrix components
as input signatures, when compared with the accuacy by
the maximum likelihood classifier.

1 Introduction
Six out ofTM's seven spectral channels are used as the
input signatures, because the ground resolution in channel
6 is different from those in other six channels. The total
number of land-cover classes was assumed to be thirteen,
and they are listed inTable I. The study site is Kanaz.awa
area in Japan, where is characterized by a blend of aqueous,urban, rural, and ecological land-cover features associatedwith rugged terrains. Ground cover conditions in this area
were identified by a joint work of the Geographical Survey
Institute of Japan and the Ishikawa Research Laboratory
for Public Health and Environment, with the aid of the land­
use map, aerial photographs, and actual examination of the

2 Neural Network Classification
In this study, we assumed a three layer neural networkmodel, consisting of 6 , 15 and 13 elements in the input
layer, the hidden layer and output layer, respectively. Sixelements in the input layer can accept the corresponding
TM's 6 spectral channel signatures. Thirteen elements inthe out put layer correspond to 13 land-cover classes.The
Sigmoid function was chosen for a semilinear activation
function.The appropriate slope value of Sigmoid function
was assumed to be 0.8 and the appropriate number of
elements in the hidden layer was 15. These values were
found from a view point of the classification accuracy andcomputational efficiency by the analysis of learning processexperiments using the training data. Nine different types oftraining signatures for the input layer were considered here:
1) Gray level signature of a randomly sampled pixel , 2)
Modes , 3) Mean(µ) , 4) Median , 5) a set of Mode,µ , andMedian, 6) a set ofµ,µ-o/2, and µ+o/2, 7) a set ofµ, µ-o/3, and µ+o/3, and 8) a set ofµ,µ-2o/3, and µ+2o/3 . We
denote a standard deviation as a. We made classification
experiments for such 9 input signature types using the above- 226 -

neural network model. The average classification accuracies
for nine input types are listed in Table II, together with theaccuracy by the conventional maximum likelihood method.
We found that the best classification accuracy is obtained
when the set of µ, µ-2o/3, and µ+2o/3 is used as input
signatures. The accuracy in this case is about 53 %, which
is better by about 5 % than in the case of the maximum
likelihood method. The calssification maps based on the
maximum likelihood classifier and the neural network using
the input set of µ, µ-2o/3, and µ+2o/3 are shown in Fig. 1
and 2.

3 Inclusion of Spatial Information
We also considered to use the spatial information on theimage for the input signatures in the neural network model.
The spatial gray level distribution of the image can be
represented by computing components of a normalized co­occurence matrix C with a size of n x n where n is a numberof gray levels. Each component of this matrix corresponds
to a probability , P 0 (i,j), that points with gray level i occur
relative to points with gray level j, with a displacement of{I= (r, 0 ), and ( i, j = 0, 1, .., n-1). The schematic diagram ofthis displacement is shown in Fig.3. In this study a moving
mask of 5x5 pixels size was used to compute P0 (i,j). It isalso necessary to requantized original gray levels into a few
levels in order to keep the size of C manageable. The
experiments of the neural net classification into 5 major
land cover classes, namely, aqueous, urban, rice field, forest
and others were performed for the gray levels in the rangeof 16, 32, 64, and 128. We found that the best classification
accuracy is obtained in the case of 32 gray levels. Assuming
a simple three layer neural network shown in Fig.4, theclassification accuracy was found to be about 78.3 % byusing co-occurence matrix elements as the input signatures,much better than the results (about 68.4 %) by either theneural network using the input set of µ, µ-2o/3, and µ+2o/
3, or the maximum likelihood method (see Table III). We
should also note that Kushardono et al. (1994) obtained aneven better classification accuracy of 84 % in the neuralnetwork using a similar co-occurence matrix approach fora different study site. The resulting classification map basedon the co-occurence matrix elements is shown in Fig.5. The
corresponding classification map by the maximum
likelihood classifier is shown in Fig.6. Although many of
misclassified pixels are seen in the forest areas in Fig.6,
they are correctly classified in Fig.5. We notice that manyfine structures in Fig.6 are lost in Fig.5 due to masking
operations in computing co-occurence matrices.

4 Conclusions
This study came to the conclusions as follows:
1) We found that the classification accuracy of Landsat TM
image by neural network using spectral data as input
signatures is as good as that by the statistical maximum
likelihood method.
2) Furthermore, a significant improvement by about 10 %in classification accuracy was obtained by the neuralnetwork using coocurrence matrix components as input
signatures, when compared with the accuacy by the
maximum likelihood classifier. This is because the spatial
information can be taken into accout by introducing theco-occurence matrix in the neural network.

Reference:
1. J.A. Benediktsson, O.H. Swain, and O.K.Ersoy,"Neural Network Approarches Versus Statistical
Methods in Classification of Multisource Remote
Sensing Data", IEEE Trans. on GRS, vol.28, no.4,
pp.540-551, 1990.
2. D. Kushardono, K. Fukue, H. Shimoda, and T. Sakata,"Characteristics of Neural Network in the Spatial
Classification based on Co-occurence Matrix", Proc. ofJapan Society of Photogrammetry Annual Meeting,pp.115-118, 1994.

- 227 -

Table I. Category Classes
1. Highly <lensed

8.

Meadow area

urban area
2. Urban area
3. Residential area
4. Highway
5. Sand area
6. Rice field

9. Deciduous forest
10. Mixed forest
11. Coniferous forest
12. River
13. Sea

7. Fields
Fig. 1. The classification map by the maximum
likelihood classifier in the case of 13

Table II. Average Classification Accuracy.

classes.

Input Signature
1) Random sampling

Average Accuracy(%)2) Mode3) µ4) Median5) Mode,µ, Median6) µ, µ-o/2, µ+o/27) 1-t, u-o/3 1-t+o/38) µ, µ-2o/3, µ+2o/3
Max. Likelihood

48.8
51.4
51.6
53.2
53.2
51.8
53.5
48.0

34.9

Table III. Average Classification Accuracy
in 5 classes (%)
co-occurence matrix
Max. Likelihood

Fig. 2. The same as in Fig.1, except it is made by
the neural neywork with a set ofµ, µ-2o/3, µ+2o/3.

78.3
68.4

(m2 ,n.J

Density j

Fig. 3. The schematic diagram of a displacement ,b=(r,0).

( m1, n1)

Density i- 228 -

■�■�Co-occurence
Matrixchl
ch2

NeuralNetworlc

ch7. �
Fig.4. The neural network model using co-occurence
matrix.

Fig. 5. The classification map by the neuralnetwork using co-occurence matrix in the caseof 5 classes.

. , excep
Fig. 6. The same as inon the maximum likelihood classifier.

�..• ·. ·- 229

A Control Design of Robotics Using the Genetic Algorithm

H. TakataR. Oguro
N. Umeda
T. Tsuji
T. Hachino
Dept.
ofElectri.
and Electro. Eng.
Research
Laboratory
Dept. of Electrical Eng.
Kagoshima University
Yaskawa Electric Co.
Kyushu Institute of Tech.
1-21-40 Korimoto
2346
Fujita,Yahatanishi-ku
1-1 Sensui-cho, Tobata-ku
Kagoshima, 890 Japan
Kitakyushu, 806 Japan
Kitakyushu, 804 Japan
Abstract

This paper presents a new and practical method fora control design of a robotic system. In general, ac­tuators in robotic systems are set with gears whosecharacteristics are elastic. Since a state feedback type
digital controller is usually used for such a robotic sys­tem, the design of the feedback gain of the controller
is important, because undesirable vibration or over­shoot in responses occur for high gains. So that de­sirable response, an output of a reference model, isdesigned initially and the feedback gains are deter­
mined so as for the response to coincide this desirable
response which is an optimization problem. The gra­dient method works to some extent, but it takes longtime to get a satisfactory result. Thus we apply thegenetic algorithm(GA) for this nonlinear optimization
problem, and as a result, the GA gives very first con­vergence. The obtained gains are useful for applica­
tions. Results of simulation are given.

1

requested from the field. Minimizing this performanceindex by the feedback coefficient is an optimization
problem. In this problem, the gradient method did
not work well, while the GA is found to show better
convergence. Some simulation results are shown togive the appropriateness of our propooal.

2

Problem Formulation

Consider a two link robot with reduction gears
which are elastic joints, then we havex = Ax+Bu
y=Cx
where the contents of A , B and C are given in Ap­
pendix, andX=[J 0 dt 0m.z

J0Introductionm.z

m.u dtA control of a robot arm with a flexible joint is in­vestigated in some papers for its undesirable vibrationof the arm [l]. Consider a two link robot with reduc­tion gears, a two mass system with an elastic joint.
Then for the application purpose, we have the 10 di­mensional system and obtain a nonlinear optimization
problem for optimal gain determination. Since now
we assume that an observer can detect all states re­quired, a state feedback controller can surely prevent
this undesirable vibration. However, although there
are many methods to determine feedback gains suchas an optimal regulator or a pole placement method, it
is another problem to determine a desirable response.
That is, if we have so many gains that we can notpredict the response for each combination of valuesof gains. In this paper, the performance index intro­
duced in the next section is seemed to be unusual and

0m1 0 az 0 az
0m.u 0m.u 0au 0au

r

where 0m.z and 0mu are the angular pooitions of the
motor of the lower and upper links respectively. 0 az
and 0au are the torsional angles in the gears at thelower and upper motors. The outputs 011 and 0z u are
the angles of the lower and upper links respectively
and satisfy the following equations:where 0m is the output of the reference model com­monly for 0m.z and 0u given by the follows:- 230 -

(3)

where Omref is the input for the model.
From the field, the following performance index is con­siderecl.

1001 ···1 0100 ···0 ······ 0100 ···0---...---...---...-

J = h1 II Om - 011 II +h2 II Om - 01u II

L a (tk)n

II a(t) 11=

2

k=l

Ki =

for suitable sampling points tk and the interval countn, and h 1, h2, h3, and h4 is weighting factors.
The problem is to determine the feeclback gains,
Kv 1, Kp 1, K;1, K,,1, K,,rn, Kvu , Kpu , K;u, K,,u , andK11vu which optimize J, where the control is given by

Here let the fitness values
F; = 1/J; (i= 1,2,, .. Af).

0
0
K,,1 K,,v 1
0
0
Kv u Kpu Ki u Kv u (Kpu + K;u)V =[step 4: Reprodu ction
The reproduction is implemented as linear searchthrough roulette wheel slots weightecl in proportion
to the fitness value of the individual gene. Prao­tically, the linear fitness scaling mentionecl below isutilizecl in order to avoid the undesirable premature
convergence. Let the minimum, average and maxi­
mum values of F.(i = 1, 2, ···M) to be Fmi n , Favg
and Fmax respectively. Then the scalecl fitness values
Ff (i = 1, 2, ···, M) are calculatecl as follows:
1) In case of Fmi n + Fmax - 2Favg ?: 0

0

K11u

J OmlreJ dt ]OmlrefMoreover, we claim that K.1
0.5Kpu from the field.

3

(6)

~

Kv 1
0

0

Ki max -Ki mi n . JC·· .
• + K,min
2r, - 1

where [Kim in, K. ma x] is the preset range of K. and JC.
is the decimal value of the binary representation of K•.
step 3: Fitness value calculation
Since we have M sets of gains (K1 K8), we can com­pute x(t) by solving (1) substitutecl by (5), so that wehave M values of the performance indices denotecl by
J;(i = 1,2,··· ,M).

(5)

Kv u

Ka

The GA is executecl as folows:
step 1: Initial population
Generate an initial population of M genes ran­
domly.
step 2: Decoding
Decode the gene by:(4)

where in general II a(t) II means:

K'J

K1

+h3 II iJ,,, II / I iJ,,1 lmax +h4 II 0,,u II / I iJ,,u lmax

Ff=
,

Favg
(D
I'i + Fmax - 2Favg )F.max - F,avg(i=l,2,···M)

Optimization by the GA~

In what follows, we write K1 Ks instead of Kv1,
Kpl, K,,,, K,,rn, Kvu , Kpu , K,,u , and K 11Du respec­tively for convenience. There are several methods toexecute the GA. It is difficult to say which method isthe best. The uniform resolution may be one of theimportant factors. The probabilities of the crossoverand mutation are also thought to be the importantfactors. Here, we set the necessary values, functions
and probabilities, by trial and error and empirically.
Initially, a gene is constructecl by allocating a r; bits
string for each gain as follows:(7)2) In case of Fmi n + Fmax -2Fa vg < 0
Fil_
- F.

Favg
_ F, . (fl - Fmi n )

avg

min

(i=l,2,···M)(B)

The reproduction is carriecl out by using this scalecl
fitness values Ff(i = 1, 2, • • • M). Namely each indi­vidual gene is reproducecl with the probability of

-zn-

M

F'flLF; (i=l,2,···,M).
j=lstep 5: Crossover
Pick up two genes randomly and decide whether
or not to cross them over according to the crossoverprobability Pc - If the crossover is required, exchange
strings in the genes at a crossing position. The crOS&­
ing position is chosen randomly. Pc is usually chosen
greater than 50%.
step 6: Mutation
Alter each bit (0 or 1) of strings in the genes accord­
ing to the mutation probability Pm . Pm is generally
designed to be quite low, for example less than few
percent.
step 7: Repetition
step 2step 6 are repeated from generation to
generation so that the fitness value of the popula­tion increases until the termination citerion is satis­
fied. The elitest preserving strategy which guarantees
that the individual gene having the best fitness value
survives in the nest generation is introduced in thisalgorithm.
Finally at the termination of this algorithm the con­trol parameters are determined by the gene with thebest fitness value over all the past generations.

~best fitness value for each generation. From this figure,it seems that there are some local extrema, although
it is difficult to confirm this.

5

Conclusions

In this paper a control design method of two linkrobot manipulator using the GA has been presented.
The GA shows at least quicker convergence to a better
solution than the gradient method does (about 1/16).
Table 1. The search ranges and Ti of the control
parameterslink control parameter search range Ti [bit]

l

u

4

Illustrative example

=✓

1600,
For the reference model (3), we put Wn
and weighting factors in the performance index (4) arechosen to be h1 h2 1,h3 h4 0.05.
The design parameters of the GA are given as fillows:(1) population size M = 30

=

=

=

=

= 120

(2) bit length of each gene l(3) crossover probability Pc=(4) mutation probability Pm

I

I

0.8

= 0.025
u

I

- 232

25
15
10
10
25
15

K,,1
Kp1
Kil
K.1
K.m

1.8182 X la2
1.8768 X 10 1
9.38407.0110 X 105-9.9323 X la3

Kvu
Kpu
Kiu

1.3812 x la2
1.7595 X 10 1
8.79759.5370 X la5-5.4052 X Ia2

K.u
KaDu

The detail about the other parameters are shown in
Table l. The results are shown in Table 2.
Fig. I shows the step responses of the lower arm po­sition of the reference model(dotted line) and the con­trol model(solid line) for the best gains in the initial
generation. Fig.2 shows the same responses but afterconvergence of the GA. Fig.3 shows the change of the

10
10
-

Table 2. Determination result of the control pa­
rameters
link control parameter determination resultl

The algorithm is terminated when e becomes less than
0.04 in our case where[0,300][0,100]0.5Kp1[la3,1CJ6]±[la2,104 ][0,300](0, 100]
0.5Kpu[la3, 1CJ6]±[la2,104 ]

K,,1
Kp1
Kil
K.1
K.n1
Kvu
Kpu
Kiu
K.u
K.nu

I

References(1] M. Matsuda, T. Murakami, K. Ohnishi, "Vibration
Suppression and Disturbance Rejection Control
of a Flexible Link Arm" ,IECON'95, pp1260-1265,(1995).

[2] J.E.Lansberry, L.Wozniak, and D.E.Goldberg,"Optimal Hydrogenerator Governor Tuning witha Genetic Algorithm", Tiuns. on Energy Conver­sion, Vol.7, No.4, pp623-630 (1992)

arm position

1.5

L

1

[3] B.Porter and A.H. Jones, "Genetic Tuning of Dig­ital PID Controllers", Eledronics Letters, Vol.28,
No.9, pp843-844 (1992)

,

0.5

,

0

Appendix
-0.5
0
0
0
0
0
0
0
0
0
0

A=

0
0
0
0
0
0
0
0
0
0

0
0
0
0
0
1.0
0
0
0
0

B=

1.0
0
0
0
0
0
0
0
0
0

0
1.0
-1.5
0
-9.8039 · 10-3
0
0
0
0
0

0
0
-1.7037 · 10 2
1.0
-4.931800001.7605 · 10- 15

1
0

0
0

-N 0
0

0 0
1 0
(N = 160)

0

0
0

1,5

arm position L_______________

--,

__:_

1

0.5
0

-0.5 aL----------------

0
0
0
0
0
0
0
1.0
0
6.5359 · 10-3--.,.0,-,. stime (secs)

Fig.2 Step response as the same as Fig.I but afterconvergence of the GA.
2

I

0

-N

�

]

-�-

-

�

(I)
·-

G=[�

0.5

0

time (secs)
Fig.I Step reponses of the lower arm position of
the reference model(dotted line) and the controlledmodel(solid line) for gains in the best gene of theintial generation.

0000
1.4286 · 10- 15
00-1.3824 -10::Z
1.0-5.609100001.8673 · 10- 13
00-1.8070 - loll
0-7.3318 -1D3

0
0
0
0
0
0
1.0
-1.5
0
-9.8039 · 10-3
0
0
1.0
0
6.5359· 10-3
0
0
0
0
0

0
0
-1.8070 · 1011
0-5.2308 · 103
00001.8673 · 10- 1 3

·�-

-

-�
rn

[J

-- (I)

_CA
.µ
rn

_ (I)

-i

.,..-

r

--

u---

Fig.3 Maximum fitness values
for each generation.

- generation
I
I
I
5 0

- 233

I O 0

Development of Practical Control Method
For an Intelligent Mobile Vehicle
M. Sugisaka 1 , X. Wang 1 and J. J. Lee2
10ita University, Oita 870-11 Japan
2

Korea Advanced Institute of Science and Technology,
Taejon 305-701 Korea

Abstract

In this paper recent research into operating an
intelligent mobile vehicle(MV), developed in Prof.
Sugisaka's laboratory at Oita University .Japan, ispresented. A complex control procedure, having a
two-input and two-output fuzzy controller as ker­nel, is used. The inputs to the fuzzy controller
are provided by a CCD camera which transforms
information on spacial objects by using an im­age processor. The MV is able to track specified
guideline types, search for object in space and rec­ognize a stimulant traffic signal. Results of labo­
ratory testing are presented.
1. Introduction

The results of latest research on a mobile vehi­cle (MV), eveloped in Prof. Sugisaka's researchlaborotoary [l], are presented. Although a num­
ber of control strategies may be applied to oper­
ate a MV, achieving good control effect remains
a difficult problem.
In this work a fuzzy controller, consisting of two
inputs and two outputs, is the kernel of the MV
control system responsible for steering and driv­
ing the vehicle. The fuzzy logic inference rules areof the IF�THEN form with twenty-eight rules
used in the experimental studies. The inputs
to the fuzzy controller are provided by a visual
controller, which obtains data from a CCD video
camera. The outputs are used to operate the driv­
ing speed and steering angle of the MV. The visual
controller is able to recognize objects and this acts

- 234

as the eye of the MV. If necessary, another con­troller, called the search controller, may be acti­
vated thus enabling the CCD video camera, which
can rotate freely both horizontally and vertically,
to search for an object or identify a traffic signal.
Many factors are considered in the overall de­sign, including the sensitivity and area of searchof the visual sensor, the harmonious relationshipbetween driving and steering, the nonlinearities ofthe performing unit, the friction and inertia.
2. Structure of Mobile Vehicle

The MV has four-coupled structure with dimen­
sions of l.5mx0.7mx0.5m and weights approx­
imately 150kg. I ts two rear wheels can rotate
frontwards and backwards about the axle shaft
and are driven by a 24V DC motor using cross he­lical gears. Besides free forwards and backwards
rotations without power, the MV's front wheels
are driven by a 24V DC stepping motor for mo­tion to the left or right, with transmission being
via gear wheels. Two 6V DC motors are employed
to rotate the the CCD camera in the horizontal
and vertical directions. The location of the front
wheels and camera are detected independently by
three encoders.
3. Control System

The MV control system is shown in Figure 1 andcomprises a fuzzy controller, a visual controller
and a searching controller.

3.3 Searching controller

3.1 Fuzzy controller
A good introduction to fuzzy control is given inreferences [2] and [3]. The five fuzzy sets NB, NM,
ZE, PM and PB are used to span the universes of
discourse of each of the two inputs, Edx={edx},
Edy={edy}, and the four fuzzy sets S, M, MB,
B are used to span the universes of discourse ofeach of the two outputs, Del={del}, Spe={spe}.
The membership functions are shown in Figure
2. The linguistic rules are implemented as thetwenty-eight sets of fuzzy conditional statementsof the form:
RI: If edx is NB and edy is S
then del is NB and spe is S;
ELSE R2: If edx is NM and edy is B
then del is NB and spe is M;
ELSE Rl5: If edx is ZE and edy is M
then del is ZE and spe is B;
ELSE R28: If edx is PB and edy is S
then del is PB and spe is M.
where
P =positive,N =negative, ZE=zero,
B=big, M=medium, S=small;edx, edy is the center of gravity of theguideline type after image processing;del, is the steering angle andspe is the driving speed.

The searching controller is used to rotate the
CCD camera either horizontlly or vertically thus
expanding significantly its field of vision.
4. Experimential Results
The outputs of the fuzzy controller are shownin Figure 3. The results of laboratory tests areshown in Figure 4 and are divided into two parts:
Fistly, the MV runs along the guideline type AMC(MD cancealed). At point B, the searching con­troller may be initiated to rotate the camera to
the left enabling the MV to run along BC. Sec­
ondly, the MV runs along the guideline type CMD(MA cancealed). At points S and P, searching
controller is likely to be initiated to rotate thecamera to the right thus enabling the MV to run
along MPD. Throughout each of these runs if the
MV identifies a red traffic signal it stops. If willthen wait for up to 28 seconds, checking the signal
every 7 seconds. If the signal disappears within
this time then the MV will continue its run, oth­erwise the complete system is stopped.
The maximum speed of the MV is about 2.0m/s.
5. Conclusions

It should be noted that the shape of the spacial
object will continuously change following image
processing [4]. The role of the fuzzy controller istherefore to identify these changes and make thecorrect decision on both the driving speed andangle of steering.
3.2. Visual controller
By means of the camera rotating upwards the
MV is capable of recognizing a traffic signal lo­
cated at a height of more than 1.5m. On seeing a
red signal the MV will stop and remain stationary
until the light disappears. The camera checks ev­
ery seven seconds and if there is no change within
28 seconds the total system will be stopped.

- 235

A complex controller incorporating a fuzzy con­troller as kernel has been successfully applied toenable a mobile vehicle to follow specfied guide­line type runs. Work is continuing on improving
the capability of the camera/image processing
stage in order to account for problems such as
luminosity changes and objects of similar colour.
Both neurocomputers and artificial life technolo­
gies [5] will also be applied to the MV control
system to make the mobile vehicle recognize andtrack moving objects and avoid obstacles using
advanced intelligence.
References[I] M. Sugisaka, et al., Running Test of a Mobile
Vehicle, Proceeding of First Asian Control Con­
ference, Vol.I, Tokyo, 439/442, 1994.

(2) Y.S. Li, J. Li, Theory and Application of Fuzzy
Control, Industry Press, B.J. 1992.
[3] M. Madan, K. Rommohan , R. Ronald , Ad­
vances in Fuzzy Set Theory and Applications,North-Holland Publishing Company, 1990.

[4] Y. Anzai, Pattern Recognition and Machine
Learning, Academic Press, 1992.
[5] D. Wasserman, Neural Computing Theory andPractice, Van Nostrand Reinold Press, 1989.

mobile vehiclevisual control

searching control

fuzzy control

steering control

velocity cotrol
signal waiting

position adjusted
runningI
ex i t.....__ _ _ _�◄�Fig. 1 The block diagram of IV control system- 236 -

NI

1 NB

µed:,:

NS

ZE

PS

PB

Pl

I

B

18

t

t

1 2 3 4 5 6 7

µ.pe

s

1

µedy

s

1

0

B

I

-5 -4 -3 - 2 -1
NB

1

NS

2

0

ZE

3

4

PS

5

PB

µdel

t

t
0

-3

-2

2

3

0

-4

4

Fig.2 The membership functions of edx, edy, spe, del

..-<
Q)
-0

edy

edy
edx

edx

Fig.3 The outputs of the fuzzy controller
D
p
I
B

C

s =-------�

Fig.4 The running ways of MV- 237 -

Fuzzy-Logic Approach to Industrial Control Problems

K J Burnham
Control Theory & Applications Centre
Coventry University
COVENTRY CVl 5FB

DJ G James
Control Theory & Applications Centre
Coventry University
COVENTRY CVl 5FB
Abstract
Increasing demands for improved profitability andproduct quality, together with a growing awareness ofthe effects of pollution on the environment, is forcing
manufacturers to closely examine their process opera­
tions. As a consequence there is currently significant
research and development activity aimed at improv­
ing control system strategies. In recent years, there
has been renewed interest in fuzzy rule-based control
strategies and this paper considers some potential in­
dustrial applications of such strategies.

1

Introduction

Manually tuned three-term proportional-integral­derivative (PID) control strategies are widely usedin practice and provide satisfactory performance formany applications. The disadvantage of such a con­troller is that it is required to be re-tuned/de-tuned foreach new operating point/range and schemes for over­coming this shortfall have included auto-tuning andlinear self-tuning strategies. However, when applied
to industrial systems exhibiting nonlinear character­istics such schemes often prove to be inadequate andfrequently underperform the basic PID controller. In
recognition of this research is undertaken, in the Con­
trol Theory and Applications Centre at Coventry Uni­
versity, into the design of nonlinear adaptive control
strategies. Particular emphasis being placed on bilin­
ear model based self-tuning control [6] [2] and fuzzy
rule-based cqntrol [5] [3] [8] strategies.
In the area of bilinear self-tuning control, where
the system model is able to capture and replicate in­put dependent plant dynamics and steady-state gain,
nonlinear adaptive controllers have been successfully
developed for high temperature heating plant appli­
cations [6]. The bilinear systems modelling approachdeveloped is also applicable to a wide range of engi­
neering, biomedical and socio-economic systems, seefor example (9] (1]. Whilst the bilinear STC approachhas distinct advantages when applied to systems forwhich local bilinearisation holds, the requirement fora priori knowledge of model structure is its main lim­
iting factor.
One of the main advantages of fuzzy rule-based ap­
proaches, over the nonlinear model based approaches,is the apparent immunity of the approach to lack ofa priori knowledge of model structure. Rather theequivalent model is embedded within the knowledge
base of the fuzzy controller. It therefore does not suf­
fer due to lack of precise structural knowledge of thesystem nonlinearities and as a consequence allows formore flexibility in its design. A good introduction to
fuzzy control is found in [1 O].

2Industrial applications

In this section two industrial applications are con­sidered, which reflect the role of fuzzy methodolo­gies within both control algorithms and supervi­
sory/decision making schemes. In the control case theapplication is that of speed control in engine testing
dynamometers and in the decision making case a fuzzy
methodology is used to optimize the start up of a com­bined cycle power generation plant.

2.1

Engine testing dynamometerAn engine testing dynamometer is a torque absorb­
ing device which is coupled directly to the drive shaftof the engine under test. In this particular case consid­
eration is given to the hydraulic dynamometer which,due to its wide range of applicability, is extensively
used in practice. Essentially such a dynamometer op­
erates on the flow of water through its working com­
partments with the output flow being the controlled
variable. It exhibits severe nonlinear behaviour in
its speed/torque characteristics which makes it not
amenable for effective implementation of model based
adaptive control methodologies. Current practice is to- 238 -

use conventional PID controllers and, since such con­trollers are tuned to provide a compromise across thefull operating range of the system, there is scope fordesigning improved controllers provided means can befound to accommodate the nonlinear characteristics
within the control algorithm. Recognising the limita­
tions of the model based methodologies the effective­
ness of fuzzy controllers has been investigated [4].

0.4

2P·2
0

�

8"0.2
-0.4

Figure 2: A linear PID control surface

+

Plant
0.4

�---+Fuzzy PD

'5

0.2

ea
c

3"0.2
-0.4

Figure 1: A fuzzy PID controller
Initially a controller, incorporating a fuzzy PD com­ponent and a conventional integrator term, which em­
ulates a well-tuned PID controller is developed; this isshown schematically in Figure 1. The product form of
the t-norm and the centre of gravity approach to de­fuzzification are adopted and triangular membershipfunctions chosen for the two linguistic variables error
e(t) and change of error l:.e(t). The resulting fuzzycontroller surface is shown in Figure 2. Whilst in sim­
ulation studies this replicates a conventional PID con­troller, in practice computational delays in the fuzzy
approach lead to a reduced performance. However, thefuzzy controller provides greater flexibility than theconventional PID controller in that the rule-base canbe adjusted to accommodate an element of schedulingof the gains of the proportional and derivative terms.
For example, recognising the need to increase the gains
at the extremities and to lower the gains in the wellcontrolled regions, the rule-base can be adjusted togive the control surface of Figure 3. When imple­
mented on the dynamometer this updated fuzzy con­troller is superior to the conventional PID controller
in some aspects, particularly in the steady-state mode,
but computational delays remain a problem in thetransient phase. For systems, such as the hydraulicdynamometer, which exhibit fast dynamics necessitat­
ing the need for rapid sampling rate, computational
aspects of fuzzy control can be a problem. Alterna­
tive fuzzification strategies are being investigated as apossible way of overcoming this problem.
An alternative to the fuzzy PID approach is toadopt a fuzzy model-based approach. In the case of the- 239

Figure 3: A non linear fuzzy control surface

hydraulic dynamometer behaviour is essentially firstorder within local operating regions, where it can bemodelled by equations of the formy(t) = -ay(t - 1) + bu(t - 1)
where y(t) and u(t) represent the speed and valve posi­tion respectively. Thus, the first step is to identify suchlinear models throughout the full operating range ofthe dynamometer; this initially being done off-line to
produce a nominal model surface. This nominal modelsurface is then updated using fuzzy logic, with u(t -1)
and y( t -1) as the antecedent linguistic variables andincorporating equally spaced triangular membershipfunctions coupled with the product form of the t-norin.
A typical resulting model surface is shown in Figure 4.
In order to accommodate the variable dead time asso­
ciated with the dynamometer the fuzzy PID controller,
previously developed, is combined within a Smith pre­dictor scheme as shown in Figure 5 which incorporates
the fuzzy model surface. Initial testing on a laboratory
based fluid level system, having similar characteris­tics to the dynamometer, have been very encouraging
and indicate potential improvement over conventionalmodel based self-tuning strategies when the estimateddelay is realistic. Work is continuing on improving thecontroller through on-line tracking of the dead time.

shorter period of time than in conventional plant. It isin recognition of such problems, coupled with the needfor a more sophisticated approach, that prompted theresearch into the design of a supervisory fuzzy logic
decision making controller (FSC).

Figure 4: A typical model surface
Plant : Delay
Model

Delay

Output+Figure 5: A Smith predictor control scheme

2.2

Combined
plant

cycle

power

generation

In order to improve thermal and economic efficiency
combined cycle power generation plant (CCP) makeuse of a combination of both gas and steam turbines.
Such plants operate by utilising the waste heat fromthe gas turbine to form steam, through heat exchange
in a heat recovery steam generator. The steam maythen be used in a steam turbine. Full details of CCP
theory may be found in [7). The increased efficiency of
CCP leads to important design and operation consid­
erations, in particular plant start up. At present start
up regimes are calculated prior to commissioning andhence may not be optimal. The start up regimes es­
sentially consist of the speed rates (ie. the ramp rates
at which to increase the speed of the rotor to its rated
speed), and load rates, (ie. the rates at which load
may be ramped onto the plant .until full rated output
is reached).
One of the most important considerations is thelevel of thermal stress experienced by those sectionsof the plant which are subjected to high temperature
variations, particularly during start up and shut down.
Whilst the thermal stress problem exists in all steam
plant special care needs to be taken with CCP. Thisis due to the difference in timescales with the start
up regimes of gas and steam turbines; the former may
well take less than fifteen minutes·to achieve full load,whilst the latter may well require two hours or morein the same situation. As a result, full steam con­ditions may be experienced by the steam turbine in a

As thermal stresses are unavoidable maximum lim­
its are imposed for each start up and shut down to en­sure that the plant design life is met. Thermal stressequations are nonlinear and the important variables
include steam temperature, steam pressure and/or
their rates of change, heat transfer coefficient between
steam and rotor surface and the dimensions of the ro­tor. ';I:'he aim of the supervisory controller is to com­plement the existing scheme by manipulating the loadrate set points so as to minimise start up times whilst
adhering to (ie. not violating) thermal stress con­straints. In order to achieve this the supervisory strat­
egy makes use of the combined heuristic features ofmodel based prediction. A supervisory load rate con­troller for a CCP has to incorporate several distinct
features. Whilst it is considered to be important tominimise start up time, it is imperative that the max­
imum thermal stress limits are respected otherwise se­vere detrimental effects on plant life expectancy may
result. Other requirements are that the load may only
be increased or held at the same level throughout theloading phase and the load rate may only be switchedbetween a set of discrete predefined rates. The present
start up regimes are open loop in nature, with respect
to thermal stress, and involve switching between thepredefined load rates at given outputs of the plant.
In the proposed scheme the inputs to the supervi­
sory controller are thermal stress and its rate of change
giving as output a 'condition value' [11) which triggers
a decision on the load rate. The condition value takes
a value on the interval [-1, 1) and will lie in one ofthree decision making regions, (i) [T1, 1], (ii) [T2, T1)
and (iii) [-1, T2]; where T1 and T2 are thresholds with
1 > T1 > 0 and 0 > T2 > -1. These decision intervals
correspond to regions where the supervisory load rate
controller will (i) increase the load rate to the next
in the sequence, (ii) keep the load rate the same and(iii) decrease the load rate to the next in the sequence.
Two other features that are included to reduce the ef­
fect of noise are (i) load rates are only altered if thecondition value lies in a decision region for m consec­
utive samples and (ii) the fuzzy logic strategy should
aim to achieve a maximum stress level j percent below
the maximum stress limit; m, j, T1 and T2 being user
defined parameters. A schematic representation of thecontroller operation is illustrated in Figure 6.

- 240 -

Due to the complexity of the problem the feasibil-

FSC

Load rate

StressStress rate(2] K J Burnham, K J Disdell, D J G James, C A
Smith, 'Developments in industrial applicationsof self-tuning', J Control Engineering Practice, 3,9, 1265-1276, 1995.

Load
Controller

StressEquation

Steam Temp

[3] J A Dunlop, Intelligent Fuzzy Controller for En­
gine Testing Systems, PhD Thesis, Coventry Uni­
versity, Coventry, 1995.

Shaft Temp

[4] J A Dunlop, K J Burnham, D J G James, P
J King, 'A self-tuning scaling method for fuzzycontrol', Proc 3rd IEEE Conf on Control Appli­
cations, 683-687, Strathclyde, Glasgow, 1994.

Figure 6: Supervisory control strategy
ity of adopting a fuzzy approach has been investigated.
Thermal stress and its rate of change are taken as theantecedent linguistic variables with values being de­termined from a thermal stress equation which relates
temperature to stress. The output or consequence de­termine the condition value which trigger decisions onload rate changes. Initial results are again encouraging
and indicate the potential for significant improvementin terms of reduced start up time (13]. Further de­velopments include adaptation of the threshold values
with load and use of radial basis function neural net­
works to classify the different start up load rates [12].

2.3[5] J A Dunlop, K J Burnham, D J G James, P J
King, 'Application of fuzzy logic based self-tuning
control, Proc IEEE/IAS Int Conf on Industrial
Automation and Control, 109-114, Hyderabad,
1995.
(6] S G Goodhart, K J Burnham, D J G James, 'Bi­
linear self-tuning control of a high temperature
heat treatment plant', IEE Proc D Control The­ory and Applications, 141, 1, 12-18, 1994.
[7] R H Hehlhofer, Combined-cycle Gas and Steam
Turbine Power Plants, Prentice-Hall, 1990.

Conclusions

The paper considers two applications where fuzzy
methodologies may be advantageous in comparison tomodel based strategies for control and decision making
as industrial plant. The results to date are encour­aging indicating the potential for implementation of
fuzzy methodologies in practice. In systems exhibiting
fast dynamics computational delay remains a problem
for further investigation. The work on fuzzy modellingis at an early stage of development and work is ongo­
ing to link the control and model surfaces coupled withalternative control strategies to the error driven PID.

Acknowledgements
The authors gratefully acknowledge the work of
their research students John Dunlop and Chris Smith
who carried out much of the work described in thispaper.

References[l] C Bruni, G Di-Pillo, G Koch, 'Bilinear systems:
an appealing class of 'nearly linear' systems intheory and applications', IEEE Trans Aut Con­trol, 19, 334-348, 1974.

[8] P J King, K J Burnham, D J G James, A com­bined rule-based and model-based adaptive con­trol scheme', Proc IEE Control '94 1484-1489,
IEE Publication 389, London, 1994.
[9] R R Mohler, Bilinear Control Process, Academic
Press, New York, 1973.
[10] W Pedryz, Fuzzy Control and Fuzzy Systems(2nd Edn), Research Studies Press, Tauton, UK,
1993.
[11] C A Smith, K J Burnham, P A L Ham, D J
G James, 'Accommodating thermal stress dur­ing the start up of a combined cycle plant', Proc
10th Int Conf on Systems Eng, Vol II, 1151-1158,
Coventry, 1994.
[12] C A Smith, K J Burnham, P A L Ham, D J GJames, 'Adaptive fuzzy control of combined cycle
plant', Proc IFAC Symp Control of Power Plants,
Cancun, Mexico, 1995.
[13] C A Smith, K J Burnham, D J G James, P A L
Ham, 'Fuzzy and model based supervisory con­troller for combined cycle plant start up using
thermal stress constraints', Proc 3rd European
Control Conf, Vol 3, 2444-2449, Rome, 1995.

- 241 -

Associative Readout from a Chaos Network
employing Chaotic ltinerancy
Takeshi yamakawa
Department of Control Engineering and Science
Kyushu Institute of Technolo gy
Iizuka, Fukuoka 820, Japan
Phone : +81-948-29-7712
Fax : +81-948-29-7742
E-mail : yamakawa@ces.kyu tech .ac.jp

An ordinary neural network can store

type nonlinear function. The combinatory

information in the form of distributed

coefficient (one of the nonlinear parameters)connective weights and thresholds,

andcan be determined to make the chaotic unit tofacilitates readout of the information in thecome up to fixed point attractor, periodic

form of training output data by applying

attractor and strange attractor (chaos). The

input data similar to but different from theconnective weights in the chaotic network

training

storing

are assigned by Hebbian learning rule. When

behavior in the training mode in the networka distorted pattern is applied to the chaotic

is dynamical, the readout behavior is static.

network, the stored pattern most closed to

Accordingly, there cannot be searching of

the distorted pattern is retrieved. When an

alternative candidates to be read out This isoverlapped pattern is applied,quite different from information retrieval in apatterns are retrieved sequentially.

input

data.

Althoughseparatedbiological brain. The brain retrieves events

When a mixed pattern or noisy pattern is

one by one, by chaining a train of thought orapplied, all the stored patterns are retrievedby analogy of different fields.

sequentially.

Thisinteresting

behavior

A group of biological neurons connectedcannot be obtained in the ordinary mutually

to each other often exhibits chaotic itinerancy.

connected networks, e.g. Hopfield network,

This is regarded as the primary behavior ofwhich is agitated by noisy signal .

chaining retrieval of stored information in theThis chaining retrieval of information isrecurrent neural network.

inherent to the chaotic network proposed

Interesting behavior of a chaotic network

here and is possibly a model of associativeis presented here, which is constructed withreadout of stored information in a humanchaotic

brain.

units

connected to

each

other

including self-feedback. The chaotic unit isconstructed with the delay block and thenonlinear block which a linear combinationof a sigmoidal function and an inverted-N- 242

Neurcomputer Control
for Tracking of Moving Objects
M. Sugisaka
Department of Electrical and Electronic Engineering
Oita University
Oita, 870-11, Japan
Abstract
We developed a new neuocomputer control technique
for tracking a moving object using a neurocomputer.
The control is produced by the RICOH neurocomputer
RN2000, which is able to learn various control laws in­stantly, in order to track the moving object within a cer­tain range of errors. This paper describes the new neuro­
computer control technique and presents the results ob­
tained from the tracking experiments.

1

Introduction

We have already developed the recognition and track­ing system of moving objects[l]-[4]. The system consistsof one CCD video camera, two DC motors in horizon­
tal and vertical axes with encoders, pulse width modula­tion(PWM) driving unit, 16 bit NEC PC-9801 microcom­
puter, and their interfaces. The recognition and tracking
system is able to recognize the pattern and size of a mov­ing object and is able to track the object within a certain
range of errors by learning control laws.
In the meanwhile the RICOH neurocomputer RN2000 has been recently devised for research anddevelopment[5],[6]. The RICOH neurocomputer RN-2000
consists of seven RN-200 digital neural network VLSI
chips, where sixteen neurons and totally 256 synapses areintegrated in a 13.73xl3.73 mm 2 VLSI chips, fabricatedby RICOH 0.8 µ CMOS technology. This chip can per­form 5.12 giga pulse operations per second. It corresponds
to effective neural computing rate of 40M CPS or CUPS.
Utilizing the neurocomputer RN-2000, we have developed
a new identification technique in order to model linear andnonlinear dynamical systems to be controlled[7],[8].
Utilizing the results obtained from the identification
experiments using the neurocomputer[7],[8], we devised a
new recognition and tracking system of moving objects[9]called a neurocomputer control system for recognition andtracking of moving objects. The new system consists of
the 32 bit microcomputer NEC PC-9801,BA, the neuro­
computer RN-2000, and the other remaining hardware
developed in the former one stated above. The main dif­ferences from the former system are:(1) The 16 bit microcomputer NEC PC-9801 is replacedby the 32 bit microcomputer NEC PC-9801,BA in order

to use the software(Nadeshiko and Yamato) for operating
the neurocomputer on the Microsoft Windows.
(2) In the former one the control laws were produced fromthe neural network systems programmed in the 16 bit
NEC PC-9801 microcomputer so that on-line learning was
not possible[8].
(3)The neurocomputer RN-2000(S specification) is newly
added to the system. Therefore on-line learning of control
laws and patterns become possible because the learning by
the neurocomputer RN-2000 is very fast.
On February 2 in 1995, we succeeded in the experi­ments in order to track both the desired values(sinusoidal,
circular, and elliptical movements) of encoders in X and Y
axes and a light from electric lamp moved by hand using
the new neurocomputer control system for recognition andtracking stated above in our laboratory. We obtained thefirst experimental results on the control studies for the
tracking using the neurocomputer control system. This
paper is a brief introduction of both the neurocomputer
control system and the results obtained from the control
studies.

2

Neurocomputer Control

The neurocomputer control system for recognition andtracking of moving objects devised in our Laboratory is
shown in Fig.I and Fig.2. The elaborate descriptions of
the system have been given in [l]-[4]. The configuration
shown in Fig.I is different from the former system[l]-[4]in the point that the neurocomputer RN-2000 is newly
connected to the microcomputer NEC PC-9801,BA. The
general view shown in Fig.2 is the same as before.
At the first stage, in the experiments of the neurocom­
puter control system simple control laws are desirable forthe purpose of both checking the performance of the hard­ware system and validity of the results. To this end, weused the neural network structure and the training data
discussed below.

2.1

Neural Network Structure

The structure of the neural network used in the neu­rocomputer RN-2000 is shown in Fig.3. The number ofneurons in the input layer is 4, the numbers of neurons- 243 -

in the first and second intermediate layers are 16 and 16,
respectively, and the number of neurons in the output
layer is 1. The inputs to the neural network are the er­rors(deviations) between the center of the CCD camera
and the desired values of the encoders in X and Y axes or
the highest intensity of a moving light from electric lampby hand as shown in Fig.4. The teaching signals are theduty ratios of the PWM. The reason why we use 4 neu­rons in the input layer is the followings.
(l)As the inputs to the neurocomputer RN-2000, the twodigits 0(equals 0) and 127(equals 1) are permitted due tothe hardware specification.
(2) The maximum error is 10 in the experiments and thisvalue is transformed into the binary number of 4 bits,
namely, 4 neurons. On the other hand, the digits ranged
from Oto 127 can be used as the output from the neuro­
computer so that one neuron is employed in the output
layer.

2.2

Data for Learning

As the learning or training data for the neural network
shown in Fig.3 in order to track a moving object, we use
the data produced from a proportional control indicted by
the dotted line shown in Fig.5. Both the real training data
and the corresponding data used for the neural networkin the neurocomputer RN-2000 are shown in Table 1. In
this table the input data to the neurons are deviations or
errors of ex and ey which are equal to 1, 2, ... , 10 and theteaching data are the duty ratios of the DC motors in bothhorizontal and vertical axes which are equal to 0.1, 0.2, ...,
1.0. The input data are transformed into 4 bit digits byusing bit transformation technique. The teaching signals
are normalized such that the maximum value 1.0 and theminimum values 0 in Table 1 correspond to 127 and 0,
respectively.
As one of criteria for the learning of the neural network,
we introduce the average error criterion J given byJ=

L IOi - T, 1/N, N = ll(patterns ),
N

i=l

(1)

where Oi is the output from the neural network and T,
is the teaching signal. We use the neural network trained
until the value of this criterion becomes less than 2.0. The
results, which were obtained from the forward processing
by the neural network trained, are shown in Table 1 us­
ing the same input data where the iteration number per
one pattern is 2378 for the learning, total learning timeis I minute and 7 seconds, and J=l.64. It is seen from
Table 1 that the data obtained from the neural network
learned are different from the teaching signals due to thehardware specification, namely, the digital neural network
VLSI with on-chip learning using stochastic pulse encod­
ing. However, the neural network learned works well forthe tracking problems as shown below.
It should be noted that in the experiments for the track­ing the control laws indicated by the solid lines in Fig.5

are used in order to compensate the nonlinear character­istics of two DC motors in the recognition and tracking
system for improving the control performance. This will
be discussed in the next Section.

3

Tracking Experiments

In the experiments for tracking, we use the neural net­
work shown in Fig. 3 in the nerocomputer RN-2000, whichis constructed by the software called Nadeshiko. The con­figuration of the neurocontroller using the neural network
is illustrated in Fig. 6. The duty ratios for two DC mo­tors in X(horizontal) and Y (vertical) axes are produced
from one neural network sequentially. In other words, atfirst, the error between the desired value in X axis( denoted
Xdesired in Fig. 6) and the value of encoder of DC motorin X axis(denoted X motor in Fig. 6) is processed by theneural network in order to get the duty ratio of X motor.
Secondly the same processing is performed in order to get
the duty ratio of Y motor.
In the whole experiments, the sampling time employed
is 100 ms by taking account of hardware specifications in
the various parts. The procedures stated above[l]-[4] arerepeated at each sampling time for the tracking experi­ments. The results are given below.

3.1

Tracking for Desired Values

We performed the experiments for tracking the desired
values(sinusoidal, circular, and elliptical movements) of
the encoders in X and Y axes using the outputs from theneural network.
At first the neural network shown in Fig. 3 is set in
the neurocomputer RN-2000 and is learned or trained asstated above by using the software Nadeshiko. Thereafter
the duty ratios for the X and Y motors are produced fromthe neural network by using the software Yamato and thecontrol programs written by C language developed in ourlaboratory. The compensated proportional control laws
were used in the programs to produce the duty ratios as
indicated by the solid lines in Fig. 5.
For the reason of space, we show only the results for
tracking the desired values of the encoders obtained fromthe elliptical movement in two dimensional space given by
Xdesired= 40sin(51rt/(180 X 100)),(2)

Ydesired= 30cos(51rt/(l80 X 100)),(3)where the unit of time t is ms.
The locus of X and Y coordinates in the elliptical move­
ment is illustrated from I second to 25 seconds by the solidline in Fig.7(a) where the dotted line is the desired value
or trajectory. The values of the encoders in X and Y axes
are shown in Fig.7(b), where the dotted lines are desired
values.

- 244 -

3.2

Tracking for a Moving Light by Hand

The procedure for tracking a moving object is shownin Table 1 in [4] and the elaborate explanations are also
given. Therefore, we show briefly how to detect the po­sition with the highest intensity of the light from a lamp
moved by hand and then explain how to track it. The
detection procedures are as follows.
(a) Read 6 bits image data of the light with 15x12=180
pixels, which is moved by hand, from CCD video camera.
(b) Write the image data into the memory in NEC PC9801,BA.
(c) Detect the position with the highest intensity of themoving light.
The tracking procedures are as follows.
(d) Calculate the duty ratios using the neural network,
which has been trained beforehand in the neurocomputer
RN-2000, for both X and Y motors sequentially.
(e) Move the system in oder to coincide the center of the
CCD video camera with the position of the hightest in­tensity using the duty ratios calculated above.
We show one of results for tracking a moving light byhand in Fig.8 where the distance between the moving lightand the CCD video camera in the system was approxi­
mately 60 cm. In this figure the loci from 8 seconds to
25 seconds are illustrated. The locus of X and Y coordi­
nates in tracking the highest intensity of the moving lightis shown by the solid line in Fig.8(a) where the dottedline is the locus of the highest intensity. The correspond­ing values of the encoders in X and Y axes are shown in
Fig.8(b).
The results obtained from tracking both the desired
values and a moving light by hand are quite satisfactory.
The followings are considered as the reasons.
The neural network structure constructed in the neuro­
computer RN-2000 is simple as shown in Fig.3.
The control law used for training the neural network isa simple proportional law as shown in Fig.5.
The control law used for calculating the duty ratios forboth X and Y motors is simple compensated as shown in
Fig.5.
The results obtained using the simple proportional con­trol law as indicated by the solid line in Fig.5 were alsosatisfactory although they are not shown in this paper.

*
*
*

4

Summary

In this paper we presented a new neuocomputer control
technique for tracking a moving object using the RICOH
neurocomputer RN-2000. The results obtained from thetracking experiments revealed that the neurocontrollersof simple structures developed in the neurocomputer RN2000 have desirable features such that they are able to
track both the desired values and the moving light by handsatisfactory by using a few training data. In other words,
the neurocontrollers developed in the neurocomputer RN2000 have knowledge or brain.

The other experimental data obtained by changing
both the structure and the training data for the neuralnetwork could not be shown in this paper for reason of
space. They are quite interesting and will be presented ina forthcoming paper. We are now developing the appli­
cations of our hardware and software obtained from theexperiments for various engineering fields.

Acknowledgments
I thank graduate student Mr. Noboru Tonoya and re­search group in RICOH Co., Ltd. Mrs. Syuuji Moto­mura, Takashi Kitaguchi, Hirotoshi Eguchi, Hiroyasu Mi­
fune, and Toshiyuki Furuta, Yutaka Ebi for their help
and valuable suggestions in order to perform this research
project.

References[1] M. Sugisaka, "Recognition and Tracking System ofMoving Objects", Patent p2020181(Japan), G05D
3100(in world}, in Japanese, Feb., 1992.
[2] M. Sugisaka, H. Kaita, M. Hara, and K. Nakao,"Recognition and Tracking System of Moving Objects
Based on Artificial Neural Network and PWM Con­trol", Proc. of the IFToNM-jc Int. Symp. on Theoryof Machines and Mechanics, Vol.I, pp.104-107, 1992.
[3] M. Sugisaka, "Recognition and Tracking System ofMoving Objects Based on Artificial Neural Network",
Proc. of the 1992 Korean Automatic Control Conj.,
pp.573-574, 1992.
[4] M. Sugisaka, M. Hara, and N. Tonoya, "Neuro-Fuzzy
Control for Recognition and Tracking System of Mov­ing Objects", Proc. of 3rd Int. Workshop on Advanced
Motion Contro� pp.1091-1098, 1994.
[5] H. Eguchi, T. Furuta, H. Horiguchi, and S. Oteki,"Neural Network Hardware with Learning Function
Utilizing Pulse-Density Modulation", Trans. of the In­
stitute of Electronics, Information and Communica­tion Engineers, Vol.J74-C-II, No.5, pp.369-376, 1991.
[6] S. Oteki, A. Hashimoto, T. Furuta, S. Motomura, T.
Watanabe, D. G. Stork, and H. Eguchi, "A Digital
Neural Network VLSI with On-chip Learning Using
Stochastic Pulse Encoding", Proc. of 1993 Int. Joint
Conj. on Neural Networks, pp.3039-3045, 1993.
[7] M. Sugisaka, S. Motomura, T. Kitaguchi, and H.
Eguchi, "Hardware Based Neural Identification-Linear
Dynamical Systems", Proc. of Int. Workshop on Intel­
ligent Systems and Innovative Computation- The 6th
Bellman Continuum-, pp.124-131, l994(Computers &Mathematics with Applications, Special Issue, to be ap­
pear 1996).

- 245 -

[8] M. Sugisaka, "Hardware Based Neural Identification:
Nonlinear Dynamical Systems", The 2nd Asia-PacificConj. on Control and Measurement(APC CM'95}, 2327 June, 1995, Wuhan-Chongqing, P.R.China.
[9] M. Sugisaka, "Neurocomuter Control System for
Recognition and Tracking of Moving Objects", to besubmitted for a Patent, May, 1995.

.l

lnpq
(Emn�XandY)➔O

4

➔

18

__

1

Fig.3 Structure of neural network in RN-2000.

.l•• •• ••••• •• •••••••••I

PC880118

Y-eda

Hgheetballilyofl.lght

i Video:1-----;.;-1 IIF Boad:: ._____, ::i ....-----,

x...

..
..
.. Parallel., _________1/0 Boad--------Panlllel
1/0 Boad''

MotDnsDrimg Cricuils

Fig.4 Errors or deviations.
cir-«

Neurocolr.,uter
RN-2000
10

Fig.I Neurocomputer control system

l>riYiJII Jlotor b
1lp ud JloD Dlrt!dlaa

#

#

#

.. 0
#

#
#
#

..

.··
.
.·

#
#

#

•

-10

cm CamaraFig.5a Nonlinear propotional control(Xa:na)-lllcoder b
lp-ud:Jlon Directlca

J>riYiJI& -- laUsltt u4 Left ltlrectlca
luocler lallslt't•Ull Left lirectlm

Fig.2 General view of recognition and tracking system.

- 246

Fig.5b Nonlinear propotional control(Ya:na).

+ -

I
�
XclllndErrors(Deviations) Teaching signals Leamedresults
0
0000
0
0
0

1
2
3
4
5
6
7
8
9
10

XIIIIIIDr

Ydeliredn

YIIIIIIDr+ RN2000

Fig.6 Configulation of neurocontroller.
(I and II show the order of processing)

10
20
30

38

50
60
70
80
90
100

76
89
102
114
127

13

40

51

64

20

10

10

-30

·-60

..

.

6

>

0

-zo

72

85
108
112
120

.,-�----- ..
.

16

20

-10

15
27
31
53
63

25

Tal>le 1 Data fur Training

30
>

0001
0010
0011
0100
0101
0110
0111
1000
1001
1010

0
-&

-10

-20·,..______.__________.

-16

0
X

25

60

-00

-10

0

10

X

20

30

40

(a) Loci of light and X and Y coordinates.

(a) Loci of desired value and X and Y coordinates.

• 40.----------------�

:.. 3020

�- 10?o -100
g-20
-30

60

100
160
200
T lae (><100 IIUNIC)

X

260-40-----------------...1
0

60
100
T lae (><tOO aaec)

150•.. 20--------------..

• 30-----------------�
=

:

20
10
..
0
k
g -10

·r:I

0

g -20

u -00

> -40,.._______________.,__,> -30'---=""-..._..___.::lli,£.._�160-�---�
260
200
0

.60

100

T1ae

(XlOO

0

->

iO

100

Tl•• (XlOO IUiec)

150(b) Values of encoders in X and Y axes.

(b) Values of encoders in X and Y axes.

Fig.7 Results of tracking desired values(l-25 sec).

Fig.8 Results of tracking a moving light(8-25 sec).

- 247 -

Pattern Recognition using Neurocomputer
Masanori Sugisaka
Department of Electrical Engineering
Oita University
Oita, 870-11 Japane-mail rnsugi@cc.oita-u.ac.jp

Abstract

This paper presents a new pattern recognition
system based on moment invariants using a neu­rocomputer. The new pattern recognition system
consists of a CCD video camera, an image pro­cessing system named FDM, a monitor, two standlights, a NEC PC-9801 microcomputer and a neu­rocomputer RICOH RN-2000. The experimental
studies in order to recognize five dynamic patternsof Japanese chestnuts were performed. From thestudies high speed of both learning and recogni­tion has been achieved compared with the former
pattern recognition system based on the softwareof artificial neural networks developed by us.
1. Introduction
We have already developed the fast pattern recog­
nition method using the moment invariant com­putation via artificial neural networks[!]. The
method was able to recognize patterns indepen­dently of rotation, translation, and size of any pat­terns. The pattern recognition method developed
used both the moment invariants of the images
obtained from CCD video camera and an artificial
neural network system constructed by software inorder to recognize the shapes or patterns of theimages.
The system consisted of two systems, the firstusing the moment invariants to perform prepro­
cessing, and the second using three-layer artificial
neural network to perform dynamic pattern recog­
nition. It was shown that the proposed method
could recognize the dynamic patterns very cor­
rectly. In the system seven moment invariants of apattern were used as the inputs in the input layerof the three layer artificial neural network with 50
units or neurons in the hidden layer and with 5
units in the output layer. The experimental stud­ies on the learning of the dynamic patterns were
performed with the image processing system.
The same system as stated above is used for the
new pattern recognition system where the soft­
ware of three layer artificial neural network with
50 units in the hidden layer and 5 units in the out­
put layer is replaced by the hardware of the neu­rocomputer RN-2000(2]-[5]. The RICOH neuro­
computer RN-2000 consists of seven RN-200 digi­
tal neural network VLSI chips, where sixteen neu­rons and totally 256 synapses are integrated in a
13.73xl3.73 mm2 VLSI chips, fabricated by RI­
COH 0.8 µ CMOS technology. This chip can per­form 5.12 giga pulse operations per second. It
corresponds to effective neural computing rate of
40M CPS or CUPS.
The simulation of the proposed system is car­ried out in the following steps. The first step
uses the image science, which is compressed in
256(16x16) pixels by preprocessing from the orig­
inal image science to the input pattern. At thesecond step, seven moment invariants are com­puted for each pattern and at the third step threemoment invariants instead of seven moment in­variants are used as the inputs of the artificial
neural network system constructed in the neu­rocomputer for all samples due to the hardware
specification of the neurocomputer. At the fourth
step the training begins with back-propagation
method by the neurocomputer. By ending the ar­tificial neural network training in the neurocom-- 248 -

puter, the outputs of the network is used to rec­ognize the dynamic patterns. In the experiments,
high speed of both learning and recognition has
been achieved, compared with .the former patternrecognition method based on the software, andthat is the advantage of the proposed system inthis paper.

2. Moment Invariants and Neural
Network
The hardware of the new image processing sys­
tem proposed for the patterns recognition is illus­
trated in Fig. I where the difference between the
new one and the former one[l] is the introductionof the neurocomputer. The moment invariants
Q1, Q2, Q3 are invariant under translation androtation [6),(1]16 16
Mpq - '"" '""iPJ·q .ri3
D. . ,- i=lj=l
��
1
16 6
µpq =
i)P(j- })qPij,

L L(i -

(1)

(2)

i=lj=l(3)(4)(5)
where Pi,j is a digital 0 or 1 function, i and j haveinteger value and i = M10/Moo and = Moi/MooThe moment invariants above will be used as
the inputs for the neural network in the neuro­
computer RN-2000. In our pattern recognition
system only the three moment invariants Q1, Q2,
Q3 are used instead of seven moment invariants
due to the hardware specification in the neuro­
computer. The artificial neural network systemconstructed in the neurocomputer is shown in Fig.2.
It consists of four layers: the input layer with 15
neurons or units, the first hidden layer with 16
neurons, the second hidden layer with 16 neurons,
and the output layer with 5 neurons which is equal
to the number of the patterns to be recognized.

3. Simulation Study
3.1 Data for Training
The experimental studies on the learning of dy­
namic patterns are performed with the image pro­cessing system shown in Figs.I and 2. Figure 3
shows the patterns which are used in the learn­ing processing and gives examples of five different
patterns. The three moment invariants of the fivepatterns are calculated for four different groups.
The locations of each pattern are different in thefour groups.
Table 1 shows the moment invariants calculated
for the patterns in each group. The moment in­variants in Table 1 are transformed into five bits
digital values in order to use the neural network
shown in Fig.2. Therefore, the maximum valueof the normalized moment invariants becomes 31.
Table 2 shows the normalized moment invariants.
The teaching signals are shown in Table 3 for eachpattern.

3.2 Training Method
It is necessary that the pattern recognition sys­
tem is able to recognize the patterns exactly as
possible as it can. There might exist effective
methods by training the neural network in theneurocomputer. However, we used the training
method as follows. The order and number of the
training are:G1(n times), G2(n), G1(n), G2(n), G3(n),G1(n), G2(n), G3(n),Jwhere Gi denotes group i and n is 15,000 in our
experiments. The synapses weights of the neu­
ral network were accumulated in the neurocom­
puter according to the order stated above. The
schematic configuration of the training process­ing is shown in Fig.4. If the number of groups isfour, the same training method as shown in Fig.4
is employed. The outputs of the neural network
trained by the method above are shown in Fig.5.
The results obtained are considered to be satis­
factory for the recognition of the five patterns.

- 249 -

4. Summarythe 2nd Asia-Pacific Conj. on Control and Mea­
surement, Wuhan-Chongqing, P. R. China, 255259, 1995.

In this paper we proposed a new pattern recog­
nition system based of three moment invariantsof a pattern using a neurocomputer. we obtained
the first experimental results for recognizing fivedifferent dynamic patterns using the new system
proposed. The experimental results confirm theeffectiveness of the proposed system and show im­
provement of both the learning and recognition
speeds, which is the main point of this paper. We
are now developing the application of our hard­ware and software developed in this study for var­ious engineering fields.

[6] B. Widrow, R. G. Winter and R. A. Bax­ter, "Layered neural nets for pattern recognition",
IEEE Trans. Acoustics, Speech, and Signal Pro­
cessing, ASSP-36, 7, 1109-1118, 1998.

1□� □ 8--,r @ J
Moallar

FDM98

CCD. c:amerallld llabtll

PC8801

Fig.l New pattern recognition system using neu­rocomputer RICOH RN-2000

References[l] M. Sugisaka and M. Teshnehlab, "Fast pat­tern recognition by using moment invariants com­putation via artificial neural networks," Control­
Theory and Advanced Technology, 9, 877-886, 1993.
[2] H. Eguchi, T. Furuta, H. Horiguchi and S.
Oteki "Neural network hardware with learning
function utilizing pulse-density modulation", Trans.
of the Institute of Electronics, Information andCommunication Engineers, Vol.J'l4C-II, 5, 369376, 1991.
[3] S. Oteki, A. Hashimoto, T. Furuta, S. Moto­mura, T. Watanabe, D. G. Stork and E. E guchi,"A digital neural network VLSI with on-chip learn­
ing using stochastic pulse encoding", Proc. of
1993 Int. Joint Con. on Neural Networks, 30393045, 1993.

Fig.2 Neural network constructed in neurocom­
puter RN-2000

□ar·
,
[)[

(4] M. Sugisaka, S. Motomura, T. Kitaguchi, andH. Eguchi, "Hardware Based Neural Identifica­
tion:Linear Dynamical Systems", Proc. of Int.
Workshop on Intelligent Systems and Innovative
Computation-The 6th Bellman Continuum-, 124131 ( Computers Math. Appli. 1996, to be ap­
pearea), 1994
[5] M. Sugisaka, "Hardware based neural identi­
fication: nonlinear dynamical systems", Proc. ofPattern4 Patterns

I

Fig.3 Examples for testing recognition of five var­ious patterns- 250 -

Patternl Patterll2 P&ttern3 P&tterd Pattern5
Q,
Q.
Q.

9.200
0.TlT
0.410

T.503
2.105
0.15'

T.220
0.133
1.403

T.808
18.9500.562

8.801
10.3003.025Group!

Q,
Q.
Q.

8.T84
0.204
0.010

T,TT9
8.418
0.416

T.115
0,090
0.5T0

6.882
12.380
0.T94

8.364
T,590
1.843

Group2

Q,

8.TT6
0.442
0.2T5

T.111
ll.122
0.218

6.931
0.019
0.5T3

T.452
18.2101.565

8.081
11.2301.843Group3

Q,

9.011
0.334
0.104

6.913
2.826
0.1T3

T.484
0.166
0.T62

6.819
9.892
0.520

6.989
9.390
2.602

Group4Q.
Q.

n= 15000 times

Q.
Q,

Fig.4 Schematic configulation of training

Table 1 Moment invariants of five patterns foreach group

P&tternl P&ttern2 P&ttern3 Pattern4 Pattern&

0
1
2
3
4

0
127
0
2
0

76
0
127
0
0

0
0
0
126
0

0
0
0
0
113

0
0
0
0
0

0
1
2
3
4

0
126
0
2
0

0
0
126
0
0

0
0
0
126
0

0
0
0
0
113

0
0
0
0
0

0
1
2
3
4

0
127
0
2
0

0
0
127
0
0

0
0
0
126
0

0
0
0
0
127

0
0
0
0
0

0
1
2
3

0
126
0
2
0

0
0
104
0
0

0
0
0
126
0

0
0
0
0
110

0
0
0
0
0

4

Q,
Qi
Qi

16
0
0

12
3
0

11
0
1

12

11

0

4

Q,
Q1
Qi

21
0
0

19
21

16
31

16

1

lT
0
1

Q,
Qi
Q,

14

12

11

12

10

Q,
Q,

28

Q,

0
0
0
0

5
0

u

a

0

18

31

18

l

4

0
0

31
2

23
0
2

21

21

1

T

19

2

31

19

GrouplGroup2
Group3
Group4

Table 2 Normalized moment invariants of five pat­terns to be used for neural network

Fig.5 Outputs of neural network trained for each
group

Yi

Ya

Ya

y,

Y•

Pattern!

0

0

0

0

0

Pattern2

127

0

0

0

0

Pattern3

0

127

0

0

0

Pattern4

0

0

127

0

0

Pattern5

0

0

0

127

0

Table 3 Teaching signals of five patterns- 251 -

Rectangles, Triangles, Robots and Transients
Philip Husbands
School of Cognitive and Computing Sciences
University of Sussex
Brighton, UK
Abstract
This paper describes an experiment in real-world
evolutionary robotics. A visuo-motor behaviour isevolved that allows the robot to distinguish between
two different targets. The mechanisms underlying the
behaviour are analysed and some surprisingly subtle
features are uncovered.

1
1.1

Introduction
Evolutionary Robotics

The basic notion of Evolutionary Robotics is as fol­lows. The evolutionary process, based on a genetic al­
gorithm [3, (Holland 1975)], involves evaluating, overmany generations, whole populations of control sys­
tems specified by artificial genotypes. These are inter­bred using a Darwinian scheme in which the fittest in­dividuals are most likely to produce offspring. Fitness
is measured in terms of how good a robot's behaviouris according to some evaluation criterion. The work
reported here forms part of a long-term study to ex­
plore the viability of such an approach in developing
interesting adaptive behaviours in visually guided au­
tonomous robots, and, through analysis, in better un­derstanding general mechanisms underlying the gen­eration of such behaviours. It is one of the strands ofthe research program of the Evolutionary and Adap­
tive Systems Group, School of Cognitive and Comput­
ing Sciences, University of Sussex. For further details
see e.g. ([1, Cliff et al 1993]).

1.2

Real World Evolution

A crucial decision in evolving robot control systems
is whether or not to use simulation at the evaluation
stage, transferring the end results to the real world.
Since an evolutionary approach potentially requires
the evaluation of populations of robots over many gen­erations, a natural first thought is that simulations

will speed up the process, making it more feasible. It
has recently been shown that control systems evolvedin carefully constructed simulations, with an appro­
priate treatment of noise, transfer extremely well to
reality, generating almost identical behaviours in thereal robot ([4, Jakobi et al 1995]. However, this exam­ple involved relatively simple robot-environment in­teraction dynamics. Once even low-bandwidth vision
is used, simulations become altogether more problem­
atic. They become difficult and time consuming toconstruct and computationally very intensive to run.
Hence evolving visually guided robots in the real world
becomes a more attractive option. The experiment de­scribed here uses a piece of robotic equipment speciallydesigned to allow the real-world evolution of visually
guided behaviours - the Sussex gantry-robot.

2

The Gantry-Robot

The gantry-robot is shown in Figure 1. The robot iscylindrical, some 150mm in diameter. It is suspended
from the gantry-frame with stepper motors that allow
translational movement in the X and Y directions, rel­
ative to a co-ordinate frame fixed to the gantry . The
maximum X (and Y) speed is about 200mm/s. Suchmovements, together with appropriate rotation of thesensory apparatus, correspond to those which would
be produced by left and right wheels. The visual sen­sory apparatus consists of a CCD camera pointing down
at a mirror inclined at 45 ° to the vertical. The mirror
can be rotated about a vertical axis so that its orien­
tation always corresponds to the direction the 'robot'
is facing. The visual inputs undergo some transforma­tions en route to the control system, described later.
The hardware is designed so that these transforma­tions are done completely externally to the processingof the control system.
The control system for the robot is run off-board on
a fast personal computer, the 'Brain PC'. This com­puter receives any changes in visual input by inter-- 252 -

3

Figure 1: The Gantry viewed from above. The horizontal
girder moves along the side rails, and the robot is suspended
from a platform which moves along this girder.

rupts from a second dedicated 'Vision PC'. A third(single-board) computer, the SBC, sends interrupts to
the Brain PC signalling tactile inputs resulting fromthe robot bumping into walls or physical obstacles.
The only outputs of the control system are motor sig­
nals. These values are sent, via interrupts, to the SBC,
which generates the appropriate stepper motor move­ments on the gantry.
The Brain PC runs the top-level genetic algorithmand during an individual evaluation, it is dedicatedto running a genetically specified control system fora fixed period. At intervals during an evaluation, asignal is sent from the Brain PC to the SBC request­
ing the current position and orientation of the robot.
These are used in keeping score according to the cur­rent fitness function. The Brain PC receives signals,
to be fed into the control system, representing sen­sory inputs from the Vision PC and the SBC. The
visual signals are derived from averaging over genet­
ically specified circular receptive patches in the cam­era's field of view.
This setup, with off-board computing and avoid­ance of tangled umbilicals, means that the apparatus
can be run continuously for long periods of time �making artificial evolution feasible. A top-level pro­gram automatically evaluates, in turn, each memberof a population of control systems. A new population
is produced by selective interbreeding and the cycle
repeats. For full technical details of the system see([2, Harvey et al 1994)).

Experimental Setup

Full details of the experimental setup for the gantry­
robot can be found in ([2, Harvey et al 1994)). This
paper also explains in full the genetic encodings used
and the control system primitives manipulated by the
GA. Experiments conducted with the gantry-robot todate have all involved relatively simple <sion basednavigation tasks. The experiment described below was
one of a series where a converged population of robots
was evolved through a series of increasingly complex
behaviours.
These were based around the evolution of control
architectures built from recurrent dynamic realtime
networks, where the primitives were the nodes in a
network, and links between them. There were no re­strictions on network topologies, arbitrarily recurrent
nets being allowed. When some of these nodes are con­nected to sensors, and some to actuators, the network
acts as a control system, generating behaviours in the
robot.
Rather than imposing a fixed visual sampling mor­
phology, we believe a more powerful approach is toallow the visual morphology to evolve along with therest of the control system. Hence we genetically spec­
ify regions of the robot's visual field to be sub-sampled,these provide the only visual inputs to the control net­
work. It would be desirable to have many aspectsof the robot's morphology under genetic control, al­though this is not yet technically feasible.
Starting from a converged population of robots that
could move forward, but little else, the first task was
to move to a large white target from random starting
points and orientations. Once this was being achieved,
the task was changed to approaching a small whitetarget and evolution continued.

4

Rectangles and Triangles

The experiment then continued with a distinguish­
between-two-targets task. Two white paper targets
were fixed to one of the gantry walls; one was a rect­
angle, the other was an isosceles triangle with the samebase width and height as the rectangle. The robot wasstarted at four positions and orientations near the op­
posite wall such that it was not biased towards eitherof the two targets. The evaluation function £3, to be
maximised, was:

- 253 -

L [.8(D1, - di.) - u(D2,, d2.)]

i=20

£3 =

i=l

(1)

where D1 is the distance of target-1 (in this case thetriangle) from the gantry origin; d1 is the distanceof the robot from target-1; and D2 and d2 are thecorresponding distances for target-2 (in this case therectangle). These are sampled at regular intervals, as
before.. The value of J3 is (D1 - d1) unless d1 is lessthan some threshold, in which case it is 3 x (D1 - d1).
The value of a- (a penalty function) is zero unless d2
is less than the same threshold, in which case it isI - (D2 - d 2 ), where I is the distance between thetargets; I is more than double the threshold distance.
High fitnesses are achieved for approaching the trian­gle but ignoring the rectangle. It was hoped that thisexperiment might demonstrate the efficacy of concur­
rently evolving the visual sampling morphology alongwith the control networks.
After about 15 generations of a run using as an
initial population the last generation of the incremen­tal small target experiment, fit individuals emerged
capable of approaching the triangle, but not the rect­
angle, from each of the four widely spaced starting
positions and orientations. The behaviour generated
by the fittest of these control systems is shown in Fig­ure 2. When started from many different positions
and orientations near the far wall, and with the tar­gets in different positions relative to e�ch other, thiscontroller repeatedly exhibited very similar behaviours
to those shown.
The active part of the evolved network that gener­
ated this behaviour is shown in Figure 3. The evolved
visual morphology for this control system is shown in­
set. Only receptive fields 1 and 2 were used by thecontroller.
Detailed analyses of this evolved system can befound in ([2, Harvey et al 1994]). To crudely sum­marise, unless there is a difference in the visual inputs
for receptive fields 1 and 2, the robot makes rotational
movements. When there is a difference it moves in astraight line. The visual sensor layout and network dy­
namics have evolved such that it fixates on the slopingedge of the triangle and moves towards it.

5

Transient Behaviour

Time plots of behaviour against this difference invisual inputs consistently revealed an interesting non­reactive feature to the robot's behaviour. Figure 4
shows such a plot. The behaviour axis (Z) is dis­cretized into simple observable motor behaviours suchas straight line motion, rotating on the spot, move­
ment in the arc of a circle and so on. The final part

- 254

....

....

\,
\,

Figure 2: Behaviour of a fit individual in the two target
environment. The rectangle and triangle indicate the po­sitions of the targets. The semi circles mark the 'penalty'{near rectangle) and 'bonus score' {near triangle) zones as­sociated with the fitness function. In these 4 runs the robot
was started directly facing each of the two target, and twicefrom a position midway between the two targets; once fac­
ing into the wall and once facing out.

of the plot, a line parallel to the time axis and termi­
nating at the point marked 'finish' at the right handside of the cube, represents the straight line motion
when the robot has fixated on the triangle edge and ismoving towards it. The parallel line above this and im­
mediately to the left represents a short lived transient
behaviour which such plots revealed always occurred
when the visual signal difference become large. Briefly,
the onset of a large difference triggers a short sharp
rotational movement which has very different conse­
quences depending on whether the robot has fixated
on a vertical or sloping edge. With a vertical edge, therotation tends to move both receptors off the target,
the visual signals become very different and rotational
behaviours ensue. However, with a sloping edge, therotation is not enough to move both receptive fields
off the target; the visual signal difference is still there
and a straight line motion follows. This is illustrated
in Figure 5. This behaviour can be interpreted as akind of 'checking' of edge orientation.

6

Conclusions

This paper has described an experiment in real­world evolutionary robotics and has shown that theV2

r.....;. c:· �----=:----;,?o::::::::-::f::_=====::::S;;;:;-.
---------------------------

Key same as in earlier network diasJam!I.

'
'
'
'
'

:----���=-?':�����:______
Figure 3: Active part of the control system that generated
fit behaviour for the rectangle and triangle experiment. Vi­
sual morphology shown inset.
evolved control networks and visual morphology fora robot engaged in a simple target distinguishing
behaviour display subtle dynamics resulting in non­reactive behaviour.

References[l] Cliff D and Harvey I and Husbands P 1993 Ex­
plorations in Evolutionary Robotics Adaptive Be­
havior 2(1) 73-110

Figure 4: Time plot of behaviour against difference in vi­

sual inputs for receptive fields 1 and 2. The time axis (X)runs left to right, the visual signal difference axis (Y) runs
bottom to top on the lower face of the cube, the behaviour
axis {Z) runs from lower to top face of the cube. See text
for further details.

[2] Harvey I and Husbands P and Cliff D 1994 Seeing
The Light: Artificial Evolution, Real Vision From
Animals to Animats 3, Proc. of 3rd Intl. Confon Simulation of Adaptive Behavior eds. D Cliffand P Husbands and J-A Meyer and S Wilson(Cambridge, Mass: MIT Press/Bradford Books)
pp 392-401[3] Holland J 1975 Adaptation in Natural and Artifi­
cial Systems (Ann Arbor: University of Michigan
Press)[4] Jakobi N and Husbands P and Harvey I 1995
Noise and the Reality Gap: The use of simula­tion in evolutionary robotics Advances in Artifi­
cial Life: Proc. 3rd European Conference on Ar­
tificial Life eds. F Moran and A Moreno and J
J Merelo and P Chacon Lecture Notes in Artifi­
cial Intelligence 929 (Berlin: Springer-Verlag) pp704-720Q-◄- ___ ,.. ______ receptivepatches0---=--.. .. ... __ - - edge of paper target

goes into rotational behaviour

small rotation1-

small rotatim

1/goes into straight-line behaviour

EDGE ORIENTATION 'CHECKING' BEHAVIOUR

Figure 5: The top part of the figure illustrates the out­

come of the transient 'checking' behaviour when the re­
ceptive fields straddle a vertical edge, and the bottom partshows the same when they straddle a sloping edge.

- 255 -

Force / Position Control of Robot Manipulator via Motion Dynamics

Cheoljoo Ham**
Woonchul Ham*** Electronic Engineering** Kunjang Technical Junior College** Kunsan, Chonbuk, Korea 573-840

Kyumann Im*
Sacheul Jeoung** Electronic Engineering* Chonbuk National Univ.
* Chonju, Chonbuk, Korea 560-756
Abstract
In this paper, we propose a new force/position con­trol scheme for motion control of a rigid robot manipu­
lator along a frictionless surface represented by a inte­gral submanifold of some distribution. The described
control scheme involves the exact cancellation of thenonlinear terms of the projected end effector dynam­
ics in the position and force controlled directions. We
introduce a new motion equation on the integral sub­manifold of a smooth distribution by using differential
geometry. The feasibility of the proposed force control
scheme is verified through a computer simulation.

1

Introduction

In this paper, we address the problem of control­ling a robot manipulator which is to track a path ona frictionless surface while exerting a force (tangen­tial and normal force) on the surface. Many manu­
facturing tasks such as deburring and polishing a sur­face require such constrained motion execution. An
interesting and informative historical perspective onsome past work in the area of robotic force control
was presented by Whitney[l]. Considerable numberof researchers have studied the problem of constrained
motion control of a rigid robot manipulator along a
frcitionless surface. Early researchers in this area, suchas, Paul and Shimano proposed decomposing the ma­nipulator into types of joints those which contributeto the force control and those which contribute to the
motion of the end effector along the surface. Raibert
and Craig[2] pursued this subdivision of tasks into po­sition and force control frame work and developed ahybrid force/position control law, where each manipu­
lator joint provides a hybrid torque which affects boththe end effector force and position. However detailed
stability analysis of the hybrid control law were not
performed. Anderson and Spong[3] combined the no­tion of impedance control with hybrid force/ position

control to allow for precise force servoing which might
be required in many applications. The dynamic equa­tions of the manipulator along an algebraic constraint
surface were identified as a singular system of differ­ential equations (or descriptor systems) by McClam­roch. Khrisnan and McClamroch[4] also considered
the design of a hybrid/force position control scheme
for flexible joint robots by linearizing them about an
operating point. Ham[5-6] proposed adaptive nonlin­
ear control of one-link flexible arm and adaptive con­trol based on explicit model of robot manipulator. In
this papaer, we derive motion equation that can beused to make the trajectory of end-effector move froman initial point to the desired point on the constrainedtask plane(or hyperplane) by using the mathematical
tools concerning vector fields of manifold. We also
consider the robust motion equation that can be used'even in the presence of uncertainties. We suggest a
new force/motion control scheme of robot manipula­
tor based on the proposed motion equation. The paper
is organized as follows. In section II, we describe theconcepts of submanifold and determistic and robust
motion equations. In section III, we formulate thedynamics of the rigid robot manipulator with an end­
effector. In section IV, we design a new force/position
control law. In section V, the feasibility of the pro­
posed a new force/position control scheme is verified
through a computer simulation.

2

Mathematical Tools

In this section, we briefly discuss the concepts of sub­manifolds and derive motion equations based on thatconcepts.
Let � be a nonsingular and has dimension d in aneighborhood U 0 of x 0 • Then the following lem�a issatisfied.
Lemma 2.1 Let � be a smooth distribution and x 0
a regular point of�- Suppose dim(�(x 0 ))- 256 -= d, thenan open neighborhood U 0 of x 0 and a se t{Ji, ..., fd} of smoo th vec t or fields defined on U 0 withthe property tha t

Taking the time derivative of V along the trajectoryof {2.1) yieldsi) the vectors fi(x), ..., fd(x) are linearly independenta t each x in U 0 •ii} Ll(x) span{f1(x), ...,fd(x)} a t each x in U 0 •-(x1 - xf (k1(t)fi(x) + ... + kd(t )fd(x))

there exist=

Remark 2.1 Moreover, every smooth vec t or field rbelonging t o Ll can be expressed, on U 0 , as

r(x)

i=l

V(x)

= 2:Ci(x)fi(x)Let x 0 , x1 be two points on a maximal submanifold
S of nonsingular d-dimensional smooth distribution
Ll(x) and define a set B 0 contained in S as follows= {xi II x-x0 II< 6,x ES}.

where 6 =II X1 - Xo 11--[((x1 - xf fi(x))

2+ ((x1 - xf h(x))+ ... + ((x1 - xf Jd (x))2 ]{3;(t ) � 0.

2

(2.5)

Let 01 be the largest invariant set of contained in set
0 2 where_n 2{x E B0 : V(x) O}, then we can find{xi}. By using LaSalle's theorem, we can find01x(t) approaches x1 as time goes to infinity.
I

=

=

=

= f(x,t ) is a stable, thenx(t) = F(x,x,t),Lemma 2.2 If x(t )

where

( . )

F x,x,t( ))= of(x,t ).X (t ) + of(x,t ) - 'Y("()
X t - f x,t '
ox'

at

=

Proof: Let z(t) :i:(t) - f(x,t), then from the above
equation we obtaini(t)Th�orem 2.1 (WITHOUT UNCERTAINTY)
Le t x0 and x1 be the points on a same maximal integral

submanifold S of the distribution Ll. Then there exist
some time func tions k1 (t), · · ·, kd(t ) such tha t the fol­lowing mo tion equa tion move the sta te trajec t ory fromx0 t o x1 wi thout devia ting from the integral submani­
fold S.

(2.1)
where fi(x), ... ,fd(x) are independen t and selected
such tha t(2.2)

Proof: The proof is based on the Lyapunov-like func­tion as follow

V(x)= 1, 2, ..., d. Then, we can write as

is also st able for any const ant 1 > 0.

Assumption 2.1 The absolute angle be tween Tx S
and Ty S is less than 1r /2 for any points x, y E B0

= span{f1(x),... ,fd(x)}.

(2.4)

Let us define

where f3i(t) > 0, i
follow

Based on the above mathematical tool, we suggest
two theorems concerning motion equation under thefollowing assumption 2.1. One is for a deterministic
motion equation which has no uncertainty and theother one is for a motion equation which has someuncertainties.

Ll(x)-(x1 - xf xdwhere c1 (x), ... , cd(x) are smoo th real-valued func tionof x, defined on U 0 •

B4

V(x)

= -,z(t).

Because 1 > 0, the above system is stable. So for anyz(O)lim z(t)t-+oo=0

and therefore we can see thatlim x(t )
t-+oo

= 0.

I

Example 2.1 Consider the following distribution,
defined on R3

Let S be integral submanifold of the above distribution
Ll. Then(2.3)- 257 -

of, xi

Consider the following points Xo = (1, 0,(0, 1,
and we choose ki(t), k2(t) as followsof(xi - xf fi(x)(l - e- 3 t)
3(xi - xf h(x)(l - e- t)

Taking the time derivative of V along the trajectoryof (2.8) yields
d

(2.6)

-(xd - xf L)k;(t) + 1J;(t))f;(x).
i=ithen, from theorem 2.1, we obtain the motion equationthat move the state trajectory from x 0 to xi as followsx(t)Let us choosek;(t) = (xd - xf f;(x) + K;tanh(si)3+

fi(x)(xi - xf fi(x)(l - e- t)
3h(x)(xi - xf h(x)(l - e- t). (2.7)

Let us define V;( x) as follows
V;(x) = -(xd - xf (k;(t) + 1J;(t))f;(x)1.5 �---�-�-�---�X2!

then, by substituting equation (2.12) into equation(2.11), we obtain

1.0 t-------<f----+�-w----._;_:······'------j------1

t---t-� "\

I

d

I

\

''

.... rI:(-s; - s;17;(t)- s;l{;tanh(s;))

V(x)

i=i

0.0 1----t--+-. ---+-.---+.----•·····-----<
1
1 Intei ral si1 bmar 1 ifold-0.5 � ·················"\</ /If the absolute value of s; is greater than l�K,, thenthe value of V; is less than zero. Therefore we canguarantee that the following inequality holds as timegoes to infinity- 1.0 �·-----1--;---�t--'��-+-----1
i -�---+I-�-� X1-1.5 �--f--+-1.5 - 1.0 -0.5 0.0
0.5
1.0
1.5

8;
l+K;

Fig.l The state trajectory of motion equation(x3 = 0).
Theorem 2.2 (WITH UNCERTAINTY)
Let x 0 and xi be the points on a same maximal integral
submanifold S of the distribution Ll. Then there exist
some time functions k1 (t), · · ·, kd(t) such that the fol­lowing motion equation move the state trajectory fromx 0 to x 1 without deviating from the integral submani­
fold S.
x(t) = k1 (t)fi(x) + k2(t)h(x) + ...+ kd(t)fd(x)+TJ1(t)fi(x) + 1J2(t)h(x) + ... + T/d(t)fd(x) (2.8)

c,,

II (xd - xf f;(x) II
II (xd - xf f;(xd) II< 1 :iK; · (2.14)

Let the angle between xd - x and f;(xd) be 0;. Then
we can obtain the following inequalities
8·

II Xd - x 1111 f;(xd) II cos0; < -'1 +K;

II Xd - x II< ?�,ip{ II f;(xd) II c;s0;(1 +Ki)} (2.15)
where I= {1, 2...d}.
Therefore if K; is selected to be large, then II Xd - x II
becomes small.
I

where l7J;(t)I < 8;, i = 1, 2, ..., d and fi(x),... ,fd(x)are independent and selected such that
(2.9)

3

Proof: The proof is based on the Lyapunov-like func­tion as follows

V(x)(2.13)

0,if ls;I < l�K,.

I

Ll(x) = span{fi(x), ... ,fd(x)}.

(2.12)

where s; = (xd - xf f; and K; > 0 , i = 1, 2, . . . d.

The state trajectory of above system is shown in Fig.1.
From Fig.1, we can check the validness of theorem 2.1.

!

(2.11)

V(x)(2.10)

Dynamics of Two-Link Robot Ma­
nipulator

In this section, we derive the dynamics of two­link rigid robot manipulator with an end-effector. In- 258 -

derivation of dynamics, we consider end-effector as
point-mass. .

,------1

Controller Design

4

In this section, we propose new force/position con­trol law for the robot manipulator based on the
motion equation discussed in section II. The newforce/position control law guarantees that closed-loopsystem has the uniformly bounded stability. The
mathematics that relate the world coordinate to thejoint angle coordinate is inherently nonlinear and canbe expressed by a nonlinear vector valued function asz..
When we apply Euler-Lagrangian equation to therobot manipulator shown in Fig.3, we obtain the iner­
tia matrix term as follows

1Du

I 2 [ mi + 4 m2 + 2m1 + m2c2 + 2m1c2]

Di2

2 1
1
I [ m2 + m2c2 + m, + m1c2]

3

3

2

:i:

-2 m2 s2q2 - m, s2q2
-21 m21 2 s2 ( q. i + q. 2) - m11 2 s2 ( q. i + q. 2 )

Cu

,2

·

·

x

=

J(q)ij_2 miglci + 2 m2glci2 + m2glci+ m1lcig + m1lc12g
1(3.3)
2 m2glci2 + m1lci29-

X

sinq2, ci2
cosq2, s2
where ci
cosqi, c2cos(qi + q2), m, denotes point mass of an end-effector.
Therefore the resulting dynamic equation of two­link robot manipulator considering the normal forceof the end-effector can be expressed as
D(q)ij_ + C(q, q)q + G(q) + J (q)Fn (q)T=uD-i(q)u.

n

(4.5)

+
+

J(q)D-i(q)C(q, q)q + J(q)D-i(q)G(q)J(q)D-i(q)JT (q)Fn (q)(4.6)
J(q)D-i(q)u.

From (4.3) and (4.6), we get==n-i(q)G(q)n-i(q)J T (q)F (q)If we premultiply J(q), then equation (4.5) becomes

1

=

(4.4)

F(z,x,t)+and the gravity vector terms as follows
G i(q)

F(H(q), J(q)q);;_ + n-i(q)C(q, ti)<i +

(3.2)

1

(4.3)

'From equation (3.4), we obtaini0= i(q)q + J(q)ij.

= J(q)ij + i(q)q

1 m2 /2s2q. + m, t2s2q. i

2

(4.2)

Now, we design the force/position control law suchthat above equation is the same as motion equationsuggested in Lemma 2.2, i.e.,

and the Coriolis and centrifugal matrix terms as fol­lows
,2

= J(q)q.

where J(q) is the manipulator's Jacobian. The end­
effector and joint accelerations are related by the fol­lowing equation
x

(3.1)

1

(4.1)

where z E R6 and q E Rn . The velocity of the end­
effector is related to the joint velocity q as follows,

Fig.2 The two-link robot manipulator.

3= H(q)+

i(q)q + J(q)D-i(q)C(q, q)q
J(q)D-i(q)G(q) + J(q)D-i(q)JT (q)Fn (q)J(q)D-i(q)u.
(4.7)

If we apply equation (4.4) to above equation, we get(3.4)

where Fn (q) is normal force(or contact force) of theend-effector on frictionless surface.

- 259 -

F(z,

x, t)

+

+

i(q)q + J(q)D-i(q)C(q, q)q
J(q)D-i(q)G(q)J(q)D-i(q)JT (q)Fn (q)(4.8)
J(q)D-i(q)u.

Let us define vector v as follow(4.9)
Then, we can find the vector v as follow by using
pseudo inverse of JT (q)v

1(J(q)J (q))- [F(x,T++x, t) - j(q)q

J(q)D- (q)C(q, cj)cj + J(q)D- (q)G(q)
1

1

T
1
J(q)D- (q)J (q)Fn (q)].

(4.10)

Therefore a new force/position controller of robot ma­nipulator as follows
u

+
+

1
T
T
D(q)J (q)(J(q)J (q))- [F(x,equation. The presented control scheme is based on
the motion equation that can be obtained by using
differential geometry. The magnitude of vector fields
that charaterize the motion equation is designed byusing Lyapunov-like function. The uniformly ultimateboundness of the control scheme is guaranteed and has
been demonstrated by a simulation. In near future,
we will study and develop the adaptive version of the
proposed control law which can be applied in real sit­uation when a part of parameters of robot dynamics
are unknown.

Referencesx, t)- i(q)q

J(q)D- (q)C(q, q)cj + J(q)D- (q)G(q)[1] Whitney, D.E., "Historical Perspectives and Stateof the Art in Robot Force Control," Proceedings

1

1

T
1
J(q)D- (q)J (q)Fn(q)].

(4.11)of the 1985 IEEE Conference on Robotics andAutomation, pp.262-268,St.Louis,MO,March 25-

28,1985.

5Computer Simulation

Computer simulations are conducted to verify thevalidity, effectiveness and performance of the proposeda new force/position control scheme. We simulated
the proposed a new force/position controller to con­trol the joint angle 0 1, 02 of the two-link rigid robot
manipulator. Table.I show the physical parameters of
two-link robot manipulator. We set the task plane ofend-effector to elliptic function (x 2 + 4y22) and set
the initial angles and velocities_ of the link 1 and link
02
0. Hence the ini­
01
0, 02
2 to 0 1== ¼,= -i, ==

tial location and velocity of end-effector is (-/2, 0) and
0 respectively. We also set the desired location andvelocities of end-effector to (parameters

m1
m2
m1
l

g

j'f, j'f)values
2
1.2
0.2
1
9.8

and 0.

[2] Raibert, M.H., J.J., "Hybrid Position/Force Con­
trol of Manipulators ," ASME Journal of Dy­
namic Systems Measurement and Control, 102,
pp.126-133 June 1981.
[3] Spong, M.W., "On the Force Control Problem for
F lexible Joint Manipulators," IEEE Trans. Au­
tomat. Contr., Vol.34,No.l, pp.107-111,January
1989.
[4] McClamroch, N. H., "Singular Systems of Dif­
ferential Equations as Dynamical Models forConstrained Robot Systems," Proceedings of the1986 IEEE Robotics and Automation Conference, April 1986 San Francisco, CA.
[5] W.Ham, "Adaptive control based on explicit
model of robot manipulator ," IEEE Trans. Au­
tomat. Contr., vol.38, Apr,pp.654-658, 1993.

units
kg
kg
kg
m
N/kg

[6] W.Ham, J.J.Lee "Adaptive nonlinear control of
one-link flexible arm," IEEE/RS] International
Conference on Intelligent Robots and Systems,

vol.I, pp.3-7, Yokohama, Japan, 1993.
[7] W.Ham, J.J.Lee "Adaptive Control Based on
Speed gradient Algorithm for Robot Manipula­tors," !ECON '94 Proceedings of the 20th In­

Table 1. Physical parameters of two-link robot
manipulator

ternational Conference on Industrial Electron­6

ics Control and Instrument., vol.2, pp.776-781,

Conclusions

Bologna,Italy,September 1994.

In this paper, we propose a new force/position con­trol scheme to perform force and position control of
the robot manipulator when the end-effector is mov­ing along a frictionless surface represented by algebraic- 260 -

[8] Isidori, "Nonlinear Control Systems : An In­
troduction ," Springer-Verlag, Berlin, New ·York,1985.

Autonomous Micro-Robot 'Kity' for Maze Contest
Jong-Hwan Kim, Myung-Jin Jung, Hyun-Sik Shim and Seon-Woo Lee
Department of Electrical Engineering
KAIST
Taejon, 305-701, Korea
johkim, mjjung, hsshim, senu@vivaldi.kaist.ac.kr

Abstract
In this paper, we propose a table-look-up immune
network which is extracted from an immune network
to solve problems that arise from autonomous micro­
robots to achieve a given goal with limited memory andcalculation capacity. This method is implemented andtested with the micro-robot 'Kity ', the size of which isless than 1 inch 3 • It is possible to generate enoughrules to make the robot achieve the goal to navigate
freely in a maze with a very small number of sensors.
Experimental results show that the immune network
can be used to control a robot in a restricted environ­ment. Kity demonstrated its efficient algorithm by tak­ing the 1st prize at the 4th International Micro Robot
Maze Contest held in Nagoya, Japan on October 6,
1995.

1

to goal point via five control points in the maze. The
key technology is to implement an autonomous micro­
robot sized less than 1 inch 3 for the achievement of agiven objective with limited memory and calculation
capacity.
The algorithm is implemented and tested with themicro-robot Kity, the size of which is less than 1 inch 3 .
With a very small number of sensors it is possible togenerate enough rules to make the robot achieve theobjective to move freely in a maze that consists ofchannels and nodes.

Introduction

Since the first mechanical devices were invented
centuries ago, the size of the parts was significantly
reduced and the precision increased tremendously.
Miniaturization of real life objects has always beena fascination for the researchers. Small models exist
for everything. Powering these small objects or giv­ing them some autonomy or even intelligence is stilla challenge left for future generations. Small mecha­
nisms are machined and assembled by different tech­niques called precision engineering, micro-mechanics
or micro-mechatronics, with this last word being used
when some control electronics is embedded into thesystem.
In this paper, a table-look-up immune network is
proposd for the proper behaviors of the autonomous
micro-robot in a maze. The table consists of antibod­
ies (behaviors) extracted from an immune network[l]to solve a problem of finding a way from start point

Fig. 1. Kity in the maze

The immune system, in particular, has various
interesting features such as immunological memory,
immunological tolerance, micro-pattern recognition,
non-hierarchical distributed structure, and so on fromthe standpoint of engineering. In addition to this, re­cent studies on immunology have clarified that the im­mune system does not only detect and eliminate thenon-self materials called antigens such as virus, cancer
cells and so on which come from inside or outside ofthe living system, but play important roles to main­
tain its own system against dynamically changing en­
vironments. Therefore, the immune system would be
expected to provide a new paradigm suitable for dy-- 261 -

namic problem dealing with unknown environments
rather than static problem. So far, several methods
have been proposed to realize phenomena of the im­mune system from standpoint of mathematical analy­
sis. However, the immune system has little been ap­
plied to engineering fields in· spite of its productive
characteristics. Based on the above facts, Ishiguro
et. al. [2] studied the immune system for the engi­
neering purpose, and applied it to the mobile robot.
They expected that there would be an interesting AI
technique for dynamically changing environment by
imitating the immune system in living organisms.
Experiment results show that immune network canbe used to control a robot in a restricted environment.
Kity demonstrated its efficient algorithm by winning
the 1st prize at the 4th International Micro Robot
Maze Contest held in Nagoya, Japan on October 6,
1995.

2

Description of Kity

Figure 2 shows the structure of Kity, an . au­
tonomous micro-robot of less than 1 inch 3 in size.

contact sensors

24.7mmLight weight of Kity makes it critical to choose ade­quate contact sensors that operate under very weakforce. The wheels are geared to the DC motors withthe gear ratio 9 : 100. Because only 'on-off' control
is possible in the system, Kity can go forward, turnleft, and right, but can take no backward motion.

3

Immune Networks

Given rich chemical environment present in a highly
evolved organism, it is inevitable that foreign organ­
isms will attempt to invade in an effort to make use of
these resources. To counteract this, in vertebrates theimmune system has evolved to identify and dispose offoreign materials. This is done in part by antibody
molecules that tag foreign materials and mark themfor eventual removal by lymphocytes, phagocytic cells,
and the complement system. The specialized portionof the antibody molecule used for identifying othermolecules is called the antibody combining region orparatope. The sequence of amino acids making up theparatope determines its shape and hence the set ofother molecules that it can react with. If the shapeof an foreign antigen molecule matches that of theparatope, the antibody can attach itself to the anti­gen, leading to its eventual demise. The regions on any
molecule (antigen or antibody) that the paratopes can
attach to are called epitopes. The set of all paratopes
is like a very large collection of keys, and the set of all
possible epitopes a very large collection of locks. Re­
cent studies on immunology have clarified that eachtype of antibody has also its specific antigen determi­
nant called idiotope•

antigen

DC motors

\I ,

CPU

epltopeImmune NetworkJ;paratope

Fig. 2. The structure of Kity.

It consists of two contact sensors, two DC motors,
two 3-Volt lithium batteries, and an i8051 compati­
ble micro-processor [3]. An i8051 processor is one ofwidely used Intel's 8 bit micro-processors for control­ling small machineries. The compatible chip has two
kilo bytes of flash memory that makes it easy to in­stall the code. To control Kity only two digital control
signal outputs to the DC motors and two digital input
channels to receive signals of contact sensors are used.

idiotope-- Stimulation

Fig. 3. Basic concept of immune network.

Based on this face, Jerene who is an immunologist
proposed a remarkable hypothesis idiotypic network
hypothesis [4]. This hypothesis explains that antibod­
ies do not exist independently in living organism, but- 262 -

communicate with each other th�ough idiotope andparatope.
This idea is schematically shown in Figure 3.
These mutual interactions (stimulation and suppres­sion) chains between different species of antibodies
form the large-scale closed chain loop which works as
a self and non-self recognizer. The important point ofJerne's idea is that the immune system does not act asa unit-level recognition system but as a system-level
recognition system.

4

□□
□,□
r7D □ □□
(a)(b)

Fig. 5. Trajectories for turning a comer and traversing in anode.

Application of Immune Network to
Kity Control

A a table-look-up immune network is employed tosolve problems that arise to achieve a given goal of
an autonomous micro-robot. The algorithm is imple­
mented and tested with the micro-robot Kity. Witha very small number of sensors it is possible to gener­
ate enough rules to make the robot achieve the goal
to navigate freely in a maze that consists of channels
and nodes.

•

Goal

•

Start

Fig. 4. Maze for 1 inch 3 robot.

Fig. 6. Numbered orientation for antibody.

turned a corner or has not yet. Figure 5 (b) shows agoing straight motion in a node. If Kity has a goodcommand of these two motions, it is able to navigate
any path in the maze by just inputting a sequence oftraversing a node, turning left at a node, or turning
right at a node. Here, let us define Mode 1 as a motionof traversing a node and Mode 2 as a motion of turning
at a node. For these motions, antibodies are defined
as in Figure 7 and Figure 8, respectively. Orientationof Kity was defined and denoted by. numbers as in
Figure 6. The number of antibodies used for Mode 1
is seven, and that for Mode 2 is nineteen. The longer
the antigen, the smoother Kity's motion at a node in
Mode 2.

Figure 4 shows the maze in which Kity should run.
Control points are where Kity should pass by at least
once. Because of limitation of the structure due tothe size, we mainly focused on solving the following
problems for Kity to navigate any path in the mazewith only the 2 contact sensors.

! L!R ! 3 ! 4 ! 5 !• ! 1 !
11

Io Io Io Id o Io 11111.-on I o I • 1 o Io Io I 1 Io 1

llllbadyZ

1 ° I 1 I O I O I 1 I O I O 1 .... .-... I • I o I o I , IO I O IO 1..-.i,3

I 1 IO IO I 1 IO IO IO I lollm-on I O I I I O I O I 1 I O IO I
onllbo6f4• Turning a corner; left or right at a node.

I O I1 I O I O IO I 1 I O I right,_°" I • I O I O I O I 1 I O I O I
antibody&• Traversing a node.
• Percepting when it is stuck by the wall becauseof the rough terrain on the playground.

Fig. 7. Antibodies for traversing a node.

The first two problems are graphically describedin Figure 5 and 6. Figure 5 (a) shows a cornering
motion. In this motion, Kity should check if it has

The last problem is an additional one. Because of
the discrepancy of the center of wheels, it is liable
to occur when Kity runs on a cha,nnel with a rough- 263 -

terrain. The problem should be considered in mind
when attending a contest.
Figure 7 shows some examples of antibodies fortraversing straight and Figure 8 for turning left. Simi­
larly, antibodies for turning right can be defined. Two
sensor.information and the orientation of Kity are used
for each antigen. L indicates the left-hand side sen­sor detects a wall, R indicates the right-hand side one
does, and each element of the rest parts is a corre­
sponding orientation. The symbol # denotes 'don't
care' condition. The behavior of antibody of maxi­
mum concentration is chosen as a proper one.

.,,_

IL 1+:J-11 ° 11 l•l•l •I •I •I 7 1

I

5

I

lo l•lo jo lo lo lo jo j oj ol ol 1 rigtnm-on lo!•§ jo jo lo l ojojoj oj 1 ol
l o I•lo lo lo lo lo lo I ol 1 I ol ol rigtnm-on lo I• f jo jojojojoj1 I oj oj o lan1body3

an1ibody4

Conclusion

Limitation in size of micro-robots makes it difficult
to realize them as autonomous ones. In this paper,to achieve an objective under a restrictive environ­ment the immune network has been used for an au­
tonomous micro-robot. By making a look-up-table ofthe network results, the calculation time could be re­duced. This table-look-up method makes it easy to
implement in a small size micro-robot like Kity. Kity
ran the shortest path in the maze with this rule ta­
ble by 52 seconds at the contest, which gave the firstwinner place to Kity.

en1ibody2

11 1° § 1° 11 1° I 0 1° 1°1 °1 °1 °1 ,...,_..,,

corner completely. This confirmation of having turneda corner is very important because Kity doesn't havea precise encoder or odometer. These points at which
the confirmation has been done are used as internallandmarks; Kity compensates its accumulated error
at these points. It was possible to generate a look-up­
table that describes the sensor input and output toactuators. With this table, the calculation time could
be reduced.
Figure 9 shows an experimental result of Kity in amaze. Kity's trajectory might be almost perfect one
that a micro-robot can have.

1 ° I•� 1° 1° 1 1 1 ° 1 ° 1 °1 °1 °1 °1

Fig. 8. Antibodies for turning left.

References[l] J. D. Farmer and N. H. Packard, "The im­mune system, adaptation, and machine learning,"
Physica 22D, pp. 187-204, 1986.
[2] A. Ishiguro, T. Kondo, Y. Watanabe and Y.
Uchikawa, "An Immnunological Approach to Dy­
namic Behavior Control for Autonomous Mobile
Robots," IEEE Con/. IROS, pp. 495-500, 1995.

Start

Goal

[3] S. K. Park, i8051 programming in C, Ohm pub­lishing company, 1995.

Fig. 9. Kity's Trajectory.

When Kity navigated in Mode 1, the concentrationof antibodies for running straight was increased suchthat it could keep Kity run straight. After traversing
a node and when reaching another one, it changes themode according to the input sequence of Mode 1 andMode 2. It was observed that when Kity ran in Mode
2 at a node, the concentration of antibodies for rota­tion was increased (at point A in Figure 5 (a)) andaft.er entering a channel (at point B in Figure 5 (b))
the concentration of antibodies for going straight wasincreased. This makes Kity notice that it has tµrned a- 264[4] N. K. Jerne, "Idiotypic networks and other pre­conceived ideas," Immunological Rev., Vol.79,
pp.5-24, 1984.

Towards Designing Artificial Neural Networks by Evolution*
Xin Yao and Yong Liu
Computational Intelligence Group, Department of Computer Science
University College, The University of New South Wales
Australian Defence Force Academy, Canberra, ACT, Australia 2600
Email: xin@csadfa.cs.adfa.oz.au

Abstract

1

Designing artificial neural networks (ANNs) for dif­ferent applications has been a key issue in the ANN
field. Although there are many training algorithms
available for learning ANN's connection weights, algo­rithms which can learn ANN's architectures are rela­
tively few. At present, ANN design still relies heavily
on human experts who have sufficient knowledge about
ANNs and the problem to be solved. As ANN's com­
plexity increases, designing ANNs manually becomes
more difficult and unmanageable. Simulated evolution
offers a promising approach to tackle this problem.
This paper describes an evolutionary approach to de­sign ANNs. We call these ANNs designed by the evo­lutionary process as evolutionary ANNs (EANNs). In
other words, EANNs refer to a special class of ANNs inwhich evolution is another fundamental form of adap­
tation in addition to learning (refers to weight train­
ing here). We use an evolutionary algorithm similar toevolutionary programming (EP) to evolve both archi­
tectures and connection weights (including biases) of
ANNs. Five mutation operators have been proposedfor our evolutionary algorithm. In order to improve
the generalisation ability of evolved ANNs, these fiveoperators are applied sequentially and selectively. We
have also used validation sets in our studies to fur­ther improve generalisation. The evolutionary algo­rithm allows ANNs to grow as well as shrink during
the evolutionary process. The evolutionary algorithm
also incorporates the weight learning process as partof its mutation process. In a sense, our EANN system
is a hybrid evolution and learning system. Extensive
experimental studies have been carried out to test our
EANN system. This paper will give some of the ex­perimental results which show the effectiveness of thesystem.

Designing artificial neural networks (ANNs) throughsimulated evolution has been investigated for many
years [1, 2, 3, 4]. It offers a very promising and au­
tomatic alternative to designing ANNs manually. The
advantage of the automatic design over the manual
design becomes clearer as the complexity of ANNs
increases. Manual design of ANNs requires the de­signer to have very good knowledge in both ANNs andthe problem to be solved by the ANN. However, suchknowledge is often unavailable for a non-ANN-expert
facing a real-world problem.
Evolutionary artificial neural networks (EANNs)[3, 4] refer to a special class of artificial neural net­
works (ANNs) in which evolution is another funda­
mental form of adaptation in addition to learning.
They provide a general framework for investigating
various aspects of simulated evolution and learning,
including the automatic design of ANNs. The generalframework ff1ake it clear where the automatic designof ANNs fits into the whole picture of a general adap­
tive system which can change its behaviours throughchanging its "hardware", i.e., weights and architec­tures, and its "software", i.e., learning rule. We will
review such a general framework [5, 6] in Section 2.
Then we discuss the issue of designing ANNs throughsimulated evolution in Section 3. Section 4 presents
our latest evolutionary system which can evolve ANN's
weights and architectures at the same time. Finally,
Section 5 gives our conclusions and indicates some fu­
ture research topics.

*This work is supported by the Australian Research Council
through its small grant scheme.

2

Introduction

A General
EANNs

Frameworkfor

Evolution can be introduced into ANNs at various
levels. At the lowest level, evolution can be intro-- 265 -

duced into weight 1 training, where ANN's weights are
evolved. This evolutionary process is similar to thelearning process in the connectionist paradigm where
weights are adjusted in order to learn certain func­
tions. At the next higher level, evolution can be in­troduced into ANN's architecture adaptation, where ·the architecture is evolved, rather than designed byhuman beings. At the highest level, evolution canbe introduced into ANN's learning rule, i.e., the rule
which specifies how to adjust weights in weight train­ing. Since the weight training has traditionally been
regarded as a learning process, the evolution of learn­
ing rules can be regarded as a process of learning tolearn (weights). A general framework ofEANNs which
includes the above three levels of evolution is given in
Figure 1 [3].
THE EVOLUTION OF LEARNING RULES

evaluation of learning rules

THE EVOLUTION OF ARCHITECTURES

tectures than that about their learning rules or a par­ticular class of architectures is pursued, it is better to
put the evolution of architectures at the highest level
because such knowledge can be used to reduce the (ar­chitecture) search space and the lower level evolutionof learning rules can be more biased towards this kindof architectures. On the other hand, the evolution oflearning rules should be at the highest level if there ismore prior knowledge about them available or there isa special interest in certain type of learning rules.
Figure 1 provides us with a common framework fordiscussing various EANN models if we interpret thesimulated evolution in a broader sense, that is, if weinterpret simulated annealing (SA), gradient descentsearch, exhaustive search, etc., as special cases of evo­lutionary algorithms. For example, the traditional
back-propagation (BP) network can be considered asa special case of our general framework with one-shot(only-one-candidate) search used in the evolution ofarchitectures and learning rules and the BP algorithm
used in the evolution of connection weights. In fact,
our general framework defines a three-dimensional
space where O represents one-shot search and +1 rep­resents exhaustive search along each axis. Each EANN
model corresponds to a point in this space, where thethree coordinates represent the three algorithms usedby the EANN model in searching for the weights, ar­chitecture and learning rules. In this paper, we will
describe an EANN model which uses one-shot algo­rithm to search for the learning rule, a variant of evo­lutionary programming (EP) to search for the archi­
tecture, and a hybrid BP-SA algorithm to search for
the weights.

3

reproduction of learning rules

Figure 1: A general framework for EANNs.
There have been some discussions on whether theevolution oflearning rules is at the highest level among
the three in Figure 1 [3, 4]. From the point of viewof engineering, the decision on the level of evolution
depends on what kind of prior knowledge is available.
Ifthere is more prior knowledge about EANN's archi1 Weights include connection weights and biases.

Automatic Design of ANNs
Through Evolution

The evolution of architectures provides an automatic
way to design ANNs. It is obvious from the generalframework in the previous section that any algorithm
can be used to implement such simulated evolution.
There are constructive and pruning algorithms whichlearn ANN's architectures and weights, but they are"susceptible to becoming trapped at structural ·1ocal
optima" [7]. In addition, they "only investigate re­stricted topological subsets rather than the complete
class of network architectures" [7]. Evolutionary al�
gorithms, like genetic algorithms, EP, and evolution
strategies, are better suited to the task of evolving
ANN's architectures [8].
Evolving ANN's architectures by evolutionary algo-- 266 -

rithms is not an easy task. The many to many map­ping between genes and phenotypes has caused two
major problems in evolving ANNs. The first problem
is the noisy fitness evaluation problem. That is, wenormally use one phenotype's fitness to approximate
its genotype's fitness. For example, the fitness of a
trained ANN (from a random set of initial weights) isoften used to represent the fitness of the ANN's archi­
tecture. Such evaluation of the architecture is noisy
because it depends on the random initialisation andthe training algorithm used. One genotype may have
many different phenotypes.
The second problem is the well-known permutation
problem (or competing conventions problem). Thatis, one phenotype may have many different genotypes.
For example, two strictly-layered feedforward ANNs
which order their hidden nodes differently are func­
tionally equivalent, but may have different genotypic
representations.
In order to alleviate the above two problems in
the evolutionary design of ANNs, we have decided to
evolve the architecture and the weights at the sametime using an EP algorithm. The simultaneous evolu­tion of the architecture and weights means that eachindividual in the population is an ANN with the archi­
tecture and weights. Its evaluation is more accurate
because we are evaluating an ANN with the architec­
ture and weights. We do not use the fitness of an ANN
with the architecture and weights to represent that of
the architecture. The use of an EP algorithm avoids
crossover operators which will not be effective due tothe permutation problem.

4

A New System for Designing
ANNs- EPNet

We have developed a new evolutionary system for de­signing ANNs automatically. The main structure of
the system, EPNet, is given in Figure 2.
The EPNet system is built upon an EP algorithm
which adopts a rank-based selection scheme [9] andfive mutations; hybrid training, node deletion, con­nection deletion, connection addition and node addi­
tion [10, 11, 12]. Hybrid training is the only mutationin EPNet which modifies ANN's weights. It is basedon a modified BP (MBP) algorithm with an adaptive
learning rate and an SA algorithm. The other four mu­
tations are used to grow and prune hidden nodes andconnections. Only feedforward ANNs are consideredin EPNet at present.
The number of epochs used by MBP to train eachObtain the newgeneration
Further training

Figure 2: The main structure of EPNet.
ANNs in a population is defined by two user-specified
parameters. There is no guarantee that an ANN will
converge to even a local optimum in a generation.
Hence this training process is called partial training.
The five mutations are attempted sequentially. If
one mutation leads to a better offspring, it is regarded
as successful. No further mutation will be applied.
In other words, only one of the five mutations will beapplied each time. The order of deletion first and addi­
tion later encourages the evolution of compact ANNs.
EPNet puts a lot of emphi,Sis on the behavioural links
between parents and offsprings. It deletes and adds
connections probabilistically according to their impor­tance in the ANN. Node deletion is done at random,
but node addition is achieved through splitting an ex­
isting node.
In order to improve the generalisation ability of
evolved ANNs, we have used two validation sets in
EPNet. The first one is used to compute the fitnessof each ANN. After the evolutionary process, all the
ANNs in the final population will be trained further by
MBP on the combined training and the first validation
set. The ANN which performs the best on the second
validation set wiU be the final output of EPNet.
We have tested our EPNet on a number of artificial
and real-world data sets, including the parity problem[10, 12] of various sizes, four medical diagnosis prob­
lems [11], and the Australian credit card assessment
problem [12]. We also tested our EPNet on anothervery difficult artificial problem - the two-spiral prob­
lem. The result of a typical run by EPNet on the- 267 -

[3] X. Yao, "Evolutionary artificial neural net­
works," International Journal of Neural Systems,
4(3):203-222, 1993.

problem is shown in Figure 3.
"""
"""

....

'

[4] X. Yao, "Evolutionary artificial neural networks,"
In A. Kent, J. G. Williams, and C. M. Hall,
editors, Encyclopedia of Computer Science andTechnology, volume 33, pages 137-170. Marcel
Dekker Inc., New York, NY 10016, 1995.

.

(a)[5] X. Yao, "Evolution of connectionist networks,"
In T. Dartnall, editor, Preprints of the Int 'l
Symp. on AI, Reasoning f3 Creativity, pages 4952, Queensland, Australia, 1991. Griffith Univ.

(b)

Figure 3: (a) The training data set of the two spiralproblem with 194 examples. (b) The decision region
formed by the ANN evolved by EPNet. There are 14
hidden nodes and 131 connections in the ANN.

5

[6] X. Yao, "The evolution of connectionist net­
works," In T. Dartnall, editor, Artificial Intelli­gence and Creativity, pages 233-243. Kluwer Aca­
demic Publishers, Dordrecht, 1994.

Conclusion

This paper addresses the issue of automatic designof ANNs. It is argued that the evolutionary ap­
proach offers a very promising and competitive alter­
native to designing ANNs manually or by a construc­tive/pruning algorithm. The evolutionary design ofANNs is discussed in a general framework of EANNs,
which provides a common basis for comparing and in­vestigating various EANN models. A new evolution­
ary system for designing ANNs, EPNet, is described.
Experimental studies were carried out for a number ofproblems. This paper reports the result on the well­known two-spiral problem. Our future work includes
parallelisation of the EPNet system and improvementof the hybrid training since it is the mutation which
consumes most of the computation time.

References[1] J. D. Schaffer, D. Whitley, and L. J. Eshelman,"Combinations of genetic algorithms and neuralnetworks: a survey of the state of the art," In
D. Whitley and J. D. Schaffer, editors, Proc. of[7] P. J. Angeline, G. M. Sauders, and J. B. Pol­lack, "An evolutionary algorithm that constructs
recurrent neural networks," IEEE Trans. on Neu­ral Networks, 5(1):54-65, 1994.
[8] G. F. Miller, P. M. Todd, and S. U. Hegde,"Designing neural networks using genetic algo­rithms," In J. D. Schaffer, editor, Proc. of theThird Int'[ Conf. on Genetic Algorithms andTheir Applications, pages 379-384. Morgan Kauf­mann, San Mateo, CA, 1989.
[9] X. Yao, "An empirical study of genetic operatorsin genetic algorithms," Microprocessing and Mi­
croprogramming, 38:707-714, 1993.
[10] Y. Liu and X. Yao, "A population-based learn­
ing algorithm which learns both architectures
and weights of neural networks," In Proc. of
ICYCS'95 Workshop on Soft Computing, pages
29-38, Beijing 100088, P. R. China, 1995. Na­
tional Research Center for Intelligent Computing
Systems. To appear in Chinese Journal of Ad­
vanced Software Research (Allerton Press, Inc.,New York, NY 10011), Vol. 3, No. 1, 1996.
[11] X. Yao and Y. Liu, "Evolving artificial neural net­
works for medical applications," In Proc. of 1995
Australia-Korea Joint Workshop on Evolutionary
Computation, pages 1-16. KAIST, Taejon, Korea,

the Int'[ Workshop on Combinations of Genetic
Algorithms and Neural 1Vetworks (COGANN-92),

pages 1-37. IEEE Computer Society Press, Los
Alamitos, CA, 1992.
[2] X. Yao, "A review of evolutionary artificial neu­
ral networks," International Journal of Intelligent
Systems, 8(4):539-567, 1993.

September 1995.
[12] X. Yao and Y. Liu, "Evolutionary artificial neuralnetworks that learn and generalise well," Sublllit­
ted to the 1996 IEEE International Conference
on Neural Networks, Washington, DC, USA. 3-6
June 1996.

- 268 -

Deadlock Resolution Using Hand-to-Hand Motion
in Group Robots
Hidenori ISHIHARA, Toshio FUKUDA, Fumihito ARAI and Naohiro HIRAOKA
Department of Micro System Engineering, Nagoya University
Furo-cho, Chikusa-ku, Nagoya 464-01, JAPANE-mail: ishihara@mein.nagoya-u.ac.jp
Abstract: This paper deals with an approach forresolving deadlock conditions in the case of carrying
loads by Distributed Autonomous Robotic System(OARS). Deadlock condition sometimes occurs when alarge number of robots move and carry loads
autonomously. Therefore, we propose an algorithm toresolve deadlock conditions by cooperative Hand-to-Hand
Motion based on only partial local information.
Through the numerical simulation, we show theeffectiveness of Hand-to-Hand Motion and consider
influences of environment.
1. Introduction

Recently, robots become playing important roles invarious fields such as industrial robots. As evolving therobotic systems, they become to be given complex
tasks. and multiple robots are applied to the several
situations. However, present robots have specificfunctions for some specific aims and lack extendibilityor flexibility to different situation from the initial
purposes at designing. The reasons are considered to be
why robots are connected by the fixed order, and why the
specific manager controls the whole robots centrally.
Centralized control is the good control method in theminority group, but it is not suitable for the system
which consists of large number of the robots because themanager must handle so much information of wholesystem that it is impossible for the manager to control
the whole system.
Recently, the distributed
autonomous robotic system (OARS) has been proposed
as a system to solve those problems! 1 1-1 7 1_ The OARS
aims at developing the intelligent system which consistsof multiple autonomous robot existing independently
each other. and which improves its efficiency according
to the cooperation of constructing robots.
We have proposed the Cellular Robotic System(CEBOT), which consists of the single-functional units,
namely cell, and can dynamically reconfigure its
structure by the connection and detachment of cells toadapt to the given circumstances or the taskslRJ -1 13 1. The
schematic drawing of CEBOT is shown in Fig. I.
We have built and tested the CEBOT Mark I-V
experimentally, and then we have evaluated the several
system architecture such as the optimal reconstruction of
the structure and the optimal distribution of knowledge
allocation1 10 1-1 13 l_ Through the previous studies, thoseresults clarify that the moving cells are important for thereconstruction and other functions. One of those reasonsis that the moving cell can play several roles such as tocarry other non mobile cells, and to expand the working- 269

area according to the movement by its sensing
capabilities. In the industrial fields, unmanned transfer
vehicle or Automated Guided Vehicle was introduced as
the mobile robots. In future, the increasing number ofsuch mobile robots from now on can be considered to be
used densely in a system. In such a system with mobile
robot, the deadlock avoidance and resolution areimportant problems to control them smoothly. In theprevious studies on the deadlock avoidance andresolution, the several methods of the single mobile
robot have been reported, but only a few methods for
multiple mobile robots have been proposed1 14 1-1 16 1. In
the OARS with the mobile robots, when the mobile
robots carry the loads as their task, the deadlock
condition of carrying the loads is easily considered tohappen often. Especially, when two robots with loads
come across on the narrow path, it is difficult to solve
such a deadlock condition. At that situation, it isconsidered to be effective in solving such a situation thateach robot changes its direction to move according tohanding over its load each other. Therefore, to resolve
deadlock condition of carrying loads, we take notice ofthe motion to hand over the loads from one robot to theother, namely Hand-to-Hand Motion, and propose theHand-to-Hand Motion strategy the as a method of thedeadlock resolution.

Fig. I Concept of Cellular Robotic System (CEBOT)2. Deadlock Problem and Deadlock
Resolution by Hand-to-Hand Motion

In the OARS including the CEBOT, since eacli
robot does not have overall global information ari-i
decides its behavior according to the partially local
information, there appear various problems ofcommunication, cooperation, competition, and collision
avoidance, etc. When we consider the task to carryloads. for example, several deadlock conditions such as
carrying of load and moving of the robots occur. If eachrobot does not accept to perform cooperatively, robots
can not clearly solve such deadlock situations. Becauseof that, the deadlock resolution requires for the robots towork the tasks cooperatively.
At the first, we define the environment and thetasks for the group robots. Several situations which thedeadlock condition happens are considered, but since it isdifficult to make the algorithm to solve all the deadlock
condition, we propose a method to solve them in thelimited situation. As the workspace, we assume thefollowing environment shown in Fig. 2:( I ) workspace consists of narrows paths where robots
can take over.
(2) there are two goals to be carried loads,(3) there are two types of loads, and different types ofloads are carried to the different goal.
In the defined environment shown in Fig. 2, several
deadlock conditions are considered to happen.

pileup load problem can be solved by handing the load
over from one robot with load to the other robot
without. Then, at the end of the queue, robots whichfinished to hand its load over to the other generally
become able to move, and the deadlock of movement canbe also solved.
Next; the case 'b' is to be considered. This casepresents that facing two robots moving in the different
direction on the narrow path are the obstacles each otheron each way ahead. The reason is that each robot tends
to go toward the opposite direction of their goals, andwhen they exchange their loads, the problem will besolved.
In those cases, handing over the loads is considered
to be a way effective to solve the deadlock condition
happening in such environment as in Fig. 2. Therefore,
this study proposes the Hand-to-Hand Motion, whichmeans the motion of handing over the loads each other,as a method of deadlock resolution. Figure 3 is theschematic drawing of hand-to-hand motion under thesituation in Fig. 2.

Load to Goal B

Fig. 3 Concept of Hand-to-Hand Motionb

C
Goal B

Except above three cases, the deadlock situation that
two robots try to hand over the load to a robot at thesame time is easily considered to occur. In such asituation, it is considered to be effective to form a one­
to-one relation between robots to hand and to be handedabiding by some agreements (i.e. protocol) using
communicationl121 . Because of that, the algorithm ofHand-to-Hand Motion using communication is defined
as followings, and is shown in Fig. 4:[Step I] a robot, namely robot A, detects the deadlock
condition based on only the partial local
information,[Step 2] robot A look for the other robot, namely
robot B, which can be transferred the loads,[Step 3] robot A informs robot B of "I would like totransfer you the load,"[Step 4] robot B receives these information,[Step 5] robot B informs robot A of "I am ready,"[Step 6] robot A receives these information,[Step 7] robot A hands over his load to robot B.

Fig. 2 Environment for the robots
First, take notice of the cases, 'a' or 'c' in Fig. 2
Under the condition 'a,' many robots move towards one
goal, and robots around the goal get deadlocked by othercoming robots even after they completed their tasks,
because the goal A can be approached from more than
two directions. When it takes much time to put the loadoff, a queue as the case 'c' is formed. In both cases,
these queues include some robots not only with a load
but also without a load. Within such a situation, the- 270 -

Time

Robol Aother the position, the IDs, the holding of load, and the
direction to go. Then each of them calculates its orderof priority to cross the intersection1 13 1 . The order ofpriority is decided as follows:( l) Robot crossing the intersection is of the highest
priority.
(2) Robot which come across no robots in his way
comes next.
(3) Robot with load does prior to that of without.
(4) Then the priority comes on robot of younger ID.
The idea to arrange this order of priority of crossing
the interaction is the same as semaphore, but is deferent
from it at the point that there's no guarantee of crossing
because the flow is not a single direction.
The following equations are used as the detection ofdeadlock.

Robot B

Transn,ission"Willin� to h;\1111-to-h:md"nReception(I),

Hand-to-hand motionI22

dk = ((xk-Xgoa1) + (Yt-Ygoa1) )Fig. 4 Flowchart of Hand-to-Hand Motion
3. Numerical Simulation

Numerical simulation is carried out for three cases;
the case using robots with hand-to-hand motion, the casewithout hand-to-hand motion but on the same basic
rules. and the case based on heuristics. Detail is given
as follows. Workspace is set up as in Fig. 2, and more
complex situations such as in Fig. 6 are used for thesimulation. Robots can not pass each other as we see
from the figure. The task is to carry the lower right loadto the upper left goal A and the upper left load to thelower right goal B.

2

(2),

where, <ik provides the distance between a robot and thegoal, when (.>.J, n) is the robot's position at that time,
k steps, and (Xgo al, Ygoa f) is the position of the goal.
In the Eq. ( 1 ), (dk-1 - dk) represents the approacheddistance to the goal during the time from k steps to (k-/)
steps, and becomes minus as coming close to the goal.
Under the deadlock condition, since robot does not moveat all, the distance of movement within 11 steps is zero,that is f = 0, and so the deadlock can be detected. Since
11 is the memorable number of steps by each robot, we
define it as 11=5. This number is defined after repeatedtrial and error.

3.1. Assumptions of Simulation
The following conditions are assumed to thesimulations:(I) each robot can moves toward 4 directions (x. -x, yand -y),(2) each robot know the absolute position in theenvironment,(3) each robot has the identification number (ID),(4) each robot does not know the map of a wholeenvironment, but know the position of goals,(5) it takes no time to communicate between robots,(6) each robot can recognize the intersection, and passes
it by the rule.
3.2. Principle of Hand-to-Hand Motion
We define the rules of basic behavior as follows:•A robot with load in deadlock conditions first looks foranother robot to hand over the load to and then hands
over to it.
•A robot without load performs random search.
•A robot reaching at the dead end moves along by theleft wall.
Each robot sees whether any other robots arecoming into the intersection when it comes just before
it. If there are other robots, they all communicate each

- 271 -

3.3. Heuristics
Considering from the environment of thesimulation and the task, there appears a heuristic rule asfollowing:
A robot in the area y-x<O divided by the diagonal as
shown in Fig. 5 moves toward x or y direction, and arobot in the area x-y<O moves toward --x or -y direction.
This rule makes a path just as an elliptical orbit throughthe two goals on the workspace.

---o

o-

t

6

y - X <0y- X > 0y=xFig. 5 Rules of Heuristics

f

750

750

500

500

ti)

__ __ _______,

250 .__

0

0.25

__._

_._

0.5

0.75

0

Occupation ratio of Robot population lo Space

Occupation ratio of Robot population lo Spacet:i) Environment 1-1 (a kind of load)
r -0--0--

IOOO

7(a) Environment 3-1 (a kind of load)-:.:.:.:-"'°

IOOO

/

s_ 750
8

0.75

0.5

0.25

j 750

ti)

500

500

250

250

Occupation ratio of Robot population to Space

0
0.25
0.5
0.75
Occupation ratio of Robot population to Space(b) Environment 1-2 (two kind of loads)
Fig. 7 Simulation results on Environment I(b) Environment 3-2 (two kind of loads)
Fig. 9 Simulation results on Environment 3

0

0.25

0.5

0.75

Table 2 shows the number of steps of finishing thetask at each environment when the number of robots is
150 (the highest occupation ratio of robot population).
It can be said that the Hand-to-Hand motion is effectivewith the large number of robots. Without Hand-to-Handmotion, there appear the cases that the task cannot becompleted even after I 000 steps passed as the number ofrobots increasing.

IOOO

f 750

ti)

500

Table 2 steps completing the task
0

0.25

0.75

0.5

Occupation ratio of Robot populalion to Space

Environment 1(a) Environment 2-1 (a kind of load)
Kinds of Load00···0···-0··· ···IOOO,....,<> .....

s_ 75087___····-,....o
....-

I

2

··-·----

Environment2

Environment32
2
II---·----- -----· ---·---

Withoul Hand-to-handmotion1000 l(J(XJ IIXXl IO(X) HXXl l()(XJ

With Heuristics

712

�-

688

371

---

With I land-to-handmotion------ r-- --•-- ------963 llXXI 1000--- -----·

946 HXXJ
643

666

758

522

500

250

'-----'-----'----ll-_...J

0

0.25

0.5

0.75

Occupation ratio of Robot population to Space(b) Environment 2-2 (two kind of loads)
Fig. 8 Simulation results on Environment 2

In order to estimate the effect of each environment,
three parameters of the environment, the number ofcorridor, average of corridor length, and average length ofpath to goal A, which are shown in Table 3, are defined
as followings:
The corridor is the path form one corner of a blockin the environment to the other corner of it along a- 272 -

This rule makes the path of the robots one-way thatmakes the robots pass near the loads and goals, and thendissolves the deadlock condition such as the case 'a' inFig. 2. For this reason, it is used as the comparison to
the Hand-to-Hand motion we propose.
3.4. Simulation Environment
The environments of the simulations are three typesof 26 x 26--square fields as in Fig. 6. Figure 6--(a) showsa simple environment distributing the path in a square,
while Fig. 6--(b) is arranged to have some longer path,and Fig. 6--(c) some dead ends. The small blocks
forming the obstruction making the path are a basic unitof representing the scale of the environment.
The evaluation of simulation results is given bycomparison of the working efficiencies as changing thenumber of robots in each workspace. The efficiency isestimated by the number of steps to complete the task ofcarrying loads to the goal. The parameters are shown in
Table I. In Table I, six kinds of environments aredefined. Environment 1--1 and Environment 1--2 use thesame map, Fig. 6--(a), but there is a kind of loads in theenvironment 1--1 and 2 kinds of loads in the environment1--2. As same way, two cases are set in both
environments 2 and 3. In the environment including
two kinds of loads, the deadlock condition is considered
to happen more than the other including a kind of loads.
Figures 7, 8 and 9 show the results of simulation.
The horizontal axis in Figs. 7, 8 and 9, the Occupation
ratio of Robot population to Space, is obtained by(number of robots)/ (number of squares of the path). At
the simulation results as shown in Figs. 7, 8 and 9, it
can be stated that the Hand-to-Hand motion strategy
solves the deadlock problem by keeping the ability ofcarrying the loads even when it becomes harder to moveas the robot population increases. While with the small
number of robots, heuristic leads the better efficiency.
The most remarkable change is obtained by theenvironment I with 2 kinds of loads as shown in Fig. 7.
This is the most general workspace though it's rather
simple, and heuristics is effective especially under thisenvironment and the best efficient value with thisenvironment is obtained by the case of heuristics. This
result has some relations to the rule that the Hand-to -­Hand motion is performed after the detection of deadlock.
More over, even under the same environment I, when itis the case with a kind of load, random search costs
wastefully. With the environment 3, extra motion ofadvance along by the left wall at the dead end, thecharacteristic of its statistic data may be of slightly
different kind, event though it can be concluded that thenoticeable difference of efficiencies indicates theeffectiveness of the Hand-to-Hand motion.

(a) Environment I for the simulation:
26 x 26--square without long paths and dead ends�"

qr ��3fib::; :e0 � 0 �..

+H•m...LLJ.

r·t1'1 L..LJ

e•

ev

I

r, LL ),rtfflf
0 0 0 0

: : ,; :�

0

a-

r

0

0

0

(b) Environment 2 for the simulation:
26 x 26--square with long paths

Table l Parameter for the simulations
Environment

l_l, 2_1,3_1

1_2, 2_2,:i_2

Work space

26X26

26X26

·--·- ----------·------ --I 00- -50 X 2---------------+---------- --·--------�----�--Kinds of Load

Number of Load
Robot population2

10, 20, 30, 40, 50, 75, 10, 20, 10, 40. 50, 75,
100, 125,150,200
100,125, 150- 273 -(c) Environment 3 for the simulation:
26 x 26--square with dead ends
Fig, 6 Environment for the simulation

side, while the average length of path to goal A is avalue taking the average of the length of the pathfrom any positions to the goal A. The average of
corridor length and the average length of path to goal
A are calculated based on the basic units in Fig. 6.
Table 3 Environmental Parameters
Environmen! I

Environmen! 2

60

40--· . -----------�--Number of Corridor

Environmen! 3···----------

49
44

-

----�-- - -------------

A vcragc of Corridor length- ·-----

A vcragc length of Path lo

G{,al

4 ()
25

5.25
>---------�--

50

25
44

4.27

4.20

----�-----

26

53

In Table 3, the difference between environment. I
and 2 is remarkable. It may be stated that there is adirect proportional interrelation between the average of
corridor length and the working efficiency. The numberof corridor is closely related with this average of corridor
length, and the number of intersections inevitably
becomes small, when, as environment 2, the average of
corridor length is long and the number of corridor islittle. Thus it becomes difficult to solve the deadlock
problems due to the slow movement of line. This
causes the inefficiency in the case by heuristics which iswithout Hand-to-Hand motion. On the contrary, as
environment l with many short corridors andintersections, less deadlock problems are caused and theresolution can be gotten easier.
From the comparison of environment I and 3, it isfound that environment 3 has smaller number of valid
corridors as it has some dead ends. According to this,the efficiency also becomes lower. From thiscomparison, there also seems to be direct proportional
interrelation between the length of path to goal and theefficiency. If we consider this together with theprevious comparison, at the first sight, it seems to beless influence on the efficiency. However, the difference
between the case of heuristics and the case with Hand-to­
Hand motion is considerable, the value of heuristics isdouble of the value of the Hand-to-Hand motion. Thenit can be read from the Table 2 that the average of thelength of the path influences to the heuristics.
4.

Conclusions

We have proposed the new method of deadlock
solution using the Hand-to-Hand Motion and have done
numerical simulation under several environments.
Through the consideration of the simulation results, it
can be concluded that the Hand-to-Hand Motion to thecarrying loads by autonomous robots is effective forboth the deadlock resolution and the earlier completion
for the task with multiple robots.
References

Mobile Robots Operating in Discrete Space,"
Proceedings of International Symposium on Distributed
Autonomous Robotic Systems (DARS'94), pp.134-161,(1994)[2]R.A.Brooks, "A Robust Layered Control System For
A Mobile Robot," IEEE Journal of Robotics andAutomation, Vol.RA-2, No.I, pp.14-23, (1986)[3]L.E.Parker, "Designing Control Laws for
Cooperative Agent Teams," Proc. of International
Conference on Robotics and Automation (ICRA'93),
pp.582-587, (1993)[4]A.Agah and G.A.Bekey, "A Novel Cognate
Architecture for Simulated Robots in an Artificial
World," Proc. ofICRA'94, pp.2309-2314, (1994)[5]F.Hara and S.lchikawa, "Effects of Population Size inMulti-Robots Cooperative Behaviors," Proc. of
DARS'92, Wako, Saitama, Japan, Sep.21-22, pp.3-9,(1992)[6] Y.Ohmori, Y.Nakauchi and Y.Anzai, "Task
Allocation Algorithm for Cooperation of Distributed
Autonomous Mobile Robots," Proc. of DARS'92,
Wako, Saitama, Japan, Sep.21-22, pp.159-165, (1992)[7] J. Wang, "On Sign-board Based Inter-Robot
Communication in Distributed Robotic Systems," Proc.
ofICRA'94, pp.1045-1050, (1994)[8] T.Fukuda and T. Ueyama, "Cellular Robotics andMicro Robotic Systems," World Scientific Series in
Robotic and Automated Systems Vol. JO, World
Scientific, pp.1-47, (1994)(9] T. Fukuda, " Structure Decision Method for Self
Organizing Robots based on Cell Structure-CEBOT,"
Proc. ICRA'89, pp. 695-700, (1989)[IOJ T. Fukuda, et. al, "Communication Method of
Cellular Robotics CEBOT as a Selforganizing Robotic
System," Proc. ofInternational Workshop onIntelligent
Robots and Systems '89 (IROS'89), Tsukuba, Japan,
Sept.4-6, pp.291-296, (1989)[11] T. Fukuda, et. al, "Concept of Cellular Robotic
System (CEBOT) and Basic Strategies for its
Realization," Computers Elect. Eng. Vol. 18, No. 2,
Pergamon Press, pp. 11-39, (1992)[12] T. Fukuda et. al, "Optimization of Group Behavior
on Cellular Robotic System in Dynamic Environment,"
Proc. ofICRA'94, pp.1027-1032, (1994)[13] A.-H Cai, T.Fukuda, et al., "Hierarchical Control
Architecture for Cellular Robotic System -Simulations
and Experiments-," Proc. of ICRA'95, Nagoya, Japan,
pp.1191-1196, (1995)[14] A. Matsumoto, H. Asama, et. al, "Communicationin the Autonomous and Decentralized Robot System
ACTRESS," Proc. of IROS'90, July 3-6, Tsuchiura,
Japan, pp.835-840, (1990)[15] S.Yuta and S.Premvuti, "Coordinating
Autonomous and Centralized Decision Making to
Achieve Cooperative Behaviors Between Multiple
Mobile Robots," Proc. of DARS'92, Wako, Saitama,
Japan, Sep.21-22, pp.173- I 8 I, ( 1992)[16] J. Wang, "Deadlock Detection and Resolution inDistributed Robotic System," Proc. of DARS'92,
Sept.21-22, Wako, Japan, pp.93-101, (1992)[I]J.Wang and S.Premvuti, "Fully Distributed Traffic
Regulation and Control for Multiple Autonomous- 274 -