(AROB 16th ’11)

Jan.27-29, 2011
B-Con Plaza, Beppu, Oita, JAPAN

Editors: Masanori Sugisaka and Hiroshi Tanaka
Publisher: ALife Robotics Co., Ltd.
Publication Date: Jan. 15, 2011
ISBN 978-4-9902880-5-1

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

Proceedings of the Sixteenth International Symposium on

ARTIFICIAL LIFE AND ROBOTICS(AROB 16th ’11)

January 27- 29, 2011
B-Con Plaza, Beppu, Oita, Japan
Editors: Masanori Sugisaka and Hiroshi Tanaka©ISAROB 2011

2nd CALL FOR PAPERSAROB 17th ‘12
January 19-21, 2012

B-Con Plaza, Beppu, Oita - JAPANHistory
The AROB was founded in 1996 under the support of Science and International Affairs Bureau, Ministryof Education, Culture, Sports, Science and Technology, Japanese Government. Since then, the symposium
organized by the AROB has been held every year at B‐Con Plaza, Beppu, Japan except AROB 5th ’00 (Oita)
and AROB 6th ’01 (Tokyo). The seventeenth symposium will be held on January 19–21, 2012, at B‐Con

Objective
Plenary Speakers
We will upload it to our
website as soon as it has
been decided.

This symposium will bring together researchers to discuss development of new technologies concerning
artificial life and robotics based on computer simulations and hardware designs of state‐of‐the‐art
technologies, and to share findings on how advancements in artificial life and robotics technologies that
relate to artificial intelligence, virtual reality, and computer science are creating the basis for exciting new
research and applications in various fields listed in the following Topics.

Topics of interest include, but are not limited


























Vice Chairs
John L. Casti
Yingmin Jia
Ju‐Jang Lee
Henrik Hautop Lund
Hiroshi Tanaka
















Workshop Chair
Jang‐Myung Lee

and also workshop, tutorial, and robotics exhibition will be held at the same time during the
symposium (AROB 17th ’12). The workshop committee and robotics exhibition committee members
will be announced soon.

Advisory Committee Chair
Fumio Harashima

Important Dates

General Chair
Masanori Sugisaka
Program Chair
Hiroshi Tanaka

Assistant General Chair
Takao Ito

Artificial brain research
Artificial living
Bipedal robot
Cognitive science
Control techniques
Evolutionary computations
Human‐machine cooperative systems
Intelligent control & modeling
Medical surgical robot
Molecular biology
Nano‐robotics
Pattern recognition
Robotics
Visualization






September 1, 2011
September 15, 2011
October 1, 2011
October 15, 2011



Artificial intelligence
Artificial mind research
Brain science
Complexity
Data mining
Fuzzy control
Human‐welfare robotics
Learning
Micromachines
Multi‐agent systems
Neurocomputing technologies
and its application for hardware
Soccer robot

Artificial life
Bioinformatics
Chaos
Computer graphics
DNA computing
Genetic algorithms
Image processing
Management of technology
Mobile vehicles
Nano‐biology
Neural networks
Quantum computing
Virtual reality

Abstract submission date
Notification of acceptance
Author’s kit except final camera‐ready papers
Final camera‐ready papers submission dateConference Language
 English
Publication
Accepted papers will be published in the Proceedings of the AROB. Extended versions of the selected papers will be
published in the international journal：ARTIFICIAL LIFE AND ROBOTICS and in the special issue on special topics ofArtificial Life and Robotics, Appled Mathematics and Computation

All correspondence related to the symposium should be addressed to:
AROB Secretariat
ALife Robotics Corporation Ltd.
1068‐1 Oaza Oshino, Oita, 870‐1121, JAPANTel / Fax: +81‐97‐594‐0181 Email: arobsecr@isarob.org

isarob.org

AROB 17th 2012

Organized By

International Organizing Committee of International Symposium on Artificial Life and Robotics (AROB)
Co‐Operated by
The Society of Instrument and Control Engineers (SICE, Japan)
The Robotics Society of Japan (RSJ, Japan)
The Institute of Electrical Engineers of Japan (IEEJ, Japan)
The Institute of Systems, Control and Information Engineers (ISCIE)The Institute of Electronics, Information and Communication Engineers (IEICE)

Program Chair

General Chair

M. Sugisaka (Nippon Bunri University and ALife Robotics Co., Ltd., Japan)



H. Tanaka (Tokyo Medical & Dental University, Japan)

Workshop Chair

Vice‐Chairs






IEEE Japan Council (IEEE Robotics and Automation Society Japan Chapter)
Japan Robot Association (JARA)
Institute of Control, Robotics and Systems (ICROS, Korea)
Chinese Association for Artificial Intelligence (CAAI, P. R. China)

J. L. Casti (International Institute for Applied Systems Analysis, Austria)
Y. Jia (Beihang University, P. R. China)
J. J. Lee (KAIST, Korea)
H. H. Lund (Technical University of Denmark, Denmark)H. Tanaka (Tokyo Medical & Dental University, Japan)



J. M. Lee (KAIST(Pusan National University), Korea)

Assistant General Chair
T. Ito (Ube National College of Technology, Japan)







M. Kam (Drexel University, IEEE President, USA)
H. Kimura (RIKEN, Japan)
P. Oh (Drexel University, USA)
M. Tomizuka (University of California Berkeley, USA)Y. G. Zhang (Academia Sinica, P. R. China)











M. H. Lee (Pusan National University, Korea)
M. Nakamura (Research Institute of Systems Control, Japan)
M. Oswald (Vienna University of Technology, Austria)
S. Rasmussen (University of Southern Denmark, Denmark)T. S. Ray (University of Oklahoma, USA)K. Shimohara (Doshisha University, Japan)
C. Taylor (University of California‐Los Angeles, USA)J. Wang (Beijing Jiaotong University, P. R. China)
C. Zhang (Tsinghua University, P. R. China)


























K. Naitoh (Waseda University, Japan)
W. Nistico (University of Dortmund, Germany)
N. Okada (Kyushu University, Japan)
S. Omatu (Osaka Institute of Technology, Japan)
H. Oogai (Waseda University, Japan)
L. Pagliarini (Technical University of Denmark, Denmark)R. Pfeifer (University of Zurich‐Irchel, Switzerland)
M. Rozailan (TATi University College, Malaysia)
S. Sagara (Kyushu Institute of Technology, Japan)
P. Sapaty (National Academy of Sciences of Ukraine, Ukraine)
J. Y. Sim (Kangnam University, Korea)
M. Svinin (Kyushu University, Japan)
K. Sugawara (Tohoku Gakuin University, Japan)
H. Suzuki (The University of Tokyo, Japan)
I. Tanev (Doshisha University, Japan)
T. Tsuji (Hiroshima University, Japan)
H. Umeo (Osaka Electro‐Communication University, Japan)
K. Uosaki (Fukui University of Technology, Japan)
K. Watanabe (Okayama University, Japan)
H. Yanagimoto (Osaka Prefecture University, Japan)
I. Yoshihara (University of Miyazaki, Japan)
M. Yokota (Fukuoka Institute of Technology, Japan)
Y. Yu (Kagoshima University, Japan)
T. Zhang (Tsinghua University, P. R. China)M. Sugisaka (ALife Robotics Co., Ltd, Japan) (Chair)
J. Wang (Beijing Jiaotong University, P. R. China)
H. Zhao (Shanghai Institute of Technology, P. R. China)

Advisory Committee Chair

F. Harashima (Tokyo Metropolitan University, Japan)

Advisory Committee






B. Eisenstein (Drexel University, Former IEEE President, USA)
R. Fischl (Drexel University, USA)T. Fukuda (Nagoya University, Japan)
F. Harashima (Tokyo Metropolitan University, Japan)
P. Kalata (Drexel University, USA)International Organizing Committee











H. Abbass (University of New South Wales, Australia)
K. Aihara (The University of Tokyo, Japan)
C. Barrett (Los Alamos National Laboratory, USA)M. Bedau (Reed College, USA)
M. Eaton (University of Limerick, Ireland)
J. M. Epstein (The Brookings Institution, USA)
A. Grzech (Wroclaw University of Technology, Poland)
H. Hashimoto (The University of Tokyo, Japan)
D. J. G. James (Coventry University, UK)
K. Kyuma (Mitsubishi Electric Corporation, Japan)

International Program Committee


























T. Arita (Nagoya University, Japan)
R. Belew (University of California‐San Diego, USA)
S‐ M. Chen (National Taiwan University of Science and Technology, Taiwan)
Y. I. Cho (The University of Suwon, Korea)
M. Chow (North Carolina State University, USA)Y. Y. Fan (University of California‐Davis, USA)T. Fuchida (Kagoshima University, Japan)
H. Hamdan (SUPELEC, France)
S. H. Han (Kyungnam University, Korea)
N. Homma (Tohoku University, Japan)
Y. Ishida (Toyohashi University of Technology, Japan)
S. Ishikawa (Kyushu Institute of Technology, Japan)
T. Iwamoto (Mitsubishi Electric Corporation, Advanced
Technology R&D Center, Japan)
J. Johnson (The Open University, UK)
H. Kinjyo(University of the Ryukyus, Japan)
M. Kono (University of Miyazaki, Japan)
K. Kurashige (Muroran Institute Technology, Japan)
H. H. Lee (Waseda University, Japan)
J. M. Lee (Pusan National University, Korea)
K. J. Mackin(Tokyo University of Information Sciences, Japan)
K. Merrick (University of New South Wales and ADFA, Australia)
C. Mi (University of Michigan‐Dearborn, USA)J. Murata (Kyushu University, Japan)
K. Nakano (The University of Electro‐Communications, Japan)

Local Arrangement CommitteeF. Dai (Tianjin University of Science & Technology, P. R. China)
Z. Ibrahim (TATi University College, Malaysia)
M. Rizon (King Saud University, Saudi Arabia)
S. Sagara (Kyushu Institute of Technology, Japan)

isarob.org

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

Artificial Life and Robotics

We invite you to publish your paperin this journal full of inventive scientific findings and state-of-the art technologies.
Two types of papers from the world are
Ⅰ. Recommended papers presented at the International Symposium on Artificial
Life and Robotics (ISAROB),
Ⅱ. Contributed papers
Contribution fee :4 pages or Under
Extra page charge￥19,950 / page (¥19,000 + 5 % consumption tax: only in Japan)￥16,800 / page (¥16,000 + 5% consumption tax: only in Japan)(Pages are counted on single-line basis as they appear in the journal.)
Shipping Cost:
domestic
overseas￥700 / delivery￥3,000 / delivery

We deliver every corresponding author (or sponsor) one copy of journal in which his/her paper is
included. W e would like to tha nk authors for their support in the form of contribution fee
payable before publication, which funds the pro duction of this journal contracted by Springer
Japan.
Order:
1. Price for co-authors¥ 5,250 (¥5,000 + 5 % consumption tax: only in Japan) per copy
(for every co-author)
If necessary, please order extra copies for co-authors to the Journal Office.
2. Subscription fee¥ 21,000 (¥20,000 + 5 % consumption tax: only in Japan) per year,
3. Purchase price for single issue¥ 8,400 (¥8,000 + 5 % consumption tax: only in Japan) per issue.
AROB Journal Office : joffice@isarob.org©ISAROB 2011

A-1

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

Artificial Life and Robotics

Our Review System

For contributed papers,
1. Duration:
Two months is our tar get time of the whole review process, however, we are trying our best to
make it as short as possible.
2. Review results:
Every contributed paper w ill be reviewed by three as signed reviewers with score ranging from1 (highest mark) to 4 (lowest mark);
1. publish with no revision,
2. publish with minor revisions,
3. possibly publish with revision and re-review,
4. reject.
Review results, a total score of the three evaluation scores, would be read as;.
3 or 4: accepted and to be published as it is,4, 5, or 6
: accepted and to be published after minor revisions,7, 8, or 9
: acceptable but subject to rewriting and re-review,
10, 11, or 12: rejected
For recommended papers,
The recommended papers are ones th at the chair of each ses sion reads before, during, and after
the symposium that year and recom mends to AR OB Journal Of fice using the sam e evaluation
score abov e. Therefore, th e quality of recommended papers is ap parent at th e tim e ofrecommendation.

We express our heartfelt gratitude to chairs and reviewers of ISAROB for theirsupport and sincere voluntary work without which this journal never exists.

AROB Journal Office : joffice@isarob.org

Artificial Life and Robotics©ISAROB 2011

A-2

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

We invite you to advertise your company
in th is in ternational jou rnal f inanced by ISAROB (In ternational Sy mposium o n Artificia l Lifeand Robotics ) and produced by a renowned publisher, Springer Japan.

Merits:・Connectivity with ambitious academic circles in the rapidly-advancing fields,
・Expanded publicity through toll electronic circulation by Springer-Japan,
・Reasonable advertising fee(1 page = ¥ 90,000, 2 pages = ¥ 150,000)(Fee+ 5 % consumption tax: only in Japan)

Please contact AROB Journal Office at ALife Robotics Corporation Ltd.
joffice@isarob.org tel/fax: +81-97-594- 0181

©ISAROB 2011

A-3

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

Intelligent Mobile Robot
Full Version
Features：Fig.1: Full
Version(Taro-1 andTrao-2)

1.Recognizes lines and moves
on or along the line
2. Recognizes the shape ofobjects
3. Can be controlled by voice(more than 20 sentences)
4. Recognizes human faces(approx. 10 people)
5. Can be controlled by theInternet and mobile phone(remote control)
Price: ¥4,000,000 ($48,000)

Size and weight of robot is halfof those shown in Fig.1
Functions of full version 1.~4.
are equipped with:・Fundamental software basedon Windows and Linux, isprovided･The robot is most suitable for
R & D (Masters and Doctorate
courses)

Specs. and style can be changed

Fig.1 Full Version(Taro-1(right),Taro2(left) and Simplified Vesion©ISAROB 2011

Simplified Version (Taro-A)

Price (Only hardware from
¥1,000,000 ($12,000)) with
fundamental software

Fig.2: Simplified
Version (Taro-A)

1068-1 Oaza Oshino, Oita, Japan 870-1121
Tel: +81-97-567-3088 Fax:+81-97-594-0181
URL:alife-robotics.co.jp E-mail ms@aliferobotics.co.jp

ALife Robotics
Corporation Ltd.

Those unit belongs to Australian
Defense Force Academy (ADFA)(
Australia), KAIST (South Korea),
TATiUC (Malaysia), others.

Fig.2 System block diagram

A-4

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

Main Specification of Taro-A・Size and weight : ½

・Weight: 28kg
・Height: 60 cm （from the floor:65cm）, Width: 28cm, Depth: 40cm
・Robot behaviors can be controlled by voice or remote control
・Behaviors of the Simplified Version of the Intelligent Mobile Robot:
1.The robot can understand human voice commands and can reply in a robot
voice . In addition, the robot can provide various information humans.
2. A human asks the robot. “What is this?” and shows an object to the robot. The
robot answers verbally. (Pattern recognition and understanding)
3. The robot can carry various things (Foods, Letters, Newspapers, Other heavymaterials (approximately 25-30kg). The robot can turn around accurately to a
specified degree.
4. The robot can climb a sloop of 15-30 degrees.
5. The robot can teach a child by answering questions from the child in pleasantplaying conditions as a private teacher.
6. The robot can teach English, Japanese, Chinese, and any other languages tohumans.

1068-1 Oaza Oshino, Oita, Japan 870-1121
Tel: +81-97-567-3088 Fax:+81-97-594-0181
URL:alife-robotics.co.jp E-mail ms@aliferobotics.co.jp©ISAROB 2011

ALife Robotics
Corporation Ltd.

A-5

of Full version ofIntelligent Mobile
Robot “Taro”
・We provide
fundamental
software for robot
by Window or Linux･Most suitable forresearch anddevelopment onmobile robot
・Most suitable forhuman being’s
partner
Price JPY 1,000,000($12,000)(*) Design andspecification will bedecided inconsultation withcustomer

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

ALife Robotics
Corporation Ltd.

This unit belongs to Electro-Communication University, Tokyo, Japan

Sales Record (From 2009)
1. ADFA, Australia
2. Electro-Communication Univ., Japan
3. KAIST, Korea
4. TATiUC, Malaysia
5. Others

1068-1 Oaza Oshino, Oita, Japan 870-1121
Tel: +81-97-567-3088 Fax:+81-97-594-0181
URL:alife-robotics.co.jp E-mail ms@aliferobotics.co.jp©ISAROB 2011

A-6

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

ALife Robotics
Corporation Ltd.

Main Specification of Fish Robot

Table 1 Functions of Fish Robot
Switch No.

Functions1Power: ON,OFF

2

Right Turn

3

Left Turn

サーボモータ4

Change of Speed

電子回路

5
6

Change of Swing
Angle of Caudal Fin

7

Random Mode

スイッチ

Comments

LOW, HIGH
Mode

100mm

4 positions

単4電池×4本

32mm

150mm

Fig.1 Fish Robot (Sizes)

Price (¥15,000≈$181)
Included:
1.Hardware and Fundamental Software

Fig.2 Picture of Fish Robot

2. PIC Writer
3.Text Book for Learning Control
Techniques Based on PIC
OPTIONS:C-MOS Camera, Remote Control Unit,
Dorsal, Pectoral, Ventral Fins can beinstalled1068-1 Oaza Oshino, Oita, Japan 870-1121
Tel: +81-97-567-3088 Fax:+81-97-594-0181
URL:alife-robotics.co.jp E-mail ms@aliferobotics.co.jp©ISAROB 2011

A-7

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

THE SIXTEENTH INTERNATIONAL SYMPOSIUM

ONARTIFICIAL LIFE AND ROBOTICS(AROB 16th '11)
ORGANIZED BYInternational Organizing Committee of International Symposium on Artificial Life andRobotics (AROB)(ALife Robotics Corporation Ltd. and Nippon Bunri University (NBU), Japan)

SPONSORED BY
Mitsubishi Electric Corporation Advanced Technology R&D Center
Oita Gas Co., Ltd.
ME System Co., Ltd.
SANWA SHURUI Co., Ltd.

CO-OPERATED BY
The Society of Instrument and Control Engineers (SICE, Japan)
The Robotics Society of Japan (RSJ, Japan)
The Institute of Electrical Engineers of Japan (IEEJ, Japan)
The Institute of Systems, Control and Information Engineers (ISCIE)The Institute of Electronics, Information and Communication Engineers (IEICE)IEEE Japan Council (IEEE Robotics and Automation Society Japan Chapter)
Japan Robot Association (JARA)
Institute of Control, Robotics and Systems (ICROS, Korea)
Chinese Association for Artificial Intelligence (CAAI, P. R. China)

SUPPORTED BY
Kyushu Bureau of Economy, Trade and Industry, Ministry of Economy, Trade and Industry
OITA PREFECTURE
Oita City
BEPPU CITY
The oita chamber of commerce and industry
Oita Joint Industrial Association
Kyodo News
Jiji Press
Oita Godo Shimbunsya
The Asahi shimbun
The Mainichi Newspaper Co., Ltd.
The Yomiuri Shimbun, Seibu
The Nishinippon Shimbun
THE NIKKAN KOGYO SHIMBUN, LTD
Japan Broadcasting Corporation Oita Station
Oita Broadcasting System
Television Oita System Corporation
Oita Asahi Broadcasting co., ltd.

©ISAROB 2011

P-1

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

GENERAL CHAIRMAN
M. Sugisaka (Nippon Bunri University and Alife Robotics Co., Ltd., Japan)

VICE-CHAIRMAN
J. L. Casti (International Institute for Applied Systems Analysis, Austria)
Y. Jia (Beihang University, P. R. China)
J. J. Lee (KAIST, Korea)
H. H. Lund (Technical University of Denmark, Denmark)H. Tanaka(Tokyo Medical & Dental University, Japan)

ADVISORY COMMITTE CHAIRMAN
F. Harashima (Tokyo Metropolitan University, Japan)

PROGRAM CHAIRMAN
H. Tanaka (Tokyo Medical & Dental University, Japan)

ASSISTANT GENERAL CHAIRMAN
T. Ito (Ube National College of Technology, Japan)

ADVISORY COMMITTEE
B. Eisenstein (Drexel University, Former IEEE President, USA)
R. Fischl (Drexel University, USA)T. Fukuda (Nagoya University, Japan)
F. Harashima (Tokyo Metropolitan University, Japan)
P. Kalata (Drexel University, USA)
M. Kam (Drexel University, IEEE President, USA)
H. Kimura (RIKEN, Japan)
P. Oh (Drexel University, USA)
M. Tomizuka (University of California-Berkeley, USA)Y. G. Zhang (Academia Sinica, P. R. China)

INTERNATIONAL ORGANIZING COMMITTEE
H. Abbass (University of New South Wales, Australia)
K. Aihara (The University of Tokyo, Japan)
C. Barrett (Los Alamos National Laboratory, USA)M. Bedau (Reed College, USA)
M. Eaton (University of Limerick, Ireland)
J. M. Epstein (The Brookings Institution, USA)
A. Grzech (Wroclaw University of Technology, Poland)
H. Hashimoto (The University of Tokyo, Japan)
D. J. G. James (Coventry University, UK)©ISAROB 2011

P-2

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

K. Kyuma (Mitsubishi Electric Corporation, Japan)
M. H. Lee (Pusan National University, Korea)
M. Nakamura (Research Institute of Systems Control, Japan)
M. Oswald (Vienna University of Technology, Austria)
S. Rasmussen (University of Southern Denmark, Denmark)T. S. Ray (University of Oklahoma, USA)K. Shimohara (Doshisha University, Japan)
C. Taylor (University of California-Los Angeles, USA)J. Wang (Beijing Jiaotong Univesity, P. R. China)
C. Zhang (Tsinghua University, P. R. China)

INTERNATIONAL PROGRAM COMMITTEE
T. Arita (Nagoya University, Japan)
R. Belew (University of California-San Diego, USA)
S- M. Chen (National Taiwan University of Science and Technology, Taiwan)
Y. I. Cho (The University of Suwon, Korea)
M. Chow (North Carolina State University, USA)Y. Y. Fan (University of California-Davis, USA)T. Fuchida (Kagoshima University, Japan)
H. Hamdan (SUPELEC, France)
S. H. Han (Kyungnam University, Korea)
N. Homma (Tohoku University, Japan)
Y. Ishida (Toyohashi University of Technology, Japan)
S. Ishikawa (Kyushu Institute of Technology, Japan)
T. Iwamoto (Mitsubishi Electric Corporation, Advanced Technology R&D Center, Japan)
J. Johnson (The Open University, UK)
H. Kinjyo (University of Ryukyus, Japan)
M. Kono (University of Miyazaki, Japan)
K. Kurashige (Muroran Institute of Technology, Japan)
H. H. Lee (Waseda University, Japan)
J. M. Lee (Pusan National University, Korea)
K. J. Mackin (Tokyo University of Information Sciences, Japan)
K. Merrick (University of New South Wales and ADFA, Australia)
C. Mi (University of Michigan-Dearborn, USA)J. Murata (Kyushu University, Japan)
K. Nakano (The University of Electro-Communications, Japan)
K. Naitoh (Waseda University, Japan)
W. Nistico (University of Dortmund, Germany)
N. Okada (Kyushu University, Japan)
S. Omatu (Osaka Institute of Technology, Japan)
H. Oogai (Waseda University, Japan)
L. Pagliarini (Technical University of Denmark, Denmark)R. Pfeifer (University of Zurich-Irchel, Switzerland)
M. Rozalian (TATi University College, Malaysia)
S. Sagara (Kyushu Institute of Technology, Japan)
P. Sapaty (National Academy of Sciences of Ukraine, Ukraine)
J. Y. Sim (Kangnam University, Korea)
M. Svinin (Kyushu University, Japan)
K. Sugawara (Tohoku Gakuin University, Japan)
H. Suzuki (The University of Tokyo, Japan)
I. Tanev (Doshisha University, Japan)
T. Tsuji (Hiroshima University, Japan)©ISAROB 2011

P-3

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

H. Umeo (Osaka Electro-Communication University, Japan)
K. Uosaki (Fukui University of Technology, Japan)
K. Watanabe (Okayama University, Japan)
H. Yanagimoto (Osaka Prefecture University, Japan)
I. Yoshihara (University of Miyazaki, Japan)
M. Yokota (Fukuoka Institute of Technology, Japan)
Y. Yu (Kagoshima University, Japan)
T. Zhang (Tsinghua University, P. R. China)

LOCAL ARRANGEMENT COMMITTEE
F. Dai (Tianjin University of Science & Technology, P. R. China)
Z. Ibrahim (TATi University College, Malaysia)
M. Rizon (King Saud University, Saudi Arabia)
S. Sagara (Kyushu Institute of Technology, Japan)
M. Sugisaka (Alife Robotics Co., Ltd., Japan) (Chair)
J. Wang (Beijing Jiaotong University, P. R. China)
H. Zhao (Shanghai Institute of Technology, P. R. China)

HISTORY
The AROB was founded in 1996 under the support of Science and International Affairs
Bureau, Ministry of Education, Culture, Sports, Science and Technology, Japanese
Government. Since then, the symposium organized by the AROB has been held every year at
B-Con Plaza, Beppu, Japan except AROB 5th ’00 (Oita) and AROB 6th ’01 (Tokyo). The
sixteenth symposium will be held on January 27–29, 2011, at B-Con Plaza, Beppu, Japan.

OBJECTIVE
This symposium will bring together researchers to discuss development of new technologies
concerning artificial life and robotics based on computer simulations and hardware designsof state-of-the-art technologies, and to share findings on how advancements in artificial lifeand robotics technologies that relate to artificial intelligence, virtual reality, and computerscience are creating the basis for exciting new research and applications in various fields
listed in the following Topics.

©ISAROB 2011

P-4

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

GENERAL SESSION TOPICS
Artificial intelligence
Chaos & Complexity
Data mining
Human-machine cooperative systems& Human-welfare robotics
Mobile vehicles
Pattern recognition & Visualization Poster
Robotics

Artificial life
Control techniques
Evolutionary computations
Image processing
Learning
Neural networks
sessionORGANIZED SESSION TOPICS
Advanced technologies& Management skills
Bio-inspired theory and application
Biomimetic machines and robots
Control and automata
Data mining
Embracing complexity innatural intelligence
Human agent interaction
toward social modification
Intuitive human-system interaction
Medical science and complex system
Special environment navigationand localization
System sensing and control

Advanced vehicle control
AI-based systems
for human awareness promotion
Computer vision and sound analysis
Control and its application
Dynamical information processingin the brain
Embracing complexity insensor system organization
Intelligent control
Intelligent systems
Learning control and robotics
Robotics and pattern recognition
Structural change detection
for ongoing time series

COPYRIGHTS
Accepted papers will be published in the proceeding of AROB and s ome of high quality papers in
the proceeding will be requested to re-submit their papers for the consideration of publication in aninternational jo urnal AR TIFICIAL LIFE AND RO BOTICS. All co rrespondence related to thesymposium should be addressed to AROB Secretariat.

AROB Secretariat
ALife Robotics Corporation Ltd.
1068-1 Oaza Oshino, Oita, 870-1121, JAPANTEL/FAX：+81-97-594-0181
E-MAIL：arobsecr@isarob.org
Home Page：http://isarob.org/©ISAROB 2011

P-5

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

ADDRESS
Takayuki Hirai
Nippon Bunri University
President

Takayuki Hirai
Nippon Bunri University
President©ISAROB 2011

It is a great p leasure fo r m e to welco me yo u all to th e Si xteenthInternational Symposium o n Artificial Life and Robo tics. I have deep
respect and great admiration for the many people who attend this event every
year and con tribute to th e development o f tech nology on Artificial Life andRobotics.
Everyone's activ ities o n th e Org anizing C ommittee o f International
Symposium on Artificial Life an d Robotics are outstanding. Especially, ourcolleague, General Chairman P rofessor Su gisaka, has m ade a l arge
contribution.
In th e un iversities o f Japan , Artificial Ro botics h as been gaining greatinterest by researchers and will develop as one of t he leading research fields,
which will sup port fu ture society. However, while m any indu stries ofArtificial Robo tics h ave arriv ed an d d eveloped, it is sai d th at th ere is still agreat demand for engineers engaged in those industries.
It is our role to teach the young of the future the technology of Artificia lLife Robotics which has been advanced and developed by all of you working
internationally, and sen d talen ted people t o so ciety. To tak e this ro le, th e
research ach ievements o f y ou attend ing th is International Sym posium arevery valuable.
It's my heartfelt hope that t his International Symposium will be successful
and all of you will have a great time here in Beppu.

P-6

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

MESSAGES
Fumio Harashima
Advisory Committee Chairman of AROB

Fumio Harashima
Advisory Committe Chairman(Pr ofessor, Tokyo
Metropolitan University)

The science and technology (S&T) on Artificial Life an d Robotics was bornin 1996, a nd i t’s been providing h uman being wi th happiness. T his S & T i s
not only important but also necessary for people living in the world to maintainhigh quality of life. Research is heart and desire of human being and the S&T
is going toward clarifying tool to achieve our objective.
I would like to congratulate researchers who work in th e fields on Artificial
Life and Robotics.

Masanori Sugisaka
General Chairman of AROB

Masanori Sugisaka
General Chairman(Professor, Nippon Bunri
University and President,
ALife Robotics Corporation,
Ltd. , Japan)©ISAROB 2011

It is my g reat p leasure to in vite you all to th e Si xteenth In ternational
Symposium on Artificial Life and Robotics (AROB 16th ‘11).
The sym posium fro m th e first (19 96) t o th e Fou rteenth (2009) wereorganized by Oita Uni versity, Ni ppon B unri U niversity(NBU), a nd ALife
Robotics Cor poration Lt d. u nder t he sponsorship o f t he Sci ence andTechnology Policy Bureau, the Ministry of Education, Science, Sports, andCulture ( Monbusho), pr esently, th e Min istry of Education, Cu lture, Spo rts,
Science, and Technology (Monkasho), Japanese Government, Japan Society
for the Promotion of Science (JSPS), The Commemorative Organization for
the Jap an World Ex position (’70), Ai r Fo rce Office of Scientific Research ,Asian Office of Aerospace Research an d Development (A FOSR/AOARD),
USA. I would like to express my sincere thanks to not only Monkasho, JSPS,
the C ommemorative Organization fo r the Jap an World Ex position (’70),
AFOSR/AOARD but al so Ja panese com panies(Mitsubishi E lectric
Corporation, Advanced Technology R &D C enter, Oita Gas C o., Lt d., M E
System Co. LTD, and Sanwa Shurui Co., LTD. for their repeated supports.
This sy mposium i s orga nized by International O rganizing C ommittee o f
AROB and is co-op erated by th e San ta Fe Institute (USA), RSJ, IEEJ,ICASE ( Now ICROS) ( Korea), C AAI ( P. R. China ), ISCIE, IEICE, I EEERobotics and Automation Society Japan Chapter, JARA, and SICE.
The AROB symposium is growing up by absorbing many new knowledge
and technologies into it. The new technologies presented in this symposium
bring happiness to human society.
I hop e t hat this sym posium wil l facilit ate th e establish ment o f aninternational research Institute of Artificial Life and Robotics in future.
Welcome and enjoy your stay in Beppu.

P-7

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

Takao Ito
Assistant General Chairman of AROB

Takao Ito
Assistant General Chairman(Professor, Ube National
College of Technology)

Given the intense co mpetition in the field of artificial life and robotics, th eeffective co ordination a nd e fficient m aintenance of i nformation exc hangetake on increased importance. The AROB international conference provides
us a unique opportunity t o enj oy t he discussions of al l new i ssues, an d t oshare experiences related to t he advanced technologies. I am pleased to put
together t his excellent program t hat rep resents s ubmissions f rom many
countries around the world.
I hop e th at y ou will en joy t his confer ence and en joy Bep pu, Oita an d theculture richness it has to offer.

Hiroshi Tanaka
Program Chairman and Vice chairmanof AROB

Hiroshi Tanaka
Program Chairman
andVice chairman(Professor, Tokyo Medical
and Dental University)©ISAROB 2011

On beh alf of th e program committee, it is my g reat pleasure and hon or toinvite you all to the Fifteenth International Symposium on Artificial Life andRobotics (AROB 15th 2011). This symposium is made possible owing to thecooperation of Nippo n Bunri University and Santa Fe In stitute. We are also
debt to Japanese aca demic associations s uch as SICE, RSJ, a nd se veral
private companies. I would like to express my sincere thanks to all of those
who make this symposium possible.
As is needless to say, th e Alife or biologically-inspired Robotics approach
now at tracts wide i nterests a s a ne w paradigm of sci ence an d en gineering.
Taking a n exa mple in the field of bioscience, the accom plishment of HGP(Human Ge nome Pr oject) a nd s ubsequent p ost-genomic com prehensive“Omics data” such as transcriptome, proteome and metabolome, bring about
vast amount of bio-information. However, as a plenty of omics data becomes
available, it becom es sin cerely recognized that the fram ework by w hichthese omics dat a can be understood t o m ake a whole picture o f life i s
critically necessary. T hus, i n the post-genomic era, bi ologically-inspired
systems approach like Alife i s expected to give one of n ew alternative ideas
to integrate this vast amount of bio-data.
This example shows the Alife approach is very promising and may exert awide influence on t he effort to develop a new paradigm for ne xt generationof life science. We hope this symposium becomes a for um for e xchange ofthe id eas of t he attend ants from v arious fields, in cluding the life scien ce
field, who are in terested i n the fu ture possibility o f b iologically-inspired
computation and systems approach.
I am looking forward to meeting you in Beppu, Oita.

P-8

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

Ju-Jang Lee
Vice Chairman of AROB

Ju-Jang Lee
Vice Chairman(Professor, KAIST)The Six teenth In ternational Symposium on Artificial Li fe an d R obotics(AROB) will b e held in Bep pu, Oita, Japan fro m Jan . 2 7h t o 29 th, 2011.
This year’s Symposium will b e held am idst th e h igh ex pectation of theincreasingly important role of the new interdisciplinary paradigm of sc ienceand engi neering represented by the fiel d of artificial life and rob otics t hat
continuously attracts wid e in terests among scientist, research ers, andengineers around the globe.
Since t he time of t he very first AR OB m eeting i n 1996, each y ear,distinguished researchers a nd t echnologists f rom arou nd t he w orld arelooking forward to attending and meeting at AROB. AROB is becoming theannual excellent forum that represents a unique opportunity for the academic
and industrial communities to meet and assess the latest developments in thisfast growing artificial life an d robotics field. AROB enables them to address
new chal lenges, s hare solutions, discuss research di rections for the future,exchange views and ideas , view the results of applied research, present anddiscuss the latest development of new technologies and relevant applications.
In a ddition, AROB of fers t he op portunity of hearing t he opi nions of wellknown leading experts in the field through the keynote sessions, provides thebases for regional and in ternational co llaborative research, and en ables toforesee th e fu ture evo lution of new scientific paradigms an d th eories
contributed by the field of artificial life and robotics and associated researcharea. The twenty-first cen tury will beco me th e cen tury of artificial life an dintelligent m achines in su pport of humankind and AROB is con tributing
through wide technical topics of interest that support this direction.
It is a great for m e as the Vice Chairman of t he 16h AROB 201 1 t owelcome everyone to this important event. Also, I would like to extend myspecial thanks to all authors and s peakers fo r c ontributing t heir re searchworks, the participants, and the organizing team of the 16th AROB.
Looking forward t o m eeting y ou at t he 16th AROB i n B eppu-Oita andwishing you all the best.

John L. Casti
Vice Chairman of AROB
It is m y g reat p rivilege t o i nvite you all to th e Sixteenth In ternational
Symposium on Artificial Life and Robotics (AROB 16th ‘11). As always at
the AR OB, the pr ogram cont ains a wide vari ety o f co ntributions t o thedeveloping areas of computer simulation of human processes, robotics, andgeneral sy stems m odeling. We a re grateful t o al l t he c ontributors for t heir
hard work in prepa ring t heir pres entations, an d l ook f orward t o the
cutting-edge research that will be described.
John Casti
Vice Chairman(Professor, International
Institute for Applied
Systems Analysis, Vienna,
Austria)©ISAROB 2011

I h ope t hat fr uitful di scussions an d exchange of ideas between researc hers
during symposium will yield new t echnological innovations for contributing
to a better life for hu mans in the coming decades. This is truly a challenging
period t o be alive, an d t he AR OB 16th conference holds the prom ise ofoffering solutions to some of our most pressing global human problems.

P-9

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

Henrik Hautop Lund
Vice-Chairman of AROB

Henrik Hautop Lund
Vice Chairman(Professor, Center for
Playware, Technical
University of Denmark)I am much honored to invite you to the Sixteenth International Symposium
on Artificial Life and Rob otics (AR OB 1 6th ‘1 1). Th e in ternational
symposium h as b een held each year si nce 1996, initially organized by Oita
University and now being organized by Nippon Bunri University.
The sy mposium at tracts an i mpressive r ange of researchers f rom a ll
continents, who all share the vision of merging research based upon artificial
life and robotics. The symposium is visionary in merging these two, scienceand en gineering disciplines, and has be come t he m ost im portant f orum forresearch into merging artificial life and robotics.
The research i n artificial life and ro botics i s ve ry im portant si nce i t bothbrings us i nsight in to ourselves as hu man bein gs an d natural system s, an dbrings u s new en gineering s olutions t hat may i nfluence o ur l ives. It i s m yhope th at you will u se th is i nsight an d opportunity to d evelop system s t hat
help humankind in socially responsible ways.
I wou ld lik e t o tak e t his opp ortunity to th ank th e g eneral ch airman o f
AROB, P rof. Masanori Sugisaka, f or being so visionary 1 6 y ears ag o t oengage in cr eating and organizing th is im portant annual ev ent fo r our
research community. It takes a lot of courage to be the first to create a novel
interdisciplinary research field suc h as the one t hat c omes from the m erge
between artifi cial life an d ro botics. Prof. Su gisaka has sho wn how being
courageous eno ugh t o eng age in th e ad venturous activ ities of m erging t wo
fields may lead to very fruitful research and to the lively research communitythat yo u are now part of with you r participation in t his in ternational
symposium.
I would also take this opportunity to thank Springer-Verlag for su pporting
this research co mmunity, and remind all participants of the Springer-Verlag
Artificial Life and Robotics Journal. Hopefully, we will see nu merous, highquality contributions to the journal as the outcome of this symposium and theresearch collaboration that may entail the symposium. Indeed, it is my hopethat you will all en gage in open and fruitful scientific discussions with yo urcolleague researchers during the symposium, and that these discussions may
open up for future research collaborations in order to bring new insight into
artificial life and robotics to the community.

Yingmin Jia
Vice-Chairman of AROB

Yingmin Jia
Vice-Chairman of AROB
(Professor, Beihang
University, P.R.China)©ISAROB 2011

The 16th International Symposium on Artificial Life and Robotics (AROB)
will b e held in Bep pu, Oit a, Jap an fro m Jan. 27th to 29th, 2011 . As avice-chairman, I am honored and privileged to invite you all to this fruitful
3-day event.
In 1996, the first Symposium was created and organized by Prof. Masanori
Sugisaka, t he ge neral chai rman o f AROB. S ince th en, the sym posium
attracts an i mpressive range of re searchers from a ll co ntinents, anddistinguished achievements have been o btained t hat s hows hi s foresight i nmerging tw o disciplines of artificial life an d ro botics. Moreov er, it isgratifying to see th at there are more and more young colleagues, researchersand eng ineers fro m th e univ ersities, research i nstitutions an d ind ustries
interested in the field. I believe there is a rapid and better development in thenear future.
Looking f orward t o m eeting y ou at t he 16th AROB i n B eppu, Oita an dwishing you enjoy your stay in Japan.

P - 10

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

Plenary talker:
PT1 : Development Outline of Humanoid Robot: HUBO II
Jun Ho Oh
Professor of Mechanical Engineering, KAIST
Director of Humanoid Robot Research Center (Hubo Lab)

Professor
Jun Ho Oh

The fu ll size humanoid robot with h eight of arou nd 1.5 m is qu itdiffer from the toy size small ones in many aspects. It should have very
stable an d well d esigned structure wit h l ittle u ncertainties. It m ust
strong e nough t o m ove i ts body wei ght b ut n ot s o heavy t o m inimize
the torques to drive the body parts. All the electrical parts and se nsors
must b e co mpact to be fit in th e enclosure of t he body. We desi gned
such kind of parts including force/torque sensors, inertia sensors and all
the driver ci rcuits, in ternal decent ralized cont rol arc hitecture andhardware. Anoth er important task is d esign walk algorithm. W alking
algorithm is composed with two parts: off-line gait pattern design andreal time stabilization control. Gait pattern design is t o find a period ic
function for each joint of leg such that humanoid robot is to walk withdesired v elocity k eeping certain level of stab ility. W e su ggested asimple fu nction c onnected with c ubic s pline a nd si ne functions withminimal number of parameters. This approach simplifies the parameter
adjustment procedure. Pl ay back of gait pattern found from the former
process, howe ver, does not guarantee the robot walks i n real practicesince there a re number of uncertainties involved in real situ ation. The
uncertainties include ground inclination, friction, un-modeled vibrationof the body. The stabilization algorithm should deal with such kind ofproblems. Hub o’s wal k al gorithm has 8 l evels of hi erarchical cont rolarchitecture to copewith the ge neral circum stances in wal kingenvironment. Th e general issu es i ncluding m entioned abo ve will bepresented.

Education:- In 1977, B.S. degree in Mechanical Engineering from Yonsie University, Korea- In 1979, M.S. degree in Mechanical Engineering from Yonsie University, Korea- In 1985, Ph.D. in Mechanical Engineering, University of California, Berkeley, USA
Professional Training and Employment:- 1985 – present
Professor of Mechanical Engineering, KAIST- 2010 – present
Distinguished Professor at KAIST- 1997 – 1998
Visiting Professor, University of Texas, Austin, USA- 2010Member of National Academy of Engineering of Korea©ISAROB 2011

P - 11

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

PT2 : Robotics:From Manufacturing to Intelligent Machine
Tzyh Jong Tarn
This presentation describes the evolution from tele-robotics to interactive
robotics and to intelligent machines due to the silicon revolution. The
impacts of this transformation to the manufacturing culture as well as theautomated manufacturing research facility areoutlined. Finally the current
trends in robotics research and new applications to intelligent life aredescribed.
Professor
Tzyh Jong Tarn

Education:-1959, B.S. degree in Chemical Engineering, National Cheng Kung University, Tainan, Taiwan,
China-1965, M.E. degree in Chemical Engineering, Stevens Institute of Technology, Hoboken, USA-1968, D. Sc. Degree in Control Engineering, Washington University, St. Louis, USA

Professional Training and Employment:-1968-1969 Postdoctoral Fellow, Washington University, St. Louis, USA-1969-1972 Assistant Professor, Washington University, St. Louis, USA-1972-1977 Associate Professor, Washington University, St. Louis, USA-1 977Professor, Washington University, St. Louis, USA-2003-2006 Chair Professor in Quantum Control, Tsinghua University, Beijing, China-2 006Director, Center for Quantum Information Science and Technology, Tsinghua
University, Beijing, China-2009-2010 Distinguished Visiting Chair Professor, National Cheng Kung University, Tainan,
Taiwan, China©ISAROB 2011

P - 12

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

PT3 : Rescue Robot Systems- From Snake-like Robots to Human Interface Fumitoshi Matsuno
Professor
Fumitoshi Matsuno(Kyoto University, Japan)

Intelligent rescue systems with information and communications
technologies (ICT) and rob otics tec hnology (R T) have be en
proposed to mitigate disaster damages, especially in Japan after the
1995 H anshin-Awaji Eart hquake. In p articular, it i s has beenstressed the importance of developing robots for search and rescue
tasks, whi ch can ac tually work i n a real disaster site. In USA the
September 1 1, 2001 terrorist
attack on New York Ci ty andWashington, DC, the hijacked plane crash in Pennsylvania, and the
Anthrax attack that immediately followed instantly changed people
attitude about safety an d s ecurity in their perso nal li ves. Pu blic
safety an d se curity problems are not li mited t o Ja pan a nd t heUnited States, since e very country has ex perienced man-made andnatural disasters in the pa st. Solutions will de pend up on new ,unconventional approaches to se arch an d rescu e. R obotics,
information and communications technologies, devices and system
integration can play an important role in providing technology thatcan contribute to Safety, Security and Rescue activities. In this talk,I would like to explain my motivation to start the development of
rescue robot system s for thedisaster resp onse a nd d iscuss
necessary te chnologies that c an accomplish s earch and resc ue
missions. I also i
ntroduce dev eloped rescue rob ots for th e
information correction and teleportation human interface.

Education:- In 1981, B.E. degree in Instrumentation Engineering from Nagoya Institute of Technology, Japan- In 1983, Master Degree from Dept of Control Engineering at Osaka University Japan- In 1986, Doctor Degree from Dept of Control Engineering at Osaka University, Japan
Professional Training and Employment:- 1986 – 1991 Research Associate, Osaka University- 1991 – 1996 Associate Professor, Kobe University- 1996 – 2003 Associate Professor, Tokyo Institute of Technology- 2003 – 2009 Professor, University of Electro-Communications, Tokyo- 2009 –Professor, Dept of Mechanical Engineering and Science, Kyoto University©ISAROB 2011

P - 13

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

Invited talker:

IT1 : Human-Robot Interaction and Social Relation
Michita Imai
(Keio University, Japan)

Associate Professor
Michita ImaiThis talk presents the factor of designing an anthropomorphic agent
such as a co mmunication robot a nd an em bodied c ommunicative
agent. C ommunication i s a co operative p henomenon between
participants. People i nvolved in a co mmunication sp eak an d h ear
actively duri ng t he c onversation. However, i t i s di fficult t o en gagethem in the communication with a robot because they seldom actively
consider the communicative intention of t he robot. This talk explains
the importance of social relationship between a human and a robot forengaging people in the communication. In particular, we consider how
the relation mak es them co nsider t he in tention of t he robot. Also , I
show what factors of the robot design elicit the social relation.

Education:- In 1994, M.S. degree in Computer Science fr om Department of Computer Science, Keio University, Japan- In 2002, Ph.D.(Eng.) in Computer Science from Department of Computer Science, Keio University, Japan

Professional Training and Employment:- 1994-1997, Researcher, Human-Interface Laboratories, Nippon Telephone Telegram (NTT), Japan- 1997-2002, Researcher, ATR Media Integration and Communications Research Laboratories, Japan- 2002-2003, Assistant Professor, Department of Information and Computer Science, Keio University, Japan- 2002-2006, Researcher, JST Prest “Interaction and Intelligence”- 2003-2005, Lecturer, Department of Information and Computer Science, Keio University, Japan- 2005-present, Associate Professor, Department of Information and Computer Science, Keio University, Japan©ISAROB 2011

P - 14

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

IT2 : Micro Nano Robotics
Tatsuo ARAI
(Osaka University, Japan)
Manipulation, se nsing, act uation, a nd a utomation i n m icro na no
scale hav e va rious a pplications a nd co ntributions i n biotechnology,
pharmacy, medicine as wel l as in industries. Micro hands are
Prof.developed
Tatsuo ARAIt o m anipulate m icro a nd n ano o rder si ze o bjects
dexterously. Micro channels fabricated on a chip, called micro TAS or
lab on chip, can provide various operations and processes for cells andtissues. Tho se activ ities in clude elab orating m achining, assem bly,
fabrication an d in tegration with ME MS technique a s well as robotics
and m echatronics. The talk will co ver mainly two t opics, m icro
manipulation system with dexterous hand and desktop bio plant, alongwith their back ground and the state of the art in Japan.

Tatsuo ARAI was born in 1952 in Tokyo. He received B.S. M.S. and PhD degrees from the University of Tokyoin 1975, 1977 , and 1986 r espectively. H e join ed th e Mech anical Engineering Labor atory, AIST , MITI (now
METI) in 1977 , and was engaged in r esearch and devel opment of new arm design and con trol, mobile robot,
teleoperation, and micro robotics. He stay ed at MIT as a v isiting scie ntist in 1986-1987. He was an adjun ct
lecturer at Chib a Universi ty in 1986-1996, an d a v isiting pro fessor at the Scien ce Universit y of Tokyo in
1996-1997. He moved to Osaka University in 1997 and since then he has been a Full Professor at the Departmentof S ystems Innovation , Gradu ate Schoo l of En gineering Science. His cu rrent research top ics are mechanism
design including parallel mechanisms, legged working robot, micro robotics for bio applications, humanoid robot,
haptic interface, and network robotics. He has published more than 300 journal and conference papers on robotics
and automation, 6 books,
and h as 37
patents in cluding foreign 8 . The publ ication list
is o nhttp://www-arailab.sys.es.osaka-u.ac.jp/publication/e_index.html
He is a m ember of IE EE, Int ernational Associ ation of Automation and Robo tics in Construction (IAARC),
Robotic Society of Japan (RSJ), Societ y of Instrumentation and Control Engineers, Japan Society of MechanicalEngineers (JSME), and o ther societies. He is currently an Editor -in-Chief of Journal of Robotics andMechatronics. He served as a Vice President of IAARC, a chair of Robotics and Mechatronics Division of JSME,a Dir ector of R SJ, a ch air of the Technical Adviso ry Com mittee of th e Destr uction of Aban doned Ch emical
Weapon of the Cabinet Office. He has been or ganizing and chairing many research committees, conferences, andsymposia for IEEE, IAARC, RSJ, SICE, and JSME.

©ISAROB 2011

P - 15

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

IT3 : Quantifying Dance Movement Waza in Nihon-Buyo
Mamiko SAKATA(Doshisha University, Japan)

Associate Professor
Mamiko Sakata

Dance is an intangible cultural asset. It is passed on from one person
to another through oral instruction, and so are the waza, the sk ills andtechniques. i nvolved i n da nce. M any t raditional da nces i n Japa n,
however, are ‘endangered species’ due to the shortage of practitioners.
We are atte mpting to create d igital archives to rec ord and store thebody m otions o f buyo using digital t echnologies, s uch as m otion
capture. We are en deavoring to so lve th is serious issu e of l osing animportant t radition. I n m y tal k, I w ould l ike t o present part of o urscientific analysis of the waza in Nihon-buyo, which have been passed
down, sometimes in silent, tacit manners.

Education:- 1999: M.S. degree in Cultural Studies and Human Sciences from Kobe University, Kobe, Japan- 2002: Ph.D. in Cultural Studies and Human Sciences from Kobe University, Kobe, Japan
Professional Training and Employment:- 1999-2001: Researcher, ATR Media Integration & Communications Research Laboratories, Japan- 2002-2005: Lecturer, Department of Nursery ,Fukushima College, Japan- 2005-2008: Lecturer, Faculty of Culture and Information Science, Doshisha University, Japan- 2008-present: Associate Professor, Faculty of Culture and Information Science, Doshisha University, Japan©ISAROB 2011

P - 16

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

IT4

Social playware for mediating tele-play interaction over distance
Henlik Hautop Lund and Tumi Thorsteinsson(Technical University of Denmark, Denmark)Professor
Henrik Hautop Lund

We s uggest t hat novel playware t echnology can f unction as amediator for playful social interaction over distance, where people are
separated by physical distance but feelthe presence of each othermediated throu gh th e in teraction with th e p layware tech nology. In
order to investigate such social playware, we developed the Playware
Soccer g ame an d tested th is with m ore than 1,000 users d uring th eFIFA W orld Cu p 2010 in Sou th Af rica. Th e test w as co nducted intownships, orphanages f or HIV/AIDS c hildren, m arkets, F IFA fa nparks, etc. alo ng with simu ltaneous tests with sim ilar set-u ps inEurope an d Asia. With th e social playwa re, players would com peteagainst each other sim ultaneously in t he thre e c ontinents, Africa,
Europe and Asia, and feel the presence of the competitors on the othercontinents expressed through the playware. The playware game is se t
up to m otivate players to e ngage in training of technical soccer skills
by receiving immediate feedback and offering challenges to players of
all skills on the soccer playing on a modular interactive wall composedof modular interactive tiles that respond with coloured light, sound andscores on the players performance. This paper outlines the concept ofsocial playwa re and physical-virtu al telep lay, an d ex emplifies th iswith the playware soccer game.

Education:- M.Sc. degree in Computer Science from University of Aarhus, Denmark- Ph.D. degree in Computer Systems Engineering from University of Southern Denmark
Professional Training and Employment:- 1992-1993 and 1994-1995, Research Assistant, the National Research Council, Rome
- 1996-1997, Research Associate (Post Doctor), Department of Artificial Intelligence, Edinburgh, UK- 1997-2000, Head of LEGO Lab- 1998-2000, Research Associate, Department of Computer Science, University of Aarhus, Denmark- 2000-2008, Full Professor, the Maersk Mc-Kinney Moeller Institute, University of Southern Denmark- 2003-2007, Member of the Danish Research Council- 2008-present, Full Professor, the Center for Playware, Technical University of Denmark©ISAROB 2011

P - 17

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

IT5 An Educational Tool for Interactive Parallel and Distributed Processing

Luigi Pagliarini1,2 and Henrik Hautop Lund1( Technical University of Denmark, Denmark)(2Academy of Fine Arts of Bari, Via Gobetti, Italy)
1

Professor
Luigi Pagliarini

In th is paper we t ry to describe how t he Modular In teractive Tiles
System (MITS) can be a valu able too l fo r in troducing stu dents tointeractive pa rallel and di stributed p rocessing pr ogramming, by
forcing and challenging programmers in designing interactive paralleland distributed systems. Indeed, MITS seems to bring a series of goals
into th e education, su ch as p arallel pro gramming, d istributedness,
communication p rotocols, master depe ndency, so ftware beha vioral
models, adaptive interactivity, feedback, connectivity, topology, islandmodeling, user and m ulti-user interaction, which can hardly be foundin other tools. We introduce the system of m odular interactive tiles asa tool for easy, fast, and flexible hands-on exploration of these issues,and through examples show how to implement interactive parallel anddistributed processing with different software behavioural models suchas open loop, randomness based, rule based, user interaction based, AI
and ALife based software.

Education:- Master Degree in Experimental Neuropsychology
Professional Training and Employment:
An Artist, Art Curator , Ps ychologist, Multim edia and Software D esigner, and a wor ldwide known as atheoretician and expert in (mainly Artistic) Robotics, A.I. and Artificial Life.
- Professor, Theories of Perception and Psychology of Shape and of Computer Art, the Academy of Fine Arts ofBari, Italy- Associate Professor, Center for Playware, Technical University of Denmark- Founder and Director, the Pescara Electronic Artists Meeting- President, the Cultural Association Artificialia- Art Director, Ecoteca- Founder of RoboCup Junior and Member of its International Committee©ISAROB 2011

P - 18

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

TIME TABLE (1/27)
Room A
1/26

13:00

(Wed)

17:00

Room B

Room CRoom D

Registration (Registration Desk)
Welcome Party (at Hotel Shiragiku)

1/27

8:00

(Thu)

9:10

Registration (Registration Desk)
OS 1 ( 5 )OS 18 ( 4 )OS 2 ( 4 )GS 4 ( 4 )Chair M. Uchida

Chair

Chair H. Tanaka

Chair S. Nakayama

K. J. Mackin

will end at 10:25
10:10
10:30

Coffee BreakOpening Ceremony (Room G)

10:55

Plenary Talk (Room G)
PT1 J. H. Oh

11:45
12:40

Chair J. M. Lee

Cha ir Y. G. Zhang
Lunch

IT1 M. Imai

GS 10 ( 7)GS 21 ( 7 )PS 1 ( 8)IT2 T. Arai

Chair H. Yamamoto

Chair

Chair J. J. Lee

B. Fu

Chair M. Nakamura
14:25
14:45

will end at 14:40
Coffee BreakOS 5 ( 5 )GS 16 ( 5 )OS 24 ( 4 )OS 17 ( 5 )Chair S. Sagara

Chair K. Ikeda

Chair M. Imai

Chair

K. Naitoh

will end at 15:45
16:00
16:05

OS 3 ( 5 )OS 4 ( 5 )GS 3 ( 5 )GS 19 ( 6 )

Chair

Chair

Chair D. E. Kim

Chair A. Nakamura

M. Yokomichi

Y . M. Jia

will end at 17:35
17:20

GS: General Session

OS: Organized Session

GS1 Artificial intelligence
GS2 Artificial life I
GS3 Artificial life II
GS4 Chaos & Complexity
GS5 Contr ol techniques
GS6 Data mining
GS7 Evolutionary computations
GS8 Human-machine cooperative systems& Human-welfare robotics I
GS9 Human-machine cooperative systems& Human-welfare robotics II
GS10 Image processing I
GS11 Image processing II
GS12 Learning
GS13 Mobile vehicles
GS14 Neural networks I
GS15 Neural networks II
GS16 Pattern recognition &Visualization
GS17 Robotics I
GS18 Robotics II
GS19 Robotics III©ISAROB 2011

PS: Poster Session

PT: Plenary Talk

GS20 Robotics IV
GS21 Robotics V
PS1 Poste r Session
OS1 System sensing and control I
OS2 System sensing and control II
OS3 Contr ol and its application
OS4 Intelligent Control
OS5 Advanced vehicle control
OS6 AI-based systems for humanawareness promotion
OS7 Advanced technologies& Management skills
OS8 Structural change detection for
ongoing time series
OS9 Computer vision and sound analysis
OS10 Intelligent systems I
OS11 Control and automata
OS12 Special environment navigationand localization
OS13 Intuitive human-systeminteraction

P - 19

IT: Invited Talk

OS14 Dynamical information
processing in the brain
OS15 Bio-inspired theory
and application
OS16 Biomimetic machines
and robots
OS17 Medical scienceand complex system
OS18 Data mining
OS19 Embracing complexity innatural intelligence
OS20 Embracing complexity insensor system organization
OS21 Learning control
and robotics
OS22 Robotics andpattern recognition
OS23 Intelligent systems II
OS24 Human agent interaction
toward social modification

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

TIME TABLE (1/28)
Room A
1/28

8:00

(Fri)

8:40

Room B

Room CRoom D

Registration (Registration Desk)
GS 5

(5)

Chair J. Murata

OS 8 ( 5 )OS 13 ( 5 )GS 15 ( 5 )Chair T. Hattori

Chair

Chair

N. Kamiura

9:55
10:15

M . Yokota

Coffee BreakPlenary Talk (Room G)
PT2 T. J. Tarn

11:05

Chair J. J. Lee

OS 7 ( 5 )GS 8 ( 4 )GS 9 ( 4 )

OS 12

(5)

Chair T. Ito

Chair

Chair

Chair

J. M. LeeT. Fuchidawill end at 12:05

N. Okada

will end at 12:05

12:20
13:15

Lunch
OS10 ( 6 )GS 2 ( 6 )OS 9 ( 6 )GS 17 ( 5 )Chair

Chair T. Arita

Chair

Chair

K. L. SuY. Yoshitomi

D. W. Lee

will end at 14:30
14:45
14:50
15:40
16:00

Plenary Talk (Room G)
PT3 F. Matsuno

Chair

H. H. Lund

Coffee BreakOS11 ( 6 )GS 7 ( 6 )GS 18 ( 5 )GS 13 ( 6 )

Chair

Chair

Chair E. Inohira

Chair

K. H. Hsia

P . Sapaty

will end at 17:15

17:30
18:30

AROB Award Ceremony (Chair
Banquet – Hotel Shiragiku (Chair Y. I. Cho)...... Address

21:00

GS: General Session

K. Naito)Y. G. Zhang / Y. Ishida/ T. Arita
J. M. Lee / K. L. Su / L. Pagliarini

OS: Organized Session

GS1 Artificial intelligence
GS2 Artificial life I
GS3 Artificial life II
GS4 Chaos & Complexity
GS5 Contr ol techniques
GS6 Data mining
GS7 Evolutionary computations
GS8 Human-machine cooperative systems& Human-welfare robotics I
GS9 Human-machine cooperative systems& Human-welfare robotics II
GS10 Image processing I
GS11 Image processing II
GS12 Learning
GS13 Mobile vehicles
GS14 Neural networks I
GS15 Neural networks II
GS16 Pattern recognition &Visualization
GS17 Robotics I
GS18 Robotics II
GS19 Robotics III©ISAROB 2011

F. Dai

PS: Poster Session

PT: Plenary Talk

GS20 Robotics IV
GS21 Robotics V
PS1 Poste r Session
OS1 System sensing and control I
OS2 System sensing and control II
OS3 Contr ol and its application
OS4 Intelligent Control
OS5 Advanced vehicle control
OS6 AI-based systems for humanawareness promotion
OS7 Advanced technologies& Management skills
OS8 Structural change detection for
ongoing time series
OS9 Computer vision and sound analysis
OS10 Intelligent systems I
OS11 Control and automata
OS12 Special environment navigationand localization
OS13 Intuitive human-systeminteraction

P - 20

IT: Invited Talk

OS14 Dynamical information
processing in the brain
OS15 Bio-inspired theory
and application
OS16 Biomimetic machines
and robots
OS17 Medical scienceand complex system
OS18 Data mining
OS19 Embracing complexity innatural intelligence
OS20 Embracing complexity insensor system organization
OS21 Learning control
and robotics
OS22 Robotics andpattern recognition
OS23 Intelligent systems II
OS24 Human agent interaction
toward social modification

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27- 29, 2011

TIME TABLE (1/29)
Room C
1/29

8:00

(Sat)

8:40

Room E

Room F

Registration (Registration Desk)OS 14 ( 5 )
Chair

10:10
10:30

Room D

H. Suzuki

GS 6 ( 5 )

GS 14

(5)

OS 19 ( 6 )

Chair

Chair

T. Yamada

Chair Y. Ishida

J. S. Shiehwill end at 09:55

will end at 09:55

OS 6 ( 7 )GS 11 ( 5 )OS 15 ( 7 )OS 16 ( 7 )Chair J. Wang

Chair M. Yasunaga

Chairwill end at 09:55
Coffee Break

Chair

K. Hashimoto

K. Watanabe

will end at 11:45
12:15
13:10

Lunch
OS 20 ( 6 )OS 21 ( 6 )GS 12 ( 7 )IT 3 M. Sakata

Chair

Chair H. H. Lee

Chair

IT 4 H. Lund

Y . Ishida

Y. G. Zhang

will end at 14:55

IT 5 L. Pagliarini
Chair Y. I. Cho

14:40
15:00

will end at 14:55
Coffee BreakOS 22 ( 7 )OS 23 ( 6 )GS 20 ( 4 )GS 1 ( 4 )

Chair

Chair

Chair

Chair

C. Zhang

M. K. Habib

will end at 16:30
16:45
17:00

OS: Organized Session

GS1 Artificial intelligence
GS2 Artificial life I
GS3 Artificial life II
GS4 Chaos & Complexity
GS5 Contr ol techniques
GS6 Data mining
GS7 Evolutionary computations
GS8 Human-machine cooperative systems& Human-welfare robotics I
GS9 Human-machine cooperative systems& Human-welfare robotics II
GS10 Image processing I
GS11 Image processing II
GS12 Learning
GS13 Mobile vehicles
GS14 Neural networks I
GS15 Neural networks II
GS16 Pattern recognition &Visualization
GS17 Robotics I
GS18 Robotics II
GS19 Robotics III©ISAROB 2011

will end at 16:00

T. Nakashima

will end at 16:00

Farewell Party ( Room F)

18:00

GS: General Session

N. Uchiyama

PS: Poster Session

PT: Plenary Talk

GS20 Robotics IV
GS21 Robotics V
PS1 Poste r Session
OS1 System sensing and control I
OS2 System sensing and control II
OS3 Contr ol and its application
OS4 Intelligent Control
OS5 Advanced vehicle control
OS6 AI-based systems for humanawareness promotion
OS7 Advanced technologies& Management skills
OS8 Structural change detection for
ongoing time series
OS9 Computer vision and sound analysis
OS10 Intelligent systems I
OS11 Control and automata
OS12 Special environment navigationand localization
OS13 Intuitive human-systeminteraction

P - 21

IT: Invited Talk

OS14 Dynamical information
processing in the brain
OS15 Bio-inspired theory
and application
OS16 Biomimetic machines
and robots
OS17 Medical scienceand complex system
OS18 Data mining
OS19 Embracing complexity innatural intelligence
OS20 Embracing complexity insensor system organization
OS21 Learning control
and robotics
OS22 Robotics andpattern recognition
OS23 Intelligent systems II
OS24 Human agent interaction
toward social modification

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

TECHNICAL PAPER INDEXJanuary 27 (Thursday)
Room G
10:55~11:45 Plenary Talk
Chair Y. G. Zhang (Academia Sinica, China)
PT1 Development Outline of Humanoid Robot: HUBO II
J. H. Oh (KAIST, Korea)
Room A
12:40~14:25 Invited Talks
Chair M. Nakamura (Research Institute of Systems Control, Japan)
IT1
IT2

Human-Robot Interaction and Social Relation
M. Imai (Keio University, Japan)
Micro Nano Robotics
T. Arai (Osaka University, Japan)

January 28 (Friday)
Room G
10:15~11:05 Plenary Talk
Chair J. J. Lee (KAIST, Korea)
PT2 Data-driven two degree-of-freedom control for a micropump and microneedle
integrated device for diabetes careRuoting Yang (Washington University, USA)
Mingjun Zhang (University of Tennessee, USA)T. J. Tarn (Washington University, USA)
14:50~15:40 Plenary Talk
Chair H. H. Lund ( Technical University of Denmark, Denmark)PT3 Rescue Robot Systems - From Snake-like Robots to Human Interface F. Matsuno (Kyoto University, Japan)

January 29 (Saturday)
Room F
13:10~14:55 Invited Talks
Chair Y. I. Cho (Suwon University, Korea)
IT3
IT4
IT5

©ISAROB 2011

Quantifying “Waza” in Nihon-Buyo Dance Movements
M. Sakata (Doshisha University, Japan)
Social playware for mediating tele-play interaction over distanceH. H. Lund, T. Thorsteinsson (Technical University of Denmark, Denmark)An Educational Tool for Interactive Parallel and Distributed Processing
L.Pagliarini (Technical University of Denmark, Denmark / Academy of Fine Arts of Bari,
Italy)
H. H. Lund (Technical University of Denmark, Denmark)P - 22

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

January 27 (Thursday)
8:00~Registration

Room A

9:10~10:25 OS1 【System sensing and control I】
Chair: M. Uchida (The University of Electro-Communications, Japan)
Co-Chair: T. Mizuno (Tokyo Polytechnic University, Japan)
OS1-1 Analysis of speech signal based on frequency distribution and intonation
Naoya Ohta, Akio Nozawa (Meisei University, Japan)
OS1-2 Color influences on human being evaluated with nasal skin temperature
Tota Mizuno, Naoki Nakategawa, Yuichiro Kume (Tokyo Polytechnic University, Japan)
OS1-3 Air speed control of airship-type fish robot
Kunihiko Sato, Masafumi Uchida (The University of Electro-Communications, Japan)
OS1-4 Regulated plus and minus power supply using approximate 2DOF robust digital control
Atsushi Saitoh, Yusuke Tsuruhori, Yoshihiro Ohta, Kohji Higuchi, Kazushi Nakano (The University
of Electro-Communications, Japan)
OS1-5 Angle-based neuro-fuzzy navigation for autonomous mobile robots
Shu Hosokawa, Joji Kato, Kazushi Nakano, Kazunori Sakurama (The University ofElectro-Communications, Japan)

14:45~16:00 OS5 【Advanced vehicle control】
Chair: S. Sagara (Kyushu Institute of Technology, Japan)
Co-Chair: M. Oya (Kyushu Institute of Technology, Japan)
OS5-1 Robust active suspension control of vehicles with measurement noises
Katsuhiro Okumura (Fukuoka Industrial Technology Center, Japan)
Hideki Wada (Shin-Nippon Nondestructive Inspection, Japan)
Yuichiro Taira (National Fisheries University, Japan)
Masahiro Oya (Kyushu Institute of Technology, Japan)
OS5-2 A method to improve stability of adaptive steering driver-vehicle systems
Shingo Tamaru, Jinxin Zhuo, Qiang Wang, Masahiro Oya (Kyushu Institute of Technology, Japan)
OS5-3 Robust controller for underwater vehicle-manipulator systems including thruster dynamics
Yuichiro Taira (National Fisheries University, Japan)
Junpei Sugino (Kyushu Institute of Technology, Japan)
Natsuki Takagi (Miyakonojo National College of Technology, Japan)
Masahiro Oya (Kyushu Institute of Technology, Japan)
OS5-4 Digital adaptive control of a winged rocket applicable to abort flight
Tomoaki Shimozawa, Shinichi Sagara (Kyushu Institute of Technology, Japan)©ISAROB 2011

P - 23

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

OS5-5 A master-slave control system for semi-autonomous underwater vehicle manipulator system
Kana Kawano, Tomoaki Shimozawa, Shinichi Sagara (Kyushu Institute of Technology, Japan)

16:05~17:20
OS3 【Control and its application】
Chair: M. Yokomichi (University of Miyazaki, Japan)
Co-Chair: N. Takahashi (University of Miyazaki, Japan)
OS3-1 Observer-based guaranteed cost control
Erwin Susanto, Mitsuaki Ishitobi, Sadaaki Kunimatsu (Kumamoto University, Japan)
OS3-2 Observer based control of a manipulator system with structured uncertainty
Chikara Aikawa, Nobuya Takahashi, Osamu Sato, Michio Kono (University of Miyazaki, Japan)
OS3-3 Analysis of manipulator in consideration of collision between link and object
Asaji Sato (Miyakonojo National College of Technology, Japan)
Osamu Sato, Nobuya Takahashi, Masahiro Yokomichi (University of Miyazaki, Japan)
OS3-4 Motion control of 2 DOF orthogonal robots with adaptive control
Kazuma Funahashi, Feifei Zhang, Masanori Ito (Tokyo University of Marine Science and Technology,
Japan)
OS3-5 Development of 6-DOF force feedback system for rehabilitation of wrist paralysis
Yasunobu Hitaka (Kitakyushu National College of Technology, Japan)
Yoshito Tanaka (Fukuoka Institute of Technology, Japan)
Yutaka Tanaka (Hosei University, Japan)
Tomonori Kato (Fukuoka Institute of Technology, Japan)

Room B

9:10~10:10 OS18 【Data mining】
Chair: K. J. Mackin (Tokyo University of Information Sciences, Japan)
Co-Chair: T. Yamaguchi (Tokyo University of Information Sciences, Japan)
OS18-1 Two-level time-series clustering for satellite data analysis
Ayahiko Niimi, Takehiro Yamaguchi, Osamu Konishi (Future University Hakodate, Japan)
OS18-2 PSP practice support system using defect types based on phenomenon
Daisuke Yamaguchi (Toin University of Yokohama, Japan)
Ayahiko Niimi (Future University Hakodate, Japan)
Fumiyo Katayama, Muneo Takahashi (Toin University of Yokohama, Japan)
OS18-3 Development and evaluation of satellite image data analysis infrastructure
Akihiro Nakamura, Jong Geol Park, Kotaro Matsushita, Kenneth J. Mackin, Eiji Nunohiro (Tokyo
University of Information Sciences, Japan)
OS18-4 Application of neural network swarm optimization for paddy field classification from remote sensing
data
Kazuma Mori, Takashi Yamaguchi, Jong Geol Park, Kenneth J. Mackin (Tokyo University ofInformation Sciences, Japan)©ISAROB 2011

P - 24

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

12:40~14:25 GS10 【Image processing I】
Chair: H. Yamamoto (Gifu University, Japan)
GS10-1 Anomalous situations detection based on confluence
Tatsuya Gibo, Shigeki Aoki, Takao Miyamoto, Motoi Iwata, Akira Shiozaki (Osaka Prefecture
University, Japan)
GS10-2 Automatic segmentation of liver region from non-contrast and contrast CT images employing
tree-structural image transformations
Masafumi Komatsu, , Guangxu Li, Hyoungseop Kim, Joo Kooi Tan, Seiji Ishikawa (Kyushu
Institute of Technology, Japan)
Akiyoshi Yamamoto (Kyushu Institute of Technology and Kyoaikai Tobata Kyoritsu Hospital,
Japan)
GS10-3 Obstacle detection using a moving camera
Shao Hua Qian, Joo Kooi Tan, Seiji Ishikawa, Takashi Morie (Kyushu Institute of Technology,
Japan)
GS10-4 Hardware implementation of a census-based stereo matching using FPGA
Jiho Chang, Sung-Min Choi, Eul-Gyoon Lim, Jae-il Cho (Electronics and Telecommunication
Research Institute, Korea)
GS10-5 Hand detection using Adaboost
Jae-chan Jeong (Electronics and Telecommunication Research Institute and University of Scienceand Technology, Korea)
Seung-min-Choi (Electronics and Telecommunication Research Institute, Korea)
Ho-chul Shin (Electronics and Telecommunication Research Institute and University of Science andTechnology, Korea)
Jae-il Cho (Electronics and Telecommunication Research Institute, Korea)
GS10-6 Improvement of early recognition of gesture patterns based on Self-Organizing Map
Atsushi Shimada, Manabu Kawashima, Rin-ichiro Taniguchi (Kyushu University, Japan)
GS10-7 Detection of a bicycle and its driving directions using HOG Feature
Heewook Jung, Joo Kooi Tan, Hyeongseop Kim, Seiji Ishikawa (Kyushu Institute of technology,
Japan)

14:45~16:00 GS16 【Pattern recognition & Visualization】
Chair: K. Ikeda (Nara Institute of Science and Technology, Japan)
GS16-1 Parts layout decision of cell production assembly line using generic algorithm system and virtual
systemHidehiko Yamamoto, Tomokazu Watanabe, Takayoshi Yamada (Gifu University, Japan)
Masahiro Nakamura (LEXER RESEARCH Inc., Japan)
Rafat Hessen Elsayed Elshaer (Zagazig University, Egypt)
GS16-2 Parallelizing fuzzy rule generation using GPGPU
Takesuke Uenishi, Tomoharu Nakashima, Noriyuki Fujimoto (Osaka Prefecture University, Japan)©ISAROB 2011

P - 25

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

GS16-3 Interactive musical editing system to support human errors and offer personal preferences for an
automatic piano- A method for searching for similar phrases using DP matching Kenichi Koga, Kentaro Minowa, Eiji Hayashi (Kyusyu Institute of Technology, Japan)
GS16-4 Subsurface imaging for anti-personal mine detection by Bayesian super-resolution with Smooth-gap
prior
Satoshi Kozawa, Takashi Takenouchi, Kazushi Ikeda (Nara Institute of Science and Technology,
Japan)
GS16-5 Construction of a sense of force feedback and vision for micro-objects:
Development the Haptic Device's controlling
J.Noda, E.Hayashi (Kyushu Institute of Technology, Japan)

16:05~17:20 OS4 【Intelligent Control】
Chair: Y-M. Jia (Beihang University, China)
Co-Chair: C. Jia (Beihang University, China)
OS4-1 Feedback stabilization of linear systems with distributed input time-delay by backstepping method
Chaohua Jia (Beijing University, China)
OS4-2 Discrete-time iterative learning control for relative degree systems: a 2-D approach
Deyuan Meng, Yingmin Jia (Beihang University, China)
OS4-3 Distributed ｒobust ｃonsensus control of uncertain multi-agent systems
Yang Liu, Yingmin Jia (Beihang University, China)
OS4-4 Switching synchronization in a heterogeneous agent network
Lei Wang, Yang Liu, Qi-ye Zhang (Beihang University, China)
OS4-5 Model matching adaptive control of time delay systems with unknown relative degree
Haixia Su, Yingmin Jia (Beihang University, China)

Room C

9:10~10:10 OS2 【System sensing and control II】
Chair: H. Tanaka (Kogakuin University, Japan)
Co-Chair: A. Nozawa (Meisei University, Japan)
OS2-1 Quantitative evaluation of body-sway caused by tactile apparent movement
Kennichi Mogi, Masafumi Uchida (The University of Electro-Communications, Japan)
OS2-2 A visual-taste interference model and the EEG measurement
Hisaya Tanaka, Yuichi Sato (Kogakuin University, Japan)
OS2-3 Dynamic analysis of dorsal thermal image
Akio Nozawa, Yuya Takei (Meisei University, Japan)
OS2-4 Feature extraction of human face image for preference database
Yu Tachikawa, Akio Nozawa (Meisei University, Japan)©ISAROB 2011

P - 26

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

12:40~14:25 GS21 【Robotics V】
Chair: B.Fu (Sahnghai Jiao Tong University, China)
GS21-1 Designing practical omni-directional mobile module in the robot hardware platform for common
use
Moosung Choi, Sulhee Lee, Byungkyu Ahn, EunCheul Shin, Kwangyoong Yang, Hongseuk Kim(Korea Institute of Industrial Technology, Republic of Korea)
GS21-2 New acoustic positioning system for under water robot using multiple frequencies
Xiu Jing Gao, Feifei Zhang, Masanori Ito (Tokyo University of Marine Science and Technology,
Japan)
GS21-3 Development of under water use humanoid robot
Yunyi Li, Eturo Shimizu, Masanori Ito (Tokyo University of Marine Science and Technology,
Japan)
GS21-4 Linear estimation method for position and heading with RDOA measurements
G. H. Choi (Yonsei Unversity, Korea)
H. S. Cheon (Changwon National University, Korea)
J. B. Park (Yonsei Unversity, Korea)
T. S. Yoon (Changwon National University, Korea)
GS21-5 Omnidirectional state-changing gripper for various objects
Kenjiro Tadakuma (Osaka University, Japan)
Riichiro Tadakuma (Yamagata University, Japan)
Hiroki Tanaka, Takuto Fukuda, Mitsuru Higashimori, Makoto Kaneko (Osaka University, Japan)
GS21-6 Object co-occurrence graph for object search in 3D environment
Puwanan Chumtong, Yasushi Mae, Kenichi Ohara, Tomohito Takubo, Tatsuo Arai (Osaka
University, Japan)
GS21-7 Study on the wall climbing robot driven by the caterpillar tracks
Jiwu Wang, Yisong Wang, Weijie Gao (Beijing Jiaotong University, China)
Sugisaka Masanori (Nippon Bunri University, Japan)

14:45~15:45 OS24 【Human agent interaction toward social modification】
Chair: M. Imai (Keio University, Japan)
Co-Chair: H. Osawa (Japan Science and Technology Agency and Keio University, Japan)
OS24-1 Social modification using implementation of partial agency toward objects
Hirotaka Osawa (Japan Science and Technology Agency, Japan)
Seiji Yamada (National Institute of Informatics, Japan)
OS24-2 Inducement of attention to agent through averting gaze from the other
Manami Suzuki, Yugo Takeuchi (Shizuoka University, Japan)©ISAROB 2011

P - 27

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

OS24-3 Behaviors for getting conscious responses
Toshihiro Osumi, Masato Noda (Keio University, Japan)
Hirotaka Osawa (Japan Science and Technology Agency, Japan)
Yuki Kuwayama (Keio University, Japan)
Kazuhiko Shinozawa (ATR, Japan)
Michita Imai (Keio University, Japan)
OS24-4 Cross-modal effects between gestures and words in human robot interaction
Takamasa Iio (Advanced Telecommunications Research Institute International and Doshisha
University, Japan)
Masahiro Shiomi, Kazuhiko Shinozawa, Takaaki Akimoto (Advanced Telecommunications
Research Institute International, Japan)
Katsunori Shimohara (Doshisha University, Japan)
Norihiro Hagita (Advanced Telecommunications Research Institute International, Japan)

16:05~17:20 GS3 【Artificial life II】
Chair: D. E. Kim (Yonsei University, Korea)
GS3-1 Demand level investigation for future domestic system
Shuki Inoue, Eiji Mimura (R&D Center The Kansai Electric Power Co., Inc., Japan)
Shingo Aoki, Hiroshi Tsuji (Osaka Prefecture University, Japan)
GS3-2 Visualizing language evolution as an emergent phenomenon based on biological evolution andlearning
Tsubasa Azumagakito, Reiji Suzuki, Takaya Arita (Nagoya University)
GS3-3 Self-organizing stability of food web that emerges from the evolution of restrictions on speciation
Hirofumi Ochiai, Reiji Suzuki, Takaya Arita (Nagoya University, Japan)
GS3-4 Modeling electrosensory system of weakly electric fish
Miyoung Sim, DaeEun Kim (Yonsei University, South Korea)
GS3-5 Simultaneous cognitive origin of life and information
Koji Ohnishi (Niigata University, Japan)

Room D

9:10~10:10 GS4 【Chaos & Complexity】
Chair: S. Nakayama (Kagoshima University, Japan)
GS4-1 Study on discrete adiabatic quantum computation in 3-SAT problems
Mohamed El-fiky, Satoshi Ono, Shigeru Nakayama (Kagoshima University, Japan)
GS4-2 Fault-tolerant image filter design using particle swarm optimization
Zhiguo Bao, Fangfang Wang, Xiaoming Zhao, Takahiro Watanabe (Waseda University, Japan)
GS4-3 (p,q)-Duffing oscillators and nonlinear energy harvesting (withdrawal)
Tokuzo Shimada, Takanobu Moriya, Hisakazu Uchiyama (Meiji University, Japan)
GS4-4 A most simple, alternative non-computer-aided proof of the Four-Color Theorem
Koji Ohnishi (Niigata University, Japan)©ISAROB 2011

P - 28

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

12:40~14:40 PS1 【Poster session】
Chair: J. J. Lee (KAIST, Korea)
PS1-1 Authentication of the reconstructed image from Computer Generated Hologram:
To use the digital watermark
Norihiro Fujii, Ken-ichi Tanaka (Meiji University, Japan)
PS1-2 An aerial handwritten character recognition based on motion direction and ratio of
stroke length
Y. Nishida (Fukui University of Technology, Japan)
K. Ogura, H. Miura, N. Matsuda, H. Taki (Wakayama University, Japan)
N. Abe (Kyushu Institute of Technology, Japan)
PS1-3 Gamer robot for the rock-paper-scissors game by hand motion recognition
Ho Seok Ahn, Dong-wook Lee, Hogil Lee (Korea Institute of Industrial Technology, Republic ofKorea)
PS1-4 Analysis method of tooth meshing condition and motion of gears
Edzrol Niza Mohamad, Masaharu Komori, Hiroaki Murakami, Aizoh Kubo (Kyoto University, Japan)
PS1-5 Research for an adaptive user’s intent-detection method for the use in rehabilitation robots
Jun-Young Jung, Duk-Yeon Lee, DongWoon Choi, In-Hun Jang, Ho-Gil Lee, HyunSub Park,
Dong-Wook Lee (University of Science and Technology and Korea Institute of Industrial Technology,
Republic of Korea)
PS1-6 Development of touch sensor system of silicone-type artificial skin for an interactive
android robot
Duk-Yeon Lee, Dong-Wook Lee, Dongwoon Che, Hyun-Sub Park, Ho-Gil Lee, Jun-Young Jung
(Korea Institute of Industrial Technology, Republic of Korea)
PS1-7 Remote control and conversation system between human and android robot via internet
Dong-Wook Lee, Dongwoon Che, Jun-Young Joung, Hoseok Ahn, Hyunsub Park, Ho-Gil Lee(Korea Institute of Industrial Technology, Republic of Korea)
PS1-8 The optimal method of searching the effective combination of dither matrix
Tsuneyasu Kato, Ken-ichi Tanaka (Meiji University, Japan)

14:45~16:00 OS17 【Medical science and complex system】
Chair: K. Naitoh (Waseda University, Japan)
Co-Chair: T. Matsuo (Kanagawa Institute of Technology, Japan)
OS17-1 The inevitability of the bio-molecules: five nitrogenous bases and twenty amino acids
K.Hashimoto, H.Inoue, K.Naitoh (Waseda University, Japan)
OS17-2 Engine for brain development: Similarity between engine and brain
H. Kawanobe, K. Naitoh (Waseda University, Japan)
OS17-3 Spatial surface wave spread network from Ambient Noise correlation
Z. Zheng, K. Yamasaki, T. Fujiwara, N. Sakurai, K. Yoshizawa (Tokyo University of Information
Science, Japan)©ISAROB 2011

P - 29

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

OS17-4 Effect of hot summer against environment which was induced by extra economical demand via
Japanese Ecological Footprint
K. Yamasaki, T. Fujiwara, N. Sakurai, K. Yoshizawa, Z. Zheng (Tokyo University of Information
Sciences, Japan)
OS17-5 Blood flow velocity waveforms in the middle cerebral artery at rest and during exercise
T. Matsuo, S. Watanabe, M. Sorimachi, M. Kanda, Y. Ohta (Kanagawa Institute of Technology,
Japan)
T. Takahashi (Asahikawa Medical College, Japan)

16:05~17:35 GS19 【Robotics III】
Chair: A. Nakamura (National Institute of Advanced Science and Technology, Japan)
GS19-1 Design of the transmitting device for motion control of robots
Masaharu Komori, Jungchul Kang, Fumi Takeoka, Yukihiko Kimura (Kyoto University, Japan)
GS19-2 Construction and basic performance tests of underwater monitoring network
Chunhu Liu, Bin Fu, Han Zhang, Lian Lian (Shanghai Jiao tong University, China)
GS19-3 On the use of human instruction for improving the behavior of RoboCup soccer agents
Yosuke Nakamura, Tomoharu Nakashima (Osaka Prefecture University, Japan)
GS19-4 Development of an autonomous-drive personal robot“An environment recognition system using image processing and an LRS”
Yasushi Kibe, Hideki Ishimaru, Eiji Hayashi (Kyushu Institute of Technology, Japan)
GS19-5 Design of robotic behavior that imitates animal consciousness－Emotion expression of robotic arm based on eyeball movement－
Koichiro Kurogi, Kei Ueyama, Eiji Hayashi (Kyushu Institute of Technology, Japan)
GS19-6 Development and experimental study of a novel pruning robot
S. Ueki (Toyota Colleges of Technology, Japan)
H. Kawasaki (Gifu University, Japan)
Y. Ishigure (Marutomi Seikou Co., Ltd., Japan)
K. Koganemaru (Gifu University, Japan)
Y. Mori (Hashima karyuu kougyou Ltd., Japan)

January 28 (Friday)
8:00~ Registration

Room A

8:40~9:55 GS5 【Control techniques】
Chair: J. Murata (Kyushu University, Japan)
GS5-1 Autonomous decentralized control scheme for large scale and dense wireless sensor networks withmultiple sinks
Masahito Maki, Akihide Utani, Hisao Yamamoto (Tokyo City University, Japan)©ISAROB 2011

P - 30

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

GS5-2 Advanced adaptive communication protocol for ubiquitous sensor networks
Shuichi Osawa, Akihide Utani, Hisao Yamamoto (Tokyo City University, Japan)
GS5-3 Lateral control of unmanned vehicle using PD controller
Young Chul Cha (Pusan National University, Korea)
Jong Il Bae (Pukyung National University, Korea)
Kil Soo Lee, Dong Seok Lee, Yun Ja Lee, Man Hyung Lee (Pusan National University, Korea)
GS5-4 A control method with pheromone information for transport system
Yoshitaka Imoto, Yasutaka Tsuji, Eiji Kondo (Kyushu University, Japan)
GS5-5 Implementation of real-time distributed control for discrete event robotic manufacturing systems
using Petri nets
Gen’ichi Yasuda (Nagasaki Institute of Applied Science, Japan)

11:05~12:20 OS7 【Advanced technologies & Management skills】
Chair: T. Ito (Ube National College of Technology, Japan)
Co-Chair: Y. Uchida (Ube National College of Technology, Japan)
OS7-1 An analysis of firm’ capacity in Mazda’s Keiretsu
R. Takida, T. Ito, S. Matsuno (Ube National College of Technology, Japan)
K. Voges (University of Canterbury, New Zealand)
Y. Ishida (Toyohashi University of Technology, Japan)
M. Sakamoto (University of Miyazaki, Japan)
OS7-2 An analysis of structure importance in Mazda’s Keiretsu
S. Tagawa, R. Takida, T. Ito (Ube National College of Technology, Japan)
R. Mehta (New Jersey Institute of Technology, USA)
H. Hasama (Fukuoka Institute of Technology, Japan)
M. Sakamoto (University of Miyazaki, Japan)
OS7-3 A centrality analysis between transactions and cross shareholdings in Mazda’s Keiretsu
E. Niki, R. Takida, T. Ito (Ube National College of Technology, Japan)
R. Mehta (New Jersey Institute of Technology, USA)
L. P. NG (Hoecheong Industries (Holding) Co., Ltd, China)
M. Sakamoto (University of Miyazaki, Japan)
OS7-4 Bottom-up pyramid cellular acceptors with four-dimensional layers
Yasuo Uchida, Takao Ito (Ube National College of Technology, Japan)
Makoto Sakamoto, Takashi Ide, Kazuyuki Uchida, Ryoju Katamune,
Hiroshi Furutani, Michio Kono, Satoshi Ikeda (University of Miyazaki, Japan)
Tsunehiro Yoshinaga (Tokuyama College of Technology, Japan)
OS7-5 Cooperating systems of four-dimensional finite automata
Yasuo Uchida, Takao Ito (Ube National College of Technology, Japan)
Makoto Sakamoto, Kazuyuki Uchida, Takashi Ide, Ryoju Katamune,
Hiroshi Furutani, Michio Kono (University of Miyazaki, Japan)
Tsunehiro Yoshinaga (Tokuyama College of Technology, Japan)©ISAROB 2011

P - 31

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

13:15~14:45 OS10 【Intelligent systems I】
Chair: K-L. Su (National Yunlin University of Science and Technology, Taiwan)
Co-Chair: J-H. Tzou (National Formosa University, Taiwan)
OS10-1 The study of path error for an Omnidirectional Home Care Mobile Robot
Jie-Tong Zou (National Formosa University, Taiwan)
Feng-Chun Chiang (WuFeng Institute of Technology, Taiwan)
OS10-2 A* searching algorithm applying in Chinese chess gameCheng-Yun Chung (National Yunlin University of Science and Technology, Taiwan)
Te-Yi Hsu (Industrial Technology Research Institute, Taiwan)
Jyh-Hwa Tzou (National Formosa University, Taiwan)
Kuo-Lan Su (National Yunlin University of Science and Technology, Taiwan)
OS10-3 Multi-robot based intelligent security system
Yi-Lin Liao, Kuo-Lan Su (National Yunlin University of Science and Technology, Taiwan)
OS10-4 Implementation of an auction algorithm based multiple tasks allocation using mobile robots
Kuo-Lan Su, Jr-Hung Guo (National Yunlin University of Science and Technology, Taiwan)
Chun-Chieh Wang (Chienkuo Technology University, Taiwan)
Cheng-Yun Chung (National Yunlin University of Science and Technology, Taiwan)
OS10-5 Fuzzy programming for mixed-integer optimization problems
Yung-Chin Lin (National Yunlin University of Science and Technology, Taiwan)
Yung-Chien Lin (WuFeng University, Taiwan)
Kuo-Lan Su (National Yunlin University of Science and Technology, Taiwan)
Wei-Cheng Lin (I-Shou University, Taiwan)
Tsing-Hua Chen (WuFeng University, Taiwan)
OS10-6 Develop a vision based auto-recharging system for mobile robots
Ting-Li Chien (Wu-Feng University, Taiwan)

16:00~17:30 OS1 1 【Control and automata】
Chair: K-H. Hsia (Far East University, Taiwan)
Co-Chair: K-L. Su (National Yunlin University of Science and Technology, Taiwan)
OS11-1 Super-twisting second order sliding mode control for a synchronous reluctance motor
Huann-Keng Chiang, Wen-Bin Lin, Chang-Yi Chang (National Yunlin University of Science andTechnology, Taiwan)
Chien-An Chen (Automotive Research and Testing Center, Taiwan)
OS11-2 Shape recognition applied in a semi-autonomous weapon robot
Chun-Chieh Wang, Chyun-Luen Lin (Chienkuo Technology University, Taiwan)
Kuo-Lan Su (National Yunlin University of Science and Technology, Taiwan)
OS11-3 Camera position estimation and feature extraction from incomplete image of landmark
Kuo-Hsien Hsia (Far East University, Taiwan)
Shao-Fan Lien (National Yunlin University of Science and Technology, Taiwan)
Juhng-Perng Su (National Yunlin University of Science and Technology and Overseas Chinese
University, Taiwan)©ISAROB 2011

P - 32

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

OS11-4 A novel variable structure theory applied in design for wheeled mobile robots
Chun-Chieh Wang (Chienkuo Technology University, Taiwan)
OS11-5 Parameter identification of Lorenz system using RBF neural networks with time-varying learningalgorithm
Chia-Nan Ko, Yu-Yi Fu, Cheng-Ming Lee, Guan-Yu Liu (Nan-Kai University of Technology,
Taiwan)OS11-6 Motion planning of a landmine detection robot
Kuo-Lan Su (National Yunlin University of Science and Technology, Taiwan)
Hsu-Shan Su, Sheng-Wen Shiao, Jr-Hung Guo (Yunlin University of Science and Technology,
Taiwan)Room B

8:40~9:55 OS8 【Structural change detection for ongoing time series】
Chair: T. Hattori (Kagawa University, Japan)
Co-Chair: H. Kawano (NTT Advanced Technology, Japan)
OS8-1 DP method for structural change detection as optimal stopping---- verification and extension --Tetsuo Hattori, Katsunori Takeda (Kagawa University, Japan)
Hiromichi Kawano (NTT Advanced Technology, Japan)
Tetsuya Izumi (Micro Technica Co., Ltd., Japan)
OS8-2 Change detection experimentation for multiple regression using ESPRT---- one variation is periodic function --Katsunori Takeda, Tetsuo Hattori (Kagawa University, Japan)
Hiromichi Kawano (NTT Advanced Technology, Japan)
Tetsuya Izumi (Micro Technica Co., Ltd., Japan)
OS8-3 Continuous change point detection for time series images using ESPRT
Katsunori Takeda, Tetsuo Hattori (Kagawa University, Japan)
Hiromichi Kawano (NTT Advanced Technology, Japan)
Tetsuya Izumi (Micro Technica Co., Ltd., Japan)
Shinichi Masuda (C Micro Co., Ltd., Japan)
OS8-4 DP method for structural change detection as optimal stopping---- experimentation in multiple regression model --Hiromichi Kawano (NTT Advanced Technology, Japan)
Tetsuo Hattori, Katsunori Takeda (Kagawa University, Japan)
Tetsuya Izumi (Micro Technica Co., Ltd., Japan)
OS8-5 Comparison of change detection methods for ongoing time series data---- extended SPRT, Chow Test, extended DP ---Hiromichi Kawano (NTT Advanced Technology, Japan)
Tetsuo Hattori, Katsunori Takeda (Kagawa University, Japan)
Tetsuya Izumi (Micro Technica Co., Ltd., Japan)©ISAROB 2011

P - 33

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

11:05~12:05 GS8【Human-machine cooperative systems & Human-welfare robotics I】
Chair: T. Fuchida (Kagoshima University, Japan)
GS8-1 EMG control of a pneumatic 5-fingered hand using a Petri net
Osamu Fukuda, Jonghwan Kim (National Institute of Advanced Industrial Science and Technology,
Japan)
Isao Nakai, Yasunori Ichikawa (SQUSE Inc., Japan)
GS8-2 Haptic warning method on steering behavior for urgent lane change situation
Jae-Woo Sim, Chi-Hoon Shin (University of Science and Technology, Korea)
Young Woo Kim (Electronics and Telecommunication Research institute, Korea)
GS8-3 Depth calculation by using Face detection ASIC
Seung Min Choi (Electronics and Telecommunications Research Institute, Korea)
Jae-chan Jeong (University of Science and Technology, Korea)
Jaeil Cho (Electronics and Telecommunications Research Institute, Korea)
GS8-4 An experimental study on interactive reinforcement learning
Tomoharu Nakashima, Yosuke Nakamura, Takesuke Uenishi, Yosuke Narimoto (Osaka Prefecture
University, Japan)

13:15~14:45 GS2 【Artificial life I】
Chair: T. Arita (Nagoya University, Japan)
GS2-1 Improved artificial bee colony algorithm for large-scale optimization problems
Ryuta Gocho, Akihide Utani, Hisao Yamamoto (Tokyo City University, Japan)
GS2-2 Simulation of self-reproduction phenomenon of cells in two-dimensional hybrid-cellular automata
model
Takeshi Ishida (Nippon Institute of Technology, Japan)
GS2-3 A physics modeling of butterfly’s flight control by GA and ANN and its over-evolution problem
R. Ooe, I. Suzuki, M. Yamamoto, M. Furukawa (Hokkaido University, Japan)
GS2-4 Emergence of behavior intelligence on artificial creature in different virtual fluid environments
Keita Nakamura, Ikuo Suzuki, Masahito Yamamoto, Masashi Furukawa (Hokkaido University,
Japan)
GS2-5 Behavior emergence of virtual creature living in complex environments
Kenji Iwadate, Ikuo Suzuki, Masahito Yamamoto, Masashi Furukawa (Hokkaido University, Japan)
GS2-6 Three-dimensional morphogenesis by cell division and death in viscoelastic amorphous computing
Eisuke Arai, Fumiaki Tanaka, Masami Hagiya (University of Tokyo, Japan)©ISAROB 2011

P - 34

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

16:00~17:30 GS7 【Evolutionary computations】
Chair: P. Sapaty (National Academy of Sciences, Ukraine)
GS7-1 A study on efficient query dissemination in distributed sensor networks- Forwarding power adjustment of each sensor node using particle swarm optimization Junya Nagashima, Akihide Utani, Hisao Yamamoto (Tokyo City University, Japan)
GS7-2 Evaluation of a competitive particle swarm optimizer in multimodal functions with complexity
Yu Taguchi, Hidehiro Nakano, Akihide Utani, Arata Miyauchi, Hisao Yamamoto (Tokyo City
University, Japan)
GS7-3 An effective allocation method of ZigBee sensor nodes using a discrete particle swarm optimizer
Ryota Saito, Hidehiro Nakano, Akihide Utani, Arata Miyauchi, Hisao Yamamoto (Tokyo City
University, Japan)
GS7-4 An implementation of probabilistic model-building coevolutionary algorithm
Takahiro Otani, Takaya Arita (Nagoya University, Japan)
GS7-5 Modeling and solution for optimization problems with incomplete information-- A general framework and an application to cruising taxi problems -Makoto Ohara, Hisashi Tamaki (Kobe University, Japan)
GS7-6 Prime number generation using memetic programming
Emad Mabrouk (Kyoto University, Japan)
Julio César Hernández Castro (University of Portsmouth, UK)
Masao Fukushima (Kyoto University, Japan)

Room C

8:40~9:55 OS13 【Intuitive human-system interaction】
Chair: M. Yokota (Fukuoka Institute of Technology, Japan)
Co-Chair: T. Oka (Nihon University, Japan)

OS13-1 A High-speed 3D image Measurement Method
Ke Sun, Yundi Yao, Cunwei Lu (Fukuoka Institute of Technology, Japan)
OS13-2 3-D face recognition method based on optimum 3-D image measurement technology
Hiroya Kamitomo, Yao Xu, Cunwei Lu (Fukuoka Institute of Technology, Japan)
OS13-3 User study of a life-supporting humanoid directed in a multimodal language
T. Oka (Nihon University, Japan)
T. Abe (Nihon Computer Kaihatsu Co. Ltd, Japan)
K. Sugita, M. Yokota (Fukuoka Institute of Technology, Japan)
OS13-4 A multimodal language to communicate with life supporting robots through a touch screen and aspeech interface
T. Oka, H. Matsumoto, R. Kibayashi (Nihon University, Japan)©ISAROB 2011

P - 35

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

OS13-5 Some consideration on user interface switching functions for the weaker at IT
Shinichi Inenaga, Kaoru Sugita (Fukuoka institute of technology, Japan)
Tetsushi Oka (Nihon University, Japan)
Masao Yokota (Fukuoka institute of technology, Japan)

11:05~12:05 GS9【Human-machine cooperative systems & Human-welfare robotics II】
Chair: N. Okada (Kyushu University, Japan)
GS9-1 Crutch gait pattern for robotic orthosis by the feature extraction
In Hun Jang, Jun-Young Jung, Duk-Yeon Lee, DongWook Lee, Ho-Gil Lee, HyunSub Park
(Korea Institute of Industrial Technology, Republic of Korea)
GS9-2 Design of personal support system in telephone correspondence using smart phone
Hiroyuki Nishiyama, Fumio Mizoguchi (Tokyo University of Science, Japan)
GS9-3 Advanced networking and robotics for societal engagement and support of elders
Peter Sapaty (National Academy of Sciences, Ukraine)
Masanori Sugisaka (Nippon Bunri University, Japan)
GS9-4 Construction of the muscle fatigue evaluation model based on accuracy of power
Akihiro Suzuki, Norihiko Kato, Yoshihiko Nomura, Hirokazu Matsui (Mie University, Japan)

13:15~14:45 OS9 【Computer vision and sound analysis】
Chair: Y. Yoshitomi (Kyoto Prefectural University, Japan)
Co-Chair: M. Tabuse (Kyoto Prefectural University, Japan)
OS9-1 Outdoor autonomous navigation using SURF features
M. Tabuse, T. Kitaoka (Kyoto Prefectural University, Japan)
D. Nakai (Kyoto Prefectural Subaru High School, Japan)
OS9-2 Facial expression recognition of a speaker using front-view face judgment, vowel
judgment and thermal image processing
T. Fujimura (Works Applications Co. Ltd, Japan)
Y. Yoshitomi, T. Asada, M. Tabuse (Kyoto Prefectural University, Japan)
OS9-3 Facial expression recognition of a speaker using vowel judgment and thermal image
processing
Y. Yoshitomi, T. Asada (Kyoto Prefectural University, Japan)
K. Shimada (Nova System Co., Ltd., Japan)
M. Tabuse (Kyoto Prefectural University, Japan)
OS9-4 A human-machine cooperative system for generating sign language animation using thermal image
processing, fuzzy algorithm, and simulated annealing
T. Asada, Y. Yoshitomi (Kyoto Prefectural University, Japan)
OS9-5 Music recommendation system using the time-series discrete wavelet transform and the fastICAK. Horiike (Taka Dance Fashion Co., Ltd., Japan)
Y. Yoshitomi (Kyoto Prefectural University, Japan)
T. Tokuyama (Shofu Inc., Japan)
M. Tabuse (Kyoto Prefectural University, Japan)©ISAROB 2011

P - 36

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

OS9-6 Music recommendation system aimed at improving recognition ability
H. Konishi, Y. Yoshitomi (Kyoto Prefectural University, Japan)

16:00~17:30 GS18 【Robotics II】
Chair: E. Inohira (Kyushu Institute of Technology, Japan)
GS18-1 DEA various method used Restricted Multiplier DEA
Shingo Aoki, Ryota Gejima, Tomoharu Nakashima (Osaka Prefecture University, Japan)
GS18-2 Efficient distributed ontology management scheme for inference in surveillance networks
Soomi Yang (The University of Suwon, Korea)
GS18-3 Recovery technique from classified errors in adjustment tasks of domestic appliances
Akira Nakamura, Yoshihiro Kawai (National Institute of Advanced Industrial Science andTechnology, Japan)
GS18-4 A visual debugger for developing RoboCup soccer 3D agent
Yosuke Nakamura, Tomoharu Nakashima (Osaka Prefecture University, Japan)
GS18-5 Development of pulse control type MEMS micro robot with hardware neural network
Kazuto Okazaki, Tatsuya Ogiwara, Dongshin Yang, Kentaro Sakata, Ken Saito, Yoshifumi Sekine,
Fumio Uchikoba (Nihon University, Japan)

Room D

8:40~9:55 GS15 【Neural networks II】
Chair: N. Kamiura (University of Hyogo, Japan)
GS15-1 Medical image diagnosis of lung cancer by revised GMDH-type neural network using various kindsof neurons
Tadashi Kondo, Junji Ueno (The University of Tokushima, Japan)
GS15-2 Neural Network application using GPGPU
Yuta Tsuchida, Michifumi Yoshioka (Osaka Prefecture University, Japan)
GS15-3 Evaluation of an optimal design method for multilayer perceptron by using the Design ofExperiments
E. Inohira, H. Yokoi (Kyushu Institute of Technology, Japan)
GS15-4 A learning method for dynamic Bayesian network structures using a multi-objective particle swarm
optimizer
Kousuke Shibata, Hidehiro Nakano, Arata Miyauchi (Tokyo City University, Japan)
GS15-5 Remarks on folding behavior of mapping capability of neural network direct controller for
many-to-one plant
Takayuki Yamada (Ibaraki University, Japan)©ISAROB 2011

P - 37

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

11:05~12:20 OS12 【Special environment navigation and localization】
Chair: J. M. Lee (Pusan National University, South Korea)
OS12-1 Optimal posture control of two wheeled inverted pendulum robot on a slanted surface
Youngkuk Kwon, Joonbae Son, Jaeoh Lee, Jongho Han, Jangmyung Lee (Pusan National University,
Korea)
OS12-2 Efficient CAN-based network for marine engine state monitoring system
Junseok Lee, Yoseop Hwang, Jaehan Jo, Jangmyung Lee (Pusan National University, Korea)
OS12-3 Localization Algorithm using Virtual Label for a Mobile Robot in Indoor and Outdoor Environment
Kiho Yu, Mincheol Lee, Junghun Heo, Youngeun Moon (Pusan National University, South Korea)
OS12-4 Potential field method applied on the navigation of multiple mobile robots with limited ultrasonicsensingChiyen Kim, Mincheol Lee, Junyoung Beak, Chibeom Noh (Pusan National University, South
Korea)
OS12-5 Adaptive tuning of a Kalman filter using Fuzzy logic for attitude reference system
Taerim Kim, Joocheol Do, Eunkook Jung, Gyeongdong Baek, Sungshin Kim (Pusan National
University, South Korea)

13:15~14:30 GS17 【Robotics I】
Chair: D. W. Lee (Korea Institute of Industrial Technology, Korea)
GS17-1 Trajectory control of biomimetic robots for demonstrating human arm movements
T. Kashima, M. Iwaseya (Tomakomai National College of Technology, Japan)
GS17-2 A study on reinforcement learning scheme in cooperative network-systems of sensor nodes andmobile robots
Koichi Hirayama, Akihide Utani, Hisao Yamamoto (Tokyo City University, Japan)
GS17-3 Towards the automation of cashew shelling operation
Son Doan Tran (Ho Chi Minh University of Technology, Vietnam)
Naoki Uchiyama, Suguru Kirita, Norifumi Yamanaka (Toyohashi University of Technology, Japan)
Phat Minh Ho (Ho Chi Minh University of Technology, Vietnam)
GS17-4 Research on the velocity variation method for precise motion transmission
Masaharu Komori, Jungchul Kang, Yukihiko Kimura, Fumi Takeoka (Kyoto University, Japan)
GS17-5 Towards the enhancement of biped locomotion and control techniques - walking pattern classification
Basak Yuksel (Mitsubishi Electric Corporation, Advanced Technology R&D Center, Japan andMiddle East Technical University, Turkey)
Kemal Leblebicioğlu (Middle East Technical University, Turkey)©ISAROB 2011

P - 38

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

16:00~17:30 GS13 【Mobile vehicles】
Chair: F. Dai (Tianjin University of Science & Technology, China)
GS13-1 Geometrical analysis and design of motion transmitting element for mobile vehicles
Masaharu Komori, Fumi Takeoka, Jungchul Kang, Yukihiko Kimura (Kyoto University, Japan)
GS13-2 Underwater unknown acoustic source localization based on Sound Propagation Loss: theory
analysis and simulation results
Han Zhang, Bin Fu, Chunhu Liu, Lian Lian (Shanghai Jiao tong University, China)
GS13-3 Position recognition system of autonomous vehicle via Kalman filtering
Dong Jin Kim (Pusan National University, Korea)
Jong Il Bae (Pukyung National University, Korea.)
Kil Soo Lee, Dong Seok Lee, Yun Ja Lee, Man Hyung Lee (Pusan National University, Korea)
GS13-4 Dynamic modeling, stability and energy efficiency analysis of crab walking of a six-legged
robot (withdrawal)
Shibendu Shekhar Roy (National Institutes of Technology, India)
Dilip Kumar Pratihar (Indian Institutes of Technology, India)
GS13-5 Motion analysis of towed vehicle on survey system for deep sea
Kohei Oshima, Etsuro Shimizu, Masanori Ito, Sadanobu Omichi (Tokyo University of Marine
Science and Technology Japan)
GS13-6 Path planning of an autonomous mobile robot considering region with velocity constraint in realenvironment
Tae Hyon Kim (Kyoto University, Japan)
Kiyohiro Goto (The University of Electro-Communications, Japan)
Hiroki Igarashi, Kazuyuki Kon, Noritaka Sato, Fumitoshi Matsuno (Kyoto University, Japan)

January 29 (Saturday)
8:00~ Registration

Room C

8:40~9:55 OS14 【Dynamical information processing in the brain】
Chair: H. Suzuki (The University of Tokyo, Japan)
Co-Chair: T. Kohno (The University of Tokyo, Japan)
OS14-1 Signal transmission in multilayer asynchronous neural networks
Wataru Kobayashi, Makito Oku, Kazuyuki Aihara (The University of Tokyo, Japan)
OS14-2 A two-variable silicon neuron circuit based on Izhikevich model
Nobuyuki Mizoguchi, Yuji Nagamatsu, Kazuyuki Aihara, Takashi Kohno (The University of Tokyo,
Japan)
OS14-3 A three-variable silicon neuron circuit
Yohei Nakamura, Kazuyuki Aihara, Takashi Kohno (University of Tokyo, Japan)©ISAROB 2011

P - 39

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

OS14-4 Theory of mind in a microscopic pedestrian simulation model
Ryo Adachi, Kazuyuki Aihara (The University of Tokyo, Japan)
OS14-5 A neural network model for categorical effects in color memory
Chihiro Imai (University of Tokyo, Japan)
Satohiro Tajima (Japan Broadcasting Corporation, Japan)
Kazuyuki Aihara, Hideyuki Suzuki (University of Tokyo, Japan)

10:30~12:15 OS6 【AI-based systems for human awareness promotion】
Chair: K Hashimoto (Osaka Prefecture University, Japan)
Co-Chair: K. Takeuchi (Osaka Electro-Communication University, Japan)
OS6-1 A Survey of AI-based systems for human awareness promotion in meta-cognition
Kiyota Hashimoto, Kazuhisa Seta, Hiroshi Tsuji (Osaka Prefecture University, Japan)
Kazuhiro Takeuchi (Osaka Electro-Communication University, Japan)
OS6-2 A multilingual problem-based learning environment for awareness promotion
Ryusuke Taguchi, Katsuko T. Nakahira (Nagaoka University of Technology, Japan)
Hideyuki Kanematsu (Suzuka National College for Technology, Japan)
Yoshimi Fukumura (Nagaoka University of Technology, Japan)
OS6-3 An effective visualization of style inconsistencies for interactive text editing
Kazuki Shimamura, Kazuhiro Takeuchi (Osaka Electro-Communication University, Japan)
Kiyota Hashimoto (Osaka Prefecture University, Japan)
OS6-4 An intelligent meta-learning support system through presentation
Kazuhisa Seta (Osaka Prefecture University, Japan)
OS6-5 A task ontology construction for presentation skills
Kiyota Hashimoto (Osaka Prefecture University, Japan)
Kazuhiro Takeuchi (Osaka Electro-Communication University, Japan)
OS6-6 Country domain governance: An analysis by datermining of country domains
Katsuko T. Nakahira, Hiroyuki Namba, Minehiro Takeshita, Shigeaki Kodama, Yoshiki Mikami(Nagaoka University of Technology, Japan)
OS6-7 Extraction and comparison of tourism information on the web
Xiaobin Wu, Sachio Hirokawa, Chengjiu Yin, Tetsuya Nakatoh, Yoshiyuki Tabata (Kyushu
University, Japan)

13:10~14:40 OS20 【Embracing complexity in sensor system organization】
Chair: Y. Ishida (Toyohashi University of Technology, Japan)
Co-Chair: Y. Watanabe (Nagoya City University, Japan)
K.
Harada (Toyohashi University of Technology, Japan)
OS20-1 Performance evaluation of immunity-based statistical en-route filtering in wireless sensor networks
Yuji Watanabe (Nagoya City University, Japan)©ISAROB 2011

P - 40

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

OS20-2 Improvement of immunity-based diagnosis for a motherboard
Haruki Shida, Takeshi Okamoto (Kanagawa Institute of Technology, Japan)
Yoshiteru Ishida (Toyohashi University of Technology, Japan)
OS20-3 Visualization of keystroke data and its interpretation
T. Samura, K. Tani (Akashi National College of Technology, Japan)
Y. Ishida (Toyohashi University of Technology, Japan)
OS20-4 Extraction of learning point by visualization of skill
Shihoko Kamisato, Yukihiro Mori, Nobuhiro Yamashiro, Kentaro Noguchi (Okinawa National
College of technology, Japan)
Yoshiteru Ishida (Toyohashi University of Technology, Japan)
OS20-5 Prediction of electron flux environment at geosynchronous orbit using neural network technique
Kentarou Kitamura, Yusuke Nakamura (Tokuyama College of Technology, Japan)
Masahiro Tokumitsu,Yoshiteru Ishida (Toyohashi University of Technology, Japan)
Shinichi Watari (National Institute of Information and Communications Technology, Japan)
OS20-6 A diagrammatic classification in a combinatorial problem: The case of a Stable Marriage Problem
Tatsuya Hayashi, Yoshikazu Hata, Yoshiteru Ishida (Toyohashi University of Technology, Japan)

15:00~16:45 OS22 【Robotics and pattern recognition】
Chair: C. Zhang (Tsinghua University, China)
Co-Chair: T. Zhang (Tsinghua University, China)
OS22-1 Application of the genetic algorithm on face recognition
Fengzhi Dai (Tianjin University of Science and Technology, China)
Liqiang Shang (Vestas Wind Technology (China) Co., Ltd, China)
Naoki Kushida (Oshima National College of Maritime Technology, Japan)
Masanori Sugisaka (Nippon Bunri University, ALife Robotics Co., Ltd, Japan)
OS22-2 Study on the disturbance rejection of Virtual Slope Walking by Stepper-2D Robot
Mingguo Zhao (State Key Laboratory of Robotics and System (HIT) and Tsinghua University,
China)
HaoDong, Naiyao Zhang (State Key Laboratory of Robotics and System (HIT), China)
OS22-3 A new method for mobile robots to avoid collision with moving obstacles
Yi Zhu, Tao Zhang, Jingyan Song (Tsinghua University, China)
Masatoshi Nakamura (Saga University, Japan)
OS22-4 Some thought for the Mckibben muscle robots
Huailin Zhao (Shanghai Institute of Technology, China)
Xiaoqing Jia (Shanghai Maritime University, China)
Masanori Sugisaka (Nippon Bunri University, Japan)
OS22-5 Vehicle 3D estimation based on time series images and prior knowledge
Haoyin Zhou, Tao Zhang, Changshui Zhang, Peng He (Tsinghua University, China)©ISAROB 2011

P - 41

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

OS22-6 Automatic drawing of correct topographical distribution of EEG rhythms based on unified suitablereference selection
Bei Wang, Xingyu Wang (East China University of Science and Technology, China)
Akio Ikeda (Kyoto University, Japan)
Takashi Nagamine (Sapporo Medical University, Japan)
Hiroshi Shibasaki (Takeda General Hospital Research, Japan)
Masatoshi Nakamura (Saga University, Japan)
OS22-7 Research on surface crack detection based on laser scanning and image processing techniques
Guangming Cai, Jiwu Wang, Mingcheng E, Wenliang Guo (Beijing Jiaotong University, China)
Sugisaka Masanori (Nippon Bunri University, Japan)

Room D

8:40~9:55 GS6 【Data mining】
Chair: J. S. Shieh (Yuan Ze University, Taiwan)
GS6-1 Development of motion analysis system using acceleration sensors for tennis and its evaluations
Takaya Maeda, Kenichirou Fuji, Hiroki Tamura, Koichi Tanno (University of Miyazaki, Japan)
GS6-2 Discriminate approach for data selection in data envelopment analysis
Akio Naito, Shingo Aoki (Osaka Prefecture University, Japan)
GS6-3 Proposal of recommender system simulator based on small-world model
Ryosuke Saga, Kouki Okamoto (Kanagawa Institute of Technology, Japan)
Hiroshi Tsuji (Osaka Prefecture University, Japan)
Kazunori Matsumoto (Kanagawa Institute of Technology, Japan)
GS6-4 Construction and analysis of purchase factor model by using creativity method
Kodai Kitami, Ryosuke Saga, Kazunori Matsumoto (Kanagawa Institute of Technology, Japan)
GS6-5 Developing a monitoring psychological stress Index system via photoplethysmography
Jiann-Shing Shieh1, Yu-Ren Chiou (Yuan Ze University, Taiwan)
Shou-Zen Fan (National Taiwan University, Taiwan)

10:30~11:45 GS1 1 【Image processing II】
Chair: J. Wang (Beijing Jiaotong University, China)
GS11-1 Simple analog-digital circuit for motion detection and its application to target tracking system
Takuya Yamamoto, Kimihiro Nishio (Tsuyama National College of Technology, Japan)
GS11-2 Analog motion detection circuit using CCD camera based on the biological vision system and its
application to mobile robot
Yasuyuki Kondo, Takumi Yamasaki, Kimihiro Nishio (Tsuyama National College of Technology,
Japan)
Toshinori Furukawa (Kurashiki University of Science and the Arts, Japan)
GS11-3 Real-time visual target tracking for Augmented Reality (withdrawal)
Donghoon Lee (University of Science and Technology, Korea)
Inhun Jang, Hyunsub Park, Moonhong Baeg (Korea Institute of Industrial Technology, Korea)©ISAROB 2011

P - 42

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

GS11-4 Detecting human flows on a road different from main flows
Mison Park, Joo Kooi Tan, Yuuki Nakashima, Hyongseop Kim, Seiji Ishikawa(Kyushu Institute of Technology, Japan)
GS11-5 Study on the height measurement based on the image processing technique
Jiwu Wang, Weijie Gao, Yisong Wang (Beijing Jiaotong University, China)
Masanori Sugisaka (Nippon Bunri University, Japan)

13:10~14:40 OS21 【Learning control and robotics】
Chair: H. H. Lee (Waseda University, Japan)
Co-Chair: H. Ogai (Waseda University, Japan)
OS21-1 A neuro PID control of power generation using low temperature gap
Kun-Young Han, Hee-Jae Park, Hee-Hyol Lee (Waseda University, Japan)
OS21-2 An efficient identification scheme for nonlinear polynomial NARX modelYu Cheng, Miao Yu, Lan Wang, Jinglu Hu (Waseda University, Japan)
OS21-3 Traffic signal control of multi-forked road
ChengYou Cui, Mizuki Takamura, Hee-Hyol Lee (Waseda University, Japan)
OS21-4 Control design methods for platooning in robot car
Ryo Takaki, Xin Zhao, Harutoshi Ogai (Waseda University, Japan)
OS21-5 Building of reverse logistics model in reusable recovery and optimization considering transportation,
inventory, and backorder costs
Jeong-Eun Lee, Hee-Hyol Lee (Waseda University, Japan)
OS21-6 Advanced pipe inspection robot using rotating probe and image processing
Ryuta Oyabu, Kentarou Nishijima, Zhicheng Wang, Harutoshi Ogai (Waseda University, Japan)
Bishakh Bhattacharya (Indian Institute of Technology, India)

15:00~16:30 OS23 【Intelligent systems II】
Chair: M. K. Habib (The American University in Cairo, Egypt)
Co-Chair: Y. I. Cho (University of Suwon, Korea)
OS23-1 Intelligent speech recognition filtering
Yong Im Cho (University of Suwon, Korea)
OS23-2 Multi robotic system and the development of cooperative navigation behaviors for humanitarian
demining
Maki K. Habib (The American University in Cairo, Egypt)
OS23-3 Development of Flexible Surgical Manipulator for Natural Orifice Transluminal Endoscopic
Surgery
Jungwook Suh, Hoseok Song, Kiyoung Kim, Jungju Lee (Korea Advanced Institute of Science andTechnology, Republic of Korea)
OS23-4 Swing-up and LQR stabilization of rotary inverted pendulum
Minho Park, Yeoun-Jae Kim, Ju-Jang Lee (Korea Advanced Institute of Science and Technology,
Republic of Korea)©ISAROB 2011

P - 43

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

OS23-5 Intelligent information retrieval system
Young Im Cho (University of Suwon, Korea)
OS23-6 A study on real-time face verification and tracking with segmented common vector
Dongkyu Ryu, Minho Park, Ju-Jang Lee (Korea Advanced Institute of Science and Technology,
Republic of Korea)

Room E

8:40~9:55 GS14 【Neural networks I】
Chair: T. Yamada (Ibaraki University, Japan)
GS14-1 A study of SVM using the combination with online learning method andMidpoint-Validation Method
Shingo Yamashita, Takeshi Yoshimatsu, Hiroki Tamura, Koichi Tanno (University of Miyazaki,
Japan)
GS14-2 Video object segmentation using color-component-selectable learning for self-organizing maps
Shin-ya Umata, Naotake Kamiura, Ayumu Saitoh, Teijiro Isokawa, Nobuyuki Matsui (University of
Hyogo, Japan)
GS14-3 Properties of localized oscillatory excitation on the non-linear oscillatory field
Ryota Miyata, Koji Kurata (University of the Ryukyus, Japan)
GS14-4 Multiple Granger causality tests for network structure estimation from time-series data
Hikaru Harima (Kyoto University, Japan)
Shigeyuki Oba (Kyoto University and Japan Science and Technology Agency, Japan)
Shin Ishii (Kyoto University, Japan)
GS14-5 A simulation study of visual perceptual learning with attentional signals
Satoshi Naito, Naoto Yukinawa (Kyoto University, Japan)
Shin Ishii (Kyoto University and RIKEN, Computational Science Research Program, Japan)

10:30~12:15 OS15 【Bio-inspired theory and application】
Chair: M. Yasunaga (University of Tsukuba, Japan)
Co-Chair: K. Yamamori (University of Miyazaki, Japan)
OS15-1 Study of computational performance of Genetic Algorithm for 3-Satisfiability problem
QingLian Ma, Yu-an Zhang, Makoto Sakamoto, Hiroshi Furutani (University of Miyazaki)
OS15-2 An adaptive resolution hybrid binary-real coded genetic algorithm
Omar Abdul-Rahman, Masaharu Munetomo, Kiyoshi Akama (Hokkaido University, Japan)
OS15-3 Neural network with exponential output neuron for estimation of physiological activities fromprotein expression levels
Kazuhiro Kondo, Kunihito Yamamori, Ikuo Yoshihara (University of Miyazaki, Japan)©ISAROB 2011

P - 44

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

OS15-4 Asynchronous migration for parallel genetic programming on computer cluster with multi-coreprocessors
Shingo Kurose, Kunihito Yamamori, Masaru Aikawa, Ikuo Yoshihara (University of Miyazaki,
Japan)
OS15-5 Classification of species by information entropy and visualization by self-organizing map
Kentaro Nishimuta, Ikuo Yoshihara, Kunihito Yamamori (University of Miyazaki, Japan)
Moritoshi Yasunaga (University of Tsukuba, Japan)
OS15-6 Digital-signal-waveform improvement on VLSI packaging including inductances
H. Shimada, S. Akita, M. Ishiguro, N. Aibe (University of Tsukuba, Japan)
I. Yoshihara (University of Miyazaki, Japan)
M. Yasunaga (University of Tsukuba, Japan)
OS15-7 Digital-signal improvement-method using Pareto optimization
S. Akita, H. Shimada, M. Ishiguro, N. Aibe, M. Yasunaga (University of Tsukuba, Japan)
I. Yoshihara (University of Miyazaki, Japan)

13:10~14:55 GS12 【Learning】
Chair: Y. G. Zhang (Academia Sinica, China)
GS12-1 Reinforcement learning with the mechanism of short-term depression for learning rate
S. Kubota (Yamagata University, Japan)
GS12-2 A method for finding multiple subgoals for reinforcement learning
Fuminori Ogihara, Junichi Murata (Kyushu University, Japan)
GS12-3 Temporal difference approach in linearly-solvable Markov decision problems
M. A. P. Burdelis, K. Ikeda (Nara Institute of Science and Technology, Japan)
GS12-4 Mutual learning of multi consciousness agent including the ego for autonomous vehicle
Atsunori Mori, Hirokazu Mastui (Mie University, Japan)
GS12-5 Reinforced learning by using a learned results of a different form robot
Nobuo Shibata, Hirokazu Matsui (Mie University, Japan)
GS12-6 The acquisition of sociality by using Q-learning in a multi-agent environment
Yasuo Nagayuki (Otemae University, Japan)
GS12-7 Effect of interaction between rules on rule dynamics in multi-group minority gameTakashi Sato (Okinawa National College of Technology, Japan)

15:00~16:00 GS20 【Robotics IV】
Chair: N. Uchiyama (Toyohashi University of Technology, Japan)
GS20-1 Design of the android robot head for stage performances
Dongwoon Choi, Dong Wook Lee, Duk Yeon Lee, Jun Young Jung, Hogil Lee (Korea Institute ofIndustrial Technology, Korea)©ISAROB 2011

P - 45

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

GS20-2 Implementation of IMU using wavelet transform and variable IIR filter
Woojin Seo, Bongsu Cho (Pusan National University, Korea)
Jaehong Lee (Hyosung Industrial Machinery, Korea)
Kwang Ryul Baek (Pusan National University, Korea)
GS20-3 Development of crawler type rescue robot with slide mechanism
Go Hirano (Kinki University, Japan)
Seiji Furuno (Kitakyushu National College of Technology, Japan)
GS20-4 Proposal of intelligence module type robot that can exchange it in seamless
Shunta Takahashi, Kohei Miyata, Yuhki Kitazono, Lifeng Zhang, Seiichi Serikawa (Kyushu Instituteof Technology, Japan)

Room F

8:40~10:10 OS19 【Embracing complexity in natural intelligence】
Chair: Y. Ishida (Toyohashi University of Technology, Japan)
Co-Chair: T. Okamoto (Kanagawa Institute of Technology, Japan)
K. Harada (Toyohashi University of Technology, Japan)
OS19-1 A systemic payoff in a self-repairing network
Masahiro Tokumitsu, Yoshiteru Ishida (Toyohashi University of Technology, Japan)
OS19-2 Effects of a membrane formation in Spatial Prisoner’s Dilemma
Yuji Katsumata, Yoshiteru Ishida (Toyohashi University of Technology, Japan)
OS19-3 Extracting probabilistic cellular automata rules from spatio-temporal patterns and analyzing
features of these rules
Takuya Ueda, Yoshiteru Ishida (Toyohashi University of Technology, Japan)
OS19-4 Emergence of observable rules in a spatial game system
K. Harada, Y. Ishida (Toyohashi University of Technology, Japan)
OS19-5 An artificial intelligent membrane for detect network intrusion
Takeshi Okamoto (Kanagawa Institute of Technology, Japan)
OS19-6 A dynamic Houjin (square) and a symmetric Houjin
Yuki Tsuzuki, Yoshiteru Ishida (Toyohashi University of Technology, Japan)

10:30~12:15 OS16 【Biomimetic machines and robots】
Chair: K. Watanabe (Okayama University, Japan)
Co-Chair: K. Izumi (Saga University, Japan)
OS16-1 Basic position/force control of single-axis arm designed with an ultrasonic motor
Keisuke Ogiwara, Fusaomi Nagata (Tokyo University of Science, Japan)
Keigo Watanabe (Okayama University, Japan)
OS16-2 Cooperative swarm control for multiple mobile robots using only information from PSD sensors
Takahiro Yamashiro, Fusaomi Nagata (Tokyo University of Science, Japan)
Keigo Watanabe (Okayama University, Japan)©ISAROB 2011

P - 46

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

OS16-3 Control of movement on stairs for a cleaning robot
Takahisa Kakudou, Keigo Watanabe, Isaku Nagai (Okayama University, Japan)
OS16-4 Jumping rhythm generator by CPG for a multi-legged robot
Masaaki Ikeda, Kiyotaka Izumi (Saga University, Japan)
Keigo Watanabe (Okayama University, Japan)
OS16-5 A bearing-only localization solved by an unscented Rauch-Tung-Striebel smoothing
Saifudin bin Razali, Keigo Watanabe, Shoichi Maeyama (Okayama University, Japan)
OS16-6 Trajectory tracking control for nonholonomic mobile robots by an image-based approach
Tatsuya Kato, Keigo Watanabe, Shoichi Maeyama (Okayama University, Japan)
OS16-7 A nonholonomic control method for stabilization an X4-AUV
Zainah Md. Zain, Keigo Watanabe (Okayama University, Japan)
Kiyotaka Izumi (Saga University, Japan)
Isaku Nagai (Okayama University, Japan)

15:00~16:00 GS1 【Artificial intelligence】
Chair: T. Nakashima (Osaka Prefecture University, Japan)
GS1-1 Adaptive co-construction of state and action spaces in reinforcement learning
Masato Nagayoshi (Niigata College of Nursing, Japan)
Hajime Murao, Hisashi Tamaki (Kobe University, Japan)
GS1-2 Autonomous acquisition of cooperative behavior based on a theory of mind using parallel Genetic
Network Programming
Kenichi Minoya, Takaya Arita (Nagoya University, Japan)
Takashi Omori (Tamagawa University, Japan)
GS1-3 A comparison of learning performance in two-dimensional Q-learning by the difference of Q-valuesalignment
Kathy Thi Aung, Takayasu Fuchida (Kagoshima University, Japan)
GS1-4 Robot path planning in unknown environment based on ant colony algorithm
Cu Xuan Tien, Young Sik Hong (Dongguk University, South Korea)©ISAROB 2011

P - 47

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

Development Outline of the HUBO2
Jun Ho Oh
Humanoid robot research center, Korea advanced institute of science and technology©ISAROB 20111

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

©ISAROB 20112

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

©ISAROB 20113

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Data-driven Two Degree-of-Freedom Control for a Micropump andMicroneedle Integrated Device for Diabetes Care
Ruoting Yang*, Mingjun Zhang**, and Tzyh-Jong Tarn** Department of Electrical and Systems Engineering, Washington University, St. Louis, MO, 63130, USAe-mail: ryang@wustl.edu, tarn@ese.wustl.edu.
**Department of Mechanical, Aerospace and Biomedical Engineering, University of Tennessee, Knoxville, TN, USA.
email: mjzhang@utk.edu.

Abstract: This paper presents a dynamic model for a micropump and microneedle integrated system for diabetes care. A
novel data-driven two degree-of-freedom control mechanism is proposed for regulating blood glucose concentration toshorten regulating time while maintaining the stability of the system in the presence of model uncertainties andunexpected disturbances. Exact feedforward linearization, gain scheduling, and data-driven planning technique areapplied to improve regulation performance as well as robustness. Simulation results indicate that the proposed control
has great potential in drug delivery problems.

1 Introduction
Diabetes mellitus is a disease in which the patient has
difficulty regulating blood glucose. Diabetes may affectthe functioning of many physiological systems, causing
everything from retinopathy and circulatory problems,
to nephropathy and heart disease. While diabetes can betreated with insulin, the dosage of insulin must be
strictly regulated - excess insulin can cause
hypoglycemia, whereas insufficient insulin can cause
hyperglycemia.
Fig. 1 shows a schematic drawing of a insulin
infusion microdevice consisting of a piezoelectric
micropump, multiple silicon microneedles, an insulin
reservoir, a membrane, wireless telemetry, and a remotecontrol component. This control system regulates blood
glucose levels by driving the piezoelectric micropump
based on glucose sensor measurements. Very generally,
sensor readings are passed via wireless telemetry to thecontroller, which then drives the micropump, causing
the release of insulin from the reservoir, through themicroneedles into the patient’s bloodstream.
The challenges of controller design mainly comefrom three areas. First, these systems usually haveserious nonlinearities, which are poorly estimated by thecorresponding linearized systems. Second, the output
measurements often have large model uncertainties,
disturbances, and slow sampling rates. Third, in theevent of an accidental insulin overdose, there is no way
to retrieve the insulin to avoid hypoglycemia.

©ISAROB 2011

Fig. 1 A schematic drawing of a micropump andmicroneedle integrated for controlled insulin delivery
Recently, advanced control mechanisms have been
applied for glucose control, including PID [1, 2], model
predictive control [3, 4], and robust control [5], among
others [6]. However, the regulating time that draws the
blood glucose concentration from a high level to thebasal level is still too long for all these methods. In thispaper, we propose a new data-driven feedback andfeedforward integrated 2DOF (Two Degree-OfFreedom) control mechanism to shorten regulating time
while maintaining the stability of the system in thepresence of model uncertainties and unexpecteddisturbances. As shown in Fig. 2, the 2DOF control
method contains two parts: (1) the feedforward control
provides the nominal control to rapidly drive the system
towards the desired goal; (2) the feedback control
stabilizes the system. It has been proven that the 2DOF
controller performs better than controllers that only use
feedback under reasonable model uncertainty[7]. The
2DOF control mechanism is said to be time-based if thereference trajectory is given by a time-driven planner. In
contrast, a data-driven 2DOF control mechanism

4The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

where Vol is the volume change and V0 is thevoltage applied to the lead zirconate titanate film with
piezoelectric coefficient d13 = 3×10-10 m/N and Poisson's
ratio µ = 0.3. The thickness and the radius of themembrane are h and a, respectively.
Papers [10] and [11], demonstrated that the flow rateincreases linearly at low actuating frequencies. Whenthe actuating frequency exceeds a critical value, though,the flow rate does not increase and may even decrease
sharply. As a result of this electro-mechanical-fluid
coupling, the membrane deflects in an undesirable way
at high frequencies[12].
If a voltage signal with changing polarity drives the
micropump, then the flow rate can be approximated as alinear function with respect to the voltage and theactuating frequency at low frequencies.

utilizes a data-driven planner to generate the reference
trajectory.

Fig. 2 The schematic of data-driven 2DOF method. ΣFF
is the feedforward controller, ΣFB is the feedback
controller, and ΣPF is the coordinate transformation
between the flat output reference zd and the output yd.
The data-driven planner is driven by an action references generated by the output measurement, instead ofdriven by time.
The data-driven 2DOF control uses exact
feedforward linearization and gain scheduling methods,
instead of exact feedback linearization, to improverobustness to model uncertainties, while has the same
regulation performance. In addition, we use a datadriven planning technique, which can further improverobustness towards model uncertainties and unexpecteddisturbances in comparison to the other 2DOF controls.
While the data-driven planner aids in robustness, it isalso advantageous compared to other planningapproaches because it does not need to replan andregenerate the reference trajectory at every samplingtime instant [8]. Since the action reference parameter iscalculated nearly at the same rate as the feedback
control, the planning process is adjusted rapidly, whichenables the planner to handle unexpected disturbances
within one control execution sampling time.
This paper is organized as follows. Dynamics modelingof the microdevice is introduced in Secion II. The datadriven 2DOF control mechanism is discussed in Section
III-V. Computer simulation is conducted in Section VI.
Conclusions and future work are discussed in the final
section.

Q pump (V0 , f ) = 2 f Vol = Ad fV0 ,where Ad = 6a 4 (5  2  )(1   )d13 /4h 2 (3  2  ) is aconstant coefficient.
If, for example, V0 = 1.5 V, a = 100 µm and h = 10
µm then according to equation (1), we will have ΔVol
= 3.675×10-3 µl. Assuming that the micropump is driven
at 100 Hz, equation (2) gives the pumping speed as
0.735 µl/s.

2.2 Microneedle
Microneedles are attractive for medical applicationsin that they are able to provide painless drug transport
pathways while at the same time largely reducing therisk of infection at injection site.
The volume flow rate of a microneedle can be
expressed as
Qneedle = ΔP / R,

R =8µL / πr4,

2.1 The micropump©ISAROB 2011(4)

In equation (4), µ is the fluid viscosity, while L and r are
the channel length and radius, respectively. We choosea straight microneedle with 100/30 µm outside/inside
diameter.
Because the tatal resistance of a microneedle array isoften smaller than the sum of the individual channel

The piezoelectric diaphragm displacement pump canbe modeled as follows [9],3a 4 (5  2 )(1   )d13V0,
4h 2 (3  2  )(3)where ΔP is the pressure drop across the channel, andR is the channel resistance for a circular channel,
where

2 Dynamics modeling

Vol =

(2)

(1)

5

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Here Q1 and Q2 stand for the insulin masses at the
injection site and the intermediate site, respectively. I is
the plasma insulin concentration with the basal value ib.
u is the subcutaneous insulin infusion rate. k and p are
the transition rate constants, and o and ke aredegradation decay rates. The parameter Vi stands for theplasma distribution volume.
The minimal model [16] has been widely accepted as
the fundamental model to describe insulin-glucose
interactions:resistances,a high needle density increases thevolume flow rate as follows,

Qneedle = ΔP / kdR,

(5)where kd is the discount coefficient.
The microneedle array is not sufficiently large toallow free flow, but it is large enough not to causesignificant resistance. Ma et al. [13] showed that for
microneedles, the flow rate is nearly linear to theactuating frequency at low frequencies. As a result, it isreasonable to assume that the microneedle array does
not impede the flow from the micropump.

2.3 Microsensor
One of the major glucose sensors is the amperometricsensor, which determines the solution concentration bymeasuring the current generated during a chemical
reaction. The amperometric sensor can be modeled asfollows [14]:Isig = c2 I sig  c1G  OS ,(6)

GS = CF(Isig – OS),(7)

X   P2 X  P3 ( I  I b ),(13)x1 = 1 x1   2 x2x2 = ( x2  Gb ) x3  P1 x2  GI
x3 =  P2 x3  P3 x4

G S  c2 GS  c1CF  G  CF (1  c2 )OSx4 = 3 x5  ke x4(8)x5 = kx6  ( p  o) x5

where 1 = c2 ,  2 = c1CF , and OF = CF (1  c2 )OS .

x6 = kx6  u

2.4 Glucose-insulin Kinetics

y = x1 ,Shimoda’s three-compartment model [15] can beused to describe the kinetics of either regular insulin ora monomeric insulin analog supplied as a continuous
subcutaneous infusion.

©ISAROB 2011(12)

where G is the plasma glucose level (with basal valueGb), X is the interstitial insulin concentration, GI is theintravenous glucose uptake, P1 is a coefficient for
glucose effectiveness, and P3 / P2 is a measure of insulin
sensitivity [16].
Combining all subsystems and letting x1 = S – OF / θ1,x2 = G – Gb, x3 = X, x4 = I – Ib, x5 = Q2, x6 = Q1, and θ3= p / Vi, we have the following sixth-order nonlinear
model. The meaning of the parameters are summarized
in the Table I.

where GS and G are the sensor and blood glucose level,
respectively.,Isig is the sensor signal,. CF is thecalibration factor and OS is the offset current. The
sensor sensitivity is characterized by the ratio of c1 andc2,
The sensor glucose model can be rewritten as= 1GS   2 G  OF ,G   XG  P1 (Gb  G )  GI

Q1 =  kQ1  u(9)Q 2 = ( p  o)Q2  kQ1(10)I = ke ( I  ib )  pQ2 / Vi

(11)

(14)

subject to 0  u (t )  10 U/h and y (t )  75 mg/dL.
The input u=CV0 is a linear function of the applied
voltage V0 according to the arguments presented insubsections A and B. In order to simplify thepresentation, let us assume C = 1.

6

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

2.5 Differential flatness

TABLE 1: Physical variables in the dynamic models

SymbolDescription

x1(mg/dL)x2(mg/dL)x3(min–1)x4(mU/L)x5(mU/Kg)
x6(mU/Kg)
P1(min–1)
P3/ P2(L/ mU)
Vi(L/Kg)u(mU/Kg/min)
Gb(mg/dL)ib(mU/L)OFOS
c1 / c2

sensor measured plasma glucose levelplasma glucose levelinterstitial insulin
plasma insulin levelinsulin mass at intermediate site
insulin mass at the injection site
glucose effectivenessinsulin sensitivity
plasma distribution volume
insulin infusion ratebasal plasma glucose levelbasal plasma insulin level
calibration factor
offset current
sensor sensitivity

Definition [19, 20]: A system is said to bedifferentially flat if there exists a set of m differentiallyindependent variables, z = [z1,..., z m ]T such that(15)y = h(x)

(16)
n

m

T
d

T
d

T
d

T
d

T
d

(u , x ) : f ( x , u ) = 0, y = h( x ).

©ISAROB 2011(20)

u   ( z , z, , z ( 1) ),

(21)

(22)

All flat systems can be transformed into a normalform

1i   2iiwith state x Î R , input u Î R and output y Î R .
In equations (15)-(16), we assume that the vector field
f(x,u) and the function h(x) are smooth.
The glucose control problem is to regulate the blood
glucose concentration from a high level to the basal
level. Exact feedback linearization based nonlinear
controls [17] can have good performance, however,these methods are sensitive to model uncertainties anddisturbances, which is a big issue in glucose control
problem. As a result, we will apply exact feedforward
linearization technique based on differential flatness,
which is more robust to the feedback linearization[18].
The feedforward linearization problem is to design acontrol ud to track a smooth reference trajectory
connecting two stationary setpoints (ud0 , xd0 yd0 ) and(udT , xdT , ydT ) within a finite time interval t Î [0, T ] .
The control and state variables satisfy the following
relationshipsTdx   ( z , z, , z ( ) ), andy  h( x)  h( ( z , z,..., z ( ) ) .

m(ud0 , xd0 ) : f ( xd0 , ud0 ) = 0, yd0 = h( xd0 ),(19)

where , , and  are smooth functions of theirarguments at least in an open subset of Rn+m(β+1), Rm(α+1),
and Rm(α+2), respectively. A vector z which satisfies theabove equations is called a flat output, and then the
output vector can be written with respect to the flat
output,Consider a class of nonlinear systemsx  f ( x, u )z   ( x, u, u , , u (  ) ),

ri 1

(23)

  rii

ki   i ( , u , u , , u ( ) ), for i  1, , miivia the Brunovsky state [21]:  (11 ,  21 , ,  r1 , 12 , ,  r2 , , 1m , ,  rm )T ,
1

2

m

(24)

where  i 1 ri  n .
By Delaleau and Rudolph [21], for the set of algebraic
equationsm i ( , u , u ,, u ( ) )  vi
i(25)

there always exists a solutionu  ( , v, v, , v ( ) ) ,(26)

where v  [v1 ,..., vm ]T and   max( i ) .
For example, consider a MIMO system

(17)
(18)

7

x1  x2  u1(27)

x2  x3  x1u1

(28)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011x3  x2  u2 .

(29)

11   21  u1

(30)

uncertainties[23], feedback controllers have been
employed in conjunction with feedforward to reduce
uncertainty-caused errors, such as sliding mode control[24], backstepping control[25], and PID control [18]. In
this paper, we propose a 2DOF controller described asfollows:

22   21   21u1  u12   21u1  u2 .

(31)

u  ( d , vd , vd ,, vd ( ) ) +K ( )(   d ),Set the Brunovsky state:   ( x1 , x2 , x2 ) . Then the
MIMO system can be transformed into

As a result,u1  11   21(32)

u2  22  11 (11   21 )  11 (11  21 ).

(33)

where K(ξ) is a scheduled gain. The gain schedulingcontrol is designed to compensate unknown model
uncertainties and avoid singularity problem, which often
occur when using linearization techniques [26, 27].
Several gain scheduling methods[8] can be chosen
dependent on the system requirements. These include
hard switching, linear interpolation, and switching withhysteresis.
The 2DOF method improves the robustness of theconventional feedback linearization[17], which is very
sensitive to model uncertainties. The feedback
linearization technique exactly cancels nonlinearities via
state feedback, while the 2DOF control method uses
exact feedforward linearization (35), which is known tobe more robust than feedback linearization in terms ofmodel uncertainties[18].
In the next section, a data-driven planning techniquewill be presented to further improve robustness towards
both model uncertainties and unexpected disturbances.

In fact, differential flatness is equivalent to dynamic
feedback linearization on an open and dense set using aclass of invertible dynamic feedbacks [20]. For SISO
systems, a differentially flat system is equivalent to astatic feedback linearizable system [22]. Fliess et al.
[20] has proved that a flat system is controllable.
Hagenmeyer and Delaleau [18] showed that if thedesired trajectory is in close proximity to the initialcondition x0, i.e.,x0  ( zd , zd , , zd( ) )   ,(34)then after applying the exact feedforward linearization
controlu  ( d , vd , vd , , vd( ) ),3 Data-driven Planner(35)

The 2DOF control can have better tracking
performance than the use of feedback control alone in
the presence of acceptable model uncertainties.[7] In
reality, model uncertainties and unexpected disturbances
can be large and result in substantial deviation of theoutput trajectory from the predefined planning trajectory.
The time-driven 2DOF control often deteriorates thisdeviation, since the time-driven planner cannot stop, butinstead continues to gives offline-computed values astime evolves. This fact will lead to poor performance,and even instability. The data-driven planner, however,refers to the current output and the planning trajectories‘stop’ and ‘wait’ the system recovered from thedisturbances. As a result, the data-driven 2DOF
controller will not deteriorate the substantial deviation
created by the model uncertainties or disturbances.
As shown in equation (17)-(18), we need plan asufficiently smooth reference trajectory connecting an

the MIMO nonlinear system (23) is equivalent to the
Brunovsky normal form (Proposition 1, [18]).

rj1  rjii

 j   j
ri

(36)

dri

2.6 Two degree-of-freedom Control
As shown in Fig. 2, the 2DOF controllers contain two
components: feedforward control providing nominal
input, and feedback control ensuring stability. The
addition of feedforward controllers can improve thetracking performance when compared with the use of
feedback controllers alone under acceptable model
uncertainties, and thereby, significantly shorten theregulating time. However, model-based feedforward
controllers alone cannot resist large model©ISAROB 2011

(37)

8

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

the output level, i.e., yd(y). The output error is zero andthe corresponding state error is much smaller than thestate error generated when using time-driven planning.
As a result, the feedforward controller can guarantee
good tracking performance.
Theorem 1 gives asufficient condition for stability of the data-driven
control approach. Very generally, this theorem implies
that the stability of data-driven control is at least thesame as the stability of the time-driven control.
Moreover, in contrast to other planning approaches[8], the data-driven planner need not replan andregenerate a reference trajectory at every sampling time
instant. In fact, the action reference parameter iscalculated nearly at the same rate as the feedback
control, meaning that the planning process is adjusted
rapidly, enabling the planner to handle unexpecteddisturbances within one control execution sampling
time.
Theorem 1 [30]: If the nonlinear system (15)-(16) is
asymptotically stable with a time-driven controller u(t),and the event s is monotonically increasing (or nondecreasing) with time t, i.e.,

initial setpoint and a terminal setpoint in the timeinterval T, and then use this trajectory to develop a exact
feedforward linearization control (35). The sufficiently
smooth reference trajectory (Fig. 3 the dotted curve) can
be constructed using a polynomial series [28] as follows,
yd (t ) = y0  ( yT  y0 ) j = r 1 a j  t / T  , t  [0, T ],2 r 1j(38)

where r is the relative degree [17] of the nonlinear
system (15)-(16), andaj =(1) j  ( r 1) (2r  1)!
, j = r  1, , 2r  1.
j  r!( j  (r  1))!(2r  1  j )!
(39)

Alternatively, we can use an exponential function (Fig.
3 the solid curve),y = ( y0  yT )(t / b  1)e  t/b  yT ,(40)

where b = T / 10.

ds / dt  0 (or ds / dt  0),(41)

Then this system is asymptotically stable (or stable)
under the data-driven controller u ( s ) .
Fig. 3 Polynomial planner (dotted) v.s. Exponential
planner (solid). In this illustratrion, we choose T = 100,y0 = 10 and yT = 0.

These reference trajectories yd(t) are sufficiently
smooth, yet steep. As shown in Fig. 4(a), if an
unexpected disturbance is applied over a short time
period t , the output error y(t) - yd(t) grows sharply, as
does the state error e = x – xd. According to [18] andequation (34), the feedforward control ud will fail totranslate the nonlinear system into a normal form. This
problem occurs because the time-driven planning
trajectory refers to the reference trajectory at time
instant t, i.e., yd(t). As a result, the time-driven
feedforward control can have poor tracking
performance, which may even lead to instability.
The data-driven planner [29, 30] is a closed-loop
planner (Fig. 2) driven by an action reference, s, which
is a non-time scalar factor generated by measurement
data. As shown in Fig. 4(b), a simplified data-driven
reference trajectory refers to the reference trajectory in©ISAROB 2011

Fig. 4 Time-driven versus data-driven planning
trajectory. A time-driven planning trajectory refers to thetime instant t, while a data-driven planning trajectoryrefers to the output level y.

The data-driven planner develops a relationshipy ,..., andbetween the output y and its derivatives y , y ( r ) . The exponential reference trajectory is easier toanalyze than the polynomial trajectory, although yT can
only be approximately reached in time T. Given the
output y, the time t can be calculated from equation

9

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011(40) directly.

4 Control Applicationt / b 1 = W ,In this section, we will constuct flatness-based
feedforward control for the insulin delivery system (14).
The output is z = x1 , so the states can be described asfunctions of z, …, z(n)(42)

where W  W (1, ( yT  y ) /( y0  yT )e) is the -1 branchof the Lambert W function, which is the solution of thefunction ( yT  y ) /( y0  yT )e = WeW . From theseequations, the first order derivative y (t ) isy (t ) = ( y0  yT )(W  1)eW 1 / b.

x1 = z ,x2 = ( z  z )/ ,(43)

x3 = ( x2  P1 ( x2  Gb ))/x2 ,If the output monotonically decreases with t, the event S
is defined as
S = y0 – y; S0 = 0,x4 = ( x3  P2 x3  P3ib )/P3 ,(44)

x5 = ( x4  ke ( x4  ib ))Vi /p,Thus, the derivatives of y can be written as functions of
S,
W ( S ) 1y ( S ) = ( y0  yT )(W ( S )  1)e/ b,

x6 = ( x5  ( p  o) x5 )/k .

In this case, the input also is a function of z , z, , z ( n 1) ,
since

(45)

y (i ) ( S ) = ( y (i  2) ( S )  2by (i 1) ( S )) / b 2 , i > 2, (46)y ( S ) = [ y ( S )  yT  2by ( S )] / b 2 ,u = x6  kx6

(47)

(48)

Hence, the system (14) is differentially flat.
The 2DOF controller can be written as

where W ( S )  W (1,[ yT  y ( S )] /( y0  yT )e) .
Keeping in mind that the output y(t) may suddenly
increase because of a short unexpected disturbance, we
examine the effect of this disturbance on the event S. In
order to keep the monototically nonincreasing condition
required for Theorem 1, as shown in Fig. 5, the event S
first increases as y decreases, but then stops evolving
after y reaches a valley yc. S resumes evolution again
only after y returns to the level of yc.

u = ( d )  K ( z ) PD(e),(49)

where K ( z ) is a hard-switching scheduled gain, andthe state  d = [ zd , zd ,..., zd( n 1) ] . The PD controller isPD(e)  K p ( z  zd )  K d ( z  zd ) .

(50)

The feedforward control ud = Θ(ξd) can be computed asx2 = ( d  1d )/ 2 ,D = 1/( x21  Gb ),x31 = [ x22  P1 x21 ]D,
x32 = [ x23  ( P1  x31 ) x22 ]D,
x33 = [ x24  ( P1  x31 ) x23  2 x32 x22 ]D,
x34 = [ x25  ( P1  x31 ) x24  3 x32 x23  3 x33 x22 ]D,
x35 = [ x26  ( P1  x31 ) x25  4 x32 x24  6 x33 x23  4 x34 x22 ]D,
x = ( x  P x )/P ,
4

3

2 3

3

x5 = ( x4  ke x4 )/3 ,x = ( x  ( p  o) x )/k ,Fig. 5 The event S stops growing until y is recoveredfrom the disturbance.

6

5

5

( d ) = x62  kx61 .

(51)©ISAROB 201110

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

sampled from 5 to 20 min. The BGL converges to thebasal level in 130 - 160 min. Clearly, the proposed
method can control blood glucose levels even for a 20
min sampling interval, which would cause poor
performance in most feedback-based controllers. The
feedforward linearization technique in the 2DOF control
largely reduces the dependence of the real-time
feedback, leading to the sampling robustness observedin our simulations.

5 Simulation Results
To demonstrate the effectiveness and robustness ofthe data-driven 2DOF control algorithm, we test
different cases by computer simulation using Matlab
and Simulink. The system parameters are set as Table II.

Table 2: System Parameters Used In The Simulation

Parameters
P1
P2
P3
Gb
o
k

Value
0.003082
0.020930.00001282
85
0.01250.25

Parameters
ke
θ1
θ2
ib
P
Vi

Value
0.267
0.33
0.33
0
0.25
0.21

5.1 Test of robustness
First of all, the robustness of the data-driven 2DOF
control method is tested by a couple of cases.

Fig. 7 The BGL is sampled from 5 min to 20 min.
The BGL converges to the basal level in between 130
min and 160 min.

Case 1: Various initial glucose input
Initial glucose inputs range from 5 to 40 mg/min at 6
- 11 min and the BGL(blood glucose level) is sampled
every 5 min. When the data-driven 2DOF control isapplied, the BGL converges to the basal level in 80 130 min, and remains above the minimal level. The
magnitude of the insulin infusion rate grows as theinitial glucose input increases. The maximum insulin
infusion reaches 7 U/h, which is under the constraints.

Case 3: model uncertainties
Two glucose inputs 40 and 3 mg/min areadministered at 6 - 11 min and 17 - 26 min, respectively.
The BGL is sampled every 5 min. The system (14) withmodel uncertainties can be written asx  f ( x, u )   f ( x, 0) ,where  f ( x, 0) implies that model uncertainties areproportional to vector field f(x,0), and  >0 is aconstant coefficient. For  values of 0.5, 1, and 1.5, the
BGL still converges under the same controller
parameters. The larger model uncertainty leads to alonger convergence time; however, one can always
adjust the gains of the PD controller (50) to achieve
desired performance.

Fig. 6 The data-driven 2DOF control approach is tested
by various initial glucose input, the BGL converges tothe basal level in 80 - 130 min, and remains above theminimal level.
Case 2: Various sampling interval
Two glucose inputs 40 and 3 mg/min are administeredat 6 - 11 min and 17 - 26 min, respectively. The BGL is©ISAROB 2011

(52)

11

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

converges to the basal level at 170 min, while the datadriven 2DOF control stabilizes the BGL at 140 min.

300
250
200

Blood Glucose Level(mg/dL)

Blood Glucose Level(mg/dL)

350

150
100
50

0

50

100

150

200
Time(Min)

250

300

350

400

50 Initial glucose uptake
200

5.2 Comparing to other control algorithms

Data-driven
2DOF
Control

100
0
0
Basal
Level
8

Insulin infusion rate(U/h)

Fig. 8 The BGL converges to the basal level subject
to model uncertainties. The model uncertainties are
assumed to be 0.5 (solid), 1(dashed) and 1.5(dotted)
times of the proposed model.

300

50

100

0

0

20

40

150

200

250

150

200

250

Time(Min)

MinimalLevel
Data-driven
2DOF
Control

6420

MPC Control

MPC Control
0

50

100
Time(Min)1) Time-driven approachWith a 10 mg/min initial glucose, a disturbance of 1
mg/min glucose is applied over the time period from 36- 47 min. The data-driven approach converges to thebasal level at 130 min, while the time-driven approachrequires 200 min - a significant, and biologically costly
delay relative to the data-driven approach.

Fig. 10 BGL converges to the basal level at 170 min
by MPC appraoch, while BGL converges at 130 min by
the data-driven 2DOF control approach.

3) PD control
Two glucose inputs 20 and 3 mg/min are administeredat 6 - 11 min and 17 - 26 min, respectively. Using theclassical PD approach with a 5 min sampling interval,
the BGL converges to the basal level at 160 min. Whenthe first initial glucose input changes to 40 mg/min,
however, the BGL significantly undershoots theminimal allowable glucose level, which may be
dangerous to the patient. For both 20 mg/min and 40
mg/min, though, the data-driven 2DOF control approachmakes the BGL converge at 140 min while at the sametime remaining will within the safety range.

Fig. 9 In the event of disturbance, the time-driven
control converges to the basal level at 200 min, while
the data-driven control converges at 130 min.

2) MPC approach
The MPC (Model Predictive Control)[3, 31] is themost extensively applied control mechanism inindustrial processing besides PID control. The Linear
MPC approach first applies local Jacobian linearization,then uses a finite-horizon optimal control as follows,p( y(k  i | k )  r (k  i)) Q( y(k  i | k )  r (k  i))
T

i =1

m

 u (k  i  1)T Ru (k  i  1)

(86)
Fig. 11 Using the data-driven 2DOF control approach,the BGL converges at 140 min and above the minimallevel in both cases, while the BGL goes underneath theminimal level using the classical PD control approach.

i =1

Two glucose inputs 40 and 3 mg/min are administered at
6 - 11 min and 17 - 26 min, respectively. Using the MPC
approach with a 5 min sampling interval, the BGL©ISAROB 201112

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Blood Glucose Level(mg/dL)4) Backstepping control with Extended Kalman Filter
Yang et al. [6] proposed a backstepping control
algorithm for the glucose control problem. Two glucose
inputs 20 and 3 mg/min are administered at 6 - 11 min
and 17 - 26 min, respectively. When we apply theclassical backstepping control approach with a 5 min
sampling interval and given full state information, we
find that the BGL converges to the basal level at 150
min, which is comparable with our proposed approach.
However, when EKF (Extended Kalman Filter) [32] isapplied to estimate the states, the BGL goes underneaththe minimal allowable level.
50250

Data-driven 2DOF
Control2000National Science Foundation under Grant No. 0328378.
The authors are grateful to the Associate Editor and to
the anonymous reviewers for their valuable criticisms
and suggestions, and to Dr. Sharon Bewick for many
useful suggestions.

REFERENCES[1]

G. M. Steil, K. Rebrin, J. Mastrototaro, B. Bernaba, and M. F.
Saad: 'Determination of Plasma Glucose During Rapid Glucose
Excursions with a Subcutaneous Glucose Sensor,' Diabetes
Technology & Therapeutics, 2003, 5, pp. 27-31[2] G. Marchetti, M. Barolo, L. Jovanovic, H. Zisser, and D. E.
Seborg: 'An Improved PID Switching Control Strategy for Type
1 Diabetes,' IEEE Transactions on Biomedical Engineering,
2008, 55, pp. 857-865[3] R. Hovorka, V. Canonico, L. J. Chassin, U. Haueter, M. MassiBenedetti, M. O. Federici, T. R. Pieber, H. C. Schaller, L.
Schaupp, T. Vering, and M. EWilinska: 'Nonlinear model
predictive control of glucose concentration in subjects with type1 diabetes,' Physiol. Meas. , 2004, 25 pp. 905-920[4] S. M. Lynch and B. W. Bequette: 'Model predictive control ofblood glucose in type I diabetics using subcutaneous glucose
measurements,' Proc. American Control Conference, 2002, pp.
4039–4043[5] F. Chee, A. V. Savkin, T. L. Fernando, and S. Nahavandi:'Optimal H ∞ Insulin Injection Control for blood glucose
regulation in Diabetic patients,' IEEE Transactions on
Biomedical Engineering, 2005, 52, pp. 1625 - 1631
[6] R. Yang, M. Zhang, and T. J. Tarn, 'Control Mechanisms for Life
Science Automation,' in Life Science Automation: Fundamentals
and Applications, M. Zhang, B. Nelson, and R. A. Felder, Eds.:
Artech House Publishers, 2007.
[7] S. Devasia: 'Should model-based inverse inputs be used as
feedforward under plant uncertainty?,' IEEE Trans. on AutomaticControl, 2002,
47, pp. 1865-1871[8] M. van Nieuwstadt: 'Trajectory Generation for Nonlinear
Control Systems,' California Institute of Technology, 1996.
[9] D. L. Polla, A. G. Erdman, W. P. Robbins, D. T. Markus, J. DiazDiaz, R. Rizq, Y. Nam, H. T. Brickner, A. Wang, and P.
Krulevitch: 'Microdevices in Medicine,' Annual Review ofBiomedical Engineering, 2000, 2, pp. 551-576[10] N. T. Nguyen and X. Huang: 'Miniature valveless pumps basedon printed circuit board technique,' Sensors Actuators A 2001,
88 pp. 104-11[11] T. Gerlach and H. Wurmus: 'Working principle and performanceof the dynamic micropump,' Sensors Actuators A 1995, 50,
pp. 135-40[12] B. Fan, G. Song, and F. Hussain: 'Simulation of apiezoelectrically actuated valveless micropump,' Smart Mater.
Struct., 2005, 14, pp. 400-405[13] S. A. Ramsey, J. J. Smith, D. Orrell, M. Marelli, T. W. Petersen,
P. de Atauri, H. Bolouri, and J. D. Aitchison: 'Dual feedback
loops in the GAL regulon suppress cellular heterogeneity inyeast,' Nature Genetics, 2006, 38, pp. 1082-1087[14] G. M. Steil, A. E. Panteleon, and K. Rebrin: 'Closed-loop insulindelivery - the path to physiological glucose control,' Advanced
Drug Delivery Reviews, 2004, 56, pp. 125-144[15] S. Shimoda, K. Nishida, M. Sakakida, Y. Konno, K. Ichinose, M.
Uehara, T. Nowak, and M. Shichiri: 'Closed-loop subcutaneous
insulin infusion algorithm with a short-acting insulin analog forlong-term clinical application of a wearable artificial endocrine
pancreas,' Frontiers Medical Biological engineering, 1997, 8,
pp. 197-211[16] R. N. Bergman, L. S. Phillips, and C. Cobelli: 'Physiologic
evaluation of factors controlling glucose tolerance in man,'
Journal of Clinical Investigation, 1981, 68, pp. 1456-1467[17] A. Isidori: Nonlinear Control Systems (Springer, New York1989)

Initial glucose uptake

0

20

40

Backstepping Control
with full states

150
100
50

Insulin infusion rate(U/h)

Basal
Level
20

0

50
Minimal
Level

100

50

100

150
Time(Min)

200

250

Backstepping Control with
Extended Kalman Filter

15
10
5
0

0

150

200

250

Time(Min)

Fig. 12 Using the backstepping control approach withfull states information, the BGL converges to the basal
level at 150 min, similar to the data-driven approach.
However, when EKF is used to estimate the states, the
BGL goes underneath the minimal level.

6 Conclusions
A new data-driven 2DOF control mechanism forcontrolling a micropump and microneedle integrated
device is presented in this paper. Compared with several
feedback techniques in literature, this approachdemonstrates much shorter regulating time for glucose
control. In addition, this method also resists more model
uncertainties and unexpected disturbances than other
2DOF controls, while has the same regulaton
performance. This work focuses on theorectial
breakthrough and validation by computer simulation.
Labortary experiments will be implemented in future
work.

ACKNOWLEDGMENT
This material is based upon work supported by the©ISAROB 201113

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011[18] V. Hagenmeyer and E. Delaleau: 'Exact feedforward
linearization based on differential flatness,' International J.
Control, 2003,
76, pp. 537-556[19] M. Fliess, J. Levine, P. Martin, and P. Rouchon: 'Flatness anddefect of nonlinear systems: introductory theory and examples,'International J. Control, 1995, 61, pp. 1327-1361[20] M. Fliess, J. Levine, P. Martin, and P. Rouchon: 'A Lie-Backlund
approach to equivalence and flatness of nonlinear systems,'
IEEE Trans. on Automatic Control, 1999, 44, pp. 922-937[21] E. Delaleau and J. Rudolph: 'Control of flat systems by quasistatic feedback of generalized states,' International Journal ofControl, 1998,
71, pp. 745-765[22] M. van Nieuwstadt, M. Rathinam, and R. M. Murray:'Differential Flatness and Absolute Equivalence of Nonlinear
Control Systems,' J. Control and Optimization, 1998, 36, pp.
1225-1239[23] Y. Zhao and S. Jayasuriya: 'Feedforward controllers and tracking
accuracy in the presence of plant uncertainties,' Proc.
American Control Conference, 1994, 1994, pp. 360-364 vol.1[24] H. Sira-Ramirez, "Sliding mode control of the PPR mobile robot
with a flexible join," in Nonlinear Control in the Year 2000. vol.
259 London: Springer, 2000, pp. 421-442.
[25] P. Martin, R. M. Murray, and P. Rouchon, "Flat systems,
equivalence and feedback," in Advances in the Control of
Nonlinear Systems. vol. 264 London: Springer, 2000, pp. 5-32.
[26] R. Yang, M. Zhang, and T. J. Tarn: 'Dynamic Modeling andControl of a Micro-needle Integrated Piezoelectric Micro-pumpfor Diabetes Care,' Proc. IEEE Conference on Nanotechnology,
Cincinnati, USA, 2006[27] F. Doyle III, J. Harting, C. Dorski, and N. Peppas: 'Control andmodeling of drug delivery devices for the treatment of diabetes,'
Proc. American Control Conf, Seattle, WA, 1995, pp. 776-780[28] A. Piazzi and A. Visioli: 'Optimal Inversion-Based Control for
the Set-Point Regulation of Nonminimum-Phase Uncertain
Scalar,' IEEE Transactions On Automatic Control, 2001, 46,
pp. 1654-1659[29] T. J. Tarn, N. Xi, and A. K. Bejczy: 'Path-Based Approach toIntegrated Planning and Control for Robotic System,'
Automatica, 1996, 32,[30] N. Xi and T. J. Tarn: 'Stability analysis of non-time referenced
internet - based telerobotic systems,' Roboics and Autonomous
Systems, 2000,
32, pp. 173-178[31] R. S. Parker, F. J. Doyle III, and N. A. Peppas: 'A model-basedalgorithm for blood glucose control in type I diabetic patients,'
IEEE Transactions on Biomedical Engineering 1999, 46, pp.
148-157[32] M. Boutayeb and D. Aubry: 'A strong tracking extended Kalmanobserver for nonlinear discrete-time systems,' IEEE Transactions
on Automatic Control, 1999, 44, pp. 1550-1556©ISAROB 201114

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Rescue robot systems-From snake-like robots to human interfaceFumitoshi Matsuno

Kyoto University, Yoshida Honmachi, Sakyo-ku, Kyoto 606-8510, Japan(Tel : 81-75-753-5232; Fax : 81-75-753-4862)(matsuno@mi.kyoto-u.ac.jp)

Abstract: Intelligent rescue systems with high information and robot technology have been expected to mitigate
disaster damages, especially in Japan after the 1995 Hanshin Awaji Earthquake and in USA after the September 11,
2001 terrorist attack on New York City. Public safety and security problems are not limited to Japan and the UnitedStates, since every country has experienced man-made and natural disasters in the past. This paper introduces adeveloped grouped rescue robot systems with high-functionality multiple mobile robots and robust/scalable
information infrastructure for searching tasks in disaster scenario.
Keywords: search and rescue, safety and security, RT and ICT,

A lot of rescue robots have been developed in Japan.
From 2002 to 2007 "Special Project for Earthquake
Disaster Mitigation in Urban Areas" (DDT Project)launched by Ministry of Education, Culture, Sports,
Science and Technology, Japan [2]. In DDT project wehave 4 mission units (MU) to accomplish our objective
as follows with considering disaster scenario (Fig. 1).

I. Introduction
Intelligent rescue systems with information andcommunications technologies (ICT) and robotics
technology (RT) have been proposed to mitigate disasterdamages, especially in Japan after the 1995 HanshinAwaji Earthquake. In particular, it has been stressed theimportance of developing robots for search and rescue
tasks, which can actually work in a real disaster site. In
USA the September 11, 2001 terrorist attack on New
York City and Washington, DC, the hijacked plane crashin Pennsylvania, and the Anthrax attack that
immediately followed instantly changed people attitude
about safety and security in their personal lives. Public
safety and security problems are not limited to Japanand the United States, since every country has
experienced man-made and natural disasters in the past.
Solutions will depend upon new, unconventional
approaches to search and rescue. Robotics, informationand communications technologies, devices and system
integration can play an important role in providing
technology that can contribute to Safety, Security andRescue activities.

Fig. 1 Overall concept in disaster scenario
1. Information Infrastructure System Mission Unit
Main systems in this MU are RF ID tags and Microservers. Tasks of this group is global informationcollection (> 10 km) using ad hoc networks, micro
servers, RF ID tags, home facilities, etc. anddevelopment of communication protocols, data
structures, etc. for data integration.
2. Aerial Robot System Mission Unit
Main systems in this MU are helicopters, airships,
balloons. Tasks of this group are global surveillance (<some km) for information collection at the initial stateof incidents and local surveillance from sky (< 200 m)
for victim search and support of ground vehicles as thesecond deployment. Fig. 2 and 3 show an autonomous

II. DDT project of rescue robot systems
Japan, which suffered Hanshin Awaji Earthquake, has
drawn on the lessons of that experience to alleviate thedamage caused by disasters in major urban areas. It is
well known that rescue in 3 hours is desirable, and thatthe survival rate becomes drastically low after 72 hours.
We should search and rescue victims from debris within
this 'golden 72 hours' [1].

©ISAROB 201115

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

helicopter (Prof. H. Nakanishi, Kyoto Univ.) and aballoon for information gathering (Prof. M. Onosato,
Hokkaido Univ.), respectively.

4.

On-Rubble Robot System Mission Unit
Main robots in this MU are crawler type, wheeled,and jumping robots. Tasks of this group are local
surveillance on the rubble pile (< 50 m) for victim
search and environmental check. Fig. 6, 7 and 8 show awheel type robot FUMA (Prof. F. Matsuno, Kyoto
Univ.), a two-tracks robot with an arm HELIOS (Prof. S.
Hirose, Tokyo Institute of Technology) and a four-tracks
robot with an arm KOHGA3 (Prof. F. Matsuno, Kyoto
Univ.), respectively.

Fig. 2 Autonomous helicopter (Prof. H. Nakanishi)

Fig. 6 FUMA
(Prof. F. Matsuno)

Fig. 3 Info-balloon (Prof. M. Onosato)
In-Rubble Robot System Mission Unit
Main systems in this MU are serpentine robots,
crawler-type robots, and sensor balls. Tasks of thisgroup are local information collection in the rubble pile(< 30 m) for victim search and environmental check.
Serpentine snake-like robots KOHGA (Prof. F. Matsuno,
Kyoto Univ.) and SORYU (Prof. S. Hirose, Tokyo
Institute of Technology) are shown in Fig. 4 and Fig. 5,
respectively.

Fig. 7 HELIOS(Prof. S. Hirose)3.

Fig. 8 KOHGA 3 (Prof. F. Matsuno)
These rescue robot systems have some sensor, forexample a camera and a laser range finder (LRF). An
operator controls a robot using a user interface based on
the transmitted information from the remote site. Theirabilities are restricted by communication performancein a practical environment.

III. Development and Integration of New
Grouped Rescue Robots System
When size of a disaster area is very large and fast
information gathering is required, robotic system has touse multiple robots to acquire information. Anotherrequirement is online data processing to use collected
information to subsequent rescuer operation. To addressthese issues, the project [3] in our group founded by
NEDO from 2006 to 2008 focuses on the following:
I. A high-functionality multiple mobile robot system.
II. A robust and scalable information infrastructure,
which includes network and GIS data system.

Fig. 4 KOHGA (Prof. F. Matsuno)

Fig. 5 SORYU (Prof. S. Hirose)©ISAROB 201116

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

lightweight obstacles and open doors by using amounted manipulator. To construct a wirelessinfrastructure, the pioneer type robot is mainly
teleoperated from a remote safe place by an operator.
Next of the mission, a number of surveyor typerobots semi-autonomously search a target building
under the information infrastructure that was alreadyconstructed by the pioneer type robots, and gather
information of damaged building and victims.
Each robot has a network camera, a fish-eye camera,a LRF, an attitude sensor, IR sensors, rotary encoders
for motors which drive tracks, and potentiometer tomeasure angles of multifunctional and flipper arms. To
control developed mobile robot platforms, following
methods are implemented.
M1. Full manual control method with virtual
bumper: A robot is controlled by sending commands
related to translational velocity and rotational velocity
directly with joystick device. Function of the virtual
bumper stops the movement of the controlled robot
when an obstacle is detected within the pre-defined areaaround the robot by LRF and IR sensors.
M2. Line trajectory trace method: An operator points
a sub-goal location of the robot on an environment map
generated by a LRF, then the robot is controlled tofollow the line created by connecting the current andsub-goal points with a certain velocity until the robot
reaches the goal point.
M3. Right/Left hand wall following full-autonomous
control method: In this mode, a robot autonomously
cruises in the building along right/left wall with keeping
a certain distance from the wall without operator'scommands.
M4 Direction oriented semi-autonomous control
method: In this mode, an operator commands desired
moving direction of a robot. The robot runs to thedirection according to compass data with avoiding
obstacles.

III. An efficient user interface and control system tooperate robots and to manage lots of information.
To design above systems, we also focused following
issues:ADAPTABILITY: There are no "same-situation andsame-environments" in a disaster target area, and asituation will change fast in the disaster. The system has
to have adaptability by a configuration and capabilities
to adapt to changing environments.
SCALABILITY: To deploy robots, the communication
infrastructure is also extensible physically. Note: only
robots can address to the disaster area, robots have tohave network building function by itself.
USABILITY: The human resource is one of the highest
cost components of the system. The system has to haveefficient/usable interface to operate. For example,
simultaneous multiple robots operation support, semiauto operation support and information presentationinterface that only shows necessary information tooperators, for each situation.
According to the above mentioned sub-themes andissues, we newly developed four elements: mobile robot
platforms, ad-hoc wireless network called Robohoc
network, user interface to control multiple robots and
GIS data server
1. Mobile robot platforms
We have developed two types of mobile robot
platforms which achieve 1.0[m/sec] driving on a flat
plane, and have high mobility and high ability. One is apioneer type robot as shown in Fig. 9 (a), and the otheris a surveyor type robot as shown in Fig. 9 (b). The

2. GUI for controlling multiple robots
Developed user interface is shown in Fig. 10. GUI
enables to control multiple robots by one operator. For
selecting a control method of a robot, the operator clicks
a control button at the upper side of GUI (1 in Fig. 10).
To select destination of command for a robot, an
operator selects a target robot in a panel located at thebottom of the GUI. Outside of the selected robot panel
is colored. For each robot, the selected control method
is indicated in the robot panel (2 in Fig. 10). The robot(a) Pioneer robot(b) Surveyor robot
Fig. 9 Developed Robot system (Prof. F. Matsuno)
function and role of each type are below.
At first of a given mission, pioneer type robots
expand the wireless network area by deploying wirelessnetwork ad-hoc nodes. The robot can eliminate©ISAROB 201117

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Followings are the requirements for thecommunication system from the system design's pointof view
R1. Use wireless communication technology tocommunicate
withmultiplemovingrobotssimultaneously and to collect information from lots ofsensors which are deployed by robots in the target area.
R2. To communicate with moving robots and static
sensors.
R3. To supply enough bandwidth to sent a video stream
from a robot.
R4. To show the up-to-date status of the network itself
to other system.
R5. To be physically extensible by robots alone without
human help.
We developed not only the software and protocols
for Robohoc network, but also the prototype ofRobohoc nodes to construct the wireless ad-hoc networkfor our field trials (Fig. 11).

panel displays 3D CG model of each robot and shows
arm position, velocity and attitude of each robot, laser
range finder data around robot and on/off status of thevirtual bumper (3 in Fig. 10). The robot panel also
displays the state of network communication for the
robot (4 in Fig. 10). The robots have two or threecameras, and images from them are displayed at theupper left side of GUI. To select which camera image tobe displayed, a camera selecting button is located at theright side of the camera image (5 in Fig. 10). Note thatan operator can select only one camera, in order not toaffect a heavy load to Robohoc network. Quality ofcamera image (frame rate and compressing rate) also
can be selected by a slider bar (6 in Fig. 10). Moreover,
the operator can choose a sending method of image data.
In first method a raw jpeg image is send directly. In theother method a jpeg image is divided into small
reconfigurable images and divided images are sent.
3. Communication system
To operate multiple robots simultaneously in theFig. 11 Robohoc node (Dr. Y. Uo)
It is necessary to communicate information such ascommands to robots and sensor data to an operator
including video images stably with considering not tobreak down all network performance. Constructed
Robohoc network is a kind of wireless ad-hoc network,
so it has a limitation of throughput. To manage thebandwidth of Robohoc network, communication data isclassified into four categories according to two
attributes, data size and responsiveness, as shown in Fig.
12. This bandwidth management mechanism is based ontime sharing of communication resource, namely not all

Fig. 10 GUI (Prof. F. Matsuno)disaster area, a sort of full-wireless network system isrequired. Moreover, it should be robust to withstandenvironment changes and dynamic extension by robots
themselves. We proposed a network system which has
the characteristics in our previous paper [4], and called
it as "Robohoc network".
There are lots of previous studies about the
networking for the robotic system and/or the sensor
network system. However, the most important issue forthe network for the disaster situation is the adaptability
to the environment rather than the performance(throughput, number of nodes, etc.). This adaptability
also relates to the network bandwidth management
because both of the network throughput and latency arevariable parameters. The system has to change the total
usage of its network to follow such changes.

©ISAROB 2011

Large data sizeVideo data

Video image forremote control

Sound data

Fast responsive communication
Robot control data

Delay allowable communication
Topological information
Sensor data

Heart beat
Small data size

Fig. 12 Categories of communication data
(Dr. Y. Uo)

18

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Earthquake as a common spatial temporal databasemanagement system for disaster risk management [5].
Fig. 13 shows a screenshot of viewer of GIS.

data have to be sent in real time and there are delayabledata.
4. GIS server
We categorized roles of GIS as two spheres, spatial
temporal database management sphere and operation
support sphere. In spatial temporal databasemanagement sphere GIS has to deal with following fourtypes information;
I1. Base map information: In most of public building
or underground space base map data exists. In thismission we assume that rescue team can get base map
data before rescue activity. If different accuracy sets ofdata are provide, the highest accuracy data have to be
selected.
I2. Information collected by grouped rescue robots:
Trajectories of mobile robots to check their behavior,time series sensor information for robots themselves,
environmental information to get from various sensors
mounted on the mobile robots to check damage
condition of the target space, and locations of accesspoints for Robohoc network placed by the pioneer
robots to manage Robohoc network condition must be
stored.
I4. Robohoc network condition: Radio field intensity,
correspondence of access points, and transmission pathmust be stored.
I5. Resources of mobile robots and access points for
Robohoc network: Remaining amount of battery ofeach robot and access point, HDD of each robot,
equipped access points of each pioneer type robot must
be stored.
In operation support sphere we have two functions
as follows;
F1. Function for making support information for
robot operators: Calibration of initial point of robots,
input of changed objects from base map information,
and alert of no-operation areas caused byground/communication condition are assumed.
F2. Function for operation support for information
administrators: Access point location planning,
removing location errors from environment map
information generated by LRF data, input no-operationand search areas are assumed.
In order to realize the requirements for GIS it iseffective to use a temporal GIS as a databasemanagement system. In this case we select our original
temporal GIS "DiMSIS" which have been developedsince immediately after Great Hanshin-Awaji©ISAROB 2011

Storedpicture
撮影した
写真の参照

Robot
Trajectory
ロボットの軌跡

Photo
point
撮影ポイントFig. 13 Screenshot of viewer of GIS(Prof. M. Hatayama)
5. System Integration
Constructed system structure is shown in Fig. 14.
The system consists of an operator PC, a GIS server anda FE (Front End) server in the operator station, Robohoc
network and robots. The GUI on the operator PC is anapplication for controlling grouped robots remotely, anddisplaying instantaneous data sending from robots. GIS
server is a database server handling global position datasuch as position of robots, rescue teams and victims inthe disaster site. FE server is an application to connectuser interface, GIS server and robots. Robohoc network
ensures communications between the operator station
and robots. To use Robohoc network, the operator
station system robots equip a special network controller.
To connect with this special network controller, theoperator station and robots are connecting each otherwithout paying attention to the Robohoc network likenormal wireless LAN network.

Fig. 14 Constructed system structure(Dr. T. Kamegawa)

19The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

wireless network to neighborhood nodes and whichrobot connects to which node.

IV. DEMONSTRATIVE EXPERIMENTS
AND TRAININGS
We carried out the demonstrative trainings to testdeveloped robot system as shown in Fig. 15. This event
was held as a special event for SICE International
Conference on Instrumentation, Control andInformation Technology 2008 (SICE Annual
Conference 2008) in Tokyo on 21st August, 2008. In
this training, two incumbent fire fighters controlled 4
robots manually, and Robohoc network was not utilized.
As a mission scenario, the pioneer type robot wascontrolled to put Rohoboc nodes on an environment,
and then the surveyor type robots were controlled toinvestigate in a building. As a result, they could
accomplish to find (dummy) victims through video
images from the surveyor type robots.

Fig. 17 Network status monitoring viewer (Dr. Y. Uo)
At first, the pioneer type robot was manuallycontrolled to put Rohoboc nodes on an environment.
Then the surveyor type robots were manually andautomatically controlled to investigate in a building.
When the robot operator found something to update to
GIS data server, the information operator inputs the data
to GIS. The information operator sometime fixes a robot
trajectory drawn in a global map. When the robot
entered new area, it means there is no previous map, theinformation operator creates a new map according to
LRF information. Finally, operators could accomplish tofind (dummy) all victims in the floor through video
images from the surveyor type robots.

Fig. 15 Demonstrative training by fire fighters
After a while, we carried out another demonstrative
experiment for rescue activities using fully developed
grouped robot systems as shown in Fig. 16.

V. CONCLUSION
In this paper, the developed grouped rescue robots
system for searching in damaged buildings is described.
The system is consisted of four elements, mobile robot
platforms, user interface, ad-hoc wireless network and
GIS server. The system was integrated so as tocomplement the element's limitations each other.

REFERENCES[1] Matsuno F (2004), Rescue Robots and Systems
in Japan, Proc. IEEE Int. Conf. on Robotics andBiomimetics (Special Invited Talk)[2] Tadokoro S (2010), Rescue Robotics: DDT Projecton Robots and Systems for Urban Search and Rescue,
Springer.
[3] Matsuno F (2008), NEDO Rescue Robot System
Project, Interactive Organized Session, Proc. SICE
Annual Conference.
[4] Fujita S, Shima K, Uo Y, Esaki H (2009), A
Decentralized VPN Service over Generalized Mobile
Ad-hoc Networks, IEICE Trans. Information Systems,
E92-D, 10.
[5] Kakumoto S, Hatayama M, Kameda H, Taniguchi T,(2009), Development of disaster management spatial
information system (DiMSIS), Proc. GIS’97, 595-598.

Fig. 16 Demonstrative training by using
Robohoc network
In the demonstrative experiments, the system washandled by three people, a commander for all system, arobot operator and an information operator, and 5 robots
were controlled. In this demonstrative experiments, thesize of target area is 57[m]x32 [m] that is larger thanprevious demonstrative trainings, and 8 Robohoc nodes
were previously set in the building. It is assumed that apart of the map of the building were available for GIS
data server. Fig. 17 shows an application of a network
status monitoring viewer that indicates connection of©ISAROB 201120

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Human-Robot Interaction and Social Relation
Michita Imai (Dept. of Computer and Information Science, Keio University)
This paper presents the factor of designing an anthropomorphic agent such as acommunication robot and an embodied communicative agent. Communication is acooperative phenomenon between participants. People involved in a communication
speak and hear actively during the conversation. However, it is diﬃcult to engage
them in the communication with a robot because they seldom actively consider thecommunicative intention of the robot. The paper explains the importance of social
relationship between a human and a robot for engaging people in the communication.
In particular, we consider how the relation makes them consider the intention of therobot. Also, I show what factors of the robot design elicit the social relation.

1

Intoroduction

Researches related to HRI (Human-Robot Interaction) investigate how to design a robot which has a anthropomorphic character, behaves autonomously, communicateswith people. And they consider what application the robot
achieves. In spite of progress of many researches, few systems use the merit of HRI. An agent which makes people anthropomorphize itself is used at ATM or MS Ofﬁce. However, it gives their functions just something extra. However, there are a lot of researchers in the ﬁeld of
HRI. This means that they consider intuitively that there
are valuable things in the interaction between people anda robot. The paper explains what are the key factors for
applying HRI technology to an interactive system.

2

Key Factors at HRI research

Key factors at HRI research depend on a type of a used
robot. There are two types of robots; one has a anthropomorphic ﬁgure and the other does not. This paper focuses on the robot which has an anthropomorphic ﬁgure
because HRI researches are related to the aspect of communications. In particular, researches about non-verbal
expressions such as gestures and gaze movements are important for HRI ﬁeld.
I write four main factors which are important in designing a robot as a interactive system.
i. Characteristic of being athropomorphized
A human-like ﬁgure enhances the tendency of anthropomorphizing a robot. The robots which have a humanlike ﬁgure are intended to utilize the characteristic. The
human-like ﬁgure makes the robot communicate with people easily using verbal and non-verbal expressions.
ii. Capability of sharing environmentswith people©ISAROB 201121

The words ”sharing environments” do not indicate simply that the robot exists in the same place with a person.
Those mean ”cognitive sharing” (it is called Joint attention in developmental psychology [3]) that the robot pays
its attention to the same thing/event as people and thatit makes people consider that both of them look at it together. The design of the robot’s behaviors is crucial toestablish joint attention [1] [5]. The robot intentionallyturns its gaze toward a target, makes eye contact with aperson, and points at it with its hand at an appropriate
timing. A research reported that the ability of establishingjoint attention in a real world is higher than CG character on a PC screen [6]．The robot is more advantageousthan the other devices in terms of sharing environmental
information. For example, it can explain items at an exhibition or a route to a place by using verbal and non-verbal
expressions. Moreover, since the body of the robot exists
physically in our environment, the robot’s expressions canbe seen by people around the robot. The embodied expressions give information to them even though they are
not a primary person in the communication.
iii. Establishing relation with people
The relation between people and a robot is signiﬁcant for
natural communication between them. People understandwhat the other said by inferring his/her communicative intention. However, they do not make the inference for an
anonymous person. Since the inference is an active mental
function, people make the inference for someone related tothem. They seldom infer the communicative intention of
the others who do not have relation with them. Although
the relation exists between friends, family, colleague, andso on, it is not necessarily a formal one. People do notunderstand actively what the other says when they do noteven have a casual relation. The study [4] found thatthe existence of the relation between a experimental participant and a robot have eﬀect on whether he/she canunderstand the robot’s utterance. The physical existenceof the robot gives it the advantage when it establishes therelation with people.

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

iv. Constraints coming from existence[2] H. Ishiguro and T. Minato. Development of androids
for studying on human-robot interaction. In ProSince a robot has a physical body, it occupies our living enceedings of 36th International Symposium on Robotics,
vironment. However, the physical body improves the abilTH3H1, 2005.
ities of sharing environments and establishing relations.
CG character can also make the relations by preparing a [3] C. Moore and P. J. Dunham. Joint Attention: Itscontext which engages a person in the interaction. HowOrigins and Role in Development. Lawrence Erlbaum
ever, it is diﬃcult for CG character to communicate with
Associatcs, Inc., 1985.
people passing by. The CG character’s abilities of attracting people is weaker than the robot’s. The HRI research [4] T. Ono and M. Imai. Reading a robot’s mind: Amodel of utterance understanding based on the theshould employ the eﬀect of the physical existence posiory of mind mechanism. In Proceedings of AAAI-2000,
tively. For example, station staﬀs can manage the ﬂow ofpages 142–148, 2000.
crowded people in rush hour by just standing. Although
nowadays robots do not have such a existence like a sta[5] O. Sugiyama, T. Kanda, M. Imai, H. Ishiguro, and N.
tion staﬀ, the HRI researches like a android science[2] will
Hagita. Three-layered draw-attention model for hureveal what is an important factor to manage people.
manoid robots with gestures and verbal cues. In Proc.
of 2005 IEEE/RSJ Intl. Conf. on Intelligent Robots
and Systems (IROS 2005), pages 2140–2145, 2005.
3 Reconsider HRI research
The question related to HRI researches is that we need [6] K. Shinozawa, F. Naya, J. Yamato, and K. Kogure.
Diﬀerences in eﬀect of robot and screen agent recoma robot as the interface of an interactive system. Thismendation on human decisionmaking. International
section discusses the issue.
Journal of Human-Computer Studies, 62(2):109–123,
Is a robot needed to just achieve a com2005.
munication with people? ?
Dose a robot need an anthropomorphic figure?
Is the anthropomorphic figure importantto refer to information in a real world?
We do not need a robot if a system just gives us information. On the other hand, we must discuss the role ofcommunications in human-society to conclude the question.
Humans do a lot of activities by communicating withthe others: establishing a relation with others, empathizing with them, behaving considering them, and sharing
an experience with them. The important matter among
them is to behave considering the others. If a human and arobot behave considering the other’s mind each other, notonly the robot takes account of the human, but also he/she
considers the robot when doing something. The bidirectional consideration makes them share their experience.
The sharing the experience diﬀerentiates the robot froma simple computer which just gives anyone the same cold
information. There is possibility that the robot can give
each individual diﬀerent information reﬂecting a sharedexperience between them.
We should reconsider the robot as the interface of aninteractive system after succeeding in developing a robot
which can interact with people based on shared experiences.

References[1] M. Imai, T. Ono, and H. Ishiguro. Physical relationand expression: Joint attention for human-robot interaction. IEEE Transactions on Industrial Electronics(ITIED 6), 50(4):636–643, 2003.

©ISAROB 201122

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

Micro Nano Robotics
Tatsuo Arai
Micro nano robotics, Tatsuo arai, Osaka University, Japan©ISAROB 201123

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

©ISAROB 201124

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

©ISAROB 201125

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

©ISAROB 201126

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

©ISAROB 201127

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

©ISAROB 201128

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

©ISAROB 201129

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

©ISAROB 201130

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

©ISAROB 201131

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Quantifying “Waza” in Nihon-Buyo Dance Movements
Mamiko Sakata
Faculty of Culture and Information Science, Doshisha University
1-3 Tatara Miyakodani, Kyotanabe City, 610-0394 Japan(Tel : 81-774-65-7703; Fax : 81-774-65-7703)(msakata@mail.doshisha.ac.jp)

Abstract: Dance is an intangible cultural asset. It is passed on from one person to another through oral instruction, andso are the waza, the skills and techniques, involved in dance. Many traditional dances in Japan, however, are‘endangered species’ due to the shortage of practitioners. We are attempting to create digital archives to record andstore the body motions of buyo using digital technologies, such as motion capture. We are endeavoring to solve thisserious issue of losing an important tradition. In this paper, I would like to present part of our scientific analysis of thewaza in Nihon-buyo, which have been passed down, sometimes in silent, tacit manners.
Keywords: Japanese traditional dance, Nihon- buyo, Motion Capture“Tenaraiko.” The girl in “Musume-Dojoji” dances withher heart filled with love, and this same passage is used
in “Tenaraiko” when the young town girl dances in aprecocious manner. The lyrics and the dance motions
are the same in both plays. However, while the amorous
attention of an adolescent girl has to be portrayed in theformer, almost a childlike innocence of a precious girlhas to be portrayed in the latter.

I. INTRODUCTION
Dance is an intangible cultural asset. It is passed onfrom one person to another through oral instruction, andso are the waza, the skills and techniques. involved indance. Many traditional dances in Japan, however, are‘endangered species’ due to the shortage of practitioners.
We are attempting to create digital archives to recordand store the body motions of buyo using digitaltechnologies, such as motion capture. We areendeavoring to solve this serious issue of losing animportant tradition. In this paper, I would like to present
part of our scientific analysis of the waza in Nihon-buyo,
which have been passed down, sometimes in silent, tacitmanners.
The purpose of this research is to examine how theperformers of Nihon-buyo differentiate the basic
motions when performing different dance roles and how
their proficiency levels affect the differentiation. We
plan to achieve our goal by analyzing their motions andevaluating the impressions generated.

III. ABSTRACTION IN MOTIONSThis research examines the aforementioned common
portions of the dances from “Musume-Dojoji” and“Tenaraiko,” namely, the three basic motions of Nihonbuyo: Okuri, Osuberi and Mitsukubi.
Okuri is about the motion of walking. Although it isa simple walking motion, the performer in “MusumeDojoji” is required to walk with her toes turned inward
and slide on the floor, while her knees and pelvic region
are relaxed Musume-Dojoji. On the other hand, in“Tenaraiko,” the performer is required to walk lightly
with short steps while pressing the legs and knees
together.

II. THE DESCRIPTION OF THE WORK

1This research examines “Musume-Dojoji” and“Tenaraiko,” both key titles in Nihon-buyo. “MusumeDojoji,” which premiered in 1753, portrays anadolescent girl who is in love with a mountain priest.
“Tenaraiko,” which premiered in 1792, portrays aprecocious downtown girl who is on her way back homefrom a temple school called Terakoya. “Tenaraiko” is aplay that was made about forty years after “MusumeDojoji,” and a passage from this earlier play is used in©ISAROB 2011

2
8

6

3

5

7

9

4

13
14

10

12
11

17

15

16
18

24

19
20

25

21 23
29
22

26
27

28

Fig.1. Positions of markers

32The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

IV. MOCAP AND THE METHOD

of the graph, while the movement is slower when thescore approaches the bottom.
Fig. 2 shows that these measurable variables werelargely divided into three groups according to eachperformer. Comparing the performers, it was found that
performers B and C, with higher proficiency levels,
moved more quickly than performer A, a dance beginner.
All performers danced “Musume-Dojoji” andWe used an optical motion capture system tomeasure the body motions from the aforementioned
dance. In our research, we placed 29 markers on thedancer's body, and her movements were measured with
10 cameras(See Fig.1.). The acquired data can beobserved as a time series of 3 dimensional coordinate
values (x, y, z) of each marker in a frame (60 fps).
1. The Experiment: Performers
The performers selected for our motion analysis
were three women who have studied Nihon-buyo. Theirexperience varies. Performer A with Nihon-buyotraining and experience of less than a year, Performer B,
17 years, and Performer C, 15 years. We asked them todance each play five times. After taking themeasurements, we asked the performers what dance
strategies they used in their renditions.
2. Feature Values for Body Motion
In order to understand the intensity of the features ofeach dance motion, its physical features were extracted
from the perspective of time, space and dynamics,
which are the components of a movement.
Result of Okuri movement analysis
From the simple walking motion of Okuri, 16
physical features were obtained: the velocity and theacceleration of the vertex of the head, right shoulder,right elbow, right fist, lower back, right knee and righttoes, the angle of the knee and the height of the lower
back. We then conducted a principal component
analysis using a total of 32 variable quantities, whichwere the mean values and the standard variations. We
extracted 5 principal components with the eigenvaluesof 1 or greater, with an accumulated contribution rate of
84.5%. We interpreted PC1 to be the variable quantity
showing “the speed of motion,” PC2 “the motion ofhand,” PC3 “the strength of the lower body,” PC4 “thestrength of the elbow” and PC5 as the “the strength of
the feet.”
The PC1 and PC2 scores were plotted on the x-axis
and the y-axis respectively, and all trials were plotted on
the x-y graph ,as in Fig. 2. PC1 is the axis showing the
speed of motion. In this Fig., the speed increases as thescore shifts toward the right side of the graph anddecreases as it shifts toward the left. PC2 is the axis
showing the motion of the hand. In the same Figure, thehand is moving more quickly as the score nears the top

Fig.2. Plot of PCA score of Okuri

Fig.3. Plot of PCA score of Osuberi

Fig.4. Plot of PCA score of Mitsukubi©ISAROB 201133

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

We plotted the PC1 and PC2 scores on the x-axis
and y-axis respectively, plotting 35 trials on the x-ygraph. This is shown in Fig. 4, and the variables plottedby each performer are shown in Fig. 5. PC1 is the axis
showing the motion of the head. In these figures, thehead moves more quickly toward the right side of thegraph and more slowly when plotted toward the left.
PC2 is the axis showing the motion of the shoulders.
The figures show that the shoulders move more quickly
and strongly when plotted near the top of the graph andmore slowly and weakly when plotted near the bottom.
Unlike Okuri and Osuberi, the quantified variablesof Mitsukubi did not divide into groups according to theperformer. Notably, only the motions of performer C in
“Tenaraiko” were very different. Regardless of the
proficiency level, PC2 (the motion of the shoulder) was
not stable, suggesting that the performers paid lessattention to their shoulders than to their heads.

“Tenaraiko” differently; however, the ways theydifferentiated their dances varied considerably fromperformer to performer.
Result of Osuberi movement analysis
From the soft curvilinear motion of Osuberi, we
obtained 12 physical features: the velocity andacceleration of the vertex of the head, right shoulder,right elbow, right fist, lower back, right knee and thetoes of the right foot, the angle of the right knee and theheight of the lower back. Again, we conducted aprincipal component analysis using a total of 24
variable quantities, which were the mean values and thestandard variations. As a result, 5 principal components
with eigenvalues of 1 or greater were extracted (theaccumulated contribution rate was 85.3%). We named
PC1 as “the speed of motion,” PC2 as “the bend of theknees,” PC3 as “the strength of the body’s center,” PC4
as “the strength of the area below the neck” and PC5 as“the strength of the feet.”
We plotted the PC1 and PC2 scores on the x-axis
and y-axis respectively; all trials were plotted on the x-ygraph. The result is shown in Fig. 3. PC1 is the axis
showing the speed of motion. In Fig. 3, the speed
increases toward the right side of the graph and it
decreases toward the left. PC2 is the axis showing thebend of the knees. In the Fig., the knee is stretched more
as it nears the top of the graph, but the knee is morebent towards the bottom, with the lower back in an
increasingly lower position.
As Fig. 3 shows, quantitative measurements of the
Osuberi motion showed similarity with Okuri, i.e., theresults were divided into three groups according to theperformer. It was also found that performers B and C,
with higher proficiency levels, differentiated the dancesfor each play. Performer A, however, was not able todifferentiate them quite as well.
Result of Mitsukubi
Mitsukubi mainly involves the motions of the head
and the shoulders, so we only calculated 4 kinds ofphysical features, i.e., the speed and the acceleration of
the vertex of the head and the shoulders. By conducting
a principal component analysis using a total of 8
variable quantities, which were the mean values and thestandard variations, 2 principal components witheigenvalues of 1 or greater were extracted, with anaccumulated contribution rate of 80.2%. PC1 wasnamed “the motion of the head” and PC2 “the motion of
the neck.”©ISAROB 2011

V. PSCHOLOGICAL EXPERIMENTS
In order to examine the type of impression perceived
from the body movement of the two plays, we
conducted a psychological rating experiment using stick
figure animation (see Fig. 1) of the motion capture data.
Twenty-four observers (7 men and 17 women)
participated in this experiment. The mean and thestandard deviation of age among the 24 observers were
21.7 and 0.98 respectively. They had no experience indance performances of any kind and no particularknowledge about dancing and traditional Japaneseculture. The animation was projected on a 50-inch
display with no sound. The stick figure animation andmuted audio were used to force the viewers to focus on
the impression expressed by the body movements alone,
discarding other factors such as facial expressions,costumes, music, etc.
After each movement was shown, the viewers wereasked to answer the questions on the response sheets. In
this rating, we employed the Semantic Differential
questionnaire. In the SD questionnaire, 10 image-wordpairs were used for rating the movements. We selected
10 word pairs which were thought more suitable for theevaluation of human body motions, based on the list
presented by Osgood [2]. The viewers rated theimpression of the movement by placing checks in eachword pair scale on a sheet.
The rating was done in 5 ranks from 1 to 5. Rank 1
was assigned to the left-side word of each word pair and

34

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Mitsukubi motion, the audience received different
impressions from different performers.
As a result of the evaluation experiment, it was
revealed that even the audience lacking any knowledgeof Nihon-buyo understood, to some extent, thedifferences in the roles played in the stories. There wasalso a tendency for the audience to perceive the“different characters” of the performers rather than the“difference in choreography” of their dances. That is tosay, it was revealed that the audience was seeing the
motions of each performer, or the difference in personal
characteristics, rather than the difference in the dance’s
choreography.

5 for the right side. Using this rating, we obtained anumerical value representing an impression for each of
the body motions from each subject.
We then conducted a principal component analysis,
PCA, based on a correlation matrix, to the mean valueof the rating value and obtained the principal
component matrix. Two significant principal
components were extracted, which were PC1—PC2. We
used the word pairs ‘natural – unnatural,’ ‘experienced –inexperienced,’ ‘like – dislike,’ etc., which are often
used to represent refinement. Hence, it was interpreted
that PC1 was a variable that related to the “refinement”
of the motion. Similarly, PC2 was related to the“activity,” as indicated by words such as ‘large motion –small motion,’ ‘adult-like –childlike,’ ‘bright – dark’ and‘happy – sad.’
We can conclude that the characteristics of the dance
motions in “Musume-Dojoji” and “Tenaraiko” arebased on two aspects - “refinement” and “activity.”
Fig. 5 is a plot of the principal component scores ofeach motion datum. Since the x-axis (PC1) represents
the degree of refinement, the performance is morerefined toward the right side of the graph and lessrefined toward the left. The y-axis (PC2) represents
activity. There is more activity toward the top of thegraph and less activity as the plots near the bottom of
the graph.
By looking at each motion, it was found that Okuri
motions were more active for every performer in“Tenaraiko” but more refined in “Musume-Dojoji.”Considering that “Tenaraiko” is the dance of a child and“Musume-Dojoji” the dance of an adolescent girl, thepsychological intentions of the performers seem to havebeen understood well by the audience. As for the
Osuberi motion, the degrees of “refinement” and“activity” were higher for “Musume-Dojoji.” As for the

VI. FUTURE POSSIBILITIESWe can get information concerning the personalityof the subject when we observe his or her body motion.
We may get various impressions from body motions.
This means that the human body motions convey
emotion and personality of the person. Personality
might be the involuntary and continuous expression ofemotions, which are peculiar to the individual.
The results of this paper could be applied to
producing a robot or CG character animation withpersonalities. Until now, many attempts have been madeto add or enhance emotional expression of robots using
linguistic communication, some simple body motions,e.g. nodding, and facial expressions. Also, changing
the design or shape of robots might be a simple way ofproviding a robot with personalities. However, we
could not find much research on giving robots
personalities with body motions.
We think changing the personalities of a robot bychanging its own body motions, and changing theexpression of the affects of the robot through body
motions are very promising areas for further
investigation.

REFERENCES[1] Hachimura K(2006), Digital Archiving of Dancing,
Review of the National Center for Digitization 8: 5
[2] Osgood, C. E. et al.(1957), The measurement of
meaning, U. of Illinois Press

Fig.5. Plot of PCA score of the psychological ratin©ISAROB 201135

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Social playware for mediating tele-play interaction over distanceHenrik Hautop Lund

Tumi Thorsteinsson

Center for Playware, Technical University of Denmark, Building 325, 2800 Kgs. Lyngby, Denmark
hhl@playware.dtu.dk
www.playware.dk

creation of playful technology, e.g. robotic toys andinteractive playgrounds, has taken its inspiration fromsuch human-machine interaction research. Based on this,the research community has also gained knowledge on
how the individual interacts and plays with suchplayware products (e.g. [4, 5]).
In our point of view, it is important to expand theplayware research to focus on the social interaction, sothat the starting point for the research and development
becomes the social interaction mediated by thetechnology rather than the individual interaction
mediated by the technology. We can define social
playware as follows: social playware is playware whichaims at creating playful social interaction between
several users.
Such playful social interaction can, for instance, be play
between children in a kindergarten mediated by aninteractive playground, multiplayer games with aphysical game platform like Nintendo Wii, andinteraction of a team of elderly performing games forhealth e.g. with Dance Dance Revolution, i.e. in all casesa free activity that the users engage in for the pleasure ofplay and social interaction in itself. Hence, this is inaccordance with the definition of play, which can be
defined as "Play is actions which we undertake andparticipate in with the purpose to create a reality-sphere
within which we are free and independently can create
and regulate moods (physical and mental states oftension) which provides us with specific, wanted
experiences (of delight), socially and individually" [6].
The definition underlines that play is submitted to freewill, and that human beings play because we want toplay. At the same time it underlines that we, in the act ofplaying, manage our lives at our own choice, as we create
the special form of lived life outside the “regular” life
where (lust for) life and happiness as the essence of playrules. By building on this play definition, the definitionof social playware simply focuses on the human desire toengage in social interactions and to live as a social being.
It can therefore also be argued that social playware isincluded by playware. Indeed, we view social playware
as a sub-discipline of playware, which however allow the
research community to direct focus on the social
interaction mediated by playware, and thereby provide

Abstract
We suggest that novel playware technology can function
as a mediator for playful social interaction over distance,
where people are separated by physical distance but feelthe presence of each other mediated through theinteraction with the playware technology. In order toinvestigate such social playware, we developed thePlayware Soccer game and tested this with more than
1,000 users during the FIFA World Cup 2010 in South
Africa. The test was conducted in townships, orphanages
for HIV/AIDS children, markets, FIFA fan parks, etc.
along with simultaneous tests with similar set-ups inEurope and Asia. With the social playware, players
would compete against each other simultaneously in thethree continents, Africa, Europe and Asia, and feel thepresence of the competitors on the other continents
expressed through the playware. The playware game isset up to motivate players to engage in training oftechnical soccer skills by receiving immediate feedback
and offering challenges to players of all skills on thesoccer playing on a modular interactive wall composed ofmodular interactive tiles that respond with coloured light,
sound and scores on the players performance. This paper
outlines the concept of social playware and physicalvirtual teleplay, and exemplifies this with the playware
soccer game.

Social Playware
Playware is defined as intelligent hardware and softwarethat creates play and playful experiences for users of allages [1]. In this paper, we suggest that novel playware
technology can function as a mediator for playful social
interaction over distance, where people are separated byphysical distance but feel the presence of each othermediated through the interaction with the playware
technology. Often, human-machine interaction is viewed
as a 1-to-1 interaction between an individual humanbeing and the technological artefact, and a lot of researchwithin the fields of human-robot interaction, social
intelligent robotics, and human-computer interaction has
put focus on the individual relationship and interaction
with the technology (e.g. [2, 3]). In many cases, the©ISAROB 201136

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

performance. The soccer game was developed together
with professional soccer players Laudrup and Høgh forpromoting playful soccer skills. For the test conductedduring the World Cup tournament, the soccer wall wascomposed of 3*4 modular interactive tiles. It is a
distributed system as each tiles has its own processor,
battery and communication to neighbouring tiles. The
distributed nature of the system aimed at allowing thesystem to be easily set up and taken down within minutes
anywhere and by anyone. Indeed, the flexibility obtainedwith a modular and distributed processing system should
provide the opportunity to bring the new playware
technology out to any township, market, and village in
Africa and on other continents since there was no
demand for any physical infrastructure whatsoever.
Often, other technological systems for physicalinteraction are characterised by being based on acentralised processing system making the systems fixed
sized (and sometimes large and bulky), and/or they arecharacterised by the demand for some kind of
infrastructure, e.g. electricity, access to screen/projector,
or similar. Examples include Lightspace, Makoto,Sportswall and even DanceDance Revolution with moreparticipants, which needs to have a centralised control
station. This makes it somewhat difficult to apply the
traditional technology for any user anywhere, since inmany places of the World, the necessary infrastructure is
not readily available to allow such technology to be
applied. This is, for instance, the case many places inAfrica, and even in a comparably developed country likeSouth Africa, where the FIFA World Cup 2010 was held,
there are townships with no electricity (Fig. 1).

further understanding on how to create social interactions
that are playful and which the users engage in for thepleasure of the social interaction.
We will exemplify the social playware in this work withphysical-virtual teleplay, which allow users to engage insocial interaction over distance mediated by the playware
technology. In such a case, we can view the playware
technology to mediate a playful tele-presence between
people interacting with each their playware tool throughwhich they sense the presence of the other people (inessence, presence removes the impression of mediationfrom a mediated experience). Examples of teleplay andsocial playware are seen in the form of massively
multiplayer online games (MMOG), which are
multiplayer video games which are capable of supporting
hundreds or thousands of players playing together
simultaneously over the Internet. However, such games
do not allow for extensive physical and naturalinteractions (other than e.g. pressing keyboard andspeaking). Some physical interaction is promoted withsome multiplayer online games, e.g. in the Japanese andKorean arcade halls, in the form of interaction with gamecards (e.g. soccer cards). For a more natural and physicalinteraction, tele-presence has been studied intensively
e.g. by Ishiguro with the studies of humanlike presence
using tele-operated androids [7, 8]. Such studies havepromoted a humanlike technology to study presence, andsome large videoconferencing set-ups have allowed for asoccer teleplay [9]. We, on the other hand, have made afirst step with a much simpler, yet playful technology formediating social interaction (in the line of the simple, yeteffective FeelLight technology for mediating social
interaction by Suzuki and Hashimoto [10]). Hence, wewill study tele-presence without a large and bulky
infrastructure, and without any anthropomorphic
expression, but a simple expression of simple lightpatterns, sound, and score. In the specific example, we
made social playware connecting players in Asia, Africa,
and Europe in a soccer game on a playware technology inthe form of modular interactive tiles.

Playware Soccer
During the FIFA World Cup 2010, we ran a RoboSoccer
World Cup in Asia, Europe and Africa, amongst otherplaces in townships, orphanages for HIV/AIDS children,
markets, etc. in South Africa. As an example ofplayware, the game is set up to motivate players toengage in training of technical soccer skills by receiving
immediate feedback and challenge players at different
levels on the soccer playing on a modular interactive wallcomposed of modular interactive tiles that respond withcoloured light, sound and scores on the players©ISAROB 2011

Fig. 1. Playware soccer in the township Atteridgeville,
South Africa, during FIFA World Cup 2010.
If, on the other hand, we take as point of departure forour technology design that no infrastructure is available,
it will lead to technology that is free from infrastructure
demands and which thereby possibly can be applied andused anywhere. It gives the possibility to bringtechnology to anybody anywhere, and thereby help in

37

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

approximately 30 hours and takes 3 hours to recharge.
On the PCB, there are connectors to mount an XBee
radio communication chip. Hence, there are two types oftiles, those with a radio communication chip (mastertiles) and those without (slave tiles). The master tile may
communicate with a game selector box (game card
reader) and initiates the games on the built platform.
Every platform has to have at least one master tile ifcommunication is needed e.g. to game selector box or aPC.
With this specification, a system composed of modular
interactive tiles is a fully distributed system, where eachtile contains processing (ATmega 1280), own energy
source (Li-Io polymer battery), sensors (FSR sensor and2-axis accelerometer), effectors (8 colour LEDs), andcommunication (IR transceivers, and possibly XBee
radio chip). In this respect, each tile is self-contained andcan run autonomously. The overall behavior of thesystem composed of such individual tiles is however a
result of the assembly and coordination of all the tiles.

contextualising both technology development andeducation in developing countries. The advantages ofsuch technology outcome may not be limited to thedeveloping part of the World, but the freedom from
infrastructure requirements may also have important
impact on the distribution and use of technologicalsolutions in the developed part of the World (e.g. forhome care in the private homes of elderly). Even in a
private garden or a football training field in thedeveloped World, the necessary infrastructure such as
electricity outlets or computer monitors may notnecessarily be available. Therefore, it is interesting toresearch the flexibility of the modular playware forallowing the technology to be set up and used anywhere
within minutes.
Hence, the playware soccer game was developed with themodular interactive tiles system [11], which is an
example of modular playware [12]. The system iscomposed of a number of modular interactive tiles which
can attach to each other to form the overall system. Each
modular interactive tile has a quadratic shape measuring
300mm*300mm*33mm. It includes an ATmega 1280 as
the main processor in each tile, and each tile cancommunicate with infra-red (IR) to its four neighbouring
tiles. Each side of a tile is made as a jigsaw puzzle
pattern (see Fig. 2) to provide opportunities for the tiles
to attach to each other. A force sensitive resistor (FSR) is
mounted as a sensor on the center of a raised platform
underneath the cover. This allows analogue measurementon the force exerted on the top of the cover.
On the PCB, a 2 axis accelerometer (5G) is mounted, e.g.
to detect horizontal or vertical placement of the tile. EightRGB light emitting diodes (LED SMD 1206) are
mounted with equal spacing in between each other on acircle on the PCB, so they can light up underneath thetransparent satinice circle.

Connectivity
In order to develop teleplay for social interaction it isimportant that the physical interactive platforms cancommunicate with each other, locally and globally, sothat the social interaction can be mediated through the
playware.
Local connection
For creating local communication between physicallyseparated groups of modular interactive tiles, andbetween a group of tiles and a PC, we used the XBee
with the ZigBee radio communication protocol. In eachgroup of tiles, there is one tile (master tile) with the XBee
radio communication chip. This tile can collect and send
information. The information can thereby becommunicated between two “islands” of tiles, i.e.
between the master in one island and the master inanother island. For communication to and from a hostcomputer, we designed an XBee USB dongle to beconnected to the host computer, which then cancommunicate with the master tile using the same
protocol.
Global connection
With the local communication allowing easy
communication between tiles and a host computer, e.g. a
laptop/netbook, we were able to relay the global
communication over laptops connected to the internet,
e.g. laptops with 3G wireless connection, so that the
teleplay could happen on platforms that communicate toeach other over the Internet. A Java program wasFig. 2. Assembly of the modular interactive tiles as a
jigsaw puzzle.
The modular interactive tiles are individually battery
powered and rechargeable. There is a Li-Io polymer
battery (rechargeable battery) on top of the PCB. A fully
charged modular interactive tile can run continuously for©ISAROB 201138

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

designed to run on the laptop, which was connected tothe tiles with the XBee. The Java program kept track of
the hits on the tiles, played feedback sounds, showed thetime and score of the game at run-time, and kept the total
score of each game. At the end of each game, theprogram sent information to a web-site that saved it
together with a username, password and location on ahighscore list which was updated immediately. The
highscore list updates would be visible on internet
connected computers at different locations, anywhere
globally, at run time.
With the design of both local and global connectivity, it
is possible to create both local and global physicalinteractive games. The local connectivity was used tocreate feedback from a local host computer in the form oftime and score displayed on a monitor, and sound from aloudspeaker connected to the host computer. The global
connectivity was used to allow feedback in the form of
run-time score updates in competition between users
playing the same physically interactive game in different
parts of the World, i.e. allowing for social interaction
over distance.

Layered Multi-Modal Feedback
For increasing the motivation to play the game, we
designed a multi-modal immediate feedback, so that theplayer would not only receive immediate feedback
directly from the tiles in terms of the changing coloured
light, but we also added sound feedback and graphicalfeedback in terms of time and score via a host computer,to enhance the system as social playware. When a player
would hit a lit tile, the light would turn off on that tileand jump to another tile, a sound would be played from aloudspeaker, and the increase in score would be shown
on a monitor. And when the game ended, the position on
the high score list would be shown on a monitor.
It is noteworthy, that the game design was made so thatthe game can run as an interesting game even without
these additional feedback modalities. Both the additional
immediate (sound, score, time) and delayed (local
highscore list and global highscore list) feedback
modalities can be added as layers on top of the basic
game that runs on the modular interactive tiles only (seeFig. 3). Hence, with this layered design of feedback
modalities, it is possible to (i) run the game as a simple
game with only the lowest level of feedback (coloured
light) on the modular interactive tiles, (ii) run it withhigher levels of feedback (sound, score, time) by adding
a laptop PC, or (iii) run it also with the highest level of
feedback (global highscore list) by adding an internet
connection. This third option (iii) was used for the
teleplay experiments to create the soccer game as a social
playware.

Soccer GameThe game content was crucial to ensure training of soccer
skills in a playful manner, so we collaborated withprofessional soccer players Laudrup and Høgh, in orderto create an appropriate game utilizing just 3*4 tiles. In
the soccer game, a specific number of tiles light up in
different colours. Each of them counts down with theireight LEDs. The player has to hit the tile before the LEDs
are all turned off, and gets points for how many LEDs areturned on at hit time, and points are multiplied by a factor
for how high the tile is positioned (row 1, 2, or 3). Also,
at random time, one of the tiles will have its LEDs
making a fast spinning pattern, indicating that if the tile ishit, a bonus round will be initiated, during which theplayer can gain extra points when hitting the tiles that are
lit up.
Preliminary testing with a number of adult players
showed that the soccer game could be set to an
appropriate difficulty level that was both easy enough toplay for all the test persons and difficulty enough that all
would be challenged to obtained higher score. Thisdifficulty level was set experimentally by investigating
the time needed for people to kick the ball and hit a tile,
so as to set the LED countdown time to an appropriate
level (the time used from all 8 LEDs were turned on,
until all LEDs were turned off, and the light would jump
to another tile).

©ISAROB 2011

Layer
5
4
3
2
1

Platform
Internet
PC monitor
PC monitor
PC loudspeaker
Tiles

Type
Global highscore list
Local highscore list
Time & Score
Sound
Light

Fig. 3. The layered multi-modal feedback design for the
playware soccer game.
The layered structure in designing feedback modalities
may resemble the layered design in much behavior-based
robotic engineering [13]. For instance, the original
subsumption architecture by R. Brooks [14] defines thatbehaviors can be designed to run in parallel on top ofeach other, starting from the design of the simplest
behaviors. Once the simplest behavior is designed,
implemented and debugged, this behavior can run byitself, and a behavior can be designed, implemented anddebugged to run in parallel on top of the simple behavior.
So forth continues the design with layers of behavior on

39

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

market, a village, an official FIFA Fan Park, a science
discovery centre, a university, a fan bar, a public park in
Soweto, etc. This variety of places was selected in orderto ensure the broadest possible test in terms of variationon the environment, the social status, the age group, theeducational level, the technology interest, and the soccer
interest of the users. Indeed, users were from 3 years old
to 80 years old (see Fig. 4), they were from orphanages
with children from families with HIV/AIDS to adultsoccer fans from high income areas, and they wereranging from people with no education to people with
university degree.
The system was designed for flexibility with the modular
interactive tiles and the layered multi-modal feedback
design, which together aimed at creating a system that
could be set up and used by anybody anywhere withinminutes. The modular interactive tiles can be viewed toprovide hardware building blocks, and the layered multimodal feedback design to provide feedback building
blocks, and simple construction with these building
blocks should give a high degree of flexibility for thedesigner and the user to create various set-ups andinteraction possibilities in an easy manner.

top of the previous ones that all run in parallel, and thelower levels continue to function as originally designed.
The design of multi-modal feedback, which we propose
here, works with the same principle. First, a simple
feedback is designed which can run by itself, and then
new layers of feedback can be added on top to run inparallel. In the present case of the soccer game, thesimplest feedback is designed to be the change of light on
the modular interactive tiles when a tile is hit. Once thisfeedback modality was designed, implemented anddebugged, on top of this, we designed, implemented anddebugged the sound modality, which would run inparallel with the light feedback. Then, on top of this, we
designed, implemented and debugged the time and score
feedback from a monitor. On top of this, we added the
local highscore list feedback. And on top of this, we
designed, implemented and debugged the global
highscore list.
As with the original subsumption architecture where
different behavior modules can run on different timescales, also with this layered multi-modal feedback
design, the different layers may run on different timescales, with the lower levels executing with the fastest
feedback time cycle and the highest levels the slowest
feedback time cycle. The lower level behaviors /feedback modalities need to give a very fast response forthe system to work, whereas the higher level behaviors /feedback modalities can give response once in a while.
The advantage of this layered multi-modal feedback
design is that it is possible to create simple layers of
feedback, that can run by themselves and work at theirown right, and then add new layers to run in parallel on
top of the previously designed layers, and when the user
is executing the system, it is possible to add/remove
layers (feedback modalities) from the top. Essentially,
this can even be done at run-time, adding and removing
new feedback modalities, since the lower levels will keep
running and working whatever is added on top of them.
This gives a high flexibility of the system for both thedesigner and the user of the system.

Fig. 4. An older man playing the playware soccer at ataxi rank in Randburg, South Africa.
In total, the system was tested with more than 1,000 users
during the FIFA World Cup 2010. The distributed natureof the system (each tile with its own processor, battery
and communication to neighbouring tiles) allowed thesystem to be easily set up and taken down. Indeed, theflexibility obtained with a modular and distributed
processing system gave the opportunity to bring the new
playware technology out to any township, market, andvillage in Africa since there was no demand for any
physical infrastructure whatsoever. It proved possible toset up the system in a very fast manner on the grounds intownships such as Soweto and Atteridgeville, in publicparks in Soweto, markets and bus station in Randburg,and in remote villages such as Phokeng. At some places,
the system was run with only part of the layered multi-

Tests
In order to explore social playware and the potential ofsuch social playware mediating social interaction, we
needed to test broad ranges of cultural differences inusers and environments. Therefore, we tested the systemsimultaneously in Denmark (Europe), South Africa(Africa), and Japan (Asia) during the FIFA World Cup
2010. For instance, in Asia the system was tested inhighly metropolitan areas, such as in Shibuya, Tokyo,
whereas in South Africa we tested in a variety of places,
including an orphanage, numerous townships, a public©ISAROB 201140

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

observe the emotional engagement and social bonding
both locally around the game, player and audience, andalso between the remote competitors who were invisible
and unknown to each other.

modal feedback, and in other places it was run with alllayers active. The layered multi-modal feedback allowed
a set-up with e.g. just layer 1 or just layer 1-4 in someplaces, and in other places to run the full system withlayer 1-5 (including global high score list via internet
connection, see Fig. 5). Therefore the system proved
flexible to make fit to the time available, the local use
and the aim of the game at a particular place with aparticular set of users.

Discussion and Conclusion
As a test of social playware, during the FIFA World Cup
2010, we ran a RoboSoccer World Cup in Asia, Europe
and Africa, most notably in townships, orphanages for
HIV/AIDS children, markets, etc. in South Africa. We
linked the events together with a novel kind of physicalvirtual live competition, which can be termed teleplay,
and which as a social playware mediated social
interaction. The teleplay took place between people inthese African environments and metropolitan fans inlarger cities in the developed World, e.g. in Tokyo,
thereby trying to create a social bond and feeling between
the fans world-wide during the World Cup through thephysical-virtual teleplay. The social bonding wasmediated through the physical football game between
players on different continents who at the same time,through the teleplay with social playware, can compete
directly and physically between the continents.
The flexibility of the modular interactive tiles and thelayered multi-modal feedback design, allowed thecreation of a system that could be set up and used byanybody anywhere within minutes, and it was therefore
possible to test the system with more than 1,000 users
during the FIFA World Cup 2010. Videos of some tests
are available at: www.playwaresoccer.com
In general, the advantages of the proposed modular,social playware can be summarized to a flexible set-up,
independence on context, runtime feedback, competition
as a motivation factor, framing of the game (World Cup
soccer where one country wants to beat another country),
audience friendly game through sounds and score which
can be followed by the audience, and where the audiencecan take on roles and feel as a part of the game (cheering,collect balls, make indications on bonus rounds, etc.).
Hence, the layered multi-modal feedback in the playware
set-up can mediate both local social interaction andglobal social interaction. The disadvantages of the set-up
as proposed here are that this is not in-game presence (thetele-presence is not immediate but delayed), it is a singleplayer game, physical differences may mean that players
are not competing on equal footing (e.g. the physicalstatus of the opponent on the other continent is unknown
to the player). Another disadvantage of the presented
study is that we only have “event-based” observations
and no long-term observations. We will elaborate further
on these issues and on social playware in general in
future work.

Fig. 5. The global highscore list on the internet(www.playwaresoccer.com).
When running the system with all five layers and indifferent locations at the same time, the system became asocial playware which mediated social interaction overdistance. Most often, social interaction would happen
around the single set-up e.g. in a township or a market
with lots of people gathering around the playware soccer
set-up cheering, helping, and interacting socially aroundthe playware and the individual player (e.g. see Fig. 1).
So it was evident from the test observations that evenwith the playware soccer set-up utilizing only the lower
layers of feedback modality, it became a social playware.
However, this was reinforced to a large degree when thegame was set up with all five layers and run in parallel at
different locations, e.g. simultaneously in the small
village of Phokeng in South Africa and in Shibuya in thecenter of Tokyo in Japan. In such cases, the players wereobserved to engage in a competition over distance: in one
location (on one continent) they would see the scores ofplayers in the other location (on another continent)
playing with the playware soccer. The players would
experiencing the high score list change minute after
minute depending on the score at their own location andthe score at the other location (visualized andcontinuously updated on the monitor next to the playware
soccer set-up). In all cases, players engaged immediately
in trying to get higher scores than in the other location,and cheering and shouts related to the scores of theremote location on the other continent allowed us to©ISAROB 201141

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Acknowledgement
The work was performed with the help of colleagues at
the Center for Playware, Technical University ofDenmark and Danish School of Education (AU). Baldur
Bjarnason, Niels Sandmann, Jacob Nielsen, Steffen
Andersen and Bjarke Pedersen made highly valuable
contributions to the development and Stine Ejsing-Duunto the concept discussions. Also thanks to localorganizers including A. Smith, Meraka Institute, B.
Windram, Junior Engineers for Africa, Royal Danish
Embassy in Tokyo, Brian Laudrup, Lars Høgh.

References[1] H. H. Lund, T. Klitbo, and C. Jessen. Playware Technology
for Physically Activating Play, Artificial Life and Robotics
Journal, 9:4, 165-174, 2005
[2] C. Breazeal. Designing Sociable Robots. MIT Press,
Cambridge MA, 2002.
[3] T. Fong, I. Nourbakhsh, and K. Dautenhahn, “A survey ofsocially interactive robots,” Robotics and Autonomous
Systems, vol. 42, no. 3–4, pp. 143-166, 2003.
[4] A. Billard. “Robota: clever toy and educational tool.”
Robotics and Autonomous Systems 42: 259–269, 2003.
[5] M. Resnick, F. Martin, R. Berg, R. Borovoy, V. Colella, K.
Kramer, B. Silverman “Digital manipulatives: new toys to thinkwith.” In: Proceedings of the CHI’98 conference on humanfactors in computing systems, Los Angeles, California, April
1998, pp 281–287[6] C. Jessen, and H. H. Lund “On play forces, play dynamics
and playware”. Unpublished manuscript.
[7] H. Ishiguro “Android science: Conscious and subconscious
recognition,” Connect. Sci., vol. 18, no. 4, pp. 319–332, 2006.
[8] D. Sakamoto, T. Kanda, T. Ono, H. Ishiguro, and N. Hagita,“Android as a telecommunication medium with human like
presence,” in Proc.2nd ACM/IEEE Int. Conf. Human-Robot
Interact., 2007.
[9] F. Mueller,A. Agamanolis, R. Picard “Exertion Interfaces:
Sports over a Distance for Social Bonding and Fun”, In
Proceedings of CHI 2003, ACM, 5:1, 561-568, 2003.
[10] K. Suzuki and S. Hashimoto. “FeelLight: A
Communication Device for Distant Nonverbal Exchange”,
Proceedings of ETP’04, New York, USA, ACM, 2004.
[11] H. H. Lund. “Modular Robotics for Playful
Physiotherapy,” in Proceedings of IEEE International
Conference on Rehabilitation Robotics, IEEE Press, 2009, 571575.
[12] H. H. Lund, and P. Marti. “Designing Modular Robotic
Playware.” In 18th IEEE International Symposium on Robot andHuman Interactive Communication (Ro-Man 2009), IEEE
Press, 2009, 115-121.
[13] R. C. Arkin. Behavior Based Robotics, MIT Press,
Cambridge MA, 1998.
[14] R. A. Brooks. “A robust layered control system for amobile robot.” IEEE Journal of Robotics and Automation,
2(1),1986, 14-23.

©ISAROB 201142

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

An Educational Tool for Interactive Parallel and Distributed Processing
Luigi Pagliarini1,21Henrik Hautop Lund1

Centre for Playware, Technical University of Denmark, 2800 Kgs. Lyngby, Denmark2
Academy of Fine Arts of Bari, Via Gobetti, 8. 70125 Bari, Italy
luigi@artificialia.com hhl@playware.dtu.dk
www.playware.dk

tool for this kind of algorithmics learning should allow
students to learn about when to utilise shared variables(e.g. in the master) and distributed variables, when to use
a scheduler (in the master), how to use semaphores forcritical sections, and for instance allow students toconfront the mutual exclusive problem [1]. Also, generalcomputer science learning about operating systems
demands learning about distributed systems, and theissues related to topology, communication, event based
control, prevention of deadlocks, data transfer, etc. (e.g.
[2]). Obviously, learning about artificial intelligence also
demands learning about distributed systems for learning
about
artificial
neuralnetworks,
evolutionary
computation, multi-agent systems, swarm intelligence,
etc., including also learning of artificial life and robotics(e.g. multi-robot systems).
A number of these computer science themes can appear
quite abstract to the engineering and computer science
student. There is clearly a need to have an educational
tool that allows the students to confront these themes in avery concrete manner. We suggest that the best way tolearn about these abstract issues is through direct handson problem solving, following the pedagogical principles
of Piaget [3] known as constructionism [4, 5, 6] andguided constructionism in the computer science literature[7]. We combine this with an approach of trying tocontextualise IT training for students by allowing them towork with building blocks [8]. Numerous experiments
have shown that the hands-on, problem-solving,
constructionism approach allow the learner to confront
abstract, cognitive problem solving in a simpler manner
through the physical representation. The feature thatdifferent representations (e.g. physical representation)
can cause dramatically different cognitive behaviour isreferred to as “representational determinism” [9]. In fact,
Zhang and Norman [10] propose a theoretical frameworkin which internal representations and external
representations form a "distributed representational
space" that represents the abstract structures andproperties of the task in "abstract task space" (p. 90).
They developed this framework to support rigorous andAbstract
In this paper we try to describe how the ModularInteractive Tiles System (MITS) can be a valuable toolfor introducing students to interactive parallel anddistributed processing programming. This is done byproviding an educational hands-on tool that allows achange of representation of the abstract problems related
to designing interactive parallel and distributed systems.
Indeed, MITS seems to bring a series of goals into theeducation, such as parallel programming, distributedness,
communication protocols, master dependency, software
behavioral models, adaptive interactivity, feedback,connectivity, topology, island modeling, user and multiuser interaction, which can hardly be found in other tools.
Finally, we introduce the system of modular interactive
tiles as a tool for easy, fast, and flexible hands-on
exploration of these issues, and through examples show
how to implement interactive parallel and distributed
processing with different software behavioural models
such as open loop, randomness based, rule based, userinteraction based, AI and ALife based software.

Introduction
Parallel and distributed processing has been an important
subject within computer science and artificial intelligencefor decades, and is one of the major focus points in mostcomputer science curricula and theoretical educational
textbooks. It is normally viewed as an important subject
to teach computer science and engineering students, since
numerous applications and systems are based on theprinciple of parallel and distributed processing, including
the Internet, cloud computing, parallel computers, multiagent systems, swarm intelligence, etc. There are
numerous important issues related to parallel anddistributed processing that a student has to learn about.
Within algorithmics, it is important to learn to what
extend parallelism can improve efficiency and what kindof algorithms can exploit parallelism. This leads, forinstance, to a demand for knowing about hierarchical andfunctional decomposition of problems. An educational©ISAROB 201143

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

objects and experiment a dialogue with the system in avery direct and non-mediated way, and hence it is viewed
as highly suitable e.g. for student training. Undeniably,
this allows for a direct hands-on experience and learning.

formal analysis of distributed cognitive tasks and to assist
their investigations of "representational effects [in which]different isomorphic representations of a common formal
structure can cause dramatically different cognitive
behaviours" (p. 88). “External representation are definedas the knowledge of the structure in the environment, as
physical symbols, objects, or dimensions (e.g., written
symbols, beads of abacuses, dimensions of a graph, etc.),
and as external rules, constraints, or relations embedded
in physical configurations (e.g., spatial relations ofwritten digits, visual and spatial layout of diagrams,
physical constraints in abacuses, etc.)” (p. 180) [9].
For the distributed processing education, we suggest
using interactive parallel and distributed processing thatallows the student to easily represent, interact with andcreate their own parallel and distributed processing
system in a physical manner. Here, we will divide thework into some of the sub-problems that the students will
have to confront and learn about through practicalimplementations.
These
sub-problems
includedistributedness, master dependency, software behaviouralmodels, adaptive interactivity, feedback, connectivity,
topology, island modeling, and user interaction.
Indeed, designing software for interactive parallel anddistributed systems means moving away from thetraditional routes and to face another way of developing
algorithms. This other programming paradigm demands
the programmer to get into a new "state of mind", which
is a most difficult thing to do. It is therefore important tohave a clear idea of the concepts and definitions
underlying this paradigm of interactive parallel anddistributed processing:

Parallel and Distributed
A computational process is called distributed [12] when asingle computational atom is on one side autonomousand on the other insufficient to determine the desired
outcome. Therefore a computational process will becalled distributed when two or more computers –communicating through any possible network - willcontribute to accomplish the very same task by sharing
different roles in a computational problem or process.
Besides that, whenever considering a distributed(computational) process, it is necessary to define thelevel of parallel vs. serial computational flow that thesystem should perform, as well as to define the“computational group” characteristics. The Parallel
computing is a form of computation in which many
calculations are carried out simultaneously, operating on
the principle that large problems can often be divided
into smaller ones, which are then solved concurrently ("inparallel"). There are several different forms of parallel
computing: bit-level, instruction level, data, and task
parallelism. Since the modular interactive tiles system ismostly dedicated to the task parallelism problem it tends
to run distributed processes in, at least, three different
ways:
Fully-Distributed,
Semi-DistributedorCentralized.

Modular Interactive Tiles System
Under an educational point of view what is really needed,
as well as would be a real additional value, is a tool thatallows for investigating and understanding parallel anddistributed processing meanwhile stressing the user
and/or multi-user interactivity component. One
possibility is the Modular Interactive Tiles System(MITS) may provide novel programmers with such a tool
and approach, since the system is based on robotic
modules with certain properties: Each robotic module hasa physical expression and is able to process andcommunicate with its surrounding environment. The
communication with the surrounding environment isthrough communication to neighbouring robotic modules
and/or through sensing or actuation. A modular robot isconstructed from many robotic modules.
The MITS approach inherits the behaviour-based robotics
methods [13] and exploits it with the belief thatbehaviour-based systems can include not only thecoordination of primitive behaviours in terms of control
units, but also include coordination of primitive
behaviours in terms of physical control units. We,

Interactivity
For interactivity, here we intend a physical and tangibleinteraction. The physical parallel and distributed systemenables the experience of physically manipulating objects
and the material representations of information. The
technology embeds physical, conceptual and cultural
constraints. The mapping between the physicalaffordances of the objects with the digital components(different kinds of output and feedback) is a design andtechnological challenge, since the physical properties ofthe objects serve as both representations and controls fortheir digital counterparts [11]. Here, we make the digital
information directly manipulatable, perceptible andaccessible through our senses by physically embodyingit.
While playing with the system, the user can takeadvantage of the distinct perceptual qualities of thesystem and this makes the interaction tangible,
lightweight, natural and engaging. Interacting with aphysical parallel and distributed system may mean
jumping over, pushing, assembling, touching physical©ISAROB 201144

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A force sensitive resistor (FSR) is mounted as a sensor
on the center of the raised platform underneath the cover.
This allows analogue measurement on the force exerted
on the top of the cover.
On the PCB, a 2 axis accelerometer (5G) is mounted, e.g.
to detect horizontal or vertical placement of the tile. EightRGB light emitting diodes (LED SMD 1206) are
mounted with equal spacing in between each other on acircle on the PCB, so they can light up underneath thetransparent satinice circle.

therefore, imagine a physical module being a primitive
behaviour. Thereby, the physical organisation ofprimitive behaviours will (together with the interaction
with the environment) decide the overall behaviour of thesystem. Hence, in a similar way to the control of robot
behaviours by the coordination of primitive behaviours,
we can imagine the overall behaviour of a robotic artefact
to emerge from the coordination of a number of physicalrobotic modules that each represents a primitive
behaviour, eventually opened to single/multi userinteraction.
The modular interactive tiles can attach to each other toform the overall system. The tiles are designed to beflexible and in a motivating way to provide immediate
feedback based on the users’ physical interaction, since
following design principles for modular playware [14].

Fig. 2. PCB and components of a modular interactive tile.

The modular interactive tiles are individually battery
powered and rechargeable. There is a Li-Io polymer
battery (rechargeable battery) on top of the PCB. A fully
charged modular interactive tile can run continuously for
approximately 30 hours and takes 3 hours to recharge.
The battery status of each of the individual tiles can beseen when switching on each tile and is indicated bywhite lights. When all eight lights appear the battery is
fully charged and when only one white light is lit, the tileneeds to be recharged. This is done by turning of the tiles
and plugging the intelligent charger into the DC plugnext to the on/off switch to recharge each tile.
On the PCB, there are connectors to mount an XBee
radio communication add-on PCB, including the
MaxStream XBee radio communication chip. Hence,
there are two types of tiles, those with a radio
communication chip (master tiles) and those without(slave tiles). The master tile may communicate with thegame selector box and initiates the games on the built
platform. Every platform has to have at least one mastertile if communication is needed e.g. to game selector box
or a PC.

Fig. 1. Modular tiles used for feet or hands interaction.

Each modular interactive tile has a quadratic shapemeasuring 300mm*300mm*33mm – see Fig. 1. It ismoulded in polyurethane. In the center, there is aquadratic dent of width 200mm which has a raised
circular platform of diameter 63mm in the centre. The
dent can contain the printed circuit board (PCB) and theelectronic components mounted on the PCB, including an
ATmega 1280 as the main processor in each tile. At thecenter of each of the four sides of the quadratic shape,
there is a small tube of 16mm diameter through which
infra-red (IR) signals can be emitted and received (fromneighboring tiles). On the back of a tile there are four
small magnets. The magnets on the back provide
opportunity for a tile to be mounted on a magnetic
surface (e.g. wall). Each side of a tile is made as a jigsaw
puzzle pattern to provide opportunities for the tiles toattach to each other. The jigsaw puzzle pattern ensurethat when two tiles are put together they will becomealigned, which is important for ensuring that the tubes on
the two tiles for IR communication are aligned. On one
side of the tile, there is also a small hole for a charging
plug (used for connecting a battery charger), including an
on/off switch.
There is a small groove on the top of the wall of thequadratic dent, so a cover can be mounted on top of thedent. The cover is made from two transparent satiniceplates on top of each other, with a sticker in between as
visual cover for the PCB.

©ISAROB 2011

Fig. 3. Assembly of the modular interactive tiles as a simple
jigsaw puzzle.

45

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

network level (i.e.: addresses and packages destination);transport level (i.e.: messages exchanges between clientsand master/s); session level (i.e.: defining andimplementing sessions in terms of priorities and processto-process communication); representation level (i.e.:
working on data-format differences); application level(i.e.: the end-user interaction and feedback); and tounderstand and implement solutions for robustness (i.e.:errors diagnosis and recovery); reconfiguration (i.e.:
modules assembling); unreliable communication (i.e.:
data loss, duplication and corruption); parallelism andconcurrency (i.e.: language non-deterministic sideeffects); fixed and expanding parallelism (i.e.: modifying
the number of involved processors).
It is also essential when teaching information distribution
to work on problems such as system connection (i.e.:
total vs. partial connection); token-passing (i.e.: how toshare and act on critical information); deadlock
prevention (i.e.: wait-die, wound-wait, etc.); memory
sharing (i.e.: how to locate the physical memory of the
distributed system); topology (i.e.: ordinary and complex
topology algorithms, initial vs. run-time topology
building, etc.); processes transfer (i.e.: distributing thework-load, speeding up calculation, hardware
andsoftware specialization amongst the system modules);centralized vs. hierarchy vs. distributed approaches (i.e.:
leaded or unleaded information flow); and run-time
adaptation (i.e.: adapting the system re/actions on-thefly).
Besides all of the above “classical” sub-problems ofcomputer science, our platform forces the educationalsession to face other aspects that software designers
should deal with when learning parallel and distributed
processing. Such sub-tasks include local and global
connectivity,
hardwaremultifaceted
topologies,
interactivity and adaptive interactivity, and multimodal
feedback.

With these specifications, a system composed of modular
interactive tiles is a fully distributed system, where eachtile contain processing (ATmega 1280), own energy
source (Li-Io polymer battery), sensors (FSR sensor and2-axis accelerometer), effectors (8 colour LEDs), andcommunication (IR transceivers, and possibly XBee
radio chip). In this respect, each tile is self-contained andcan run autonomously. The overall behavior of thesystem composed of such individual tiles is however a
result of the assembly and coordination of all the tiles.
The modular interactive tiles can easily be set up on thefloor or wall within one minute. The modular interactive
tiles can simply attach to each other as a jigsaw puzzle,
and there are no wires. The modular interactive tiles canregister whether they are placed horizontally or
vertically, and by themselves make the software games
behave accordingly.

Fig. 4. Physical interaction with the modular interactive tiles
placed on the ground.

Also, the modular interactive tiles can be put together ingroups (i.e.: tiles islands), and the groups of tiles may
communicate with each other wireless (radio). For
instance, a game may be running distributed on a groupof tiles on the floor and a group of tiles on the wall,
demanding the user to interact physically with both thefloor and the wall.

Theoretical Aspects of Interactive Paralleland Distributed Processing
Interactive parallel and distributed systems programming
demands the student programmer to shape specificabilities, and we believe that the MITS can simplify thislearning process. We will present a number of theinteractive parallel and distributed sub-problems that a
student needs to learn about, and we believe MITS
provides an open tool for facing all the aspects ofprogramming both low and high level programming or
front and back end representation.

Connectivity
To materialize a proper interactive parallel anddistributed platform, the modular interactive tiles system
has to implement both a local connection system through which the hardware cells communicate to theneighbourhood and propagate such information from side
to side – and a global connection device – through whichto connect with neighbour platforms and any external
tool.

Classical parallel and distributed processes subtasks
Coding parallel and distributed processes stressprogramming and understanding of different levels, suchas: physical level (i.e.: bit transmission); data link level(i.e.: packages, transmission errors and recovery);

Hardware Multifaceted Topologies
Since the modular interactive tiles system implies the useof run-time de/attachable modules, the emphasis on
hardware/software topology is quite strong and it
demands a big effort to comprehend the programming©ISAROB 201146

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

execution to that it is possible, in some cases, toeventually modify the users’ behaviour itself [16].

and dealing with such structures. In our model we wereable to identify three specific subtypes of topologies:
1. Regular, that is a one-block (i.e. any given group of
hardware cells attached in a contiguous way and sharing
a single master cell) platform with modules attached in asquared or rectangular shape;
2. Irregular, which is a one-block platform, which can bearranged in any desired shape. Nevertheless hardware
cells have to be continuous (i.e. the assembling does notreveals discontinuity and there is not any isolated cell orgroup of cells);
3. Islands Configurations, that is a platform made with
two or more one-blocks (i.e. as defined above in point 1,
and 2). It makes no difference whether master cells
communicate amongst each others, through an external
device, or do not communicate at all.

Multimodal Feedback
When talking about HMI we kind of committed
ourselves to the "how you give is more important of what
you give" motto. Therefore, in recent years we pushed
our research towards software and tools that can bothgive and get feedbacks from the user(s).
When developing software for modular interactive tiles
we constantly try to provide the user with an immediate
feedback (e.g. LED, experience report) as well a delayedor long term feedback (e.g. adaptivity, documentation
software). For the immediate feedback from modular
interactive tiles we use light (LED) configuration or
colours. In addition to that, anytime there is a need for astronger or a more complex or long-run “signal”, weinterface the modular interactive tiles with external
devices in a layered mode, where each layer of feedback
can be added/removed freely on top of each other This iswhat we call Layered Multi-modal Feedback [17]. The
external devices we use can be “passive” as visionoriented feedback (e.g. screen, projector, etc.), soundoriented feedback (e.g. loud speakers, buzzers, etc.), or“active” such as computational devices that through anexternal communication (e.g. radio and internet) run ananalysis or link the user action to specific databases.
In conclusion, to manage and teach the many features ofparallel and distributed programming we need to run on asystem, which is robust, reliable and easily
reconfigurable. This is where we believe that the MITS
can express a certain degree of efficiency, besides ofbeing ideal in shifting the level of representation from thevery abstract representation to an empiricalrepresentation. Therefore, in the following paragraph we
provide examples, which attempt to show how one canaccess the above-described aspects in a fast,comprehendible and easily generalizable way.

Interactivity
Implementing software for modular interactive tiles
implies designing, or at least dealing with a quite relevantinteractive scenario, since in most cases the use of thesoftware itself relies on the users’ physical andcontinuous action. The software designer will have todeal with completely different requirements accordingly
to single-user or multi-user targeted software. Often, thesoftware designer will also have to hypothesize a large
variety of behavioural situations, even including
situations (according to our personal experience) where asingle-user platform will be used by many users, or amulti-user software will be run by a single user.
Adaptive Interactivity
The way we approach interaction in such a modular anddistributed model leads beyond the classic idea of
human-machine interaction (HMI), and is of fundamental
importance since it prospects and applies - under both
physical and cognitive circumstances – user adaptation
and user adaptivity. First of all, our model beingarchitecturally reconfigurable – eventually run-timereconfigurable – represents by itself the essence ofadaptation. In addition, being focused on users’ physicalaction, such a system can be easily tailored to users’
activity, either in real time or in the long run. To reachsuch a goal, modular interactive tiles can be programmedusing many different strategies that also depend on thequality and quantity of feedback the software designer iswilling to exchange with the users. (Feedback andmultimodal feedback will be introduced in the next
paragraph). Indeed, in more then one case we showed[15, 16] that using modular interactive tiles we could
detect some of the users’ characteristics, and therefore
adapt the software execution to those. Last but not theleast, in further tests it has been shown that by capturing
the users’ provisory attitude and adapting the software©ISAROB 2011

Implementations Examples
As a first step the teacher/tutor should introduce students
to the hardware platform (Figures 2, 3, and 4) and ask theclass to implement all the needed protocols for obtaining
a robust, efficient and reliable parallel and distributed
system. This would require and encourage students toface the basic algorithms and protocols that the subtasksof parallel and distributed systems need (e.g.: physicallevel, data link level, network level, transport level,session level, representation level, etc.).
Once such a start-up system is obtained (from thestudents work or from the pre-made system), a second
step could be, for example, testing the system by working
on problems such as application, robustness,

47

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011predefined or random based – is governed by a specificrule or set of rules. The simplest case we can think of is
the one where, given any machine state and configuration(e.g. two tiles) those states which are ON turns OFF andthose states which are OFF turns ON. Of course, we candesign a much more complex setting but, essentially, thisis the logic that is used in rule based software.
On the other hand, when introducing the interaction
element in rule based software we obtain a more dynamic
scenario denoted by the fact that the rules and users arecoactive and contribute step by step to the system state.
Such a situation can be clearly observed in the American
Football game (Figure 7).

communication, system connection, token-passing,deadlock prevention, parallelism, reconfiguration,
memory sharing, topology, and process transferring.
The MITS model is ideal for implementing all of theabove challenges since the hardware components areminimalistic and the distributed system complexity canbe developed and tested in a quick and easy manner(Figure 5).

Fig. 5. Examples of different topologies

Once students have reached this new level ofcompetencies, the tutor can drive their attention to ahigher level of representation and ask them to implement
end-user interaction based applications, such as in thefollowing examples.
Fig. 7. American Football, a sequence of 5 states

Games Examples.
Once a specific topology is chosen, the software
engineering student can implement and run a large
variety of tasks (here we start by considering examples toapply to a semi-distributed, single user application on aregular topology platform).

This is one-against-one game where, given a, say, 5(width) per 2 (height) cluster of modular interactive tiles,
such interactive software is made so that at the beginningof the game the platform extremes appear activated (i.e.
light on) and of two different colours (i.e. blue in one
extreme and red in the other extreme). By squeezing thetiles, the user “pushes” the color/activation forward in therow (i.e. switches off the squeezed tile and switches on
the adjacent one towards the opponent). The user who
first pushes its color to the opposite extreme of the gameplatform wins the game.

Open Loop and Randomness based software.
The simplest case, a naïve one, could be the following
Easy Game (Figure 6).

User-interaction based software.
The user-interaction based program is, per se, aninteractive software conception in which the user directly
contributes to the next machine state (i.e. tiles color or
activation). Such a software model is quite similar to theinteractive version of the rule based software – since theuser itself cannot determine the machine states if notaided by some underlying algorithm. It only differs fromthat in terms of strain used on increasing the user role andcontribution to the next machine state, and the attempt toreduce the rule component. A good example could be the
Final Countdown game (Figure 8). In the Final
Countdown the tiles platform can vary both in aspect andsize, since the game components behave all in the verysame way. It consists of a number of tiles that, when thegame is initiated, all of the tiles are fully lighted (i.e. anycolor would do). After initialization and with a given
interval (e.g. one second) they all start to “fade-out”
switching OFF one of their 8 light bulbs after the other ina clockwise sequence. If one of them gets completely

Fig 6. Easy Game, a sequence of 7 states

In Easy Game the light is “passed” from one module toeither an adjacent or a distant one (i.e. with a predefined
open loop algorithm or randomness based one). In boththe above cases the software cycling is endless and weneed to introduce the interactivity level (e.g. the gamefinishes when the user hit the lighted tile) to stop it, andby doing so transforming the two into very young
children games. When the user press a tile, then the
dynamics somehow stops and the tiles freeze in aparticular pattern, until the user presses the lighted tile
again, and the light shift sequence will start again.
Rule(s) based software.
One step further is a rule-based software characterized by
the fact that pattern sequence - which can either be©ISAROB 201148

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

OFF the game is over. To restore a single tile to theinitial state, the user has to squeeze it. The wider is theplatform the more important becomes the strategy users
bring into play to keep the game alive.

Acknowledgements
The authors wish to thank the Centre for Playware
colleagues for discussions of the concept andimplementations.

References[1] Harel, D. Algorithmics, Addison-Wesley, 1987
[2] Silberschatz, A., Peterson, J., Galvin, P. Operating System
Concepts, Addison-Wesley, 1991
[3] Piaget, J. and Inhelder, B. La psychologie de L'enfant. Paris,
P.U.F, 1966.
[4] Papert, S. Mindstorms: Children, Computers, and Powerful
Ideas. Basic Books, New York, 1980.
[5] Papert, S. Constructionism: A New Opportunity for
Elementary Science Education. A proposal to the National
Science Foundation, 1986.
[6] Martin, F. “Ideal and Real Systems: A Study of Notions ofControl in Undergraduates Who Design Robots”. In Y. Kafai
and M. Resnick (Eds.), Constructionism in Practice: Rethinking
the Roles of Technology in Learning, MIT Press, MA, 1994.
[7] Lund, H. H. “Robot Soccer in Education”. Advanced
Robotics Journal, 13:8, 737-752, 1999.
[8] Lund, H. H., and Sutinen, E. “Contextualised ICT4D: a
Bottom-Up Approach”, Proceedings of 10th International
Conference on Applied Computer Science, WSEAS, Japan,
2010.
[9] Zhang, J. The Nature of external Representations in
Problem Solving. Cognitive Science 21:2, 179-217, 1997.
[10] Zhang, J., Norman, D.A. Representations in Distributed
Cognitive Tasks. Cognitive Science 18: 87-122, 1994.
[11] Ishii, H. Ullmer, B. Tangible Bits: Towards Seamless
Interfaces between People, Bits and Atoms. In Proceedings of
CHI: Human Factors in Computing Systems. pp. 234–41, 1997.
[12] Rumelhart, D., McClelland, J., et al., 1986, Parallel
Distributed Processing, vol. I, Cambridge, Mass.: MIT Press.
[13] Brooks, R. “A robust layered control system for a mobile
robot”. IEEE Journal of Robotics and Automation, 2(1):14--23,
1986.
[14] Lund, H. H. and Marti, P. "Designing modular robotic
playware," the IEEE Int. Workshop Robots Human Interactive
Commun Toyama, Japan. Sep. 27-Oct. 2., IEEE Press, 2009.
[15] Hammer, F. Derakhshan, A., Hammer, F., and Lund, H.
H. “Adapting Playgrounds for Children's Play Using Ambient
Playware”. In Proceedings of IEEE Intelligent Robots andSystems (IROS’06), IEEE Press, Hong Kong, 2006.
[16] Thorsteinsson, T. and Lund, H. H. “Adaptive Modular
Playware”. (in press).
[17] Lund, H. H. and Thorsteinsson, T. “Social Playware formediating tele-play interaction over distance” to appear inProceedings of 16th International Symposium on Artificial Lifeand Robotics, ISAROB, Japan, Jan. 2011.
[18] Neumann, J. v. “The general and logical theory of
automata,” in L.A. Jeffress, ed., Cerebral Mechanisms in
Behavior – The Hixon Symposium, John Wiley & Sons, New
York, 1951, pp. 1-31.

Fig. 8. Final Countdown, a sequence of 6 states

A.I. and ALife based software.
The A.I. and ALife based software are, again, acomplication of what we defined as rule based systems.
Essentially they rely on the same principles, both for theautonomous and the interactive version, although thequality of the computational experience is much higher interms of software behavioral equality/variety,un/predictability, and etc. Further, since modular
interactive tiles tend to resemble pixel-made structures it
seems to easily incorporate a consistent number ofclassical and modern A.I. paradigms. A good example is
the Cellular Automata (i.e. CA), a discrete model used incomputability theory and many different fields, which
consists of a regular grid of cells, each one with a finite
number of possible states (e.g. ON, OFF), that canchange their state accordingly with their neighborhood
activation states [18]. We, first, implemented one of themost famous CA algorithms, the Conway’s Game of Lifeon modular interactive tiles and, after that, added theinteractive aspect.

Conclusion
We developed the concept of interactive parallel anddistributed processing in order to put focus on thephysical interaction with parallel and distributed system,and to highlight the many challenges that the student
programmers might face in understanding and designinginteractive parallel and distributed systems.
It is our belief that a system like the modular interactive
tiles is a tool for easy, fast, and flexible learning andexploration of these challenges, e.g. as shown with theexamples of how to implement interactive parallel anddistributed processing with different software behavioral
models such as open loop, randomness based, rule based,
user interaction based, AI and ALife based software.
Indeed, MITS provides an educational hands-on tool thatallows a change of representation of the abstract
problems related to designing interactive parallel anddistributed systems, so that students can learn aboutclassical and modern aspects of parallel and distributed
systems.

©ISAROB 201149

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Analysis of speech signal based on frequency
distribution and intonation
Naoya OHTA and Akio NOZAWA
Meisei University, 2-1-1 Hodokubo, Hino, Tokyo, JAPAN(Tel : +81-02-591-5895; Fax : +81-42-591-5895)(akio@ee.meisei-u.ac.jp)

Abstract: In this study, the atmosphere as a communication mode of human was defined by time information of speech.
The objective of this study was the development of quantitative measurement technique of the atmosphere as KANSE
I in a group. The communication mode of the group was indexed by using a time frequency distribution analysis on con
versation voice. In particular, a speech duration time was focused. And the correlation between the time series of the sp
eech duration time and a communication modality was evaluated. Consequently, the difference of the communication mode between man and woman was seen in the speech duration time. And a difference of the characteristics of speech wa
s found in the duration time and the frequency.
Keywords: atmosphere, communication, speech duration time, frequency

aspects of understandability, an effect of learning, an
ambition and an attitude of teacher, which are evaluatedby students. Because of these factors, objective
evaluation of the class becomes increasingly important.
The activity of the class was evaluated as atmosphere.
Understanding of refining of student evaluation system
is considered become one of indicators.
In recent years, the various applications that was
introduced the sound recognition technology wasdeveloped[4]. Speech dialogue system as target to
information guidance was practical use. Many of those
speech dialogue is remain handle only language
information contained in speech. Therefore, uniform
response is shown against any opponent. Dialogue
between humans is and non speech information
obtained from ears and eyes. There are also non-verbal
information contained in speech. They are born deep
interaction by use together. That is similar about
recognition of communication mode including
atmosphere. Factor to form an atmosphere wasinterpersonal relationship and emotional relationshipand power relationship and interests relationship.
Atmosphere is difficult captured properly by machine.
In addition, atmosphere is difficult captured properly byhuman with no prior knowledge. But human is cancatch sensuously an atmosphere. Because it is interpret
atmosphere as communication mode of group based atnonverbal information of facial expression, tone,gesture not only explicit language information incontent. Interpret of nonverbal information by machineis studied extensively by area of human-machine system.

I. INTRODUCTION
The atmosphere of field was unexpressed explicitly
in language of emotional relationship in situation ofinterpersonal relationship and social group, power
relationship, and interest. Atmosphere was idiomatic
phrase of Japanese which indicating many elements ofrelationship without expression. Awaking atmosphere is
implicit. This ability is called social intelligence bypsychology [1]. Reading atmosphere is cognition ofcommunication mode. In particular, subtlety incommunication is technique which learned ininterpersonal psychology. That is, the skill of humanrelations is not innate target in interpersonal psychology.
In recent years, student evaluation system has been
introduced into many schools [2]. The objective of thesystem is to find the problem of the class in order torefine the class. However, the questionnaire was writtenby the student whose grade was in the hands of teachers[3]. The evaluation by those students may tend to berather high. Thus, the evaluation doesn't have muchvalue for the objective. The introduction of the system
in school is seen as just the evidence of an enthusiasm
for refining the class. That is, an alibi. A basis of theevaluation has variety. The relationship between theeffect of learning and the satisfaction for the classdoesn't always has a correlation. Sometimes, they could
have an inverse relationship. Because teachers tend tohave the priority to the satisfaction of students andignore the effect of learning. This is like putting the cart
before the horse. The student evaluation system has the©ISAROB 201150

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table1. Stats of speech duration time

Study example as object for communication mode ofgroup is little. In addition interpret of verbal information
concerning communication mode is become premise tointerpretation of context whole of communication.
Therefore complexity isn't avoided. This study,
atmosphere as communication mode of between
humans was defined by time information of speech.
Purpose of this study is develop quantitative
measurement technique of atmosphere as sensitivity ofgroup. As result of this study is improvement of student
evaluation system. Quantitative evaluation of theatmosphere is considered become one of evaluation
indicator concerning communication mode as group. In
addition, by communication mode visualization andquantify is function as one of indicator grasp familiarity
density in talks. This is brings the efficiency of
information transmission. This study was focused inrelationship of speech duration time and speechfrequency. This study is analysis of speech signal basedon frequency distribution and intonation.

Fig.1. time series of speech duration time (News(M))

Fig.2. time series of speech duration time (News(F))

III. ANALYSIS PROCEDURE
Audio Source sample used at analysis was News(M)of 10kinds. And News(F) of 7 kinds. And Intr(F-M) of 6
kinds. And Intr(F-F) of 8 kinds. Audio source sample ofnews recitation is 17kinds in total. Audio source sample
of interview is 14 kinds in total. Absolute value
processing was administered for recorded voice signal.
And get at envelope of peak value. Speech duration
time and the corresponding speech time was recorded attime series. Voice signal per one sound source was 300
seconds. 10 seconds of first is silence part forenvironmental noise level evaluation. Extract of speech
is performed as standard of environmental noise level.
Threshold was about 3 times of the average
environmental noise level. And tried visualization ofcommunication mode by conversation sound analysis.
This study was focused in speech duration time. The
conversation breaks were detected as standards silencepart of constant time. This study conversation break was
regarded silence part more than 0.2 second.

II. EXPERIMENT METHODOLOGY
In this study, conversation sound was used as
analysis object. Recorded audio signal at PC was
analyzed as digital data. Audio signal was converted by
PCM method to A/D. Audio signal was recorded in fileat WAVE form. sampling frequency was 22.050kHz.
Recorded file was quantized at 16bit. In addition, thenumber of channels was one (monaural). Speech soundof record object was not included acoustic of soundeffect except speech. The conversation to become object
is idealized the speech of professionals trained
concerning speech. But securement of subjects wasdifficult. So, audio source sample was acquired
equivalent audio source from internet. Audio source
sample is read sound of male announcer in radio news
(News(M)) of internet delivery. And, read sound offemale announcer in radio news (News(F)) of internet
delivery. And, interview from female to male (Intr(FM)) in internet delivery program. And, interview fromfemale to female (Intr(F-F))in internet delivery program.

©ISAROB 2011

IV. RESULT AND DISCUSSION
Table.1 is result of the stats concerning speech
duration time. Fig.1 is result of time series of speech
duration time (News(M)). Fig.2 is result of time seriesof speech duration time (News(F)). Fig.3 is result oftime series of speech duration time (Intr(F-M)). Fig.4 is51

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig.3. time series of speech duration time(Intr(F-M))
Fig.5. Time frequency distribution of speechof news recitation

Fig.4. time series of speech duration time(Intr(F-M))
result of time series of speech duration time (Intr(F-F)).
Fig.5, 6 is result of Time frequency distribution of
speech of news recitation and interview. Fig.7, 8, 9, 10
is result of each frequency analysis. From result of table
1, number of speech of men of the news recitation is130.82 times. Number of speech of female of the news
recitation is 152.14 times. Number of speech of female
is lot of than men. Summation of time series of speech
duration time is 190.96 second by news recitation of
male. Summation of time series of speech duration time
is 195.41 second by news recitation of female. Averageof speech duration time is 1.47 second by news
recitation of male. Average of speech duration time is1.34 second by news recitation of female. Maximum ofspeech duration time is 5.03 second by news recitationof male. Maximum of speech duration time is 4.70
second by news recitation of female. Average of volumeof speech duration time is 131.08 db by news recitationof male. Average of volume of speech duration time is98.45 db by news recitation of female. Female is seen
tend extension of speech duration time than men. For
result of male of news recitation of time series of fig.1,
amplitude is seen fit into within constant. In addition,
male of news recitation of fig.1 is bigger than amplitude
female. For result of female of news recitation of timeseries of fig.2, amplitude is seen fit into within constant.
Amplitude of female of fig.2 is seen to smaller thanamplitude of male of fig.1 Summation of speech
duration time concerning news recitation wasn't seen
both male and female significant difference. Both

©ISAROB 2011

Fig.6. Time frequency distribution of speechof interview
difference of news recitation is showing at distribute ofspeech duration time. This report, news contents is notcontrol. Therefore, difference between male and female
in distribute of speech duration time is cannot
disaffirmance possible of factor of contents. Result ofnews recitation of both male and female is seen constant
amplitude within the range. Speech of news recitation isspoken by constant rhythm easy to hear. So, amplitude
is considered become constant. Number of speech ofinterview of female for male is 5.79 times. Number ofspeech of interview of female for female is 6.01 times.
Summation of time series of speech duration time ofinterview of female for male is 177.79 second.
Summation of time series of speech duration time ofinterview of female for female is 174.66 second.
Average of speech duration time of interview of female
for male is 0.76 second. Average of speech duration
time of interview of female for female is 0.80 second.
Max of speech duration time of interview of female formale is 5.79 second. Max of speech duration time ofinterview of female for female is 6.01 second. Averageof volume of speech duration time of interview offemale for male is 64.33 db. Average of volume ofspeech duration time of interview of female for female
is 66.79 db. Number of speech and max time of52

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig.7. frequency analysis (News(M))

Fig.9. frequency analysis (Intr(F-M))

Fig.8. frequency analysis (News(F))

Fig.10. frequency analysis (Intr(F-F))
acknowledged. In addition, speech frequency andspeech duration time was ascertained possible indicate afractal structure. Fractal dimension of news recitation of
speech as analysis target is constant. News recitation isthinking became constant fractal dimension by spoke
trained announcer. Fractal dimension in interview ismultiplicity. Therefore Fractal dimension in interview isconsidered possible to be one of the indicators to
capture the communication mode.

interview is greater than the news recitation. But
summation and average of speech duration time issmaller than the news recitation. Principal speech of
conversation of interview is 2 man. And conversation ofinterview is accrual communication. And conversationis hard to break. And speech duration time is inferred
extending. Accretion of principal speech is not bringaccretion of speech time. Information concerning
speech duration time as per the communication mode isindicating difference arise. From result of time
frequency distribution of fig.5, 6, speech of short time isincreased frequency of occurrence. Conversely longspeech of duration time is low frequency of occurrence.
Time frequency distribution of speech is inferred takeon fractal structure. From result of frequency analysis of
fig.7, 8, 9, 10, power spectrum density is inverse
proportion f of frequency. And this is become
relationship in 1/f fluctuation.

REFERENCES[1]John D. Mayey, Peter Salovey, David R.Caruso :"Emotional Inteligence : Thaeory, Finding and
Implications," Psychological Inquiry, Vol15, No3,
pp.197-205(2004)[2]Yumiko M, Kensaku S, Toshio O, Akihumi J, Akiko I,
The Enforcement and the Tabulation of the Class
Evaluation
Questionnaire(inJapanese).
Tokyo
Management College, Vol.17, pp.47-58(2009)[3]Yoshiaki H, Anonymous course evaluation and final
evaluation of each class registered (in Japanese).
International financial and administrative Vol.23, pp.4758(2009)[4]Yosuke A, Kiyoshi S, Atsushi K, A Study for a
Scalability Evaluation Model of Spoken Dialogue
System(in Japanese), Information Processing Society ofJapan, Vol. 46, No.9, pp.2269-2278(2005)

V. CONCLUSION
In this study, focusing at non-verbal part of speech.
Object is to visualize communication mode from
implicit called atmosphere. Conversation sound asanalysis target was applied time frequency distribution.
In particular, focused at speech duration time. And
evaluate is concerning of time series of the speech
duration time and the communication mode. As result,difference of speech mode concerning to sex was©ISAROB 201153

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Color Influences on Human Being
Evaluated with Nasal Skin Temperature
Tota MIZUNO, Naoki NAKATEGAWA and Yuichiro KUME
Tokyo Polytechnic University, 1583 Iiyama, Atsugi, Kanagawa 243-0297 Japan(Tel : +81-46-242-9508; Fax : +81-46-242-9508)(mizuno@mega.t-kougei.ac.jp)

Abstract: Since colors affect human conditions physiologically and mentally, various studies about the color influences
have been conducted. Most of the studies have been focused on psychological aspects. In this study, we investigate thecolor influences physiologically by use of infrared thermography. With this technique, subjects wear no devices, andobjective values are expected to be obtained to evaluate the color influences. Nasal skin temperatures were measured
by infrared thermography under yellow, red and blue illuminations. As a result, significant temperature change wasobserved under red illumination.
Keywords: color, nasal skin temperature, visual analog scale, infrared thermography

Nasal skin temperature can be measured by infrared
thermography, which is featured with non-contact.
When physiological and mental influences by color stimulation are measured, the non-contact device has anadvantage that the subjects do not wear any sensors.
Previous color influence studies have been conductedwith brain wave or heartbeat measurements [1]. No
studies about nasal skin temperature to evaluate color
influences have been reported.
In this study, we investigated the color influences
psychologically and physiologically by use of infrared
thermography. The purpose of this study is to clarify
nasal temperature changes in terms of color illuminations, and the relationship between psychological evaluation and nasal temperature.

I. INTRODUCTION
Since colors affect human conditions physiologically and mentally, various studies about the color influences have been conducted [1,2]. For an example, longwavelength lights, such as red or yellow, excite humanbeings. These colors raise the awakening level or thegrip strength. On the contrary, the short wavelength
lights, such as blue or green, are the sedation colors.
While a number of studies have been supporting thesepoints, a lot of negative data have also been reported.
The conclusion has not been clarified yet [3].
In our previous studies, mental work-loads (MWL)
have been examined with physiological and psychological measurements under arithmetic calculation tasks.
It was shown that changes of nasal skin temperatureare remarkable as a physiological index [4,5]. In generally, nervous systems about brain waves or the heartbeats respond quickly to change the tasks, but this reaction does not continue for long period. Living bodies
tend to be adapted to tasks quickly. On the other hand,the nasal skin temperature changes more slowly alongwith the task. In nasal skin, there are many arteriovenous anastomoses (AVAs) which are sympathetic nervous
systems. Nasal skin temperature changes slowly withincrease and decrease of the bloodstream with the antagonism of a sympathetic and the parasympathetic nerve.
This reaction continues for long period to stimulations
which affect an autonomic nervous system. From thisviewpoint, nasal skin temperature is more suitable forthe evaluation for a long period for mental stimulations
about MWL [4,5].

©ISAROB 2011

II. EXPERIMENTThe relationship of physiological and psychologicalindices was investigated, when environmental color(red, blue, yellow) was changed.
1. An evaluation index
Visual Analogue Scale (VAS) was used as a psychological index, while nasal skin temperature was used asa physiological index. Figure 1 shows an example of
VAS. Subjective senses and feelings can be measured
by marking a position on 10cm long scale characterized
by a pair of opposite words or phrases at the both ends.
This method is featured with little individual differencesin the understanding of the description, and shorter timeto perform measurements. In this study, seven pairs ofwords for VASs were employed.

54

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig.1. Visual Analogue Scale

Fig.3. Experimental environment

Fig.2. Nasal skin temperature
Fig.4. Experimental protocol

There were “Unpleasant-Pleasant”, “Distracted-Concentrate”, “Unhappy-Happy”, “Fatigue-Vigor”, “Cold-Hot”,“Dark-Light” and “Sleepy-Awake”. For the evaluation,
we used the VAS value differences between before andafter color illuminations.
Figure 2 shows a change of nasal skin temperature.
As the physiological index, we used temperature differences between forehead and nasal skin. Nasal skin temperatures are well affected by the activities of autonomous nerves. In addition, forehead skin temperatures arelittle affected by the activities of autonomous nerves.
Skin temperatures are affected by ambient conditions.
The ambient influences can be reduced by subtracting
forehead skin temperature from nasal skin temperature.
Therefore, relationship of sympathetic system and theparasympathetic system can be measured indirectly withtime changes of temperature differences between forehead and nasal skins. (Afterward, difference of forehead
and nasal skin temperature is called NST.)In previous studies, the time change of NST were
employed for various evaluations [4-8]. As an example,
a time changes of NST is shown in Figure 2. In this experiment, NST is denoted by T(t), and t is the time after
starting color illumination.
Ptime is the time when NST reaches the maximum
between starting color illumination and the time when
NST is less than NST average in Rest1. Ptime is thetime which is from starting color stimulation to appearance of physiological human changes. We defined maximum temperature displacement by ∆Tmax (∆Tmax =T(tave)-T(t1)). This ∆Tmax is a total variation which is
influence by the color illumination. In addition, we defined the maximum of the greatest temperature decreaseby ∆Tdmax. (∆Tdmax = max (T(t)-T(t+60s))).

©ISAROB 2011

This ∆Tdmax is the maximum value which is affectedby the color illumination for minutes. In this experiment,
we used these Ptime, ∆Tmax and ∆Tdmax as evaluation
measures. In addition, NST data were statistically analyses to study individual differences. NSTs were normalized as follows: Dispersion=1 and Average=0.
2. Experiment method
The experimental system is shown in Figure 3. A
subject sat in front of a gray booth. In this booth, ambient colors were changed by LEE Filters (Dfilter (yellow (104), red (166) and blue (161))) and fluorescentlamps (Toshiba (FLD6500)) on the ceiling of the booth.
Horizontal illuminations of all colors were 500 ± 10%(lx) in the booth. Infrared thermography device(NEC/Avio TVS-200EX) was placed at 1m horizontal
distance from nose of the subject. The thermal image
size was 320 × 240[pixel], and the temperature resolving power was 0.08 degree centigrade. Skin emissivity
was 0.98, and sampling period was 1sec.
Figure 4 shows the experimental protocol. A subject
took rest (REST1) for three minutes, sitting and opening
the eyes. After this rest, color illumination started andcontinued for five minutes. After color illumination,
another rest (REST2) for three minutes was taken in thebooth. The booth inside was illuminated by white lightduring the rests. VAS was carried out under the color
illumination in first and last one minute. This protocol
was performed with each color three times. The subjects
were normal male adults aged from 21 to 24. This experiment was based on Helsinki Declaration (1964) for
ethical consideration.

55

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table.1. Results of VAS (VAS Value (SD))
VAS

Yellow

Red

Blue

Unpleasant-Pleasant0.4(1.1) -0.1(1.4) 0.1(2.0)

Distracted-Concentrate
Unhappy-Happy
Fatigue- Vigor
Cold-Hot
Dark-Light
Sleepy-Awake-0.1(1.5) -1.1(1.6) 0.7(1.5)
0.1(1.3) -0.4(1.3) 0.7(1.4)
0.5(2.0) -0.7(1.0) 0.1(1.7)
0.4(1.9) 0.5(1.8) 0.0(1.1)
0.5(1.6) -0.2(2.0) 0.2(2.3)-0.1(1.8) -1.5(2.7) 0.9(1.5)
Fig.5. NST in yellow illumination

3. Result and Discussion
Table 1 shows averages and standard deviations of
VAS values of all subjects. Most of the VAS values
were positive under blue and yellow illuminations, and
6 of the 7 VAS values were negative under red illumination. An analysis of variance and multiple comparisons(Bonferroni method) were performed statistically to test
significant difference in each item of VAS values by thecolors. In two VAS values of “Distracted-Concentrate”
and “Sleepy-Awake”, there was a significant differenceswere shown in red and blue (p<0.05). From this result, it
is understood that colors affect human conditions mentally. In particular, Red affects the VAS values of negative side among the three colors.
Figure 5-7 show average of NSTs of all subjects.
The NSTs were maintained or increased for Ptime after
starting color illuminations, and decreased after Ptime.
Each NST was slightly different in the change by a color.
The NST changed after starting each color illumination.
The t-tests of statistical analysis were performed between temperatures of the starting and temperatures ofthe ending at the tasks under all color illuminations. As
a result, in the both temperatures, there was a significant
difference (p<0.001). From this viewpoint, it was shownthat color illuminations affect human conditions physiologically.
Table 2 shows averages and standard deviations of
Ptime,ΔTmax and ΔTdmax. Red was the most effective color physiologically, since Ptime was short, Δ
Tmax and ΔTdmax for red were the biggest of threecolors. Although, NSTs of several subjects obviously
were affected for color illuminations, the statistical significant differences were not observed between NST
and colors. It is thought that there were no significant
difference for state of subjects about the physical condition, mental condition and the taste color illuminations.
Table 3 shows the results of correlations between©ISAROB 2011

Fig.6. NST in red illumination

Fig.7. NST in blue illumination
Table.2. Results of Ptime, ΔTmax, ΔTdmax
ColorsYellow
Red
Blue
Ptime (s) (SD) 104.2(84.0) 76.1(62.8) 82.5(103.4)
ΔTmax (SD)ΔTdmax (SD)1.8(1.1)
1.2(0.5)

2.4(0.8)
1.5(0.7)

2.0(1.1)
1.1(0.3)

VAS value and NSTs (Ptime, Δ Tmax, Δ Tdmax).
There were intermediate and strong correlations (fourVAS values for yellow, eleven VAS values for red andsix VAS values for blue). As a result, red is most influential color physiologically and psychologically. Particularly, high correlation coefficient 0.82 was observed
between “Unpleasant-Pleasant” and Δ Tmax for red.
From this viewpoint, the NST decreases when red illumination is felt unpleasant.

56

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table.3. VAS vs NST(Ptime, ΔTmax and ΔTdmax)
ColorsYellow

Red

VAS

Ptime

ΔTmax ΔTdmax

Ptime

Unpleasant-Pleasant

0.26

-0.50

0.25

0.14

0.82

Distracted-Concentrate
Unhappy-Happy
Fatigue- Vigor
Cold-Hot
Dark-Light
Sleepy-Awake

-0.02
0.34
0.25
0.27
0.34
0.21

-0.35
-0.18

-0.33
0.38
-0.24
0.25
-0.17
-0.19

-0.66
-0.41
-0.68

-0.72
-0.33

-0.76
-0.74

-0.07
0.04

-0.70

ΔTmax ΔTdmax

Ptime

ΔTmax ΔTdmax

0.52

-0.29

-0.02
0.32

-0.16
-0.06

0.15
-0.03

-0.44
-0.41
-0.48

-0.54
-0.59

-0.51

-0.44
-0.49

-0.11
0.00

0.48
-0.50

0.03

-0.35

0.41

infrared thermography total/physiological conditions.

However, there were correlations between “UnpleasantPleasant” andΔTmax in yellow and blue illuminations.
On the contrary, positive correlations were shown in red.
This result cannot be explained by the conventional
hypothesis that NST decreases with unpleasant feeling.
This result can be explained as follows: NST decreases “fast” down with unpleasant feeling. Red is the mostinfluential physiologically and psychologically at threecolors and there was a negative intermediate correlation
between “Unpleasant-Pleasant” andΔTdmax of red.
In previous studies, the stimulations by tasks or
loads to subjects were calculation problem [4,5], metal
scratch noise [7] and driving task[8]. The reaction of
NST for these stimulations decreased fast immediately
after starting the stimulation. However, NST decreased
after Ptime in this experiment. In comparison between
the results of previous studies and this experiment, thecolor illumination might be less influential to humanconditions physiologically and mentally than the stimulations previously used.

-0.32
0.00
0.01
-0.14

measure

-0.16
-0.23
-0.39
0.01
0.38
-0.07
-0.23

human

men-

REFERENCES[1] H.kubo, Y.Inoue, The Influence of Lighting by Used
Chromatic Light on Physiological and Psychological
Responses, Illum, Engng, Onst, Jpn, Vol.92, No.9,
pp.645-649, 2008[2] Y.Kaku, K.Momose, M.Saito, Physiological effects
and psychological effects of color light, Color
Science Association of Japan, Vol.31 SUPPLIMENT, pp20-21, 2007
[3] T.Oyama, M.Saito, An introduction to Color
Theory: “Kansei” and the Psychology of Color,
University of Tokyo Press, 2009[4] K.Hioki, A.Nozawa, T.Mizuno, and H.Ide, Physiological Evaluation of Mental Workload in Time
Pressure, T.IEEJ Trans.EIS, 127(7), pp.1000-1006,
2007.
[5] T.Mizuno, S.Nomura, A.Nozawa, H.Asano and
H.Ide, Evaluation of the Effect of Intermittent Mental Work-Load by Nasal Skin Temperature, IEICE,
J93-D(4), 535-543, 2010.
[6] Advanced Industrial Science and Technology Institute for Human Science and Biomedical Engineering, Handbook of Human measurement, Asakura publisher, 2003.
[7] H.Zenju, A.Nozawa, H.Tanaka, H.Ide, Estimation of
Unpleasant and Pleasant States by Nasal Thermogram, IEEJ Trans. EIS, Vol.124, No.1, pp.213-214(2004)[8] R.Sakamoto, A.Nozawa, H.Tanaka, T.Mizuno, andH.Ide, Evaluation of the Driver’s Temporary Arousal Level by Facial Skin Thermogram –Effect of Surrounding Temperature and Wind on the Thermogram-, IEEJ Trans.EIS, 126(7), 804-809, 2006.

III. CONCLUTION
In this study, we investigated the color influences
physiologically by use of nasal skin temperature measured by infrared thermography under yellow, red andblue illuminations. As a result, it was suggested that themental and physical conditions are affected by the colors difference. Significant temperature change was observed under red illumination.
It is necessary to examine color influences mental
and physiological with more subjects, more the VAS
items and more color illuminations.
In order to clarify the relationship between NST andhuman mental/physiological conditions, further studies
are required. It is expected to develop a novel system by©ISAROB 2011

Blue

57

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Air speed control of airship-type fish robot
Kunihiko Sato* and Masafumi Uchida***The Graduate School of the University of Electro-Communications**The Department of Electronic Engineering, The University of Electro-Communications
1-5-1, Chofugaoka, Chofu, Tokyo, Japan(Tel: +81-42-443-5173; Fax: +81-42-443-5173)(Email: sato@zidane.ee.uec.ac.jp)

Abstract: This paper deals with progress in the motion performance and control of a Balloon Fish Robot (BFR). The
BFR is a fish-type airship robot that derives its propulsion from vibrations of two joints. The BFR is moved in threedimensional space by an actuator. We consider the thrust of the BFR and measured it with a force sensor. The purposeof this research is to construct an equation of motion for the BFR and to control its air speed.
Keywords: Airship Robot, Fish Robot

I. INTRODUCTION
Recently, there has been progress in research on
airship robots that float in the air. An airship robot has
the advantage of being able to move in three dimensions
without being influenced by obstacles on the ground andwithout disturbing people's work. Moreover, because anairship uses a gas that is lighter than air to obtainbuoyancy, the energy necessary for floatage is less thanthat of other types of flying robots, such as airplane andhelicopter robots. Airship robots have been studies forapplications in disaster relief and indoor patrol andsurveillance systems [1].
On the other hand, there have been advances in
research on the movement of fish, which have superior
propulsive performance in water. There has been somework on applying fish-type underwater robots tooffshore surveying and underwater surveys of natural
resources [2][3]. In addition, a robot that employs the
propulsion principle of aquatic organisms in the air, notwater, has been developed. An example is the mantatype flying robot (Air Ray) developed by FESTO Co.
developed.
We developed an airship robot based on a fish's
propulsion principle, called the Balloon Fish Robot(BFR). The purpose of our research is to construct anequation of motion for the BFR and to control its air
speed.

Fig.1. Structure of the BFR

Fig.2. Appearance of the BFRof three parts: the head, the body, and the caudal fin. The
head and the body are formed of aluminum film balloons
filled with helium, giving the BFR a buoyancy of about
300 grams.
The BFR has two joints. One is a servo motor-driven
joint between the head and the body, and the other is aspring joint between the body and the caudal fin. The
servo motor moves symmetrically by various angles andat various frequencies and the BFR vibrates smoothly toobtain thrust. This movement is generally referred to as a
fish's wriggling motion [4].

II. Structure and equation of motion of BFR
1. Structure of the BFR
Fig. 1 shows the structure of the BFR, and Fig. 2
shows a photograph of the BFR. The BFR is composed©ISAROB 201158

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig.4. Thrust measurement systemdifference in the position of the rotation axis in the realmachine and the thrust measurement experiment.
Fig.3. Real machine experimental system
1. Real machine measurements
We measured the propulsion of the BFR
experimentally using an actual robot. Fig. 3 shows thereal machine measurement system. Four tracking points
were set on the BFR, and images were captured as the
BFR advanced. The movement of the BFR was
measured as two-dimensional coordinates.

2. Equation of motion of the BFR
Zhang constructed an equation of motion for anairship under the following three conditions [5]:1. The airship is a rigid body.
2. There is no rotary motion about the three axes.
3. There is no air flow.
The propulsion direction of the BFR is taken as the xaxis, the horizontal direction is taken as the y axis, andthe vertical direction is taken as the z axis. Since we deal
with propulsion of the BFR, it is assumed that there is no
movement along the z axis. Because the wriggling
motion is a symmetric motion, the y-axis components ofthe thrust cancel each other out. The x-axis componentof the thrust determines the propulsion of the BFR. The
equation of motion in the x-axis:m

2. Thrust measurement
In this study, the thrust of the BFR was measured
with a force sensor. Fig. 4 shows the thrust measurement
system. The force sensor, the body, and the caudal fin of
the BFR were fixed to a wood post (prop), and the force
generated by the wriggling motion was measured withthe force sensor. Similarly, the movement of the head,
which vibrates along with the wriggling motion, wasalso measured.
3. Thrust correction
There is a difference in the position of the rotation
axis in the real machine and in the thrust measurement
experiment, as illustrated in Fig. 5. In the thrustmeasurement experiment, the rotation axis determines
the position of the servo motor. In this case, the length of
the BFR that contributes to propulsion is the entire
length of the BFR. The length of the head is defined as
L1, and the length of the body is defined as L2.

1d2dx   CS x  F cos  , (1)
2
2
dt
dt

where m is the mass of the BFR, F is thrust,  is theangle of the wriggling motion,  is the density of air, C
is the air resistance coefficient, and S is the projectedarea of the BFR. The thrust F is measured to construct
the equation of motion of the BFR.

The rotation axis moves in the actual machine, as shownin Fig. 5. Therefore, the length of the BFR that
contributes to propulsion is shorter in both the head andthe body. There is a difference in the thrust produced in
the real machine and the thrust measurement experiment.
In this case, the length of the head that contributes topropulsion is defined as nL1, and that of the body isdefined as mL2 (0 < n< 1, 0 < m < 1). The numericalvalues of n and m are the coordinates of the rotation axis
in the head and the body where the wriggling motion of
the BFR occurs. The two points are the points that exist
between two tracking points of the head and the body inthe real machine experiment, where the distance moved
is minimized.

III. MEASUREMENT SYSTEM
We focused on the thrust of the BFR and measured itwith a force sensor. We considered two parameters of the
BFR’s wriggling motion: the oscillation angle and theoscillation frequency. To determine the equation of
motion, we measured the thrust. In addition, the actual
robot movements were measured, and the results werecompared with those of the thrust measurement
experiment. The thrust is corrected because there is a

©ISAROB 201159

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig.6. Numerical values of n and m
Fig.5. Difference of rotation axis

oscillation speed. In a preliminary study, we found thatthe velocity of the BFR was highest when the oscillation
angle was 45 degrees and increased as the oscillation
speed increased. Therefore, we set the following
experimental conditions: the oscillation angle was 45
degree, the oscillation frequency was 0.5 Hz, which was
the maximum speed of the servo motor, the measurement
time was 15 s, and the sampling frequency was 30 fps.

The numerical values of n and m were calculated
from the results of the real machine experiment. The
values of n and m were varied from 0.1 to 0.9 in steps of0.1, and the values that minimized the distance moved
were identified. The numerical results are illustrated inFig. 6. From the results, we defined n = 0.5 and m = 0.2.
Therefore, in the real machine, the length of the headthat contributes to propulsion is 0.5L1, and that of the
body is 0.8L2.
In short, because the thrust produced is different in
the real machine and the thrust measurement experiment,
it is necessary to correct the thrust with the values of nand m.
First, the thrust of the head is corrected. Because
n = 0.5, the rotation axis of the head is at the center ofthe head. A positive thrust and a negative thrust cancel
each other out in the wriggling motion in one cycle.
Therefore, the thrust that the head generates is 0.
Next, the thrust that the body generates is corrected.
The thrust is proportional to the volume of air pushed
out behind by the wriggling motion. In this case, wherem = 0.2, the length of the body that contributes topropulsion is 0.8L2, and a body length of 0.2L2 generates
a negative thrust. As a result, the thrust that the body andthe caudal fin generate is corrected by a factor of 0.6. In
addition, the angle of the wriggling motion is 45 degree,
but because the rotation axis and the head move, theangle of the wriggling motion becomes smaller than 45
degree. From the measurement results, the angle of thehead was 15 degree, and the angle of the body was 30
degree during the wriggling motion. Therefore, theoscillation angle of the body is assumed to be 30 degreein the thrust measurement experiment.

2. Thrust measurement experiment
The thrust of the BFR was measured with the force
sensor. We set the experimental conditions according to
the thrust correction. The oscillation angle was 30 degree,
the oscillation frequency was 0.5 Hz, the measurement
time was 15 s, and the sampling frequency was 1024 Hz.

V. RESULTS
The velocity of the BFR was calculated from theresults of the real machine experiment and the thrustmeasurement experiment.
The distance moved by the tracking points on the
BFR was measured from the results of the real machine
experiment, and the velocity of the BFR was calculated
every 0.3 s.
The results of the thrust measurement experimentwere substituted into the equation of motion of the BFR,
and the velocity of the BFR was calculated. Fig. 7
shows the numerical results. In Fig. 7, the horizontal axis
corresponds to time and the vertical axis corresponds tothe velocity of the BFR. In the thrust measurement
experiment, the velocity of the BFR was calculated using
the thrust values before correction and after correction.
The bottom graph shows the difference between the
results of the real machine experiment and those after
thrust correction. In Fig. 7, because the thrust wascorrected, the calculated value become close to thevelocity of the real machine. As a result, it was possible
to construct an equation of motion closer to that of thereal machine by considering the movement of the realmachine. The maximum velocity of the BFR was about
0.6 m/s for the real machine and the equation of motion,

IV. EXPERIMENT
1. Real machine experimentImages of the BFR during propulsion were captured
with a camera. The wriggling motion of the BFR is
oscillating movement having an oscillation angle and©ISAROB 201160

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011of the real machine by considering the movement of thereal machine.

VIII. FUTURE TASKS
Tasks to be implemented in future work include
considering the change in the projected area, conducting
experiments using other wriggle motion patterns,
repeating the motion experiments for a BFR with
pectoral fins, constructing an equation of motion, andcontrolling the air speed.

Fig.7. Velocity of the BFRACKNOWLEDGMENTS

showing good agreement. However, even after the thrust
had been corrected, it was confirmed that the differencein velocity compared with the real machine was about
0.3 m/s at most. Therefore, there must be force that was
not taken into account in constructing the equation of
motion.

A part of this study was supported by the HAYAONAKAYAMA Foundation for Science & Technology andCulture and a Grant-in-Aid for Scientific Research(Basis Research (C), Assignment Number: 21500511).

REFERENCES

VI. DISCUSSION[1] Kazuhiro Hosoi, Masanori Sugimoto, “AnAutonomous Security System Using Multi-Flying
Robots,”The 18th Annual Conference of the JapaneseSociety for Artificial Intelligence, 2C1-04, 2004.
[2] Junzhi Yu, Lizhong Liu, Long Wang, Min Tan, De
Xu, “Turning Control of a Multilink Biomimetic Robotic
Fish,”IEEE TRANSACTIONS ON ROBOTICS, Vol. 24,
No. 1, pp. 201-206, February 2008.
[3] Junzhi Yu, Long Wang, Min Tan, “Geometric
Optimization of Relative Link Length for Biomimetic
Robotic
Fish,” IEEE
TRANSACTIONSONROBOTICS, Vol. 23, No. 2, pp. 382-386, APRIL 2007.
[4] Motomu Nakashima, Kyosuke Ono “
Dynamics of
Two-Joint Dolphinlike Propulsion Mechanism (1stReport, Analytical Model and Analysis Method),”TheJapan Society of Mechanical Engineers, 62-600(B), pp.
3044-3051, 1996.
[5] H. Zhang, J. P. Ostrowski, “Visual Servoing with
Dynamics: Control of an Unmanned Blimp,”Proc. IEEE
Int. Conf. Robotics and Automation, pp. 618-623, 1999.

There was a difference between the results observedwith the real machine and those obtained with theequation of motion. In Fig. 7, the velocity periodicallydecreases in the calculation results for the real machine.
The cause of this is thought to be the change in the
projected area. In this study, the projected area of the
BFR was assumed to be constant in calculating thevelocity. In practice, however, the projected area changes
over time because the head of the BFR moves during thewriggling motion. The reason why the velocity of thereal machine decreases is that the projected area
increases periodically.
Similarly, it is also necessary to consider the force
that the pectoral fins generate during the wriggling
motion.
In this research, the velocity of the BFR was
calculated for one pattern of wriggling motion; in future,
it will be necessary to calculate the velocity for otherwriggling motions.

VII. CONCLUSION
The aims of this study were to measure the thrust of
the BFR and to construct an equation of motion. We
measured the thrust with a force sensor and constructed
the equation of motion by comparing the results withthose for real machine movement. As a result, it waspossible to construct an equation of motion closer to that

©ISAROB 2011

61

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Regulated Plus and Minus Power Supply Using
Approximate 2DOF Robust Digital Control
Atsushi Saitoh1 ,Yusuke Tsuruhori2 , Yoshihiro Ohta3 , Kohji Higuchi4 , Kazushi Nakano5
1,2,3,4,5
The University of Electro-Communications,
1-5-1 Chofu-ga-oka, Chofu, Tokyo 182-8585, Japanemail:saitoh@francis.ee.uec.ac.jp, tsuruhori@francis.ee.uec.ac.jp, ohta@francis.ee.uec.ac.jp,
higuchi@ee.uec.ac.jp, nakano@ee.uec.ac.jp
Abstract– A plus and minus switching power supplies areneeded for many applications, for example audio power
supply. Since the output voltage changes of such power
amplifiers which are loads are large, the power supply voltagse are changed largely. Usually, in order to suppress
the change, a capacitor with large capacity is used at theoutput end. If the capacities are made small, the power
supplies can be compacted. In this paper, it is shown thatthe capacities can be made small using a robust digital control using an approximate 2DOF. The derived controller isactually implemented on a DSP. It is demonstrated fromexperiments that the power supplies can be compacted by
the robust controller.
Fig. 1. Plus and Minus power supply

I. Introduction
The DC-DC converter is used for a switching power
supply of an plus and minus power amplifier for many
applications, for example an audio amplifier. This power
supply is needed to be compacted. In usual power supply, in order to suppress an output voltage change for loadchange and input voltage change, a capacitor with large
capacity is used for the output end. It can be considered that that the capacity is made small as one meansof a compact the audio power amplifier. The authors proposed the method of designing a approximate 2-degreeof-freedom (2DOF) controller of DC-DC converter[1] asa robust digital controller. The digital controller makes
the control bandwidth wider, and at the same time makes
the change of the output voltage very small at suddenchanges of load and the input voltage. This robust digital
controller was applied to control one side power supply[2].
In this paper, it is shown that the output voltage changeof the both sides plus and minus power supply can fully
be suppressed and the power supply can be compacted by
the capacitor with small capacity using this type robust
digital control. The robust digital controller is actually
implemented on a DSP. It is demonstrated from experiments that the capacity can be made small by the robust
controller.

Fig. 2. Plus side model of power supply

object. The triangular wave carrier is adopted for the
PWM switching signal. The switching frequency is set
at 100[KHz] and E is 14.5[V], n1 : n2 = 4 : 13. The
LC circuit is a filter for removing carrier and switching
noises. C0 is 1000[µF] and L0 is 70[µH]. In Fig.1, a plus
side and a minus side are symmetrical, and the plus sidemodel is shown in Fig. 2. The state equation of Fig. 2
is derived. If the frequency of control signal u is smaller
enough than that of the carrier, the state equation of the
DC-DC converter at a resistive load in Fig.1 except forthe controller in DSP can be expressed from the state
equalizing method as follows :II. Plus and Minus power supplyn

The configuration of the audio power supply is shown in
Fig. 1. The part except for the controller is a controlled©ISAROB 201162ẋ = Ac x + Bc u
y = Cx(1)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig. 3. Controlled object with input dead time Ld (≤ T )Fig. 5. Model matching system using only voltage (output) feedback

These parameter changes can be considered to be equivalent disturbances qu and qy inputted into the controlled
Fig. 4. Equivalent disturbances due to load variations (parameter object from the exterior as shown in Fig. 4. If the transvariations) and model matching system with state feedback
fer functions between the equivalent disturbances and theoutput voltage can be made small, it will become the control system hardly influenced of the input voltage change
where
·
·
¸
¸
¸ and the load changes. Therefore, if the controller is de·
1
0− C01RL
signed so that this transfer function may become small,
eo
C0
Ac =
Bc = K px =0
the compact audio power supply will be attained even if
i
− L10
−R
L0
L0
the capacity C0 is small.
£
¤
Vi N2
1 0
C =
u = ei y = e0 Kp =Cm N1III. Robust digital controller

R0 is the total resistance of coil and ON resistance of
FET, etc., and the value is 0.015[Ω]. When realizing a
digital controller by a DSP, a delay time exists between
the starting time of the sampling operation and the outputting time of the control signal due to the calculation
and AD/DA conversion times. This delay time is considered to be equivalent to the input dead time which exists
in the controlled object as shown in Fig. 3. Then thestate equation of the system in Fig.3 is expressed as follows :nxdw (k + 1) = Adw xdw (k) + Bdw v(k)(2)
y(k) = Cdw xdw (k)A. Configuration of control system

First, the transfer function between the reference input
r and the output y is specified as follows :Wry (z)=where, n1 and n2 are the zeros for the discrete-time control
object (2). It shall be specified that the relation of H1 andH2 , H3 becomes |H1 |À|Re(H2 )|, |H1 |À|Re(H3 )|. Then
Wry (z) can be approximated to the following first-ordermodel:

where·

xd (k)
ξ2 (k)

¸

·

¸

x(k)
xd (k) =xdw (k) =ξ1 (k)
·
·
¸
¸
Ad Bd
0
Adw =
Bdw (k) =
1
0
0
·
¸
R
L
eAc T eAc (T −Ld ) 0 d eAc τ Bc dτAd =00· R T −Ld¸Ac τe Bc dτ
0
Bd =
1
¤
£
£
¤
Cd 0
Cd = C 0
ξ1 (k ) = u(k )Cdw =It can be considered that the input voltage E and theload change by change of the amplitude of a voice signal etc. is parameter changes for the controlled object.

©ISAROB 2011

(1 + H1 )(1 + H2 )(1 + H3 )(z − n1 )(z − n2 )(z + H4 )(3)(1 − n1 )(1 − n2 )(z + H1 )(z + H2 )(z + H3 )(z + H4 )Wry (z) ≈ Wm (z) =1 + H1z + H1(4)This target characteristics Wry (z) ≈ Wm (z) is specified
so that the bandwidth of the audio power supply becomesas wide as possible.
Applying a state feedback
v
x∗

=
=

−F x∗ + GH4 r[y x2 ξ1 ξ2 ]T(5)

and feedforward
ξ1 (k + 1) = Gr(6)to the discrete-time controlled object as shown in Fig. 4,
we decide F = [F (1, 1) F (1, 2) F (1, 3) F (1, 4)] and G so

63

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

From eqs.(12) and (13), it turns out that the characteristics from r to y can be specified with H1 , and the characteristics from Q to y can be independently specified withkz . That is, the system in Fig. 6 is an approximate 2DOF,
and its sensitivity against disturbance becomes lower with
the increase of kz .
If an equivalent conversion of the controller in Fig. 6 iscarried out, the approximate 2DOF digital integral-type
control systems will be obtained as shown in Fig.6. In
Fig. 7, the parameters of the controller are as follows :Fig. 6. System reconstituted with inverse system and filter

k1that Wry (z) becomes eq.(3). Th current feedback is usedin Fig. 4. This is transformed to voltage and control input feedbacks without changing the pulse transfer function
between r − y by an equivalent conversion. The following
relation is obtained from Fig. 4:−F (1, 2)x2 (k)=−Ad (1, 1)x1 (k)−F (1, 2)(x1 (k + 1)
Ad (1, 2)
Ad (1, 3)ξ1 − Bd (1, 1)η)−(7)

If the current feedback is transformed equivalently using
the right-hand side of this equation, the control systemwith only voltage feedback as shown in Fig. 5 will be obtained. The transfer function WQy (z) between this equivalent disturbance Q = [qv qy ]T and y of the system inFig. 5 is desfined as¤£
Wqv y (z) Wqy y (z)(8)
WQy (z) =The system added the inverse system and the filter to thesystem of Fig.5 is constituted as shown in Fig. 6. In Fig.
6, the transfer function K(z) becomesK(z) =

kzz − 1 + kz

(9)The transfer functions between r − y and Q − y of thesystem in Fig. 6 are given by
y

=

y

=

z − 1 + kz
1 + H1Ws (z)rz + H1 z − 1 + kz Ws (z)z−1
z − 1 + kz
WQy (z)Q
z − 1 + kz z − 1 + kz Ws (z)

(10)
(11)

where
Ws (z) =(1 − H2 )(1 + H3 )(z − n1 )(z − n2 )(z − H2 )(z + H3 )(1 − n1 )(1 − n2 )©ISAROB 2011

≈

y

≈

1 + H1rz + H1z−1
WQy (z)Q
z − 1 + kz

(13)

F (1, 1 + F (1, 2)F F (1, 1) + ((−F (1, 4)
F (1, 2)F F (1, 4))(−F (1, 2)/F F (1, 2)))
(GH4 + GFz )(kz /(1 + H2 ))F (1, 2)/F F (1, 2) + G(kz /(1 + H2 ))F (1, 3) + F (1, 2)(F F (1, 3)) k4 = −Fz
Gkz ki2 = (GH4 + GF z)kz
G kr2 = GH4 + GFz(15)= −Ad (1, 1)/Ad (1, 2)= Ad (1, 2)= −Ad (1, 3)/Ad (1, 2)= −Bd (1, 1)/Ad (1, 2)= −F (1, 4) − F (1, 2)F F (1, 4)

In the audio amplifier power supply, since a plus side
and a minus side is symmetrical, the controller which controls only the plus side is designed. A model for the controlled object of the plus side becomes as shown in Fig.
2. The robust control system to this controlled object becomes as shown in Fig. 7. This system is an approximate
2DOF system and can specify independently the characteristics between the reference value and the controlled
output, and the characteristics between the disturbanceand the controlled output. If the parameter of the controller is set up so that the transfer function between the
disturbance and the controlled output may become small,the control system will become low sensitivity very much.
Therefore, even if the capacity is small, the large loadchanges can be suppressed. The same controller of Fig.
7 is applied to the minus side of Fig. 1. The averaged
control input is switched to the plus side and minus side.

(12)Here, if Ws (z) ≈ 1, then Eqs.(9) and (10) become, respectively,
y

k2
k3
ki1
kr1
where
F F (1, 1)
F F (1, 2)
F F (1, 3)
F F (1, 4)
Fz

=
−
+
=
=
=
=

IV. Experimental studies
The sampling period T are set at 10[µs] and the input
dead time Ld is about 0.999T [µs]. The nominal value of
RL is 4[Ω]. We design a control system so that all the
specifications are satisfied. First of all, in order to satisfy
the specification on the rising time of startup transient
reponse, H1 , H2 , H3 and H4 are specified as

(14)

H1 4

64

=

−0.9 H2 = 0.1 + o.1i H3 = 0.1 − 0.1i

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig. 7. Approximate 2DOF robust digital control system

Fig. 10. Experimental results by PI control at C0 = 1000[µF]Fig. 8. Experimental results by the approximate 2DOF robust
digital control at C0 = 1000[µF]H4=−0.999 kz = 0.15

Then the parameters of controller become
k1
k4

=
=

Fig. 11. Experimental results by PI control at C0 = 220[µF](16)14.063 k2 = −11.605 k3 = −0.022209−0.20448 ki = −1.3752 kiz = 1.1427 (17)We used the DSP(TI TMS320LF2808). Fig. 8 and Fig.
9 show the output voltage changes by the approximate
2DOF robust control at C0 = 1000[µ] and C0 = 220[µ],respectively. From these results, it turns out that output voltage change hardly changes even if capacity be-comes very small. Fig. 10 and Fig. 11 show the output voltage changes by PI control at C0 = 1000[µ] andC0 = 1000[µ], respectively. It turns out that the output
voltage changes are very large compared with the robust
control. Moreover, the output voltage changes are oscillating at C0 = 220[µ]. These results show that the robust
control by approximate 2DOF is eﬀective in compact audio power supply.
V. Conclusion
In this paper, the robust digital control was performed
by DSP. Consequently, even if it made capacity small, it
was shown that output voltage changes of the both sidesof the plus and minus power supply can be suppressedalmost like the time when capacities are large. A future
subject is designing the robust digital controller which canmake the value of output voltage change smaller.
References[1][2]

Fig. 9. Experimental results by the approximate 2DOF robust
digital control at C0 = 220[µF]©ISAROB 2011

65

K. Higuchi,K. Nakano, K. Araki and F. Chino, ”NEW ROBUST CONTROL OF PWM POWER AMPIFIER,” IFAC
15th Triennial World Congress(CD-ROM), 2002.
T. Kanai, K. Higuchi, K. Nakano, T. Kajikawa, A. Shimizu, O.
Yoshizawa, ”Compact Size of Audio Power Supply using RobustControl,” SICE 7th Annual Conference on Control Systems,
pp.62-1-2, March, 2007.

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Angle-based neuro-fuzzy navigation for autonomous mobile robots
Shu Hosokawa, Joji Kato, Kazushi Nakano, Kazunori Sakurama
Dept. of Electronic Eng.,The University of Electro-Communications, Japan(Tel : 81-42-443-5190; Fax : 81-42-443-5183)(hosokawa@francis.ee.uec.ac.jp)

Abstract: This paper presents a neuro-fuzzy navigation method for mobile robots based on local sensors mounted
on the robot. This method is not sensitive to sensor noise, and is able to automatically adjust the internal parameters given by the teacher signal. However, most of previous studies dealt with two-wheel driven robots have focused on the acceleration control of their wheels. For this reason, it is difficult to generate teacher signals for robots with many actuators such as omnidirectional mobile robots. In this paper, we propose a method of neurofuzzy navigation based on control of rotation speed of the robots. We demonstrate the validity of our proposed
method through simulations and experiments.
Keywords: Omnidirectional mobile robot, Neuro fuzzy, Robot navigation, Local sensor

I. Introduction
Recently, the autonomous mobile robots working in
dangerous environments such as disaster sites have beenstudied. These robots cannot get information about theiroperating environments in advance. For this reason,these robots require navigation methods to a target point
based on information from local sensors mounted on the
robots.
There are some methods based on the fuzzy inference techniques using local sensors on the robots tonavigate to a target point [1][2]. Those methods can
reduce the effect of observation noises from local sensors, but adjustments of the parameters by experts arenecessary. Automatic parameter turning methods which
incorporate neural network structures into fuzzy inference systems have been proposed in [3][4]. These controllers are able to adjust the parameters automaticallyby giving desired moving patterns. However, most ofprevious studies deal with the two wheel robots considering the acceleration of the motors as input. But when
applying these methods to more complex systems (forexample: snake-like robots, omnidirectional mobile
robots and so on), the complexity makes it difficult togenerate teacher signals.
In this paper, we propose a method using the rotation speeds of the robots as the input for the robot. By
using the rotation speed instead of the acceleration ofeach actuator, we can generate the teacher signals forthe controller. The validity of the proposed method isdemonstrated through simulations and experiments.

Figure 1 Omnidirec- Figure 2 Obstacle sensor
tional mobile robot
placements
robot, 𝑉𝑦 is the velocity at the lateral direction of therobot, and 𝑤𝑟 is the velocity of the rotation of the robot.
In this study, the rotation speed of each wheel is controlled by a suitable controller (for example: PID), andis assumed to accurately rotate according to the reference value. The following equation shows the relationship between the moving velocity of the robot and thevelocity of each wheel:𝑤1sin(𝜃)𝑤2
sin(𝜃)
1[𝑤 ] = [𝑅𝑤 −sin(𝜃)3𝑤4−sin(𝜃)𝑅
𝑉𝑥
𝑅] [ 𝑉𝑦 ]
𝑅
𝑤𝑟
𝑅

(1)

where R is the radius of the robot, and 𝑅𝑤 is the radiusof the wheel. In this study, we design a controller using
two pieces of information; the distance between the
obstacle and the robots, the angle of the target pointfrom the front direction of the robot.

II. Omnidirectional mobile robot
Figure 1 shows an omnidirectional mobile robot inthis study, where 𝜃 is the angle between y-axis and awheel, 𝑉𝑥 is the velocity at the front direction of the©ISAROB 2011

− cos(𝜃)cos(𝜃)
cos(𝜃)− cos(𝜃)66

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Figure 2 shows the arrangement of fifty obstacle detection sensors mounted on the robot. Those sensors are
divided into five groups of three adjacent sensors {rightbehind, right front, front, left front, left behind}. The
output value of each sensor group is selected to take asmallest value in the groups. The angle of the target
direction is given from the front of the robot. In thispaper, we assume a constant velocity at the lateral direction for simplicity.

III. Angle-based neuro-fuzzy navigation
This section, explains the proposed method of anglebased neuro-fuzzy controller. At first, we explain thecontroller structures and how to adjust the parameters of
the fuzzy membership functions.

Figure 3 Angle-based neuro-fuzzy controller
The third layer represents the fuzzy controller based on
the if-then rules, which has the connection with the second layer as the antecedent, and the connection withthe fourth layer as the consequent. As for the if-then rule,
there are 160 rules from five fuzzy sets of the target
direction and two fuzzy sets of five groups of obstacle
sensors. The fourth layer represents the output linguistic
grades. The fifth layer is the output of the controller,
which gives the center of gravity defuzzification of
fuzzy sets of the four layers. The center of gravity is
calculated using the lowest grade in the antecedent ofeach fuzzy rule:

1. Controller structure
Figure 3 shows the structure of the proposed anglebased neuro-fuzzy controller. This controller consists ofa five-layer neural network. The first layer receives thetarget direction and distance information from the obstacle sensors. The second layer converts an input value
to a fuzzy grade using the fuzzy inference logic. Membership functions of the target direction consist of five
fuzzy sets {right behind, right front, front, left front, left
behind}, in the form of triangular type functions:𝑝𝑖𝑗 = {

1−

2|𝑢𝑖 −𝑚𝑖𝑗 |𝜎𝑖𝑗, 𝑚𝑖𝑗 −𝜎𝑖𝑗
2< 𝑢𝑖 < 𝑚𝑖𝑗 +

𝜎𝑖𝑗
2

𝑦=

(2)

0.001, otherwise

𝑞𝑘 = min*𝑝1 𝑘1 , 𝑝2 𝑘2 , 𝑝3 𝑘3 , 𝑝4 𝑘4 , 𝑝5 𝑘5 , 𝑝6 𝑘6 +where i is a input number, j is a fuzzy set number, 𝑝𝑖𝑗
is the fuzzy grade, 𝑚𝑖𝑗 is the center value of the membership function, 𝑢𝑖 is the input value, 𝜎𝑖𝑗 is the widthof the membership function. Otherwise, the membershipfunction of the obstacle sensor consists of two fuzzy
sets to determine the distance from the obstacle. A Ztype function in (3) is adopted as the membership function that represents the distance between the robot andthe obstacle:𝑝𝑖𝑗 =2|𝑢𝑖 −𝑚𝑖𝑗 |Learning algorithm
We explain a parameter adjustment law adopted byour proposed method to use teacher signals. The proposed method has the parameters; the center value offuzzy membership function and the width of fuzzy
membership function. In this study, we update only thecenter values of fuzzy membership functions.
Adjustment of the parameters is performed by using
the evaluation function described by the following equation:

(3)

𝜎

𝑝𝑖𝑗 =
{

©ISAROB 20111−

2|𝑢𝑖 −𝑚𝑖𝑗 |
𝜎𝑖𝑗

(6)

2.

1−
, otherwise
𝜎𝑖𝑗{
while, a S-type function in (4) is adopted as the membership function that represents the closeness between
the robot and the obstacles.
0.001, 𝑢𝑖 ≤ 𝑚𝑖𝑗 − 𝑢2
1, 𝑢𝑖 > 𝑚𝑖𝑗(5)

where y is the output value of the rotation value, k is thenumber of fuzzy rule sets, 𝑣𝑘 is the center value of the
fuzzy membership function, and 𝑞𝑘 is the fitness of thek-th fuzzy rules.

𝜎0.001, 𝑢𝑖 ≥ 𝑚𝑖𝑗 + 𝑢
2
1, 𝑢𝑖 < 𝑚𝑖𝑗∑160
𝑘=1 𝑣𝑘 𝑞𝑘∑160
𝑘=1 𝑞𝑘1
𝐸 = (𝑦 − 𝑦̂)22

(4)

, otherwise

67

(7)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

where E is the error, 𝑦̂ is the teacher signal. This evaluation is based on the least mean square method. Z as
the median to be updated, the update formula is expressed as
𝒁(𝑡 + 1) = 𝒁(𝑡) − 𝜀

𝜕𝐸
𝜕𝒁

(8)

where 𝜀 is the learning rate, 𝒁(𝑡) is the parameter
vector at the update number 𝑡. Each parameter is updated by the following equation:
𝜕𝐸
𝑚𝑖𝑗 (𝑡 + 1) = 𝑚𝑖𝑗 (𝑡) − 𝜀
(9)
𝜕𝑚𝑖𝑗
𝜕𝐸
(10)
𝑛𝑠 (𝑡 + 1) = 𝑛𝑠 (𝑡) − 𝜀𝜕𝑛𝑠Figure 4 Training environment
Table 1 Simulation parameters
Robot radius
10 cm

where, if 𝑞𝑘 = 𝑝𝑖𝑗 and 𝑝𝑖𝑗 ≠ const,
𝜕𝐸
=
𝜕𝑚𝑖𝑗
−2 {(y − 𝑦̂)160
𝑣𝑘 ∑160
𝑘=1 𝑞𝑘 − ∑𝑘=1 𝑣𝑘 𝑞𝑘

sign(𝑢𝑖 − 𝑚𝑖𝑗 )𝜎𝑖𝑗otherwise(∑160
𝑘=1 𝑞𝑘 )2

Working space
Measuring range of the
obstacle sensor
Translation velocity
Control cycle}

405 cm×605 cm
0 cm ~ 100 cm
20 cm/s
200 ms

×

𝜕𝐸
=0
𝜕𝑚𝑖𝑗

.

Figure 5 Trajectory for Figure 6 Trajectory after
training data
parameter adjustment.
And also, if 𝑣𝑘 = 𝑛𝑠 ,𝜕𝐸𝑞𝑘
= (𝑦 − 𝑦̂) 160
𝜕𝑛𝑠∑𝑘=1 𝑞𝑘otherwise
𝜕𝐸=0𝜕𝑛𝑠
Teacher signals are used in expressions to produce
more updates from the history of the human-robot operation. The teacher signals has the three types of parameters; the angle of the target point from the front directionof the robot, the distance from the obstacles were measured by the local sensor mounted on the robot, and thevelocity of the rotation of the robot were controlled by ahuman. Parameter adjustment is repeated until the error
falls below the set value 𝛿 (0 < 𝛿).

©ISAROB 2011

Figure 5 Results in various environmentschangedIV. Simulation and experimental results
This section shows simulation and experiment results of our proposed method. First, we explain how to
generate the teacher signal in the training environment,
and adjust the parameters of fuzzy membership functions. Next, we show some different environments. Finally, we demonstrate the results of the actual robot.

68

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

our proposed method through simulations and experiments.

REFERENCES[1] Eugenio Aguirre and Antonio Gonzalez (2003), A
fuzzy perceptual model for ultrasound sensors applied
to intelligent navigation for mobile robots. Applied Intelligence 19, 171-187[2] S.M.Raguraman, D.Tamilselvi and N. Shivvakumar(2009), Mobile robot navigation using fuzzy logic controller. International conference on control , automation,communication and energy conservation[3] Kai-Tai-Song and Liang-Hwang Sheen (2000), Heuristic fuzzy-neuro network and its application to reactive
navigation of a mobile robot. Fuzzy Set and Systems
110, 331-1340[4] Anmin Zhu and Simon X. Yang (2007), Neurofuzzy-based approach to mobile robot navigation in
unknown environments. IEEE trans. on Systems, Man,
and Cybernetic – Part C; Applications and Review
37(4), 610-621

Figure 6 Actual results
Figure 4 shows the environment in the simulation.
Table 1 is the list of the parameters used in these simulations. The simulation purpose is to move a robot to agoal position on the left side of each environment fromthe start at the right side without colliding against obstacles.
Figure 5 shows the trajectory of the robot operated
by a human teacher. Figure 6 shows the trajectories of
the robot after the parameter adjustment based on theteacher. By adjusting the parameters of fuzzy membership functions, we can see that the robot reaches thetarget without colliding against obstacles.
Next, we show the result in some different environments with pre-adjusted parameters in the previous environment. The starting position and the target point are
the same as the previous environment. Figure 7 shows
the results in the four simulation environments in different shapes, where the numbers of obstacles are different.
We can see that the robot reach the target in all the environments without re-adjusting the parameters.
Finally, we apply our proposed method to the actual
robot. We prepare a simulated environment created by
the teacher signals shown in Figure 5. The parameters ofmembership functions are obtained by the simulation.
Figure 8 shows the trajectory obtained by the actual
robot. We can see that the actual robot reach the target
without colliding with obstacles.

VI. Conclusion
In this study, we proposed an angle-based neurofuzzy navigation method. We control the rotation speedof robots instead of the acceleration of each motor. Thus,
a controller structure was too compact to simplify thegeneration of teaching signals even when applying therobot to a complex structure. We showed the validity of©ISAROB 2011

69

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Quantitative Evaluation of Body-swayCaused by Tactile Apparent Movement
Kennichi Mogi* and Masafumi Uchida***The Graduate School of Electro-Communications, The University of Electro-Communications**The Graduate School of Informatics and Engineering, The University of Electro-Communication
1-5-1, Chofugaoka, Chofu, Tokyo, Japans(Tel : 81-42-443-5173; Fax : 81-42-443-5173)(mogi@zidane.ee.uec.ac.jp)

Abstract: The recognition of tactile apparent movement is normally a subjective sense. However, when applying tactile
apparent movement to an engineering system, quantitative evaluation is necessary. In a previous study, we examined
the body-sway caused by tactile apparent movement under fixed experimental conditions; however, the body-sway
characteristics were not fully investigated. In this study, we investigated the body-sway caused by tactile apparent
movement under fixed experimental conditions. We focused on biological information and body-sway, and comparedcases where the apparent motion was recognized and not recognized. Our findings will help to improve theperformance of systems that use tactile apparent movement.
Keywords: Tactile Apparent Movement, Body-sway, EMG,
The recognition of apparent movement is normally asubjective sense. On the other hand, when applying
tactile apparent movement to an engineering system,
quantitative evaluation is necessary. In many previous
studies, however, evaluation of the recognition of
apparent movement was based on statistical techniques,
and quantitative evaluation was difficult.
In our previous study, we reported the body-sway ofan upper limb caused by tactile apparent movement
under fixed experimental conditions [4]–[6]. If
quantitative evaluation of the body-sway becomes
possible, we will be able to evaluate the tactile apparent
movement. The characteristics of the body-sway wereanalyzed using an averaged electromyography (EMG)
waveform when apparent movement was recognizedbased on frequency analysis. In that study, however, thebody-sway was not fully evaluated quantitatively.
In this study, we focused on the frequency
characteristics of body-sway and investigated the bodysway caused by tactile apparent movement.

I. INTRODUCTION
The tactile sense of our skin is one sense that isbeing investigated for sensory substitution because thepossibility of losing the tactile sense is lower than thepossibility of losing sight, hearing, and other senses due
to congenital or acquired causes. Sensory substitutionmeans substituting a remaining sense for a lost sense.
For example, a tactile display can transmit information
using the tactile sense instead of the visual sense.
Apparent movement is a sensory illusion mainly
studied and reported in the field of vision. However, it is
known that apparent movement also appears in thetactile sense. When two stimuli appear one after anotherwith an appropriate time lag, we sense movement of thestimulus; that is to say, we sense that one stimulus
moves to the other stimulus.
The frequency characteristics of tactile apparent
movement have been investigated [1]–[3]. Ueda et al.
proposed a tactile display that combined tactile apparent
movement and phantom sensation [7],[8]. Park et al.
designed aperson identification authentication system
based on a tactile stimulus using a tactile display [9].
When applying tactile apparent movement to a tactile
display in which stimulation elements are arrayed, thenumber of stimulation elements increases as a result of
increasing transinformation [10],[11]. By applying
apparent movement to tactile displays, it should bepossible to realize smaller displays.

©ISAROB 2011

II. MEASUREMENTS
Fig. 1 shows the measurement system used in ourstudy. It consists of two PCs and five components: (1)an EMG measurement system, (2) a three-axis
acceleration sensor, (3) a tracking system, (4) astabilometer, and (5) a vibrotactile stimulation
presentation system.

70

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig. 1. Measurement system.
Fig. 3. Arrangement of 5 tracking markers.

Fig. 4. Arrangement of four vibrators and accelerometer.

Fig. 2. Arrangement of 5 electrodes and body earth.
The sampling frequencies were different in eachsystem. In detail, biological information and three-axis
acceleration were acquired via the same A/D converter,
whose sampling frequency was 2000 Hz. The tracking
system’s sampling frequency was 60 Hz, and thestabilometer’s sampling frequency was 80 Hz. PC1 wassynchronized with PC2 via a synchronization signal that
PC2 generated.
The number and positions of the EMG measurement
channels were defined as shown in Fig. 2. The number
of EMG channels was 7. The sampling conditions areshown in Table 1.
The tracking system consisted of a CCD camera(Library Corp.: GE60). Positions of tracking markers
are shown in Fig. 3. The tracking points were the upper
limb, shoulder, body, and below the ear lobe. Tracking
results were recorded on PC2 as coordinate data (X, Y).
The three-axis acceleration sensor was worn on the rightwrist.

©ISAROB 2011

Fig. 5. Drive waveform of vibrator.
A stabilometer (Nitta Corp.: BPMS) measured thesubject’s center of gravity while standing still. The
posture on the stabilometer was based on stabilometry
defined by the Japan Society for Equilibrium Research.
The vibrotactile stimulation presentation system
consisted of four vibrators (Audiological Engineering
Corp.: Skin Stimulator). The subject held four vibrators(SS1, SS2, SS3 and SS4) as shown in Fig. 4. The
vibrators were driven by the drive waveform shown in
Fig. 5. The drive waveform had four parameters, T, TS,
TC, and . In this study, tactile apparent movement was
generated by adjusting only the parameter . The otherparameters, T, TS, and TC,

71

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

upon recognizing apparent movement, a significant
difference should be observable in(a) Subject 1(b) Subject 2(a) Subject 1(b) Subject 2(c) Subject 3(d) Subject 4
Fig.6. Averaged waveform of TP10ch(c) Subject 3(d) Subject 4
Fig.7. Averaged spectrum of TP10ch
the spectrum at a specific frequency compared with thecase where the apparent movement is not recognized.
As preprocessing, the EMG was filtered from 20 Hz
to 500 Hz and subjected to envelope processing, theacceleration value was filtered from 0 Hz to 45 Hz, andthe velocity of the tracking points and the center ofgravity were analyzed using the finite difference method.
Then, the preprocessing data for 3000 ms (for center ofgravity only: 2500 ms) from the beginning of thestimulation was averaged. The frequency spectrum wascalculated from the preprocessing data, averaged over
all trials for every . The spectral intensity of eachfrequency was compared to examine the difference
between recognized and non-recognized apparent
motion, and a statistically significant difference wasfound.

were fixed at T = 100 ms, TS = 4 ms, and TC = 3000 ms.
One trial was assumed to consist of driving vibrators
SS1 to SS4 in sequence. One set consisted of 30 trials,
and ten sets in total were performed.
The value of parameter  when the subject felt theapparent movement the most is m. The values ofparameter  representing the difference threshold for theapparent movement are L and H. They were measured
by employing an adjustment method used in
psychophysics measurements. In this study, vibrotactile
stimulations each composed of 0, L, m, and H were
presented to the subject. The body-sway caused by thetactile apparent movement was evaluated by comparing
and examining the subject's reaction at this time. The
parameters 0, L, and H do not cause any apparent
movement while m causes tactile apparent movement.
Here, the relation of the  parameters is 0 < L < m <
H. One of the  parameters was selected randomly ineach trial, and a vibrotactile stimulation was presented
to the subject using that parameter. In the measurement,
the subject stood on the stabilometer, with their eyes
closed, wearing noise canceling headphones andholding their dominant arm horizontally.

IV. RESULTS
Table 1 shows the results of psychophysics
measurements. The parameter  that each subject
selected was different, indicating that the recognition oftactile apparent movement is subjective.

III. ANALYSIS

Table 1. Results of the psychophysics measurement.
L [ms]m [ms]H [ms]Subject 1
20
70120
Subject 2
1070120
Subject 330
70110
Subject 4
1080160
As one example of the result, Fig. 6 shows the average
waveform of the preprocessing data for the tracking
point 10ch (TP10ch, Fig. 3). It is thought that TP10ch

The body-sway caused by apparent movement reported
in the previous study caused the upper limb to move inthe direction in which the stimulation image was moved
when the subject recognized an apparent movement. In
the present experiment, the subject held his or her upper
limb horizontally. If body-sway is caused, it is expectedthat the arm would center on the shoulder and shake
periodically. Therefore, we focused on the frequency ofthe measurement data. When periodic shaking appears©ISAROB 201172

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

contains a lot of information about the body-sway
because TP10ch is an element in the same direction as
the direction in which the stimulation image is moved.
From Fig. 6, subject 1 and subject 4 exhibited periodic
shaking. This result suggests the possibility of the bodysway having periodicity. Fig. 7 shows the averaged
spectrum of TP10ch. From 0 Hz to 10 Hz, the differencein spectral intensities by a factor of 2 or more between
the recognized and non-recognized apparent motions for
subjects 1 and 4 was statistically significant (P < 0.01, ttest). From the above result, it is possible that the bodysway has periodicity because both subjects showed asignificant difference at a specific frequency. Here, thechannel on which the frequency with a significant
difference existed was extracted from all channels, andthe results are shown in Tables 2–4. Table 2 shows theresult of comparing m with 0, Table 3 shows the resultof comparing m with L, and Table 4 shows the result ofcomparing m with L. From the results, besides TP10ch,
other channels are expected to show a periodic response.
However, these are not reactions common to all subjects.

This is thought to be due to the fixed parameter T in thepresented stimulation, because the parameter T is an
important parameter causing apparent movement.
Therefore, it will be necessary to conduct an experimentthat considers the influence of parameter T in the future.

V. CONCLUSION
In this study, we investigated the body-sway caused bytactile apparent movement. Focusing on the frequency
characteristics of the body-sway, we tried to evaluate
the body-sway quantitatively. The results show thepossibility that periodic shaking occurred whenapparent movement was recognized. In future work, theT parameter of the presented stimulation will be
examined, and quantitative evaluation of the body-sway
caused by tactile apparent movement will be attempted.

REFERENCES[1] J.H. Kirman: "Tactile apparent movement: The effects ofinterstimulus onset interval and stimulus duration", Perception& Phychophysics, Vol. 15, pp. 1-6 (1974).
[2] J.H. Kirman: "Tactile apparent movement: The effects ofnumber of stimulators", Journal of Experimental Psychology,
Vol. 103, pp. 1175-1180 (1974).
[3] J.H. Kirman: "Tactile apparent movement: The effects ofshape and type of motion", Perception & Psychophysics, Vol.
34, No. 1, pp. 96-102 (1983).
[4] K. Kawahara, Y.-I. Park, and M. Uchida: "Characteristic
Extraction of EMG with Recognition of Tactile Apparent
Movement", IEEJ Trans. SM, Vol. 129, No.6, pp. 187-188(2009) (in Japanese).
[5] Y.-I. Park and M. Uchida: "Analysis of Surface EMG with
Apparent Movement of Tactile sense", Proc. of the SICE
Annual Conference 2009, 2C12-6 (2009).
[6] Y.-I. Park and M. Uchida: "Swing Analysis of Body-parts
Motion Accompanied by Apparent Movement", The
International Symposium on Artificial Life and Robotics 2010
(AROB 15th), OS1-4 (2010).
[7] S. Ueda, M. Uchida, A. Nozawa, and H. Ide: "A Tactile
Display Used Phantom Sensation with Apparent Movement
Together", IEEJ Trans. FM, Vol. 127, No. 6, pp. 227-284(2007) (in Japanese).
[8] Y.-I. Park and M. Uchida: "A Tactile Display by Using
Phantom Sensation with an Apparent Movement based on
Electro-stimulation", Proc. of the International Conference on
Electrical Engineering (ICEE), CD-ROM, P-131 (2008).
[9] Y.-I. Park and M. Uchida: "System of personal
identification by using tactile stimuli", Artificial Life andRobotics, Vol. 13, No. 1, pp. 209-213 (2008).
[10] C. Domenici and D. Derossi: "A stress-componentselective tactile sensor array", Sensors and Actuators, Vol. 13,
pp. 97-100 (1992).
[11] B.L. Gray and R.S. Fearing: "A surface-micromachined
micro-tactile sensor array", IEEE Int. Conf Rob. and Auto.,
Vol. 1, pp. 1-6 (1996).

Table 2. Extracted channels for m vs. 0
Subject
TP
CW
EMG
Acc
1
ch3,ch4ch1ch1, ch2 ch1, ch2
ch7,ch8
ch9,ch102ch6------3ch7--ch6, ch7
ch1
4
ch5, ch8
--ch1
ch1
ch10

1
2
3
4

1
2
3
4

©ISAROB 2011

Table 3. Extracted channels for m vs. L
TP
CW
EMG
Acc
ch3,ch4--ch6--ch3,ch9
ch1,ch2--ch2
ch8--ch6,ch7--ch3,ch5--ch2,ch5--Table 4. Extracted channels for m vs. H
TP
CW
EMG
Acc
ch9
--ch2
--ch5,ch6
ch2,ch5--ch1,ch2
ch7------ch1
ch6
--ch7
---

73

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A visual-taste interference model and the EEG measurement
Hisaya Tanaka and Yuichi Sato
Department of Information Design, Kogakuin University
1-24-2, Nishi-shinjuku, Shinjuku-ku, Tokyo, 163-8677, Japan(Tel : 81-3-3340-0890; Fax : 81-3-3348-3486)(hisaya@cc.kogakuin.ac.jp, j206051@ns.kogakuin.ac.jp)

Abstract: Taste cognition is interfered by visual information; however the mechanism has not been clarified. We
assumed the interference model in the process of taste and visual information. The model was tested with frequency
analysis on EEG and the button response time. The tasks were matched/miss-matched between taste and visual
information of orange or apple juice. There were changes in α waves that originated in visual processing of a juice
package and changes in β waves that originated in taste processing. There was the possibility with the parallel
processing mechanism in the visual-taste interference.
Keywords: taste cognition, visual information, interference, EEG
on the tongue onto sensory nerve fibers, and the fivetaste (sweet, salty, delicious, bitter and sour)
information is taken over to the taste area in the brain.
These two kinds of information in the above areintegrated in the brain and the integrated information isrelated to memory, leading to taste cognition (Fig.1).

I. Introduction
There are studies using facial expressions [1] or skintemperature on the nasal area [2] as a method to give an
objective evaluation on taste. There are also various
papers that handled taste and the brain [3] [4].
However, those papers that handled the brain and taste
have not reached an objective evaluation of products.
In this paper, therefore, a method to give an objective
evaluation on taste as well as products presented wasreviewed based on the relationship between taste andthe brain. Spontaneous brain waves were measured inorder to develop the taste processing model in this paper,by focusing on integrated processing of vision and taste.
The taste processing model was designed to measure theinteraction of visual information and taste, including
whether information obtained from people’s eyes
influences taste cognition, or only taste influences taste
cognition.

MemorySensory
Visualorgan (eyes) information
Integrated
processing

Sensoryorgan(tongue)

Fig.1. Proposed taste processing model
2. Verification of hypotheses with brain waves
The following three hypotheses are considered basedon the taste processing model.
If responses of
spontaneous brain waves are successfully obtained inaccordance with these hypotheses, it will be useful tosupport development of the taste processing model.
Spontaneous brain waves can be divided into threebands including α waves (8-13Hz)，β waves (14-30Hz)
and θ waves (4-7Hz). α waves appear when eyes areclosed, when relaxed or not nervous, or when brain
activities are calm. β waves appear when stress is
accepted or during calculation. θ waves appear in thecourse from awakening to onset of sleep. It is also
known that α waves decrease when visual information is
given, in comparison with the time at rest [4]. In
addition to the above, we considered that the volume of

II. Interference between taste and vision
1. Proposal of taste processing model
The process to reach taste cognition ininterdependence between vision and taste was proposedas the “taste processing model” in this experiment.
Information received from two sensory organs, i.e., eyesfor vision and the tongue for taste, is communicated to
the nerve center. Images entered through eyes arerecognized in the vision area, the information is sent to
the temporal association cortex, and the images seen areinterpreted as meaningful. Taste entered from thetongue transmits from the receptor called the taste buds

©ISAROB 2011

Taste
information

Taste
cognition

74

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

thoughts occurring in the brain when taste is given(difficulty in relating short-term memory from vision or
long-term memory to taste) was involved in theincrease/decrease of α waves and β waves in this paper.
If these three hypotheses are verified with experiments,
the taste processing model will be successfully
developed. Hypotheses at this time were preparedbased on brain waves including α, β and θ waves whentaste information of water was given.

↑BeginFig.2: Timetable for the experiment
Table 1 includes Tasks 1 to 11. Enough time wasmaintained between task and task in the experiment, andthe oral cavity was rinsed with water after and
immediately before taste stimulation. To avoid thesame sample to be input in a row, Tasks 1 to 5 and Tasks
6 to 11 in the experiment were conducted at least with
one-week interval.

III. EXPERIMENTS
1. Experiment method
Subjects were four healthy males (age: 21-22). The
experiment was conducted in a lab which was not ashielded room, and lights were turned off during theexperiment, i.e., the condition of a dark room. First ofall, subjects sat on chairs, the experimenter presented atable with the names of six samples (water, apple,orange, strawberry, banana and pear) to ensure taste
cognition, and indicated one of them would be put into
the mouth. The subjects were instructed to identify
which sample was put into the mouth. They were topush down the switch at hand immediately after theyknew what sample it was (considered that taste wasrecognized). The reason for pushing down the button
is that the time from completion to integrate vision withtaste until taste cognition as shown in Fig.1 isconsidered to be interrelated with the time until pushingdown the button by the subjects. Therefore, after
explaining about the experiment, we asked them toavoid movements except pushing down the button orteeth grinding during the experiment.
After full
explanation, subjects were instructed to put on anelectrode cap to measure brain waves. They were also
asked to hold the hose (φ5×7 mm) connected to the One
Shot Measure 30ml (Sentec D-19002BK) in the mouthusing the lips, to enable input of a fixed amount. (Thehose was not sticking to the face, and the sample was
not visible.) Since the photo of the package is shown
before giving the sample (taste stimulation) in thisexperiment, the task is in the order of visual stimulationand taste stimulation. Eyes were opened when visual
stimulation was given, and closed when it ended.
Visual stimulation was given for five seconds, andanother five seconds were provided for the time to closethe eyes. Then, input of taste began. The sample was
input without notifying the subject of the timing of input.
The timeline of the experiment is indicated in Fig.2.

©ISAROB 2011

↑Begin input↑End↑Begin presentation(5seconds）↑Push the button↑End presentation(5seconds)

Table 1: Tasks in the experiment
Task No.
Visualpresentation
Experiment
samples

1

2

3

O

A

x

x

x

W

4

5

6

7

8

9

10

11

x

x

O

A

O

A

O

A

O

A

O

A

A

O

W

W

In regards to symbols in Table 1, O representsorange juice, A represents apple juice and x represents
no stimulation. Orange juice and apple juice used in
the experiment was “Tropicana 100% Orange Juice”
and “Tropicana 100% Apple Juice.” Packages forthese two kinds of juice were photographed with adigital camera for presentation. With the responsetime like in Tasks 3, 4 and 5 as the reference where only
the sample was given, Tasks 8, 9, 10 and 11 wereconsidered to take longer for integrated processing
when the subjects respond, leading to a long button
response time. On the other hand, the button responsetime was considered to be shorter for tasks which seem
to take less time for integrated processing.
2. Measurement method
Neurofax EEG1100 (Nihon Kohden) was used tomeasure brain waves, and FOCUS (Nihon Kohden) wasused for analysis.
31 exploring electrodes inaccordance with the International 10-20 method arearranged on the electrode cap (Electro Cap) (refer toFig.3). Unipolar induction was used for measurementwith both earlobes (A1 and A2) as reference electrodes.
Experiments were completed within 60 minutes after
the electrode cap was worn (Fig.3). In regards to brain
waves extracted, the band with α waves was at 8-13Hz,
β waves at 14-30Hz and θ waves at 4-7Hz. Theintervals to analyze brain waves included from the
beginning of visual stimulation presentation to thecompletion of visual stimulation presentation, and from75

the beginning of taste stimulation presentation to button
pushing by a subject. The FFT was applied to theseintervals and the content of brain waves in all channels
were averaged to obtain the content rate for these threebands.

含有率[%]
Content
Content
rate[%]
rate[%]The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

50
40
30
20
10
0

θ
α
β

Task3(x,Task3(x, W)W)
タスク3（×，水）Task4(x,Task4(x, O)O)
タスク4（×，オ）

Task5(x,A)
Task5(x,A)
タスク5（×，リ）

Fig.4: Taste and electroencephalographic response(average ± standard deviation)

2.
4.

Hypothesis 2: Presence of visual presentationand electroencephalographic response
Next, tasks with taste information only are comparedwith tasks where the same visual and taste information
are presented such as visual information of apple andtaste information of apple (Tasks 6 and 7). For
Hypothesis 2, the volume of thoughts decreases upontaste cognition, since integrated processing of vision andtaste goes easier with tasks with visual information.
Thus, we considered that α waves increase and β waves
decrease in comparison with brain waves when only
taste is presented. As a result of the experiment, thecontent rate for α waves and β waves decreased whenTasks 4 and 6 were compared, and increased whenTasks 5 and 7 were compared. θ waves have atendency to increase when vision is presented rather
than presenting taste only. A significant difference was
not recognized in Student’s t-test (significancelevel=0.05; two-sided test) for Tasks 4 and 6 as well as
Tasks 5 and 7. Based on the above results, it isassumed that in the case of Tasks 4 and 6, α waves
decreased and β waves increased as a result of relating
to long-term memory, and in the case of Tasks 5 and 7,
α waves increased and β waves decreased as a result ofrelating to short-term memory (Fig.5).

Results

Fig.3: Measurement system

IV. RESULTS
Hypothesis 1: Taste andelectroencephalographic response
Tasks with taste presentation only are compared.
For Hypothesis 1, it was considered in the cases ofsamples with and without taste that α waves decreaseand β waves increase because the burden of taste
integration increases for the sample with taste, as aresult of the load from the work to relate to long-term
memory. As a result of experiment, the content rate ofα waves tended to decrease and the content rate of β
waves tended to increase for samples with taste incomparison with samples without taste, when Tasks 3
and 4 as well as Tasks 3 and 5 were compared, and thecontent rate of θ waves decreased. This is because thevolume of thoughts increased more for the process toscreen orange or apple out of the given hints (water,
apple, orange, strawberry, banana and pear) than thecase of water.
Significant difference was notrecognized in Student’s t-test (significance level=0.05;
two-sided test) of brain waves for either Tasks 3 and 4
or Tasks 3 and 5. Based on the above, the tendency ofthe hypothesis was observed but statistically significant
differences were not recognized (Fig.4).

©ISAROB 2011

含有率[%]
Content
rate[%]

1.

50
40
30
20
10
0

θ
α
β
Task4(x,O)
タスク4（×、オ）

Task6(O,O)
タスク6（オ、オ）

Task5(x,A)
タスク5（×、リ）

Task
7 (A, A)
タスク7（リ、リ）

Fig.5: Presence of visual presentation andelectroencephalographic response (average ± standard
deviation)

76

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Hypothesis 3: Electroencephalographic
response to match/mismatch of taste andvisual presentations
Next, tasks where the same vision and taste arepresented are compared with tasks where different
vision and taste are presented (Tasks 8 and 9). For
Hypothesis 3, we considered the case where taste
information does not match vision information given,e.g., visual information of apple and taste information oforange. In this case, integrated processing becomes
extremely difficult and the volume of thoughts
increases; therefore we considered that α waves
decrease and β waves increase compared with the casewhere the same visual and taste information is given.
First of all, the content rate of α waves decreased, thecontent rate of β waves increased and θ waves also
increased in the case of Task 8 (A, O) compared with
Task 7 (O, O). On the other hand, the content rate of α
waves and β waves decreased and θ waves increased in
the case of Task 9 (O, A) compared with Task 6 (A, A).
Significant difference was not recognized in Student’s ttest (significance level=0.05; two-sided test) of brain
waves for Tasks 7 and 8 as well as Tasks 6 and 9. It isconsidered that α waves decreased with short-term
memory in regards to the mismatch (Fig.6).
含有率[%]
Content
rate[%]

3.

・・・Processing

20
10・・・Processing

Integration

タスク9（リ，オ）
Task9(A, O)

タスク7（リ、リ）
Task7(A, A)

Pear
Strawberry

OrangeWater

Button pushing

BananaComparison

relating to memory
Apple

time

Fig.8: Restructured taste-vision processing model (inthe cases of Hypotheses 2 and 3)

タスク8（オ，リ）
Task8(O, A)

Fig.6: Electroencephalographic response (average ±standard deviation) in match/mismatch of taste andvisual presentations

The model under Hypotheses 2 and 3 was indicatedin Fig.8. Increase/decrease of α waves and β waves
was considered to change depending on long-term
memory and short-term memory used to relate tomemory when the same taste information as visual
information is given. On the other hand, when taste
information different from visual information is given, α
waves decreased while β waves did not increase, byrelating to long-term memory without using short-term
memory; therefore the model was restructured.

VI. CONCLUSION
We measured spontaneous brain waves for the taste
processing model. In these experiments, a tendency ofα wave decrease and β wave increase was observed fortasks with taste information only in comparison withtasks where taste information of water only waspresented under “4.1 Taste and electroencephalographic
response.” Based on these results, the vision-taste
processing model was developed. It is possible toexplain from this model that integrated processing is
conducted in the interaction of vision and taste. We
hope to carry out statistical review by increasing thenumber of subjects as well as the number of trials in the
future.

V. DISCUSSIONS
Based on the results of experiments at this time,visual information as well as specific work in integrated
processing is added to the taste processing model(Fig.1).
The model under Hypotheses 1was indicated in thetaste-vision processing model in Fig.7. The decreaseof α waves and increase of β waves can be explainedwith the increase in the volume of thoughts at the timeof screening taste. Based on this, the model was
restructured.

©ISAROB 2011

memory retention

Screening

Taste
informationfrom visual
information to
taste
information

Discrimination

Visualinformation0
タスク6（オ、オ）
Task6(O, O)

Water

Fig.7: Restructured taste-vision processing model (inthe case of Hypothesis 1)

θ
α
β

30

Orange

time

50
40

Strawberry

BananaButton pushing

Taste
information

memory retention
Pear

Apple

Screening

Visualinformation

Discrimination

Integration

77

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

We had assumed that β waves of brain waves
covered at this time increase/decrease as a result ofrelating taste to memory; however a significant
difference was not recognized in any task. We will
confirm whether or not a significant difference isrecognized by calculating brain waves with the content
rate. Higher brain activities might not be occurring
when taste is obtained if there is no difference in β
waves; therefore review is desired. In this paper, amodel was developed by focusing on vision and taste
and conducting experiments. It is needless to say thatwhen an object is tasted, not only vision and taste but
also smell is an important sense, and we plan to review
the influence of smell on the brain in our future studies.

REFERENCES[1] Masamitsu Okumura, Kunihito Kato, Kazuhiko
Yamamoto: “Study on Change of Facial Expression
with Taste Stimulation,” The Institute of Image
Information and Television Engineers Technical Report32(35), 53-56, 2008/8[2]Toru Furuichi, Aoki Nozawa, Hideo Ide:“Comparison between Amount of Gustatory Sense
and Nasal Skin Temperature,” IEICE Technical Report
MBE2006-50, 2006/09[3] Kiyota H, Morita K, Murayama N, “Evoked
Potential to Gustatory Stimuli in Man,” TECHNICALREPORT OF IEICE MBE98-120, 1999/01[4] Hiroaki Uematsu, Kentaro Hirata, Hiroaki Morino,
Sanae Takasaki, Yasuo Morodomi, Michiaki Mito:
Analysis of Emotional Response to Visual Stimulation
with β Wave Mapping,” IEICE Technical Research
Report MBE, ME and Bio Cybernetics, 103(637), 65-70,
2004/01[5]
Yuichi Sato, Hisaya Tanaka: Measurement of
Spontaneous Brain Waves to Develop Taste Processing
Model,” The Institute of Electrical Engineers of Japan
Research Report 2009/12©ISAROB 201178

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Dynamic analysis of dorsal thermal image
Akio Nozawa and Yuya Takei
School of Science and Engineering, Meisei University, Tokyo, Japan(Tel: +81-42-591-5895; Fax: +81-42-591-5895)(Email akio@ee.meisei-u.ac.jp)

Abstract: A dynamic analysis was subjected to thermal images of dorsal of the foot in this study. A
psychophysiological effect of a facial massage by aesthetician was evaluated. First, psychophysiological effectsof facial massage were assessed on proprietary stress test. Physiological indices measured were alpha-wave
power spectrum, dorsal skin temperature variations and high frequency component of heart rate variability. STAI,POMS (Brief Form) and amount of sensory awareness was administered to evaluate for psychological status. The
aspects of the amount of sensory awareness were comfortable, awakening and effect of massage. Secondary, we
assessed stress response on thermal image of dorsal of foot. Thermal image of dorsal of foot was measured byinfrared thermography device.
Keywords: Thermal image, Peripheral skin temperature, Dynamic analysis, Comfort evaluation, Physiology meas
urement, Stress test

study a dynamic analysis was subjected to thermal
images of dorsal of the foot. A psychophysiologicaleffect of a facial massage by aesthetician was evaluated.
First, psychophysiological effects of facial massage
were assessed on proprietary stress test. Physiologicalindices measured were alpha-wave power spectrum,
dorsal skin temperature variations and high frequency
component of heart rate variability. STAI, POMS (Brief
Form) and amount of sensory awareness wasadministered to evaluate for psychological status. The
aspects of the amount of sensory awareness werecomfortable, awakening and effect of massage.
Secondary, we assessed stress response on thermal
image of dorsal of foot. Thermal image of dorsal of footwas measured by infrared thermography device

I. INTRODUCTION
Comfort evaluation has been performed by sensory
evaluation method or physiology index. Sensory
evaluation method has been widely used toquantitatively evaluate the tendency of preference andfeeling of the user [1-5]. Though the sensory evaluation
method is easy-to-use, possible issues regarding
examinee’s individual variability on interpretation ofevaluation words have been indicated; a temper of
examinee affects the evaluation [6-8]. And unconscious
mental stress isn’t able to evaluate. Physiological index
is provided by bioinstrumentation, which is measured
by person. Physiological index is changed byphysiology condition and mind condition. Physiology
index is able to evaluate serial quantitative andobjective in comfort evaluation. However, corporeity onelectrode measurement such as electroencephalogram(EEG) and electrocardiogram (ECG) may give physicaland mental stress. Noncontact and unconfined andnoninvasive measurement can be measured by infrared
thermography device. Measurement with infrared
thermography device will be projected to reduce
physical and mental stress. Peripheral of body is nasal,
hand, foot and such. Peripheral skin temperature (PST)
is regulated by the sympathetic nervous system. We
have evaluated phychophysiological condition such as
mental stress evaluation and comfort evaluation by
peripheral skin temperature. Peripheral skin temperature
was measured by infrared thermography device. In this©ISAROB 2011

II. EXPERIMENTAL
Experimental equipment set-up and electrodearrangement for scalp EEG are shown in figure 1.
Experiments were executed in a measurement room. An
infraredthermographysystem(TVS–200EX,
AVIONICS) was installed 1 m in front of subject.
Thermograms of dorsal of foot were created with 1-s
sampling periods. Image resolution of thermograms was
320×240 pixels, and room temperature was set at26±1.0 degrees Celsius. Infrared emissivity of skin is_=0.98. The subject was in a seated position in a restingstate. EEG was recorded at a sampling frequency of 200
Hz using a biological amplifier/sampler (5102 EEG
HEAD BOX, NF Electronic Instruments) and digital

79

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table.1. Overall evaluation resultsof indices in facial massage.
Interpretation

Significanceα-wave
PST
HF
POMS
STAI
VAS #1
VAS #2Brain activity

n.s.
n.s.
P**
N+
P+
P**
N+

Sympathetic acceleration
Parasympathetic acceleration

Depressive moods
Trait anxiety
Comfortable
Arousallead from Pz was calculated every 5 s by FFT using1024 data points at a sampling frequency of 200 Hz.
The -waves were defined as the frequency componentsof the EEG in the frequency range from 8 Hz to 13 Hz.
The power of α-wave was created as the sum of thefrequency components in the frequency range.
The PST time series was calculated as follows.
Thermal images of the right first toe region of the footwere extracted from the thermogram of dorsal of foottime series. PST time series were created as a cascade of
spatial average temperature for pixels in the interested
area of each thermal image.
HF time series was calculated as follows. A source
R-wave interval time series was extracted from ECG
time series by using threshold processing. Temporally
equidistant R-wave intervals (HRV) were derived byresampling process in frequency of 20 Hz after cubic
spline interpolation. The power spectrum time series of
HRV was calculated every 1 s by fast Fouriertransformation (FFT) using 512 data points at asampling frequency of 20 Hz. Finally, HF time series
was created as a summation of discrete frequency
components in power spectrum time series of HRV, inwhich the frequency range of HF was 0.15 Hz to 0.4 Hz.
POMS-short form, which had been shortened andtranslated into Japanese, was administered. The POMSshort form comprised of 30 questions about the current
mood state. These 30 questions were classified in 6 subscales: T-A (‘tension and anxiety’), D (‘depression anddejection’), A-H (‘anger and hostility’), V (‘vigor’), F(‘fatigue’), and C (‘confusion’). The subjects selected
the raw score from one of five values (0, 1, 2, 3 and 4,
where 0 = no such mood state and 4 = extreme moodstate). These raw scores in each sub-scale were thenadded to generate each sub-scale score.

Fig.1. The measurement systemand electrode arrangement for EEG.

Fig.2. Protocol of the experiment.
signal processor unit (5101 PROCESSOR BOX, NF
Electronic Instruments). Electrodes used for scalp EEG
was Pz, based on the international 10-20 method, and areference electrode was A1. Electrodes used for ECG
were put on the superior margin of the sternum and the
cardiac apex, based on modified Lead NASA, in orderto reduce the artifacts of EMG from the ECG. A
common electrode for both EEG and the ECG was put
on Cz. An auditory stimulus of scratch sound was
reproduced by PC as a control stimulus. The protocol ofthe experiment is shown in figure 2. Subjects were
nineteen 18- to 23-year-old healthy women. Subjects
were well-informed about the experiments and theobjective of this study before participation forexperiments. The experiment was performed once par a
subject. The measurement was not begun until thesubject had been in the room for at least 15 minutes inorder to habituate to the room temperature. The
measurement consisted of eye-closed resting periods(period R1, R2 and R3), eye-opened evaluation periods
for evaluation of sensory awareness (period V1, V2 andV3), eye-opened period under the controlled stimulationof the mental work task (MAT) and eye-opened period
under the facial massage operation (period TEST).
The power spectra time series were calculated asfollows. The power spectrum time series of the EEG©ISAROB 2011

Indices80

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

The STAI is a 20-item scale that measures acutelevel of anxiety. The subjects selected the raw scorefrom one of four values (1, 2, 3, and 4, where 1 = not at
all and 4 = very much). A summary score is obtained byadding the weight of each item. The STAI scoresindicate an increase in response to situational stress anda decline under relaxing conditions. In this study, STAIJYZ, which regards Japanese cultural factors better, wasused. The STAI-JYZ exhibits acceptable internalconsistency and test-retest reliability. The aspects of theamount of sensory awareness were comfortable,
awakening and effect of massage. Each data wasexpressed as ‘means ± standard error (SE)’. Wilcoxon
signed-rank test was performed to evaluate them.

III. RESULTS AND DISCUSSION
1. Psychophysiological effects of facial massage
Table.1 shows the result of the statistical test
representing the psychophysiological effect of the facial
massage. "P" means positive response for theinterpretation of each index, and "N" means negative
response. Each physiological index was normalized byperiod R2. R2 was a baseline for the statistical test andwas compared with period R3 by the test. -wave ratio
was 0.067±0.054 which indicates that a brain activity
had been maintained by having performed a facial
massage. PST ratio was 0.631±0.490 which represented
that had no significant changes in the massage. It isconsidered that this is because of rather large standard
error between subjects. HF ratio was 1.406±0.094 andincreased remarkably after the massage (p<0.01). Thisshows that the parasympathetic activity was enhancedby the facial massage. POMS score was compared ineach scale before and after experiment. Most scores,
which indicate negative feeling, tended to decline after
the experiment. Especially, T-A (‘tension and anxiety’)significantly declined from 42.68±1.747 to 39.58±1.294(p<0.05), A-H (‘anger and hostility’) dropped
38.05±0.609 to 37.16±0.158 (p<0.1), and also F(‘fatigue’) was decreased 40.47±1.035 to 37.32±0.769(p<0.05). Similarly, the score for anxiety in the STAI
significantly declined from 43.79±1.829 to 40.32±2.013
following the massage (p<0.1). Amount of sensory
awareness was compared before and after TEST.
‘Comfortable’ gathered from 0.503±0.015 to
0.621±0.029 (p<0.01). ‘Awakening’ declined from0.563±0.033 to 0.517±0.045 (p<0.1). The ‘Anxiety’ in
the STAI and negative moods in POMS were

©ISAROB 2011

Fig.3. Thermal image of dorsal of right footFig.4. A temperature change in each area.
significantly decreased following the massage. These
results suggested that the facial massage had strong
effects on stress alleviation or psychological relaxation.

81

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

from the maximum to the minimum. In the trail, acenter of gravity was in the part of the ankle at the
beginning. The ankle has little variations in temperature
which is shown in figure 3. And the temperature of theankle tends to be high. The center of gravity of theinterested area moved towards a tiptoe in the middle ofperiod TEST.

IV. CONCLUSION
In this study, an activity of the autonomic nervous
system was evaluated by dynamic analysis in peripheral
skin temperature which shows activity of the autonomic
nervous systems. Peripheral skin temperature in thedorsal of the foot was measured by infrared
thermography. A facial massage was used to promote
comfort. The rise of peripheral skin temperature by the
inhibition of the sympathetic nerve activity was foundin the area of tiptoe. Variations in temperature weregreatest in the area of tiptoe. A similar change intemperature was seen in the area except the ankle.

V. REFERENCES

Fig.5. A trail of the center of gravity of
the interested area in PST.

[1] Parente, M. E., Gambaro, A. and Ares, G.: Sensory Characterization of Emollients. J. Sens. Stud. 2
3(2): 149-161, 2008[2] Stern, P., Valentova, H. and Pokorny, J.: Relations between Rheological and Sensory Characteristics
of Cosmetic Emulsions. Seifen-oele-Fette-Wachse 1
23(7): 445-448, 1997
[3] Tanaka, Y. and Sukigara, S.: Evaluation of ”Shittori” Characteristic for Fabrics. J. Text. Eng. 54(3):75–81, 2008[4] Senoo, M., Takemoto, Y. and Jingu, H.: Change
in Affections by Continuous Use of Skincare Cos
metics. KANSEI Eng. Int. 3(3/4): 31–36, 2002
[5] Armanini, L. and Aucar, B.: Tactile Evaluationsof Cosmetic Ingredients. Seifen-Oele-Fette-Wachse
118(20): 1247–1254, 1992
[6] Kusakari, K., Yoshida, M., Matsuzaki, F., Yanaki, T., Fukui, H. and Date, M.: valuation of postappl
ication rheological changes in cosmetics using
a novel measuring device: Relationship to sensory
evaluation. J. Cosmet. Sci. 54(4): 321–333, 2003
[7] Dykes, P. J.: What are meters measuring? Int. J.
Cosmet. Sci. 24(4): 241–245, 2002
[8] Wang, S., Kislalioglu, M. S. and Breuer,M.: The effect of rheological properties of experimental moisturizing creams/lotions on their efficacy and perc
eptual attributes. Int. J. Cosmet. Sci. 21(3): 167–18
8, 1999

2. Dynamic analysis of dorsal thermal image
Thermal image of the measured dorsum of the rightfoot is shown in Fig.3. Images were recorded at 300 s,
330 s, 390 s, 450 s, 510 s, 570 s, 630 s, 690 s, 690 s and720 s each. The ankle, which is not peripheral part, hadsmall variations in temperature, while the temperaturein tiptoe changed rather large. These variations intemperature were analyzed in detail. Thermal images ofthe dorsum of the right foot were divided into squared
areas of 9 x 14 pixels each. Fig.4 shows time evolutionof variations in temperature of each divided area. Large
variations in temperature were shown in the areas of
tiptoe, while there were a small temperature changes in
the areas of ankle. It was indicated that the variations intemperature were different in parts. However, similarchanges in temperature were seen generally. And thetime in which has greatest value in temperature of eacharea was different. Fig.5 indicates a trail of the center ofgravity of focused area of a certain temperature in range
from period R2 to period R3. The range in temperature
was defined as follows. Highest temperature wastracked in each area from period R2 to period R3. The
maximum and the minimum values were collected fromthe tracked highest temperatures of each area. Then, therange in temperature was defined as rank in the top 20%©ISAROB 201182

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Feature extraction of human face image for preference database
Yu Tachikawa and Akio Nozawa
Graduate School of Meisei University(Tel: 81-42-591-5895; Fax: 81-42-591-5895)(you@biel.ee.meisei-u.ac.jp)

Abstract: In this study, aimed for a preference judgment for the face image of the person by the single trial ERP.
Cognitive task imposed on subjects with human faces. At the time of the image presentation of the key push it down,
and impose a problem the time response time (RT).200ms to 400ms after image presentation time of peak positive
potentials appear focused ± 100ms. Calculated from the maximum amplitude ± 100ms latency and area of potential.
P300 is analyzed to extract frequency components, 1.28 seconds for ERP 2-3Hz amplitude spectra obtained.1 Analyzed
EEG frequency components appear to recognize the human face image and asked the potential spectrum. Spectral
correlation potential in the area and potential. Correlated with latency and RT. Single-trial ERP waveforms wereextracted correctly.
Keywords: ERP, P300, preference, face image, feature extraction.
infraredthermographsystem(TVS-200EX,
AVIONICS) was installed 0.7m in front of subject.
Facial skin thermograms were created with 1s sampling
periods. Image resolution of thermograms was
320×240 pixels, and room temperature was set at 26 ±1.0 degrees Celsius. a seated position in a resting. An
electrode headpiece (Pasteless Electrode Helmet, Brain
Function Laboratory) and set of headphones wereplaced on the subject. EEG was recorded at a sampling
frequency of 200 Hz using a biological plifier/sampler(5102 EEG HEAD BOX, NF Electronic Instruments)
and digital signal processor unit (5101 PROCESSOR
BOX, NF Electronic Instruments). Electrodes used for
scalp for scalp EEG were Pz, based on the international
10-20 method, and a reference electrode was A1.
Electrodes used for ECG were put on the superior
margin of the sternum and cardiac apex based on amodified Lead NASA in order to reduce artifacts of
EEG from ECG. A common ground electrode for both
EEG and ECG was put on Cz. EEG was recorded at asampling frequency of 200 Hz using a biologicalamplifier/sampler (5102 EEG Head Box, NF Electronic
Instruments) and digital signal processor unit (5101
Processor Box, NF Electronic Instruments).The protocol
for the experiment is shown in figure2. Subject was 22
year-old healthy men. Subject was well informed about
the experiments and the objective of this study before
participation experiments. The day of experiment could
be before or after the day scheduled. The measurement
was not begun until the subject had been in the room forat least 15 minutes to habituate to the room temperature.
In this study, subjects performed discrimination task of
the face images. The subject carried out a discrimination

I. INTRODUCTION
When a human being runs a social life, ability toread the will of another person, and to take theappropriate behavior is very important as towards otherpeoplecommunicativecompetence.
Emotional
intelligence EQ is famous in recent year[1]. ”KANSEI”
is socialized in the same way of frequent occurrencefor ”Human being serious consideration” and ”KANSEIsociety”. Communication is transmission of ”KANSEI”.
If ”KANSEI infomation” has gotten across to machine ,It should have gone of smooth communication between
person and person. Preference is one of the factors ofhuman action. KANSEI information communication
technology is expected of used to human preference. It
will be the application as quite new communications
tool. Event Related Potential (ERP) is caused cerebral
evoked potential reaction of the mental work of thesubjects [2]. Preference is involved with attention and acognitive level. There have been various investigationsof preference to the event [3]. ERP was evaluatedsensitivity information about the preference. However,face images were not evaluated. That was aimed at
estimating the correlation of the face image aspreference because quantity of characteristic of the face
image was extracted.

II. EXPEROMENTAL
Experimental equipment setup and electrodearrangement for scalp EEG are shown in figure1.
Experiments were executed in a measurement booth
divided with partition walls of 1.8m in height. An©ISAROB 201183

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

task to distinguish images shown every 0.5 second in
the measurement. The images consist of the faces of the
man and woman. The target image is told to a subject
before the measurement. The target image wasdisplayed on the display of the PC. Subjects was
instructed to push the key as quicklyas possible.
Response time (RT) was defined as the time of pushingdown the key on presenting stimulation. The
presentation frequency of a target image and the nontarget image follows (Non-target) : (Target) = 1:1. In
addition, subjects carried out POMS (short form) andSTAI-JYZ and VAS (pleasant - unpleasant) before andafter the measurement pleasant. Afteran experiment,
preference of each image was measured by VAS. The
measurement was not begun until the subject had beenin the room for at least 15 minutes to habituate to theroom temperature. The experiment was conductedduring the day except with in 2 hours after eating, a waslimited to be conducted once daily for each subject. And
an oral explanation about experiment contents, purpose
and investigation object were given by a document, andthe subjects confirmed an agreement for the experimentcooperation by a signature. The ERP is the reaction thatis caused when a percept. A component of ERP called
P300 is related to the cognition of stimulus.P300 canrepresent reaction corresponding to the discriminationof the stimulus quantitatively. The ERP is expresseddefinitely so that discrimination of the stimulation is
performed easily. ingredient called P300 wave related to
the recognition develops, and ERP are shown in figure 3.
An areal voltage and latency of the ERP were focused toevaluated quantity of the preference for the face image.
In this study, P300 was defined as 300ms the positive
peak potential in range of 300±100ms. It was reported
that P300 and preference have positive correlation.
Positive areal potential in range of±100ms of thelatency of the peak was calculated. Areal voltage andlatency of the peak are assumed from the time when
showed the positive peak of±100ms.Preference for the
stimulation of the subjects have been assessed withareal voltage and latency of the peak. Calculated fromthe maximum amplitude ± 100ms latency and area of
potential. Areal Voltage and Amplitude spectrum areshown are shown in figure4. P300 is analyzed to
extract frequency components, 1.28 seconds for ERP 17Hz amplitude spectra obtained. ERP is reaction of
induced by mental work to impose on task of subject.

©ISAROB 2011

Fig.1. Measurement system and EEG electrodearrangement

Fig.2. Experiment protocol

Fig.3. Latency and Areal Voltage of P300
600

Latency(ms)Amplitude spectrum (uV)

18

12

6

400

200

0

0
0

60

Areal Voltage(ms・μV)

120

0

200

400

600

RT(ms)

Fig.4. Areal Voltage and Amplitude spectrum

84

800

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

the mouth with a standard value for a standard price at
the interval of the eyes of right and left. The correlationof the psychology index was estimated as the numericalvalue extracted

An ingredient called P300 wave related to therecognition develops, and ERP for the discrimination
problem by the sight stimulation can arrest reaction
corresponding to the discrimination of the stimulation
quantitatively. ERP is developing that a difference of thestimulation is big. In this study, preference of subject
was measured based on the areal voltage and latency of
ERP. Furthermore, P300 was extracted by frequency
analysis. Characteristic of an ERP pattern provided fromtarget stimulation, expression of the positive electric
potential is accepted as of 300ms by the stimulation
presentation and is called P300 wave generally. Analysisof the ERP,
evaluation are usually made by
extracting the amplitude of P300, an area, quantity ofcharacteristic such as the top latency. The ingredientrelated to discrimination should be included during the
response button exercise start time to push it down fromthe time of the sensory receptor excitement by thepresentation stimulation. In addition, the evoked
potential should develop for positive electric potential
because P300 wave affects it in one way or another forcognitive process. It is quantity of characteristic of the
ERP to reflect these two businesses and refers to thisparameter. Method of evaluating on brain waves isreading of P300 of top latency of positive electric
potential 250ms-500ms interval. About ERP and theconnection of the preference, the amplitude value andan amplitude spectrum and correlation of the preferenceof P300 were reported [4]. Imitated this in this study,
attention to ± 50ms of the top latency of the positiveelectric potential of the single trial ERP wave pattern, an
amplitude spectrum is demanded of 1-7Hz from thetime when the positive greatest amplitude for theamplitude value and the ERP of 1.28 second time ofP300 in the section of ± 50ms was showed. The
amplitude value and an amplitude spectrum wereassumed on evaluation value. Preference for stimulationof the subject was evaluated. The amplitude value,
amplitude spectrum, RT and latency relations is shownin figure 3. Than a precedent study, the preference has
relation to an electric potential area of the amplitude
value of P300 for latency. Than a precedent study, thepreference has relation to an electric potential area of
the amplitude value of P300 for latency. Even this study
imitates, the preference as latency and an electric
potential area of P300 were estimated. An expression asa lineament was extracted. The extraction method of theexpression compares eyebrows width and the width of©ISAROB 2011

III. RESULTS AND DISCUSSION
An electric potential area when the face image of the
man a target assumed and an amplitude spectrum and
RT and correlation of the latency are shown in figure
5.Equilateral correlation (r=0.406,n=50, *p <0.05) was
accepted between an electric potential area andamplitude spectra in the case of a man a target.It is
supposed that an electric potential area and extraction of
amplitude spectrum P300 wave based for top latency
were made from P300 wave between 250-500ms. As aresult, an electric potential area and an amplitude
spectrum for an index were used when correlation of thepreference was judged. Because the electric potential
area was based for top latency, latency was used for an
evaluation index of the preference equally.Latency andpreference on correlation when the face image wasmade of the man a target are shown in figure 6.Negative
correlation (r=-0.271, n=50, *p <0.05) was accepted
between a target liking it in the case of a man withlatency. When a subject discriminated the image of thewoman from it, as for meaningful correlation having
come out to the face image of the man than a woman, itis thought that taste did not show a remarkabledifference because an object is the opposite sex.A target
used it for an index of the image extraction as a result ofabove more because the taste in case of the man was
based on ERP. Eyebrows width and the width of themouth and the ratio of the interval of both eyes areshown in figure 7.Eyebrows width and the ratio and thetaste of the interval of eyes did not have the correlation.
The width of the mouth and the ratio and the taste of theinterval of eyes did not have the correlation.The graph
was divided which was into top and bottom 15 items of
the polarity of the preference valuated are shown infigure 8.Correlation (r=0.418, n=15, *p) equilateral
between the width of the mouth of the image which
taste was reflected on weakly and the ratio with theinterval of eyes and taste > 0.05) was accepted.If thewidth of the mouth was wide, for the face image of theman, it followed that it was favorable for the image.As
for the width of the mouth being wide, the subject takes
the expression of the person as "a smile". Therefore it

85

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 201120

Latency(ms)Amplitude spectrum (uV)followed that a subject had a favorable impression froman image.

600

25

15
10

400

VI. CONCLUSION
200

5In this study, the preference for the face ofthe
subject was evaluated. Brain waves and the correlation of the psychology index were accepted when u
sed ERP. An expression was extracted for quantityof characteristic of the image. width of the mouth
was used for quantity of characteristic of the expre
ssion. As a result, correlation was accepted by area
and the taste of the width of the mouth.

0

0
0

60

120

0

180

200

Areal Voltage(ms・μV)

400

600

800

RT(ms)

Fig.5. Areal Voltage and Amplitude spectrum

800

REFERENCES

Preference

600[1]John D. Mayer, Peter Salovey, David R.Caruso :“Emotional Intelligence : Thaeory, Finding, and I
mpliications,” Psychological Inquiry, Vol15, No3, pp.197– 215(2004)[2]Y. Nageishi and Shimokochi: “Endogenous com
ponents of Event Related Potentials (ERP) and principai-component analysis),” Clinical brain waves，Vol.26, No.10, pp.623– 628(1984)[3] Mariko FUNADA, Miki SHIBUKAWA, Satoki P.
NINOMIYA: “Objective Measurement of Event Re
lated Potentials‘Changes,” Ergonomics,Vol38,pp.53
8– 539(2002)[4] Kazuya TANOUE, Nobuki MURAYAMA, Tomohiko IGASAKI,RyujiESHIGE,andYukariNOGUCHI:”exmination of simulus methods and frequency analys
is for Communication by using EEG”Science Citati
on Index. MBE,Soceiety fir Medical and BiologicalEngineering 2004(1), 83-86, 2005-01-30

400

200

0
0

200

400

600

800

Latency(ms)Fig.6. Latency and Preference

Fig.7. Preference and Width of the eye brow
and mouth

Fig.8. Preference and Width of the eye brow
and mouth©ISAROB 201186

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Observer-based Guaranteed Cost Control
Erwin Susanto1 , Mitsuaki Ishitobi2 and Sadaaki Kunimatsu3
1,2,3
Department of Mechanical Systems Engineering
Kumamoto University, Kumamoto 860-8555
Tel: +81-96-342-3866
E-mail: 1 095d9213@st.kumamoto-u.ac.jp, 2 mishi@kumamoto-u.ac.jp, 3 kunimatu@mech.kumamoto-u.ac.jp

Abstract: This paper presents a design scheme of a minimal order observer-based guaranteed cost controller
for uncertain linear systems. The perturbations are assumed to be described by structural uncertainties. An
iterative linear matrix inequality (ILMI) approach is used to design the observer-based controller since theproblems contain inverse relations. We modify the algorithm of Matsunaga et al by optimizing a suﬃciently
large initial guaranteed cost. This method can be implemented by LMI control toolbox of Matlab. Finally,
a numerical example is given to illustrate the eﬀectiveness of the proposed method.
Keywords: robust control, guaranteed cost control, minimal order observer

I. INTRODUCTION

with
FAT (t)FA (t) ≤ I, FBT (t)FB (t) ≤ I

Considerable attention to the problem in robust
stability analysis and robust stabilization of uncertain systems has been attracting many authors forseveral last decades. One approach to this problem is the guaranteed cost control method which
not only achieves the stability of the uncertain system but also guarantees an adequate level of performance via linear matrix inequality (LMI) techniques (Lien [2], Won and Park [3]).
Although the controller is usually constructedby using state variables, it may not be possible tomeasure all the states of the system in many cases.
The observer-based control is probably well suitedand better than the state control feedback in suchsituations.
Since inverse relations among variables appear,
this paper concerns a design method of a minimalorder observer-based guaranteed cost controller via
an iterative linear matrix inequality (ILMI) technique under an assumption that the statisticalproperties of the initial state variables are known.

where DA , DB , EA , EB are constant real-valued
known matrices with appropriate dimensions, andFA (t) and FB (t) are real time-varying unknown
continuous and deterministic matrices.
We further assume that the initial state variablex(0) is unknown, but their mean and covarianceare known
E [x(0)] = m0
(4)
[
]
T
E (x(0) − m0 )(x(0) − m0 ) = Σ0 > O (5)where E [·] denotes the expected value operator.
The problem considered here is to design a minimal order observerż(t) = Dz(t) + Ey(t) + F u(t)x̂(t) = P z(t) + W y(t)and a controlleru(t) = K x̂(t)D = A11 + LA21 , P T + W C = In ,[]A11 A12
F = T B, T A − DT = EC, A =,A21 A22[]T[]
P = In−m 0 , T = In−m L

Consider a continuous-time uncertain systemẋ(t) = (A + ∆A(t))x(t) + (B + ∆B(t))u(t) (1)(2)

where x(t) ∈ ℜn is the state vector, u(t) ∈ ℜr isthe control input vector, y(t) ∈ ℜm is the measured output vector, A, B, C are known constantreal-valued matrices with appropriate dimensions,and C is restricted to the form of C = [O Im ].
Matrices ∆A(t) and ∆B(t) denote real-valued matrix functions representing parameter uncertainties. It is assumed thatso as to achieve an upper bound on the following
quadratic performance index[∫ ∞]E [J] = E
(xT (t)Qx(t) + uT (t)Ru(t))dt (9)0associated with the uncertain system (1) and (2)where Q and R are given symmetric positivedeﬁnite matrices.

∆A(t) = DA FA (t)EA , ∆B(t) = DB FB (t)EB (3)©ISAROB 2011

(8)

with

II. PROBLEM STATEMENT

y(t) = Cx(t)

(6)
(7)

87

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

1 T
1/2
(Y Σ12 + ΣT12 Y ), Σ22 = [v 1 , v 2 , · · · , v m ]
2
[
]
[
]T
Σ11 Σ12Σ0 =, eik = 0Tk−1 1 0Ti−kΣ21 Σ22

III. MAIN RESULTS

Θ3 =In this section, a suﬃcient condition is established for the existence of a minimal orderobserver-based guaranteed cost controller for theuncertain system (1) and (2). Here, it is assumed
that the feedback gain matrix is
K = −R−1 B T S1

has a solution S1 > 0, S2 > 0, X > 0, Y , Z, ζ > 0,
δ > 0, ϵ > 0, ϵinv > 0, θ > 0, θinv > 0, µ > 0,µinv > 0, νinv > 0, γ0 , γ1 , γ2 , γ3 , γ4 which satisfy
the relation ϵ−1 = ϵinv , θ−1 = θinv , µ−1 = µinv
and S1−1 = X, then the minimal order observerbased control law (6)-(8) with (10) is a guaranteed
cost controller which gives the minimum expectedvalue of the guaranteed cost[]
E [J ∗ ] = E xT (0)S1 x(0) + ξ T (0)S2 ξ(0) (15)(10)

where S1 is a symmetric positive-deﬁnite matrix.
The main result of this study is given by Theorem 1.
Theorem 1. If the following matrix inequalities optimization problem; min {γ0 + γ1 + γ2 + γ3 + γ4 }subject toTTΛ0 XEAXEA
XT
 EA X −ζI
0
0 

<0
 EA X 0 −θI
0 X
00 −Q−1Λ̄0 GT1

GT1

GT2

GT3

GT3

GT4



where ξ(t) = z(t) − T x(t) is the estimated error of
the minimal order observer.
Remark 1: Since (11) and (12) have a constraintof the relationship of the inverse, ILMI approachis introduced to solve (Ghaoui et al [4], Cao et al[5]).
Before giving a proof of Theorem 1, a key lemma
is introduced (Mahmoud and Zribi [6]).
Lemma 1. Let D and E be matrices of appropriate
dimensions, and F be a matrix function satisfying
F T F ≤ I. Then for any positive scalar α, thefollowing inequality holds(11) G1 −δI 0
0
0
0
0 


 G1 0 −µI
0
0
0
0 


 G2 00 −θinv I
0
0
0 
<0


 G3 0
0
0
−ν
I
0
0
inv


 G3 0000−µinv I 0 
G4 0
0
0
0
0
−R

DF E + E T F T E T ≤ αDDT + α−1 E T E. (16)Proof of Theorem 1.
Equations (1) and (6)-(8) yield the closed-loop system
[
] [
][
]
ẋ(t)
Φ1 Φ2
x(t)
=
(17)
Φ3 Φ4
ξ(t)
ξ̇(t)

(12)
n
∑

eTnk Θ0 enk < γ0 ,

k=1
m
∑

eTmk Θ2 emk < γ2 ,

k=1

m
∑
k=1
m
∑

eTmk Θ1 emk < γ1

whereeTmk Θ3 emk < γ3 (13)

Φ1 = A + ∆A(t) + (B + ∆B(t))K

k=1

Φ2 = (B + ∆B(t))KP
Φ3 = −T ∆A(t) − T ∆B(t)K
Φ4 = D − T ∆B(t)KP−γ4 v T1 Y T v T2 Y T · · · v Tm Y T..  Y v 1 −S2
. 



..  < 0..
 Y v2
.
. 


 .
.. 
.
..
 ..
. 
Y vm · · ·· · · · · · −S2

Deﬁne a candidate of Lyapunov function as(14)

V (t) = xT (t)S1 x(t) + ξT (t)S2 ξ(t)then, the time derivative of (18) along to (17) is
calculated asV̇ (t) = wT (t)Ωw(t) − (xT (t)Qx(t) + uT (t)Ru(t))

where
T
T
Λ0 = AX + XAT − BR−1 B T + ζDA DA+ ϵDB DBTT
+ϵinv BR−1 EB
EB R−1 B T + δDB DB
−1 T
−1 T
+νinv BR EB EB R B

Λ̄0 = S2 A11 + AT11 S2 + Y A21 + AT21 Y T
Y = S2 L, Z = [S2 Y ] ,

(19)
where

[

]
[
]
Λ1 Λ2
x(t)
w(t) =, Ω=ξ(t)ΛT2 Λ3

Λ1 = S1 (A + ∆A(t)) + (A + ∆A(t))T S1−S1 BR−1 B T S1 + Q − 2S1 ∆B(t)R−1 B T S1

T T
G1 = EB R−1 B T S1 P, G2 = DAZ
T T
T
G3 = DB Z , G4 = B S1 P
1
Θ0 = (S1 (Σ0 + m0 mT0 ) + (Σ0 + m0 mT0 )T S1 )211
Θ1 = (S2 Σ11 + Σ11 S2 ), Θ2 = (Y Σ21 + ΣT21 Y T )2
2©ISAROB 2011(18)

Λ2 = −S1 ∆B(t)R−1 B T S1 P − ∆AT (t)T T S2+S1 BR−1 ∆B T (t)T T S2
Λ3 = S2 D + DT S2 + P T S1 BR−1 B T S1 P
+2S2 T ∆B(t)R−1 B T S1 P

88

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Under the condition

where
Ω<0

[

Σ11 Σ12
Σ0 =Σ21 Σ22(20)]

equation (19) leads toHere, we consider positive scalars γ0 , γ1 , γ2 , γ3 , γ4satisfying the following inequalitiesV̇ (t) < −(xT (t)Qx(t) + uT (t)Ru(t)) < 0 (21)for any x(t) ̸= 0 and the closed-loop system is
asymptotically stable.
Applying lemma 1., pre- and post-multiplying
by diag(S1−1 , I) on both sides, denoting X = S1−1 ,Y = S2 L, ϵinv = ϵ−1 , θinv = θ−1 , µinv = µ−1 ,νinv = ν −1 , and using Schur Complement lead to(11) and (12).
Then, integrating (21) from 0 to T and as T
tends to the inﬁnity yields∫ ∞J=(xT (t)Qx(t) + uT (t)Ru(t))dt0T< x (0)S1 x(0) + ξ T (0)S2 ξ(0) = J ∗trS1 (Σ0 + m0 mT0 ) < γ0
trS2 Σ11 < γ1
trS2 LΣ21 < γ2

(30)
(31)
(32)

trS2 Σ12 LT < γ3
trS2 LΣ22 LT < γ4

(33)
(34)

Minimizing γ0 + γ1 + γ2 + γ3 + γ4 results in giving min E [J ∗ ]. By recalling tr(AB) = tr(BA),1/2(30)-(33) lead to (13). Next, by denoting Σ22 =[v 1 , v 2 , · · · , v m ], (34) is calculated as(22)

trS2 LΣ22 LT∗= v T1 Y T S2−1 Y v 1 + v T2 Y T S2−1 Y v 2+ · · · + v Tm Y T S2−1 Y v m

where J denotes the guaranteed cost. Here, we
consider the optimal expected value of the guaranteed cost. It is calculated as[][]
E [J ∗ ] = trS1 E x(0)xT (0) + trS2 E ξ(0)ξ T (0)Y v1
 Y v2 ][= v T1 Y T v T1 Y T · · · v T1 Y T S2−1  .  < γ4
 .. 

(23)

Y vm
A relation between mean and covarience of x(0) is
given by[](24)
Σ0 = E x(0)xT (0) − m0 mT0(35)
Further, Schur complement derives (14) from (35).
Q.E.D.
It is noted that the inequalities (11) and (12)cannot be solved directly by LMI because they contain the scalars ϵ, ϵinv , θ, θinv , µ, µinv , and two
matrices S1 , X which satisfy the relation S1−1 = X,
ϵ−1 = ϵinv , θ−1 = θinv , µ−1 = µinv . There are anumber of algorithms available in literature, andwe apply the cone complementarity linearization
approach (Ghaoui et al [4]) to propose the algorithm as follows.

Substituting (24) into (23) yields
E [J ∗ ] = trS1 (Σ0 + m0 mT0 )[]+trS2 E (z(0) − T x(0))(z(0) − T x(0))T(25)
Here, it is readily seen that[]E (z(0) − T x(0))(z(0) − T x(0))T= T Σ0 T T + (z(0) − T m0 )(z(0) − T m0 )T(26)Step 0: Set kmax , γmin and κ.
Step 1: Choose a suﬃciently large initial γ such that
there exists a feasible solution to LMI conditions[]
S1 I> 0, γ0 + γ1 + γ2 + γ3 + γ4 < γ,I Xϵϵinv > 1, θθinv > 1, µµinv > 1,
inequalities (11)-(14)Hence, (25) leads to
E [J ∗ ] = trS1 (Σ0 + m0 mT0 ) + trS2 (T Σ0 T T+(z(0) − T m0 )(z(0) − T m0 )T ) (27)Here, it can be assumed that an initial value of aminimal order observer z(0) satisﬁes the following
equation without loss of generality.
z(0) − T m0 = 0(28)

Substituting (28) into (27) yields
E [J ∗ ] = trS1 (Σ0 + m0 mT0 )+trS2 (Σ11 + LΣ21 + Σ12 LT + LΣ22 LT )(29)

©ISAROB 2011

Step 2.1 : Set γ̄ = γ, k = 0, i = 1, set S1 (k) = S1 ,X(k) = X, ϵ(k) = ϵ,ϵinv (k) = ϵinv , θ(k) = θ,θinv (k) = θinv , µ(k) = µ, µinv (k) = µinv .
Step 2.2 : Solve the following LMI problem
tk =Minimize(tr[S1 (k)X + X(k)S1 ]) +ϵ(k)ϵinv + ϵϵinv (k) + θ(k)θinv + θθinv (k) +89

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011µ(k)µinv + µµinv (k))
subject to[]
S1 I> 0, γ0 + γ1 + γ2 + γ3 + γ4 < γ,I Xϵϵinv > 1, θθinv > 1, µµinv > 1,

100

optimal value of guaranteed cost90

inequalities (11)-(14)Step 3.1 : If k < kmax and tk > 2n + 6 + κ then set
k = k + 1 and go to 2.2.

70
60
50
40
30
20
10
0

Step 3.2 : If k ≤ kmax , tk ≤2n + 6 + κ, LMI conditions are satisﬁed, and γ(0.5)i >γmin thenγ̄=γ̄-γ(0.5)i . Else if γ(0.5)i ≤γmin then exitand γ̄ is an optimal value.

2

4

6

8

10

12

14

16

18

iteration

Fig. 1: Trajectory of optimal guaranteed cost γ.

V. CONCLUSION

Step 3.3 : If k<kmax , tk ≤2n + 6 + κ, LMI conditions
are not satisﬁed, i̸=1 and γ(0.5)i >γmin thenγ̄=γ̄+γ(0.5)i . Else if γ(0.5)i ≤γmin then exitand γ̄ is an optimal value. Else if i = 1 thenexit and no optimal solution is obtained.

A guaranteed cost observer-based control problem concerned on a minimal order observer has
been discussed. A suﬃcient condition for the existence of state feedback guaranteed cost controllers
is derived on the basis of the ILMI approach tosolve inverse relation. A numerical example isgiven to illustrate the proposed method.

Step 3.4 : If k=kmax , tk >2n + 6 + κ, i̸=1 andγ(0.5)i >γmin then γ̄=γ̄+γ(0.5)i .
Else ifγ(0.5)i ≤γmin then exit and γ̄ is an optimalvalue. Else if i = 1 then exit and no optimal
solution is obtained.

References

Step 4 : Set i = i + 1 and return to 3.1.

[1] D. Matsunaga, M. Ishitobi and S. Kunimatsu (2009), An ILMI approach to guaranteed cost controllers with a minimal orderobserver. Proc. of ICROS-SICE International
Joint Conference 2009:583–588.

This algorithm allows the optimal value γ̄ can bereached faster than that of Matsunaga et al [1]because the correction is not ﬁxed but depending
on iteration i.

[2] C. H. Lien (2004), Robust observer-based control of systems with state perturbations via
LMI approach. IEEE Trans. on AutomaticControl Vol.49(8):1365–1370.

IV. AN ILLUSTRATIVE EXAMPLE
Consider a system with

−3 0 −2 03
 0 −2 0 −1 
 2 



A=
 1 0 0 0  , B =  −6  ,0 1 0 0
1[]
C = O2 I2 , m0 = 04 , Σ0 = I4 , R = 9,[]
0.1I2 O2
Q = diag(7, 15, 1, 3), DA =
,
O2 O2
[
]
0.3I2 0.3I2
EA =, DB = diag(0.3, 0.1, 0.3, 0.1),
O2 O2
[
]T
EB = 1 −1 1 −1 .

[3] S. Won and J. H. Park (1999), Design ofobserver-based controller for perturbed timedelay systems. JSME International Journal C
Vol.42(1):129–132.
[4] L. E. Ghaoui, F. Oustry and M. AitRami(1997), A cone complementarity linearization
algorithm for static output-feedback and related problems. IEEE Trans. on AutomaticControl Vol.42(8):1171–1176.
[5] Y. Cao, J. Lam and Y. Sun (1998), Static output feedback stabilization: an ILMI approach.
Automatica Vol.34(12):1641–1645.

Applying
Theorem1,with
kmax =200,
γmin =0.0001, κ=0.000001 and initial γ=100,
we obtain a solution[]−0.1844 −0.0440
L=,−0.0074 −0.2493[]
K = −0.3837 −0.4460 0.5278 −0.4720 ,[6] M. S. Mahmoud and M. Zribi (2003), Guaranteed cost observer-based control of uncertain time-lag systems. Computers and Electrical Engineering Vol.29(1):193–212.

γ̄ = E [J ∗ ] = 16.8893.

©ISAROB 2011

80

90

20

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Observer based control of a manipulator systemwith structured uncertainty
Chikara AIKAWA, Nobuya TAKAHASHI, Osamu SATO and Michio KONO
Faculty of Engineering, University of Miyazaki
1-1 Gakuenkibanadai-Nishi, Miyazaki, 889-2192, JAPANTel & Fax : +81-985-58-7419
E-mail : ntaka@cs.miyazaki-u.ac.jp
Abstract : In this paper, we show an uncertain model of a two link RR manipulator with uncertainties in thetwo rotation angles of each joint, and show the extended system with uncertainty also in an output matrix. For
this system, we apply a guaranteed cost control method based on a linear upper bound. Parameter tuning of
γi in the linear upper bound is effective to design a feedback gain which have appropriate characteristics. In
the numerical simulation, we show an advantage that the state observer is effective to reduce the influence of
signal noise in state vector.
Keywords : Uncertain system, Guaranteed cost control, Observer based control

1

Introduction

Let us consider the following uncertain system.
The guaranteed cost control (GCC) is one of an effective
approach to design a robust control system. This method
is an extended version of the linear quadratic regulator(LQR) that is one of the efficient method for designing
control system, and it is stated as an essential concept in
the modern control theorem. However, LQR method isformulated as nominal form, thus, it is weak for the effectof the disturbance which is caused by the uncertainty ofthe system model, secular distortion, signal noise, ans so
on. The system performance is degraded by these disturbance effects.
For such a problem, under the assumption that an uncertain parameter variation is in an admissible closed set,
the GCC method guarantees the upper bound of the performance index variation [1]. Takahashi et al. extended
the GCC method to the case with uncertainty in an output matrix [2], and they proposed the modeling method
for the uncertain inverted pendulum car system which include uncertainty in a pendulum angle and apply the guaranteed cost control method [3]. In this paper, we will propose a method of the GCC to the system with structured
uncertainties in input, state and also output matrix. And
show the effectiveness of the parameter tuning in the upper bound. At last, we will apply the state observer toconsider the influence of disturbance.

2= A(ξ)x(t) + B(ζ)u(t)= C(ψ)x(t)(1)

where, state, output and input vector are x ∈  n , y ∈m and u ∈ l , respectively. Uncertain state matrix
A(ξ) ∈ n×n , input matrix B(ζ) ∈  n×l , and output
matrix C(ψ) ∈ m×n are defined as
A(ξ)
B(ζ)
C(ψ)

= A0 += B0 += C0 +

p

i=1
q

j=1
r


ξi Ai , |ξi | ≤ 1(2)

ζj Bj , |ζj | ≤ 1(3)

ψk Ck , |ψk | ≤ 1

(4)

k=1

where, A0 , B0 and C0 represent the system structure that
could be included in the linear system model. We callthese matrices as nominal element. A i , Bj and Ck represent the uncertain system structure that could not be included in the linear system model. We call these matrices
as uncertainties. Where p, q and r are numbers of the corresponding uncertainties, ξ i , ζj and ψk are indeterminate
scalar parameters which represent the scale of uncertainties and included a bounded closed set. These parameters
are used to normalize the structures of uncertainties.
Here we consider the GCC problem for the system of
eq. (1) which has uncertainty in an output matrix. The performance index function consists of the quadratic forms of

Formulation of the GCC problem

In this section, we will show the GCC problem with parameter variation in state, input and also output matrix.

©ISAROB 2011

ẋ(t)
y(t)

91

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

the input vector u(t) and the output vector y(t).
J(y, u, ξ, ζ, ψ)
 ∞
 T

y (t)Qy(t) + uT (t)Ru(t) dt=

Let us define a state vector x(t) and an input vector u(t)are⎡⎤
θ1 (t)⎢ θ̇1 (t) ⎥u1 (t)⎢⎥
x(t) = ⎣, u(t) =θ2 (t) ⎦u2 (t)
θ̇2 (t)(5)0where Q ∈ m×m ≥ 0 and R ∈ l×l > 0 are weighting
matrices of input and output vector, respectively. In virtueof the uncertain structure (2), (3) and (4), the linear upperbound becomes:

An input matrix A(ξ) and an output matrix B(ζ) are obtained as following:
⎡
⎤
0
1
0
0
⎢ h̄11 /hD 0 h̄12 /hD 0 ⎥⎥A(ξ) = ⎢
⎣
0
0
0
1 ⎦
h̄21 /hD 0 h̄22 /hD 0⎡⎤00⎢ h22 /hD −h12 /hD ⎥⎥B(ζ) = ⎢⎣⎦00−h12 /hD h22 /hD

UL (A(ξ), B(ζ), C(ψ), P, Q, R)
p

 −1

γi P + γi ATi P Ai + P Rs P + Qs (6)
=
i=1

where
Rs =

q


(Bj R−1 B0T + B0 R−1 BjT )

(7)

where

j=1

Qs =

r


(C0T QCk + CkT QC0 + CkT QCk )hD = h11 h22 − h212

(8)

k=1

By applying this upper bound, we have stochastic algebraic Riccati equation (SARE) based on the linear upperbound:(A0 + γI)T P + P (A0 + γI) + C0T QC0 + Qsp
−P (Rn − Rs )P +γi ATi P Ai = O

(9)

i=1

p

where γ = 1/2
gain is obtained as:

−1
−1 T
B0 .
i=1 γi , Rn = B0 R

F = −R

3−1

Feedback©ISAROB 2011

= ∆c1 (−h12 g(m1 lG1 + m2 l1 )+ ∆c2 m2 glG2 (h11 − h12 ))h̄22= ∆c1 ∆c2 m2 glG2 (h11 − h12 )=2
I1 + m1 lG1

2+ I2 + m2 (l12 + lG2+ 2∆c2 l1 lG2 )

B0T P

h12
h22

=
=

2
I2 + m2 (lG2
+ ∆c2 l1 lG2 )2I2 + m2 lG2From A(ξ) and B(ζ), we can obtain deterministic elements A0 and B0 as (∆θ1 , ∆θ2 ) = (0, 0). A1 and B1 are
obtained in the condition of (∆θ 1 , ∆θ2 ) = (max1 , 0) andA2 and B2 are obtained in the condition of (∆θ 1 , ∆θ2 ) =(0, max2 ). Where maxi is a maximum uncertainty of therotational angle in the joint i.

4

Table 1 : Parameters of the Manipulator

g
τi

h̄12

+ ∆c2 m2 lG2 (h22 − h12 ))= ∆c1 ∆c2 m2 glG2 (h22 − h12 )

h̄21

h11

From the past research [4], we have a dynamics of uncertain LTI system of a two link RR manipulator. The link
1 is connected to the base with a rotation joint 1 and thelink 2 is connected to another end point of the link 1 witha rotation joint 2. Each joints and arms have physical parameters illustrated in table 1.

θi
mi
Ii
li
lGi

= ∆c1 g(h22 (m1 lG1 + m2 l1 )h11 , h12 and h22 are given as follows:

Skeletal form of the uncertain modelParametersh̄11

Meaning [unit]

State observer

In this section, we consider the application of an identical state observer to the uncertain system (1). x̂(t) is thestate of the observer (estimation of the plant state), y(t)is output from the plant, and u(t) is the input from thecontroller to the plant, respectively.

Angle of the joint [rad]Mass of the arm [kg]Inertia moment of the arm [kg · m 2 ]Length of the arm [m]Distance from the joint to the centerof gravity of the arm [m]Gravity [m/s2 ]Input torque to the joint [N· m]˙x̂(t)= (A − KC)x̂(t) + Gy(t) + Hu(t)where, K is observer gain.

92

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

5

Numerical example

the parameter tuning of γ i in the linear upper bound to design a system which have an appropriate characteristic ofclosed loop system, in the case of the system have only
one uncertainty. Here we show the result with two uncertainties of Ai where p = 2.

5.1 Uncertain system
In this section, we will show the numerical example. Herewe consider that the joint 2 is passive, thus the input matrix
B(ζ) becomes⎡⎤0⎢ h22 /hD ⎥⎥B(ζ) = ⎢⎣⎦0−h12 /hD

Degradation [%]
1.8
1.4
1
0.6

The values of the physical parameters are illustrated as intable 2.
Table 2: Parameters
Parameter Value Parameter
m1
1
m2
I1
0.03
I2
l1
0.3
l2
lG1
0.15
lG2
g
9.8

0.1
1
10
γ 1 100
1000
10000 0.1

Value
1
0.03
0.3
0.15

10
1

γ2

Figure 1 : Degradation of the Performance Index
At first, we solve the SARE on the any point of γ 1
and γ2 in 0.1 ≤ γi ≤ 10000 (i = 1, 2), and calculate the performance index as J LQR = xT0 PLQR x0
and JGCC = xT0 PGCC x0 . The weighting matrices are
Q = diag(1, 1) and R = 1. From these results, we examine the ratio of the degradation of the performance index.
This result is illustrated in the figure 1 by double logarithm 3D-chart, which x- and y-axis are logarithmic andz-axis is plotted with a linear scale. In figure 1, the surface illustrates the degradation ratio of the performanceindex between GCC and LQR. The contour line on the zplane denotes a line of J LQR and JGCC are equal. The
minimum value takes J GCC /JLQR = 0.6568 at a point(γ1 , γ2 ) = (46.8750, 15.6250). This point is surroundedby the square contour line that the corner is round. Insideof this square, the GCC method provides a good result.
But in the outside region, a reversed result is provided.

For the above parameters, we have uncertain system asfollows:⎤⎡
0.0000 1.0000
0.0000 0.0000⎢ 30.3093 0.0000 −12.1237 0.0000 ⎥⎥A0 = ⎢⎣
0.0000 0.0000
0.000 1.0000 ⎦−28.2887 0.0000
50.5155 0.000⎤⎡
0.0000⎢9.6220 ⎥⎥B0 = ⎢⎣
0.0000 ⎦−17.8694
For the disturbance ∆θ 1 = ∆θ2 = 0.08, uncertain system
becomes:⎤⎡
0.0000 0.0000
0.0000 0.0000⎢ −0.0969 0.0000
0.0388 0.0000 ⎥⎥A1 = ⎢⎣ 0.0000 0.0000
0.0000 0.0000 ⎦0.0905 0.0000 −0.1616 0.0000⎤⎡
0.0000 0.0000
0.0000 0.0000⎢ 0.0056 0.0000
0.1059 0.0000 ⎥⎥A2 = ⎢⎣ 0.0000 0.0000
0.0000 0.0000 ⎦−0.0168 0.0000 −0.3192 0.0000⎤⎤⎡⎡
0.00000.0000⎢ 0.0000 ⎥⎢ −0.0228 ⎥⎥⎥⎢B1 = ⎢⎣ 0.0000 ⎦ , B2 = ⎣ 0.0000 ⎦0.00000.0686

norm(F)
95
85
75
65
55
0.1
1

1000

10

γ1

100
1000
10000 0.1

100
10
1

γ2

Figure 2 : Norm of F GCC
Next, we will show the comparison of the norm of feedback gain in the same region. In figure 2, the contour lineon the z-plane denotes a results of the LQR method that
FLQR = 68.8696. The minimum value of the norm of
feedback gain is norm(F GCC ) = 55.2322 on the same

5.2 Parameter Tuning γ i of the linear upper boundIn this paper, we use the linear upper bound for the SARE.
In the past research [2], we had shown that the effective of©ISAROB 2011

1000
100

93

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 20116

minimum point of figure 1. These results have a similartendency.

For the system which haves two uncertainties, we showed
the advantage of the state observer for the influence of disturbance and parameter tuning of linear upper bound.

5.3 Numerical solution of SRAE
Here we show and compare the results of GCC and LQR.
The following results is the solution of the SARE (Propsed
method) on the minimum point (γ 1 , γ2 ).
⎤⎡214.9665 54.3720 163.5782 31.4218⎢ 54.3720 13.7897 41.6448 7.9989 ⎥⎥PGCC = ⎢⎣ 163.5782 41.6448 128.8797 24.5399 ⎦31.4218 7.9989 24.5399 4.6930

References[1] S. Chang and T. Peng (1972), Adaptive Guaranteed
Cost Control of System with Uncertain Parameters,IEEE Transactions on AC, 17(4):474-473[2] N. Takahashi, M. Kono and O. Sato (2009), Guaranteed Cost Control of the Uncertain System with Parameter Variations in an Output Matrix, Proceedingsof the 1st Japan-Korea Joint Symposium on Dynamics & Control, 139-142[3] N. Takahashi, O. Sato and M. Kono (2009), Robust Control Method for the Inverted Pendulum System with Structured Uncertainty Caused by Measurement Error, Artificial Life and Robotics 14[4] J. Hara, N. Takahashi, J. Kato, O. Sato and M. Kono
(2010), A study of Guaranteed Cost Control of theManipulator with Passive Revolute Joint, Proceedings of the 15th International Symposium on Artificial Life and Robotics, 910-913[5] S. Yamamoto, Q. Shen and H. Kimura (1993),
Quadratic Stabilization for a class of Uncertain Linear Systems and its Application to a Magnetic Levitation system, Transactions of the Society of Instrument and Control Engineers, 29:334-339[6] N. Takahashi, M. Kono, K. Hiranuma and O. Sato
(2000), Generalization of Guaranteed Cost Control,
Transactions of the Japan Society of Mechanical Engineers (Series C), 66(645):1531-1536 (Japanese)[7] M. Kono, N. Takahashi, O. Sato and A. Sato (2002),
Generalization of Guaranteed Cost Control - Extension to the case with Cross Terms, Proceedings of
the SICE Annual Conference 2002, 542-545[8] A. Sato, O. Sato, N. Takahashi and M. Kono (2008),
A Study of Manipulator with Passive Revolute Joint,
Artificial Life and Robotics, 13:31-35

Feedback gain is
FGCC = [ −38.3216 − 10.2522 − 37.8073 − 6.8958 ]Eigenvalues of the closed loop system are:( −7.8958 ± 2.3148i, −4.3929 ± 0.5460i )The LQR result (Ordinary method) is follows.
⎤⎡
309.6216 80.0140 250.8530 45.7467⎢ 80.0140 20.8360 65.4442 11.9295 ⎥⎥PLQR = ⎢⎣ 250.8530 65.4442 210.2729 37.8890 ⎦45.7467 11.9295 37.8890 6.9139
Feedback gain is
FLQR = [ −47.5721 − 12.6897 − 47.3509 − 8.7623 ]Eigenvalues of the closed loop system are:( −23.0041 − 4.3278 ± 0.7210i − 2.8177 )5.4 Comparison of the numerical simulation
Here we will show the numerical simulation of uncertain
systems. Now, we compare the performance index function value of eq. (5) with the difference of system composition i) with/without observer, ii) with/without disturbance. The simulation is calculated by Euler’s methodwith step time is 0.01, time interval (0, 30) with an initial state value is x(0) = [ 1 0 1 0 ]T and initial observerstate value is ẑ(0) = [ 0.1 0 0.1 0 ]T . The disturbanceis added in the state vector (x 2 , x4 ) = (θ̇1 , θ̇2 ) as a signal
noise of uniformly random number between (−0.08, 0.08)
on every step time.
Table 3: Comparison of the degradation
No Noise with Noise Degradation [%]LQR1 1108.2367 1214.8047
1.0962
GCC1 1203.4899 1317.1795
1.0945
LQR2 1916.9598 2621.1953
1.3674
GCC2 2866.6822 3257.1295
1.1362
In table1, LQR1 and GCC1 are the result without observer. LQR2 and GCC2 are the result with observer. It
express the GCC have high performance value index but
have more robustness than the LQR method. In the casewith observer, this tendency becomes more remarkable.

©ISAROB 2011

Conclusion

94

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Analysis of Manipulator in Consideration of Collision
between Link and Object
Asaji SATO1 , Osamu SATO2 , Nobuya TAKAHASHI2 and Masahiro YOKOMICHI2
1

Dept. of Mechanical Engineering, Miyakonojo National college of Technology, Miyakonojo 885-8567, Japan.
Dept. of Computer Science and Systems Engineering, Faculty of Engineering, University of Miyazaki, Miyazaki 889-219, Japan.

2

Abstract: In this paper, equations of motion of a manipulator are derived in consideration of characteristics of driving
source. Considering the collision between the link and object, and considering the active motion to absorb kinetic
energy of the object, trajectories for saving energy are calculated by iterative dynamic programming. And, the dynamic
characteristics of manipulator controlled based on the trajectory for saving energy are analyzed theoretically andinvestigated experimentally.
Keywords: Manipulator, Trajectory, Dynamic Programming, DC Motor, Minimum Energy .

1

Introduction

two degrees of freedom as shown in Figure 1, which isable to move in a vertical plane, are as follows.

For the purpose of enlarging the work space about
carriage work, it is necessary for studying about thethrowing motion and catching motion of themanipulator. In a previous report [1], a casting
manipulator is introduced, and it has large work spacecompared with its simple mechanism. But, theconsideration of energy consumption is not enough.
And, evaluation of robotic mechanisms subjected to
impact load are investigated [2]. But, energy
consumption is not considered.
In previous report by the authors [3][4]，trajectories
for saving energy about the throwing motion ofmanipulator, were easily calculated by iterative
dynamic programming. And, dynamic characteristicsof the system were analyzed.
In this paper, equations of motion of a manipulator
are derived in consideration of the characteristics of DC
servomotors, and a performance criterion for savingenergy is defined in consideration of energy consumptionof the driving source. When the manipulator is operated
in a vertical plane, the system is highly non-linear due togravity，and an analytical solution can not be found. Then,a numerical approach is necessary. Considering thecollision between the link and object, andconsidering the active motion to absorb the kinetic
energy of the object, the trajectories for energy saving
are calculated by iterative dynamic programming. The
dynamic characteristics of manipulator controlled basedon above mentioned trajectory are analyzed theoreticallyand investigated experimentally.

2

⎡ A11
⎢A
⎣ 21
where

(1)

A11 = a1 , A12 = a3 cos (θ1 − θ 2 ) , A21 = A12 , A22 = a2
A13 = τ 1 − a3θ2 2 sin (θ1 − θ 2 ) − a4 cos θ1 − Fl1 cos θ1

A23 = τ 2 + a3θ12 sin (θ1 − θ 2 ) − a5 cos θ 2 − Flc

a1 = m1lg12 + I G1 + m2l12 ,a3 = m2l1lg 2 ,a2 = I G 2 + m2lg 2 2

a4 = ( m1lg1 + m2l1 ) g ,a5 = m2glg 2

y
lg 2

m2

θ2

l1
lg 1

m1

θ1
x

τ2

τ1

(a )

mb

y
F = kξ

l1

k
θ1

ξ

lc

(b)

Modeling of manipulator

The dynamic equations of the manipulator with©ISAROB 2011

A12 ⎤ ⎡θ1 ⎤ ⎡ A13 ⎤⎢ ⎥=A22 ⎥⎦ ⎣θ2 ⎦ ⎣⎢ A23 ⎦⎥

Fig.1

95

Mechanism of manipulatorybxThe Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 1 Parameters of the manipulator
Parameter Value Parameter
Value

The kinetic energy of the mechanism and object isK=

1 2 1  2
1
a1θ1 + a2θ 2 + a3θ1θ2 cos (θ1 − θ 2 ) + mb y b 2 ,
2
2
2

( 2)

l1 (m)lc (m)
lg1 (m)lg2 (m)m1 (kg)m2 (kg)mb (kg)rb (m)and potential energy is
1 2
kξ + mb g yb .
2( ξ ; displacement of spring )And, absorbed energy by motor is( 3)

E a b = − ∫ τ 1 ⋅ θ1 d t − ∫ τ 2 ⋅ θ2 d t .

( 4)

U = a4 sin θ1 + a5 sin θ 2 +The applied voltage of the servomotor is( 5)

e j = b1 j θ j + b 2 j θj + b 3 j τ j + b 3 j τ f j sign(θ j )

0.10
0.01
0.05
0.01
0.02
0.02
0.02
0.01

Given input data

b1 j = kv j + ( Ra j / kt j ) Dm j , b 2 j = ( Ra j / kt j ) Im , b 3 j = Ra j / kt j ,ia j : electric current of the armature ,Ra j : resistance of armature ,I m j : moment of inertia of armature,
Dm j : coefficient of viscous damping.

θ i j ,θ i j , θ f j ,θ f j , T , N , Δ θ
Calculate initial searching region
n ←1

(A)

Then, the electric current is
i a j = (e j − k v j θ j ) / R a j .

(6)2

E = ∑ ∫ (e j ⋅ i a j )dt .

(7)
n>30

j =1

Yes

3 Simulation of the manipulatorEnd

We shall take the parameters of the system asshown in Table 1.
Figure 2 shows a flow chart for iterative dynamic
programming method. In frame (A), the trajectory forsaving energy is searched by dynamic programming [3].
In frame (B), the searching region is shifted tominimize the consumed energy, and width of theregion is changed smaller.
Figure 3 shows the trajectory for searching,and initial trajectory for searching is expressed as(θ − θ ) ⎪⎧1 − cos ⎛ π t ⎞⎪⎫
ij

2

⎨
⎪⎩

⎜⎜
⎟⎟ ⎬
⎝ t f ⎠ ⎪⎭

Fig.2

No

(B)
Shif the searching region
to minimize the energyand( Δθ ) n +1 ← C ⋅ ( Δθ ) n
C = 0.5 ～ 0.7

n>30 → Δθ <0.02radn> 30 → Δ θ <0. 02 ra d

Flow chart for simulation(8).

Figure 4 shows the motion of an object to fall free.
Initial height is yb = 0.15 ( m ) , and velocity is yb = 0 .
When t = 0.128 ( s ) , the object contacts with the link.
And then, yb′ = 0.069 ( m ) and yb = −1.25 ( m s ) .
Under the condition that θ1i = 0.1 and θ1 f = 0.05 ( rad ) ,optimal trajectory is calculated by IDP method.
The performance criterion is2

E ′ = ∑ ∫ (e j ⋅ i a j )dt + C 1 ∫ F ⋅ dt + C 2 y b − y ′bn ← n+1

Search the trajectories
for saving energy by D.P.

And, the consumed energy is

f j

1.7×10-5
1.7×10-5
500
0.046
0.046
7.9×10-50.0013
3.5

Start

where

θ j = θi j +IG1 (kgm2)
IG2 (kgm2)k(N/m)
kt1 (Nm/A)
kv1 (Vs/rad)Dm1 (Nms/rad)τf1 (Nm)
Ra1 (Ω).

Fig.3

Trajectory for searching

Fig.4

Motion of object to fall free

(9)

j =1

In Figure 4, the solid line shows the motion of
object whose kinetic energy is absorbed by the motor.

©ISAROB 201196

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011(a) Link is not actuated

(b)
Fig.5

Link is actuatedMotion of the object and the mechanism
Figure 5 shows the motion of the object and themechanism. In Figure (b), link 1 is actuated, andkinetic energy of the object is absorbed. And, theresponse about conservation of mechanical energy are
shown in Fig.6.

5 Experimental results

Fig.6(a) Front view
Fig.7

©ISAROB 2011

In this section, the results of fundamental experimentare shown to examine the effectiveness of modelingfor the simulations.
Figure 7 shows an experimental apparatus. Slideboard is tilted (π 3) measured from the level surface.
And, two Laser displacement meters are installed formeasuring the passing time of the object.
The parameter of the link 1 and the motor are
shown in Table 1, and the motor (rated 24 V, 60W) areon the frame, and sampling time of the control is 0.002
s. The feedback gain for angular displacement is 50
V/rad, and the feedback gain for angular velocity is
0.5 Vs/rad.
Figure 8 (a) and (b) show experimental results
about motion of the link and object. In Figure (a),the link is fixed at initial position, and the object
bounds high. In Figure (b), the link is actuated along
the trajectory calculated by IDP method, and theobject bounds low.
Figure 9 show are experimental results about the
response of the link and motor. About the angular

Mechanical energy(b) Side view

Experimental apparatus

97

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011displacement and angular velocity, experimental
results ( blue line ) are similar to the theoreticalresults ( red line).
From these results, it is considered that modelingfor simulation is effective.

6

Conclusions

The results obtained in this paper are summarized
as follows.
(1) It is considered that the active motion to absorb
the kinetic energy of object is possible byanalyzing the relative motion about the collision
between the link and object.
(2) From experimental results, it is considered thatmodeling for simulation is effective.

t =0

References[1] H.Arisumi, K.Yokoi, T.Kotoku et al (2002),
Casting manipulation (Experiments of swing andgripper throwing control), International Journal of
JSME, Vol.45, No.1 C, 267-274.
[2] H.Tachiya, K.Urano et al (1996), Characteristics
evaluation and comparisons of robotic mechanisms(2nd report, Evaluation of robotic mechanisms
subjected to impact load, Journal of JSME (inJapanese), Vol.62 No.598, 2395-2402.
[3] A.Sato, O.Sato, N.Takahashi and M.Kono (2006),
Trajectory for saving energy of direct-drive
manipulator in throwing motion, Artif Life
Robotics, 11:61-66[4] A.Sato, O.Sato, N.Takahashi and M.Kono (2008),
A study of manipulator with passive revolute joint,
Artif Life Robotics, 13: 31-35

t = 0.16

t = 0.08

t = 0.33 ( s )t = 0.25(a) Link is fixed at initial position

t =0

t = 0.16

t = 0.08

t = 0.33 ( s )t = 0.25(b) Link is actuated along the trajectory calculated by IDP method
Fig.8

Experimental results (motion about the link and object)

0.2

0.4

0.6
t (s)

θ・ (rad/s)

-0.5
10
0

5
0

0.2

-5

0.2

0.4 t (s) 0.6

-10
Fig.9

©ISAROB 2011

e (V)

0

10

Experimental
TheoreticalLaser
displacement meter（V）θ (rad)0.5

4

Displacement meter 1
Displacement meter 2

2
0

0.2

Experimental results ( response of link and motor)

98

0.4 t (s) 0.6

0.4 t (s) 0.6

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Motion Control of 2 DOF Orthogonal robots with Adaptive Control
Kazuma.Funahashi , Feifei.Zhang and Masanori.Ito
Tokyo University of Marine Science and Technology,
2-1-6 Echujima, Koto-ku, Tokyo 135-8533 , JAPAN(Tel : 81-3-5245-7422; Fax : 81-3-5245-7422)
Abstract: This study considers motion control of 2 DOF orthogonal robot that is crossed two industrial linear robots
with adaptive control under the various load condition. We constructed experimental facility with putting 0.5m lengthlinear robot on a 1m length linear robot and confirmed performance under various conditions with applying adaptive
control of fast sampling and control interval. The results show a prescribed control performance can be satisfied.
Therefore, adaptive control can be expected as a useful control method for industrial, high-speed positioning robot.

Keywors:

linear robot, motion control, adaptive control

I.

which the weak point is thought to be improved with the
Saying concretely, we constructed experimental
system with crossly putting 0.5m length linear robot on a
1m length linear robot, and moved both robots at thesame time with predetermined pattern under the changeof carrying weight with adaptive control. Through thoseexperimental researches, we confirmed the effectivenessof adaptive control.

INTRODUCTION

The adaptive control that can reflect the change ofcharacteristics of the control targets was thought as aneffective control method for modern control system.
However it requires many and complex calculation in thecontrol for real time estimating of the target model andadjusting control signal, then control interval becomes
rather large. It is a weak point of adaptive control. So,
main current of the control theory moves to the robust
control. On the contrary, the conventional PID control isused in industrial fields because of the easiness and highspeed. However, needs to high productivity or accuracy
in production system require high level of automatic
control system which can comply with a change ofcharacteristics of target or environmental condition in
those days instead of conventional PID control.
In the fields of industrial robot, it becomes moreevident, so we treated the motion control that can satisfy
the predetermined condition under the change of load forlinear transfer robot with applying adaptive control ofhigh performance of computer.

II.

Figure 1 shows the configuration of experimental
system. Each linear robot is composed of servomotorwith resolver, ball screw, carrying table, motor driverand controller. The carrying load is set on the carrying
table of upper robot. In this system, we used a personal
computer as the controller and control program isdesigned with using Simulink toolbox of MATLAB by
Mathworks. Here, the torque control signal is calculatedwith adaptive control algorithm based on the position
signal with resolver and output to motor through driver.

Fig.1 Experimental system©ISAROB 2011

EXPERIMENTAL SYSTEM

99

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

III.

The control problem of precise positioning robot is tomake the angle of the motor accurately follow to thetarget signal. It is well known in PID control, that thecontrol system contains a PI speed control loop, and a P
position control loop. In this study, we use adaptive
control instead of PI control for the speed control loop,
as shown in Fig. 2, the purpose of which is to absorb the
response speed change caused the load change .
The RLS method (recessive minimum mean square
method) is used as the model identification, and themodel order is assumed 2 in consideration of thecomplexity and rapidity of response of the system.
Some referred expression is as follows.
Model :

Fig.2 Photo of Experimental system
The specifications of main components are shown in
Table 1.
Table 1 Specifications of main components

Lower Linear Robot

1)

Stroke:1000mmMax. Carrying Load : 30kg
Carrier: Ball Screw(Lead: 20mm/rev.)
Power Source: AC SERVO MOTOR(Output; 200W, Max. Rpm; 3000,
Max. Torque;0.64Nm)
Reduction Gear Ratio : 1/3
Position Sensor: Resolver(Resolution; 16384 pulse/rotation)y(k )  a1 y(k  1)  a2 y(k  2)  b1u(k  1)  b2u(k  2)
  T (k )(k )(1)(Where, y( ) and u( ) indicate plant output and control
input, respectively, and  a1 a2 b1 b2 T  y(k  1) y(k  2) u(k  1) u(k  2)T

(2)
The parameter vector
follows.

 (k) is calculated repeatedly as

PT(k  1)(k )2 1) a (k )a1 (ka
3 a4T (k )
1


(
k
)
P
(
k

1
)

(
k
)
  y(k  1) y(k  2) u(k  1) u(k  2)TP(k  1)(k ) T (k )P(k  1)
(2)
P(k )  P(k  1) 
1   T (k )P(k  1)(k )Upper Linear Robot
Stroke: 500mmMax. Carrying Load : 10kg
Carrier:Ball Screw(Lead: 20mm/rev.)
Power Source:AC SERVO MOTOR(Output; 200W, Max. Rpm; 3000,
Max. Torque;0.64Nm)
Reduction Gear Ratio : 1/3
Position Sensor: Resolver(Resolution; 16384plase/rotation)

 (k)  y(k)  T (k) (k)

(3)
(4)
(5)

,with the following initial conditions.

  1 1 1 1T
PI

Fig.3 Control system©ISAROB 2011

CONTROL DESIGN

100

(6)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Lastly, the expression of the control input is as follows.

u (k ) um (k )  b2u(k  1)  a1 y(k )  a2 y(k  1)
b1

IV.

Figure 6 shows the results of the motion following tolinear patterns of different speed, one is 94rad/8sec(300mm/8sec) and the other is 47rad/8sec (150mm/8sec).

(7)

RESULTS AND EVALUATION

Here we show the results of experiments and makeevaluation for them.
In the following figures, we only referred to theresults of motion on lower robot when both robots weremoved in the experiment. The sampling rate of thoseexperiments was 0.001 second.
Figure 4 and Figure 5 show the results of the
motion following to sine wave under the load of 0kg and5kg. Here, continuous line is set value, and dash line anddot line are responses for 0kg and 5kg lords, respectively.
・ Fig.6 Comparison of linear motion(Moved distance: 150mm and 300mm)
Figure 7 and Figure 8 show the starting and stopping
process to the order of moving 150mm/ 8sec forevaluation. We tried this test for the case of 300mm (94radian of total rotation). The result is omitted here
because they are nearly equal to the results of 150mm(47 radian).

Fig.4 Response to 4πsin(π/2･t)・ Fig.7 Detail at starting of 150 mm movement･ Fig.5 Response to 4πsin(π･t)©ISAROB 2011101

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

accomplished in a laboratory and having big difference
to actual industrial use. Decreasing the level of error,
optimization of initial parameter for adaptive control, or
simplifying the algorithm and speed up of calculation forsmall computer are the subjects for actual use.
Now we are continuing study for the level-up of
accuracy with improving position and speed control,
especially optimizing the initial value of θ, and the
speed-up of calculation with combining speed andposition control and/or decreasing the degree of referringmodel.
・ Fig.8 Detail at stopping of 150 mm movement

REFERENCE[1] System Identification for Control, Shuuichi Adachi,
Tokyo Denki Univ.
[2] Theory ＆ Practice of Adaptive Control Systems, I.
D. Landau and M. Tomizuka, OHM Co., Ltd.

In those results, the dash line and dot line in eachfigure show almost same responses. This means that thedifference of load does not influence to the results.
Especially, the difference of stopping position between
the carrying weight of 0kg and 5kg is very small (fromFig.8, it is 0.03mm). We think this is the allowable
margin of error because it is the level of machinery error.
All these merits obviously come from adaptive control.
Figure 7 shows a small shaking of response at justafter starting. This is allowable, because it is an initialand usual process of adaptive control, and shows that
control system has started learning for targets, anddisappears immediately.
On the other hand, it is seen that the delay of
response becomes remarkable depending on a frequency
of input signal. In this case, the delay is connecting tocontrol error directly. Also, the positioning error at
stopping point (see, Fig.8) is about 0.3mm, this islarger than the demanded accuracy (0.04mm).
We think these problems are depending on the
structure of control system, because the position control
loop use the conventional P control. How to control bothspeed and position of high speeded robot with adaptive
control is our next subject.

V.

CONCLUSION

This study shows that the performance of transfer
system with linear robot can be improved with adaptive
control. We can ensure the desired performance to thelinear robot under specified range of load condition. The
weak point of adaptive control that is not so fast is
improved to the level of robot motion control withcomputer performance. Of course, the study was©ISAROB 2011102

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Development of 6-DOF Force Feedback System
for Rehabilitation of Wrist Paralysis
Yasunobu HITAKA*, Yoshito TANAKA**, Yutaka TANAKA*** and Tomonori KATO *** Kitakyushu National College of Technology
Tel : +81-93-964-7244
Email hitaka@kct.ac.jp** Hosei University*** Fukuoka Institute of Technology
Abstract: The increasing number of paralyzed persons and a shortage of the physical therapist is becoming an
increasing problem, focusing attention on rehabilitation support. Noting the increase in wrist paralysis, we aredeveloping corresponding rehabilitation support. Our basic approach centers on force feedback system based on theparallel 6-degree-of-freedom (DOF) Stewart platform. In this paper, we focus on the generation of a desired force
which is a resultant force of the six cylinder forces of the parallel mechanism. Also we focus on drawing a 3D
computer graphic (CG) model which shows harmonic movement with the parallel mechanism in virtual space. It is the
feedback information for wrist rehabilitation. In this paper, we show that the experimental results of the desired force generation and the 3D-CG model moves harmonically with the parallel mechanism assigned distance place via
the Local Area Network.
Keywords: Stewart type parallel mechanism, six degree of freedom,
individual cylinders must be calculated. They arecalculated by solving six simultaneous equations
representing static force relationships.
This paper also focuses on drawing a 3D computer
graphic (CG) model showing synchronized movementwith the parallel mechanism in virtual space. It is the
feedback information for wrist rehabilitation. The
therapist can recognize the situation of treatment at a
distant place on real time by viewing the movement of
the 3D-CG model. The model is drawn from the 6-DOF
attitude information. It is calculated by solving direct
kinematics of the parallel mechanism with data on
cylinder length. From the calculation result, the PC
draws a similar parallel mechanism which shows thesynchronized movement with the real one in a virtual
workspace displayed.
The desired force generation and 3D-CG modeldrawing can be achieved for the parallel mechanism
assigned a distance place via LAN by using the TCP/IP
connection.
This paper is organized as follows: Section 2
introduces the system configuration. Section 3 focuseson controlling force act on the platform gripper andgives the results of force control experiments. Section 4
focuses on drawing a 3D-CG model of the parallelmechanism and shows that the 3D-CG model moves
harmonically with the parallel mechanism assigned
distance place via the Local Area Network. Section 5 isconclusions.

I. INTRODUCTION
For the cases of persons afflicted with paralysis
increasing, due to adult disease, traffic accidents andaging, the shortage of physical therapists is a growing
problem requiring rehabilitation support. In our focus
on wrist paralysis, we considered rehabilitation support,
requiring multiple degrees of freedom (DOF) andcompactness, taking into account the parallel Stewart
platform, which generates 6-DOF force and provides
high power and rigidity [1]. It applies 6-DOF force towrist of a patient through a gripper on the platform forrehabilitation therapy instead of the physical therapist.
In our past research, we developed a prototype of
the parallel Stewart platform with six pneumatic air
cylinder actuators and achieved the thrust control ofindividual pneumatic cylinder by a pulse widthmodulator (PWM) control [2]. The goal of our researchis that one physical therapist can treat multiple patientsat the same time by remote controlling the parallelmechanisms. To achieve the purpose, it is necessary thatthe parallel mechanism can output the desired force on
the platform gripper. It is also necessary that thephysical therapist can see the situation of rehabilitation
treatment.
This paper focuses on controlling force act on theplatform gripper. It is a resultant force of six pneumatic
cylinder forces. Thus, to generate a desired force on theplatform gripper, the component forces generated by©ISAROB 2011103

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Ⅱ. SYSTEM CONFIGURATION

physical therapist can treat multiple patients at the sametime by sending rehabilitation commands to theindividual parallel mechanisms of the patients.

Figure 1 shows the force feedback configuration we
propose, which consists of a pneumatic parallelmechanism and two personal computers (PC). The one
is for the physical therapist and the other is for a patient.
The parallel mechanism is connected with the patientside PC through I/O board. These PCs communicate
with each other by TCP/IP via LAN. When a desired
force which will act on the platform gripper is input at
the therapist side PC, six cylinder forces corresponding
to the desired force are calculated by solving six
simultaneous equations representing static force
relationships. The calculated cylinder forces are sent to
the patient side PC and translated into PWM signals andsent to twelve solenoid valves mounted the parallelmechanism. Each cylinder has two valves and one isused for pushing the cylinder and the other is for pulling.
The force resulting from six cylinders should beequivalent to the desired force.

Ⅲ. FORCE GENERATION
To generate a desired force on the gripper, cylinders
must generate the component forces of output by the six
equations representing the relationship of static force acton the gripper, shown in Figure 2.

w

Fig2. Analytical model of parallel mechanism
These are i-th cylinder force Fi ( i = 1,..., 6 ) , desired
output force L , desired moment w and force ofgravity M . These force vectors have coordinates,, wx , wy , wz
and
Fix , Fiy , Fiz
Lx , L y , Lz
( 0, 0, −mg ) for the fixed frame, m is platform mass
and g is gravity acceleration.
The static force relationship for translational
movement is as follows:(

Fig1. Force feedback configuration

(

)

(

)

6

∑F + L + M = 0 .

(1)iWhen the parallel mechanism is moved, the patientside PC detects the six cylinders length by the six
location detectors, each of which is mounted on thecylinder. The patient’s PC sends the cylinder length data
to the physical therapist side PC. The therapist side PC
calculates the parallel mechanism attitude by solving thedirect kinematic equations with the cylinder data. Fromcalculation results, the therapist side PC draws a 3D-CG
model of the parallel mechanism. It shows the
movement synchronized with the real parallelmechanism in a virtual work space displayed. The CG
model in virtual space is drawn using OpenGL.
By this way, the physical therapist can see thesituation of rehabilitation treatment in detail even if heisn’t around the patient. Moreover, if multiple parallelmechanisms are prepared, it is expected that one©ISAROB 2011

)

i =1

For rotational movement, the relationship is as follows:6∑ (r × F ) + (R × L) + (R × M) + w = 0 .
i

i

(2)

i =1

Eq.(1) and Eq.(2) are the six simultaneous equations forcylinder force Fi , and solving them by using following
relationship(Fix , Fiy , Fiz ) =Filix 2 + liy 2 + liz 2(lix , liy , liz ) ,the component forces Fi of the desired output L anddesired moment w are obtained. When a desired force
is input at the physical therapist’s PC, Fi arecalculated by above analysis, sent to the patient’s PC. At

104

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

patient’s PC, they are translated into PWM signals tocontrol the solenoid valves [2].
The force control experiments are conducted andresults are presented. Figure 3 shows the overall view offorce control experiments for the pneumatic parallelmechanism.

Eｘperimental result(Push)

Proofreading graph

1.4

Voltage[V]

1.2
1.0
0.8
0.6
0.4

Bridge circuit

0.2
0.0
0.0

Strain gauge

10.0

20.0

30.0
Force[N]

40.0

50.0

60.0

Fig.6: Experimental results (z direction (-))
Figure 5 shows the results for plus direction and Fig.
6 for minus direction. Initial positioning is adjusted as
neutral position (cylinder expansion and contraction:15mm) and supply pressure is 0.15Mpa. For both
directions, measurement and proofreading results
coincide well confirming that arbitrary force isgenerated in the z direction.

Parallel mechanism

AmplifierFig.3: Parallel mechanism experiments

Ⅳ. 3D-CG DRAWING BASED ON DIRECTKINEMATICS
To draw a 3D-CG model showing the movement
synchronized with the parallel mechanism, it isnecessary 6-DOF attitude information of the platform.
The parallel mechanism has six location detectors
detecting each cylinder length. Thus, by solving thedirect kinematics of the parallel mechanism with dataon cylinder length, 6-DOF attitude can be obtained.
Figure 3 is i-th cylinder vector diagram. o is theorigin of the fixed frame, o′ is the origin of the motion
frame assigned at the centroid of the platform. It showsa relationship as follows:

Fig4. Force controller interface
Figure 4 is the interface of the force input on thephysical therapist’ PC. Inputting components of adesired force along x, y and z direction, corresponding
duty rates for the twelve solenoid valves are calculated
and sent to the I/O board. Also, these values are
displayed on the interface.
Eｘperimental result(Pull)l i = AimT + R − Bi

Proofreading graph

Where, l i is a cylinder vector, R is a position vectorof centroid of the platform, and Bi is a position vectorof lower attachment point and there vectors are withrespect to the fixed frame. AimT denotes upperattachment point with respect to the motion frame andthe coordinate transform matrix T is

1.4

Voltage[V]

1.2
1.0
0.8
0.6
0.4
0.2

cos θ cosψ
T = sin φ sin θ cosψ − cos φ sinψ cos φ sin θ cosψ − sin φ sinψ

0.0
0

10

20

30
Force[N]

40

50

60

Fig.5: Experimental results (z direction (+))©ISAROB 2011

(3)

105

cos θ sinψ
sin φ sin θ sinψ + cos φ cosψ
cos φ sin θ sinψ − sin φ cosψ− sin θ 
sin φ cos θ cos φ cosθ The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

, where, φ , θ , and ϕ are “roll”, “pitch”, and “yaw”
angles of the platform with respect to the motion frame.
Ai

o′

R

o

Bi

Fig3. Vector diagram for i-th cylinder
Here, letlif be the measured value of i-thcylinder vector and α be the vector of the platform
attitude and define a function f ( α ) as follows:f ( α ) = ( f1 ( α ) , f 2 ( α ) , f3 ( α ) , f 4 ( α ) , f5 ( α ) , f 6 ( α ) )fi ( α ) = Aim AimT + RRT + Bi BiT+ 2 RT T AimT − 2ψ BiT T AimT − 2 RBiT − lif 2 = 0

α = ( x, y, z , φ ,θ ,ψ )T

i = 1,..., 6(4)Equation (4) is a couple of nonlinear 6 dimensional
equations and it is difficult to derive the solution α by
any analytical ways. Thus, by using the NewtonRapthon method which is one of a numerical method,
approximate solution of α is derived.
n
Let α ( ) be an approximate solution which is in then times iterations and applied the Newton-Rapthon
method to the Eq. (4), following equation is obtained.

(

M α(

n +1)

)

( )

− α ( ) = −f α ( )nnFig7. Experiments of harmonic movement

Ⅵ. CONCLUSION
In this paper, for the development of the force
feedback system applied to the wrist rehabilitation
support system, generating desired force act on theplatform gripper was detailed. Also, drawing a 3D-CG
model showing the movement synchronized with theparallel mechanism in virtual space was detailed. Thesecan be achieved for the parallel mechanism assigned a
distance place via LAN by using the TCP/IP connection.
In projected works, we plan to measure force forother direction, to display detail information on the 3DCG model, like force value, vector image of force, etc.

(5)

Where, M is a matrix of the partial differentiation ofnf α( ) .
By solving Eq. (5) iteratively, approximate solutionof α can be derived.
By this way, PC of the force feedback system
calculates the 6 DOF attitude of the platform. From theresult, it draws the 3D-CG model of the parallelmechanism.
Figure 7 is set of the still images of the 3D-CG
model and the parallel mechanism. These pictures are
clips of the experiment of the synchronized movement.
In this figure, the parallel mechanism is moved byhand. It is shown that the 3D-CG model moves
synchronized with the parallel mechanism.

( )©ISAROB 2011

References[1] Stewart, D., A Platform with 6 degrees of freedom,
Proc. Inst. Mech. Eng., Vol.180, Part I, No.15, pp.371386, 1965-66.
[2] Hitaka, Y., Tanaka, Y., Tanaka, Y., Ishii, J.,
Development of 6DOF Force Feedback System with
Pneumatic Parallel Mechanism, Proceedings of the 15th
International Symposium on Artificial Life andRobotics in Beppu, Japan(2010).

106

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Feedback Stabilization of Linear Systems with Distributed Input
Time-delay by Backstepping Method
Chaohua Jia1,2
1 School of Mathematics and Systems Science, Beijing University of Aeronautics and Astronautics;
2 LMIB of the Ministry of Education, Beijing 100191, P. R. China(jiach@buaa.edu.cn)

Abstract: The stabilization problem of a linear time-invariant system with lumped and distributed delays in the control
will be investigated by the backstepping method. A transformation is introduced firstly to reduce the system with
distributed input delay into a system with lumped input delay. The transformation kernel can be expressed explicitly
through solving a Cauchy problem of ODEs. Then the backstepping arguments presented by Krstic and Smyshlyaev in[1] can be applied to work out a feedback control for the original system, where the key point is to model the lumped
delay by a first-order hyperbolic partial differential equation.
Keywords: time-delay systems, distributed delay, backstepping method, reduction transformation.

I. INTRODUCTION

Then (1) is reduced to a delay-free system()ẏ(t) = Ay(t) + B0 + e−Aτ B1 u(t).

In many engineering systems, the variation of thesystem state depends on past states. Such a character iscalled time-delay. Time-delay systems are also known
as systems with aftereffect or dead-time, hereditarysystems, which have attracted the attention of many
researchers because of their importance and widespreadoccurrence. They are special infinite dimensional systems, and researches indicate that the time delays leadto some complexity. On one hand, the time delays may
deteriorate the control performance or even cause theinstability of a dynamic system. On the other hand,several studies have shown that voluntary introductionof delays can also benefit the control.
The stabilization for time-delay systems is a topic ofgreat importance and has received increasing attention[2], [3], [4], [5]. Smith-predictor is well-known for
designing linear feedback controllers for systems withdelay. However, this method is confined to stable plants
because it implies pole-zero cancellations. Moreover,Smith predictor is sensitive to parameter errors. The
model reduction method is another important approachto deal with the systems with input delays, where theso-called Artstein model reduction is often involved.
Briefly speaking, the model reduction is a transformation
through which one can simplify a dynamic systemwith input delay into an equivalent delay-free system.
Considering the following linear system with lumped
input delay,ẋ(t) = Ax(t) + B0 u(t) + B1 u(t − τ ),It is straightforward to compute a state feedback control
u(t) = Ky(t), provided that (A, B0 + e−Aτ B1 ) is stabilizable. Thus the system (1) can be stabilized by the control
law
(
)
∫ t
−Aτ
A(t−s)
u(t) = K x(t) + ee
B1 u(s)ds .
(3)t−τ

Kwon and Pearson were the first who clearly put
forward the reduction transformation (2) for a systemwith lumped input delay [4]. In [4], the authors applied
the receding horizon method to the stabilization of
linear time-invariant systems with lumped input delay. It
was shown how the receding horizon control suggested
the reduction transformation (2). Afterwards, Artsteinestablished the general abstract theory of the reduction
method in [2]. According to this theory, one can transform general linear time-varying systems into delay-freesystems. It has been shown by many researches thatthe reduction provides a strong tool for manipulating
systems with delays in the controls, even for linear
systems with time-varying lumped input delay [2].
It has also been pointed out in [5] that any lumped
time-delay U(t) = u(t − τ ) can be represented by aclassical transport equation:{
τ ∂∂t v(x,t) + ∂∂x v(x,t) = 0, x ∈ [0, 1],v(0,t) = u(t), v(1,t) = U(t).
In other words, we can use a first-order hyperbolic partial differential equation to model the lumped delay. A
problem of boundary feedback stabilization of first-order
hyperbolic partial differential equations was consideredin [1], where the authors applied the backstepping(1)

introduce the new variable y(t) defined by−Aτ

y(t) = x(t) + e©ISAROB 2011

∫ t

t−τ

eA(t−s) B1 u(s)ds.

(2)

107

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

where the transformation kernel matrix P(·) : [0, τ ] →Rn×m is to be determined later. Now one can calculate
easily thatmethod to design controllers. As an example of theapplications, the authors studied the stabilization of alinear time-invariant system with lumped input delay by
combining the backstepping design for hyperbolic PDEs
with the backstepping design for linear ODEs.
Motivated by the researches mentioned above, wewill investigate the stabilization problem of a linear
time-invariant system with lumped and distributed delays in the control by the backstepping method. We
first transform the linear system with distributed input
delay into a system with lumped input delay, where
the transformation kernel is determined by a Cauchy
problem of ODEs. This transformation kernel can be
expressed explicitly. Then the procedure presented in[1] can be applied in our circumstance. Concretely,
we model the lumped delay by a first-order hyperbolic
partial differential equation, and replace the resulting
system in the first step by an ODE-PDE cascade. After
introducing a backstepping transformation, we obtain thetarget system whose origin is exponentially stable. Thus
we can work out the corresponding feedback control law
for the original system.
The rest of this paper is organized as follows. In
section 2, the linear time-invariant system with lumped
and distributed delay under consideration is given. The
feedback stabilization control law is deduced in detail
based on an integral transformation and the backstepping
method, and the main result is presented. The conclusionand some remarks are given in Section 3.

∫ τẆ (t) =0+(B1 + P(0))u(t) + (B2 − P(τ ))u(t − τ )+AW (t)Choosing such a transformation kernel matrix P(·) that{Ṗ(s) − AP(s) + D(s) = 0, s ∈ [0, τ ],(6)
P(0) = −B1 ,we then get thatẆ (t) = AW (t) + (B2 − P(τ ))u(t − τ ).

P(s) = −eAs B1 −D(s)u(t − s)ds,

W (t) = X(t) −−W (t) = X(t) +©ISAROB 2011

t−τ

∫ t

eA(t−s) B1 u(s)dsτ∫ t ∫ t−t−s
t−τ 0

B = B2 + eAτ B1 +eA(t−s−ξ ) D(ξ )u(s)d ξ ds. (9)∫ τeA(τ −s) D(s)ds,

(10)

0

which depends on the constant time delay τ . We thenget the following linear system with lumped time-delay,(4)Ẇ (t) = AW (t) + Bu(t − τ ).

(11)

In one word, we have reduced the system (4) with
distributed input delay into the system (11) with only
the lumped input delay through the transformation (9).
Secondly, we can go a step further to rewrite thesystem (11) asẆ (t) = AW (t) + Bv(0,t),(12)
vt (x,t) = vx (x,t), x ∈ (0, τ ),v(τ ,t) = u(t),where v(0,t) = u(t − τ ) just gives the input in (11). In
the sequel, we discuss under the hypothesis: there exists
such a matrix K that (A + BK) is Hurwitz. It has been
proven in [1] that the orgin of the following system is
exponentially stable ,Ẇ (t) = (A + BK)W (t) + Bϕ (0,t),(13)
ϕt (x,t) = ϕx (x,t),ϕ (τ ,t) = 0.

∀t ∈ [−τ , 0].

P(t − s)u(s)ds,(8)

For the sake of simplicity, we define the matrix B as

In order to design a feedback controller which stabilize
the system (4), we take the following procedure.
Firstly, inspired by the reduction method in [2], [4],we introduce an integral transformation as follows,∫ t∀s ∈ [0, τ ],Hence the integral transformation (5) isis the vector of control input. The constant matrices A ∈Rn×n and B1 , B2 ∈ Rn×m are known. The lumped delay
appears in the term B2 u(t − τ ), while the
distributed de∫lay is represented by the integral term 0τ D(s)u(t − s)ds.
The known function D(·) : [0, τ ] → Rn×m is assumed tobe continuous, and the positive constant τ denotes the
known time delay.
It is to be noticed that X(t) is only the state of lumped
portion of the delay system. The complete state of thesystem (4) at time t is {X(t); u(s),t − τ ≤ s < t}. Thenthe initial conditions are assumed to beu(t) = u0 (t),eA(s−ξ ) D(ξ )d ξ ,0where X(·) ∈ Rn denotes the state vector, and u(·) ∈ Rm

X(0) = X0 ,0+AW (t).

Ẋ(t) = AX(t)∫ + B1 u(t) + B2 u(t − τ )0∫ swhich combined with (7) yields)(∫ τeA(τ −s) (.s)ds u(t − τ )Ẇ (t) =B2 + eAτ B1 +We now consider the following linear time-invariant
system,τ(7)

Obviously, the solution of the Cauchy problem (6) reads

II. Design of feedback controllers and main results+(Ṗ(s) − AP(s) + D(s))u(t − s)ds

(5)

Aiming to transform the system (12) into the target

108

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

system (13), introduce the so-called backstepping transformation,

which together with (9) implies |X(t)| ≤ c1 e−c2 t . This
completes the proof.

ϕ (x,t) = v(x,t) −III. CONCLUSIONSIn this paper, we present a feedback stabilization control law for linear time-invariant systems with lumped
and distributed input delay. The system under consideration in this paper is a general model for linear timeinvariant systems with delay in the control input. The
result obtained in this paper can cover the stabilization
result for linear systems with lumped input delay onlyin [4].
In fact, the system (4) degenerates to the system (1)if D(·) ≡ 0. And the feedback control u(t) given in (18)now reads
(
)
∫ t
Aτ
A(t−s)
u(t) = K e X(t) +e
B2 u(s)ds ,
(21)

∫ x
0

k(x, θ )v(θ ,t)d θ − Q(x)TW (t). (14)The integral kennel k(x, θ ) and Q(x) is determined by
the following system,{kx (x,t) + kt (x,t) = 0, k(x, 0) = Q(x)T B,(15)Q′ (x) = AT Q(x), Q(0) = K T .
Please refer to [1] for the details. It is not difficult toobtain that()Tk(x,t) = KeA(x−t) B, Q(x) = KeAx .
Thus the backstepping transformation (14) reads(∫ x)
ϕ (x,t) = −KeA(x−θ ) Bv(θ ,t)d θ + eAxW (t)t−τ0+v(x,t)which seems to differ by a factor eAτ from (3) derived
in [4]. If examining (20), we then find out that (20)is a new transformation which reduce the time-delayed
system (11) into the delay-free system(16)

Noting u(t) = v(τ ,t) in (12) and ϕ (τ ,t) = 0 in (13), wethen set x = τ in (16) to get
(
)
∫ t
Aτ
A(t−s)
u(t) = K e W (t) +eBu(s)ds .
(17)

Ż(t) = AZ(t) + Bu(t).

t−τThis transformation differs also by a factor eAτ fromthe reduction transformation (2), which coincides withthe difference between (21) and (3). This relation was
discovered too in [7] through discussing the relations between continuous reduction transformation and discreteone.

Hence we actually obtain the feedback control for theoriginal system (4),∫ t ∫ τeA(t+τ −s−ξ ) D(ξ )u(s)d ξ ds)
∫ tA(t−s)e
B2 u(s)ds (18)+K e X(t) +u(t) = Kt−(τ t−sAτ

t−τ

IV. ACKNOWLEDGMENTS
This work was supported by the National Natural
Science Foundation of China (Grant No. 10626002).

Now we are ready to state the main result on thestabilization for the system (4).
Theorem∫ 1: Let K be such a matrix that A + B2 K +eAτ B1 K + 0τ eA(τ −s) D(s)Kds is Hurwitz. The linear
time-invariant system (4) with distributed input delay
can be stabilized by the feedback control u(t) given in(18).
Proof We would like to prove that
∃c1 , c2 > 0,s. t.

|X(t)| +∫ tt−τ

REFERENCES[1] M. Krstic and A. Smyshlyaev, “Backstepping boundary control for first-order hyperbolic PDEs and application to systems with actuator and sensor delays,” Sys.
Control Lett., Vol. 57(9), pp. 750-758, 2008.
[2] Z. Artstein, “Linear systems with delayed controls: a
reduction,” IEEE Trans. on Automat. Contr., Vol. 27(4),pp. 869-879, 1982.
[3] K. Gu and S.-I. Niculescu, “Survey on recent resultsin the stability and control of time-delay systems,”
Transactions of ASME, Vol. 125, pp. 158-165, 2003.
[4] W. H. Kwon and A. E. Pearson, “Feedback stabilization of linear systems with delayed control,” IEEE Trans.
on Automat. Contr., Vol. 25(2), pp. 266-269, 1980.
[5] J. Richard, “Time-delay systems: an overview ofsome recent advances and open problems,” Automatica,
Vol. 39(10), pp. 1667-1694, 2003.
[6] S.-I. Niculescu, Delay Effects on Stability: A RobustControl Approach, Springer-Verlag, Heidelberg, Germany, 2001.
[7] B. Liu, M. Haraguchi and H. Hu, “A new reductionbased LQ control for dynamic systems with a slowly
time-varying delay,” Acta Mech. Sin., Vol. 25(4), pp.
529-537, 2009.

|u(s)|ds ≤ c1 e−c2 t . (19)In the sequel, c1 , c2 stand for generic positive constants,
whose values may change from line to line. In view ofthe form of u(t) given in (17), let u(t) = KZ(t) with
Z(t) = eAτ W (t) +

∫ t

t−τ

eA(t−s) Bu(s)ds.

(20)

It is easy to check that Z(t) satisfies
Ż(t) = (A + BK)Z(t).
By the assumption on K, it is known that
|Z(t)| ≤ c1 e−c2 t
for some positive constants c1 , c2 . So we can conclude
after some simple calculations that
∫ t

t−τ

|u(s)|ds ≤ c1 e−c2 t .

Naturally, |W (t)| decays exponentially in view of (20),©ISAROB 2011

109

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Discrete-Time Iterative Learning Control for Relative Degree Systems:
A 2-D Approach
Deyuan Meng and Yingmin Jia
The Seventh Research Division and the Department of Systems and Control,
Beihang University (BUAA), Beijing 100191, China(E-mail: dymeng23@126.com, ymjia@buaa.edu.cn)

Abstract: This paper is devoted to the two-dimensional (2-D) design problem that arises from discrete-time iterative
learning control (ILC). For linear time-invariant (LTI) systems with well-defined relative degree, a unified ILC algorithm
is considered which provides wider freedom for the updating law formation. It demonstrates that an appropriately defined
variable, together with the tracking error, can be employed to establish the Roesser systems based 2-D description of the
ILC process. This enables both asymptotic stability and monotonic convergence of the relative degree ILC systems tobe achieved. In particular, conditions for the monotonic convergence are described in terms of linear matrix inequalities(LMIs), which directly give formulas for the updating law design.
Keywords: Discrete-time iterative learning control; relative degree; monotonic convergence; linear matrix inequality.

I. INTRODUCTION
Iterative learning control (ILC) is known as an effecttechnique for systems operating repetitively over a fixed
time interval. The key feature of ILC is a fundamentally
two-dimensional (2-D) process [1], with evolution in two
independent directions. In order to take into account theentire dynamics of an ILC, the 2-D analysis approach isfound to be a good alternative which can be implemented
based on the Roesser’s type 2-D systems (see, e.g., [2][7]). Hence, the well-developed theory of 2-D systems
can be employed to deal with the ILC design. However,
the existing 2-D analysis approach is only applicable tothe ILC design for such systems with relative degree thatis not more than one. This is because it is difficult toestablish the 2-D model description of ILC systems withhigher-order relative degree. In fact, the system relative
degree plays a significant role not only in the ILC design
but also in the ILC convergence analysis [8]. Moreover,
the area of monotonically convergent ILC design has
seen relatively little activity when it comes to addressing
systems with higher-order relative degree.
In this paper, the 2-D design approach is investigatedfor discrete-time ILC with relative degree. It shows thatthe 2-D Roesser systems can be established to describe
the entire dynamics involved in ILC with well-definedsystem relative degree. Based on the 2-D system theory,
a convergence analysis of ILC can be directly presented,
and a necessary and sufficient condition can be provided,
which is dependent only upon the first non-zero Markov
parameter matrix. Moreover, after giving the relationshipbetween two sequential iteration tracking errors from the2-D Roesser systems, the monotonic convergence of ILC
can be obtained by applying the bounded real lemma [9].
In this case, it shows that the monotonically convergent©ISAROB 2011

ILC can be designed through the linear matrix inequality(LMI) technique, and formulas can be presented for thecontrol law design. Finally, a simulation test is proposed
to illustrate that the 2-D approach can be used to address
monotonically convergent ILC with relative degree.
Notations: I and 0 denote the identity matrix and thezero matrix with required dimensions, respectively; M >0 (respectively, M < 0) denotes a symmetric positive(respectively, negative) definite matrix; an asterisk (F)denotes a term that is induced by symmetry. Matrices,
if their dimensions are not explicitly stated, are assumed
to be compatible for algebraic operations. For a given
vector xk (t), let q be a shift operator such that q : xk (t) →qxk (t) = xk (t + 1), and ∆ be a difference operator suchthat ∆ : xk (t) → ∆xk (t) = xk+1 (t) − xk (t).
II. ILC SYSTEM DESCRIPTION
Consider the system over t ∈ {0, 1, · · · , T − 1} (shortfor t ∈ [0, T − 1]) and k ∈ Z+ :xk (t + 1) = Axk (t) + Buk (t)yk (t) = Cxk (t), xk (0) = x0 , ∀k(1)where xk (t) ∈ Rn is the state, uk (t) ∈ Rm is the input,yk (t) ∈ Rl is the output, and (A, B, C) is the constantsystem matrix pair of appropriate dimensions.
It is assumed that system (1) has a relative degree ofr ≥ 1 which is defined as follows.
Relative Degree: The relative degree r of system(1) is an integer which can be characterized by thefollowing conditions:
1) CAi B = 0 for all i < r − 1;
2) CAr−1 B 6= 0 and is of full-row rank.
To deal with the relative degree r, the ILC considered•

110

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

in this paper uses an updating law given by

and yk (t) = Cxk (t) leads toruk+1 (t) = uk (t) + ∑ Ki qi ek (t)ek+1 (t) = ∆ek (t) + ek (t)= ∆ [yd (t) − yk (t)] + ek (t)= −∆yk (t) + ek (t)= −C∆xk (t) + ek (t).

(2)

i=0

(7)

where ek (t) = yd (t) − yk (t) is the tracking error, and Ki ,i = 0, 1, . . . , r, is an m × l gain matrix to be designed.
The trajectory yd (t) is the desired output to be tracked
over [r, T + r].
The objective of this paper is to address convergenceand design problems of the ILC system (1) and (2) bydeveloping a 2-D approach under the Roesser systems
framework. To this end, we assume that the initial reset
condition is satisfied, i.e., qi yk (0) = qi yd (0) and, without
loss of generality, qi yk (0) = qi yd (0) = 0 is considered,
where i = 0, . . . , r − 1.

where ∆xk (t), by using the first equation of system (1),satisfiesq∆xk (t) = ∆qxk (t)= ∆xk (t + 1)
(8)
= ∆ [Axk (t) + Buk (t)]= A∆xk (t) + B∆uk (t).

III. MAIN RESULTS

Following the same steps repetitively, it can be developed further from (9) that

Since ∆xk (0) = 0, the use of (8) can yield∆xk (t) = Aq−1 ∆xk (t) + Bq−1 ∆uk (t).

r−1

A. 2-D System Representation∆xk (t) = Ar q−r ∆xk (t) + ∑ A j Bq− j−1 ∆uk (t)First of all, the 2-D Roesser systems based approach
will be developed for ILC with a system relative degreer ≥ 1. To this end, let us denoteη1k (t) = Ar q−r ∆xk (t)




r−1 j



(t)
=
A j BKi q−( j−i)−1 ek (t)
η

2k
∑
∑



j=0 i=0




r−1 r


 η3k (t) = ∑ ∑ A j BKi qi− j−1 ek (t)(3)
j=0 i= j+1



r−1


r
−r+ j

ek (t)
 η4k (t) = ∑ A BK j q


j=0



r−2 r−2



i+1
− j−1

ek (t)
 η5k (t) = ∑ ∑ A BKi− j q

j=0

r−1

∆uk (t).

The ILC law of (2) can be rewritten asr∆uk (t) = ∑ Ki qi ek (t).

(11)

i=0

Then insert (11) into (10) to get
r−1

r

j=0

i=0

∆xk (t) = η1k (t) + ∑ A j Bq− j−1 ∑ Ki qi ek (t)r−1 r= η1k (t) + ∑ ∑ A j BKi qi− j−1 ek (t)j=0 i=0

r−1 j= η1k (t) + ∑ ∑ A j BKi q−( j−i)−1 ek (t)(12)

j=0 i=0

holds when r = 1. For the variables of (3), some properties are given as follows.
Lemma 1: Consider ηik (t) defined in (3), where i = 2,
3, 4, and 5. Then

r−1

+∑

r

∑ A j BKi qi− j−1 ek (t)j=0 i= j+1= η1k (t) + η2k (t) + η3k (t).
In view of (5) and by inserting (12) into (7), it follows
immediately thata) η2k (t) can be rewritten as
r−1 r−1(4)

ek+1 (t) = −C [η1k (t) + η2k (t)] −Cη3k (t) + ek (t)¡¢(13)= −Cξk (t) + I −CAr−1 BKr ek (t)j=0 i= jb) η3k (t) can be such that

where
(5)

c) η4k (t) and η5k (t) satisfy(6)η4k (t) + η5k (t) = Aη2k (t).
Proof: The proof can be immediately derived withsome algebraic operation and, thus, is omitted here.
Now with Lemma 1, let us consider the 2-D representation of ILC systems. The use of ek (t) = yd (t) − yk (t)©ISAROB 2011

(10)

− j−1

j=0

where ∑i−1
j=i (·) j , 0, ∀i. Particularly, η5k (t) = 0 always

Cη3k (t) = CAr−1 BKr ek (t).

j= η1k (t) + ∑ A Bq

j=0 i= jη2k (t) = ∑ ∑ Ai BKi− j q− j−1 ek (t).

(9)ξk (t) = η1k (t) + η2k (t).

(14)

From (13), it is obvious that an iterative equation about
the tracking error is obtained, which can reflect the ILC
system dynamics along the iteration axis k. With thisfact, ξk (t) will be further discussed in order to disclose
the time-domain dynamics involved in the ILC system(1) and (2).
To describe the ILC system dynamics along the time111

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

axis t, next let us consider ξk (t + 1) = qξk (t). Towards
this end, compute qη1k (t) and then insert (8) and (11)to obtain£¤qη1k (t) = q Ar q−r ∆xk (t)= Ar q−r q∆xk (t)= Ar+1 q−r ∆xk (t) + Ar Bq−r ∆uk (t)r= Aη1k (t) + Ar Bq−r ∑ K j q j ek (t)
j=0

r−1

= Aη1k (t) + ∑ Ar BK j q−r+ j ek (t) + Ar BKr ek (t)j=0= Aη1k (t) + η4k (t) + Ar BKr ek (t).

(15)

And with (4), qη2k (t) is computed by
r−1 r−1qη2k (t) = q ∑ ∑ Ai BKi− j q− j−1 ek (t)j=0 i= jr−1 r−1

r−1

j=1 i= ji=0= ∑ ∑ Ai BKi− j q− j ek (t) + ∑ Ai BKi ek (t)r−2 r−2

r−1= ∑ ∑ Ai+1 BKi− j q− j−1 ek (t) + ∑ Ai BKi ek (t)j=0 i= ji=0r−1= η5k (t) + ∑ Ai BKi ek (t).
i=0

(16)

Use (6), (15) and (16) to obtainqξk (t) = qη1k (t) + qη2k (t)r= Aη1k (t) + η4k (t) + η5k (t) + ∑ Ai BKi ek (t)i=0r= A [η1k (t) + η2k (t)] + ∑ Ai BKi ek (t)
i=0

r

i

= Aξk (t) + ∑ A BKi ek (t).
i=0

(17)

Thus, based on (13) and (17), the following 2-D Roesser
model can be established:
·
¸ ·
¸·
¸
A
ξk (t + 1)
ξk (t)∑ri=0 Ai BKi=(18)
ek+1 (t)−C I −CAr−1 BKr ek (t)which clearly describes the two independent dynamics
involved in the ILC system (1) and (2), as claimed in[2]-[4], [6]. For the 2-D model of (18), the use of (3)and (14) yields
r−1 jξk (t) = Ar q−r ∆xk (t) + ∑ ∑ A j BKi q−( j−i)−1 ek (t)j=0 i=0

r−1 j= Ar ∆xk (t − r) + ∑ ∑ A j BKi ek (t − ( j − i) − 1).
j=0 i=0Note that ∆xk (0) = 0 holds, and qi yk (0) = qi yd (0) implies yk (i) = yd (i), i.e., ek (i) = 0, for i = 0, 1, . . . , r − 1.
Hence, it is obvious that ξk (r) = 0 holds for k ∈ Z+ .
©ISAROB 2011

Consequently, the boundary conditions for (18) are:ξk (r) = 0 for k ∈ Z+ and finite e0 (t) for t ∈ [r, T + r].
(19)
Remark 1: From (18), it is clear that Roesser models
can be developed with ξk (t) and ek (t) to describe the 2D processes resulting from the ILC system (1) and (2).
This implies that a general 2-D framework is established
for ILC systems with relative degree. For the particularcase where r = 1, it can be easily shown that the 2-D
Roesser system (18) provides an alternative approach todescribe the ILC systems that have been considered in,e.g., [2]-[4].
B. Convergence Analysis of ILC
With the development of 2-D system representation,
both asymptotic stability and monotonic convergencecan be considered for ILC systems with relative degree.
First, the following result is given for the asymptotic
stability of ILC.
Proposition 1: Consider the ILC system (1) and (2)of the general relative degree r ≥ 1. Then the tracking
error ek (t) converges asymptotically to zero as k → ∞ ifand only if the matrix I¡− CAr−1 BKr is¢ stable, i.e., thespectral radius fulfills ρ I −CAr−1 BKr < 1.
Proof: With the 2-D system theory applied to (18)and (19), the proof is immediate. For more details, see[3] and [4].
Remark 2: From Proposition 1, it is obvious that theasymptotic stability of ILC depends only upon the first
non-zero Markov parameter matrix CAr−1 B, regardlessof the system relative degree r.
Next, the following result is presented for the monotonic convergence of ILC.
Proposition 2: Consider the ILC system (1) and (2)of the general relative degree r ≥ 1. Then the tracking
error ek (t) converges monotonically to zero when k → ∞in the sense of the L2 -norm if there exist scalars ε1 > 0,ε2 > 0 and matrices Q > 0, Xi , i = 0, . . . , r, that satisfy
the following LMIs


−Q
T

QA

r
T
∑ X BT Ai T
i=0 i
0ε1 ≤ ε2
(F)
−Q
0
CQ

(F)
(F)
−ε1 I
ε2 I +CAr−1 BXr



(20)

(F)
(F) 
 < 0.
(F) 
−ε1 I

(21)
If the LMIs of (20) and (21) are feasible, then the gain
matrices are given by
Ki = −ε2−1 Xi , i = 0, . . . , r.
(22)
Proof: With the bounded real lemma (see, e.g., [9])applied, the proof can be proved based on the use of (18)and in the same way as in the proof of [8, Theorem 3]
and, thus, is omitted here.
Remark 3: Proposition 2 implies that although theasymptotic stability of the updating law (2) depends only
on the selection of Kr , its other learning gains can help
112

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Clearly, it is easy to show that CB = 0 and CAB has fullrow rank, resulting in system (1) with a relative degreeof r = 2.
To perform the simulation, the zero initial control input u0 (t) = 0 is used, and the following desired trajectory
is considered:¸·¸ ·20 − 20 cos(0.02π t)
yd1 (t)yd (t) ==yd2 (t)6 · 10−10t 5 − 1.5 · 10−7t 4 + 10−5t 3
where t ∈ [0, 100]. Accordingly, yk (t) = [y1k (t), y2k (t)]T
is denoted for the sake of notations. Then solve LMIs
(20) and (21) with r = 2 to derive
0.04340.1665
K0 = −0.0119 2.4293 
0.0078 −1.0000

0.2855 −0.2664
K1 = −0.0354 1.1861 −0.0007 −2.4293

0.5883 −0.0000
K2 = −0.3300 0.2664  .
0.0309 −1.3525
Now using such gain matrices, we perform the ILC
system (1) and (2), and show the test results in Figs.
1 and 2. Fig. 1 shows the evolution of tracking errors
kyd1 (t) − y1k (t)k2 and kyd2 (t) − y2k (t)k2 with respect
to the iteration number k, and Fig. 2 shows the time
evolution of the reference trajectory yd (t) and actual
output yk (t) for k = 1, 3, 5. From Figs. 1 and 2, it isclear that the ILC process converges monotonically. It
illustrates that the proposed 2-D approach can effectively
address the design of ILC with system relative degree.
V. CONCLUSIONSIn this paper, the 2-D design approach to discrete-time
ILC with relative degree has been discussed. It has been
©ISAROB 2011

3

2

10

10

2

1

10

10

1

0

10

||yd2(t)−y2k(t)||2

10

0

10

−1

10

−2

−2

10

10

−3

−4

10

10

−4

10

−1

10

−3

10

−5

0

5

10

15

20

25
iteration k

30

35

40

45

10

50

0

5

10

15

20

25
iteration k

30

35

40

45

50

Fig.1. Left: The error between yd1 (t) and y1k (t).
Right: The error between yd2 (t) and y2k (t).
40

1.2

yd1

35

yd2

y11

1

y21

y13
30

y23

y15
Reference and output for k=1,3,5

In this example, system (1) is considered with matrices given by:


0
1
0
0
0
0

0
0
1
0
0
0 



0
0
0
1
0
0 


A=
0
0
0
0
1
0 



0
0
0
0
0
1 
−0.0043 0.0004 0.12 0.149 −0.71 −1.7


0 0 0
0 0 0
1 0 0B=0 1 0
0 0 1
0 0 0·¸
0 1 0 0 0 0
C=.
0 0 0 0 0 1||yd1(t)−y1k(t)||2

IV. NUMERICAL EXAMPLE

shown that the asymptotic stability of ILC is dependent
only upon the first non-zero Markov parameter matrix.
Moreover, sufficient conditions have been provided interms of LMIs to guarantee the monotonic convergenceof ILC and give formulas for the updating law design.
For ILC designed through solving LMIs, its effectiveness has been verified finally through simulation test.

Reference and output for k=1,3,5

to achieve the monotonic convergence of ILC to ensure
good performance.

25

20

15

y25

0.8

0.6

0.4

0.2

10

0

5

0

0

10

20

30

40

50
time t

60

70

80

90

100

−0.2

0

10

20

30

40

50
time t

60

70

80

90

100

Fig.2. Left: yd1 (t) and y1k (t) for k = 1, 3, 5.
Right: yd2 (t) and y2k (t) for k = 1, 3, 5.
ACKNOWLEDGMENTS
This work was supported by the NSFC
(60727002, 60774003, 60921001, 90916024), the MOE(20030006003), the COSTIND (A2120061303), and the
National 973 Program (2005CB321902).
R EFERENCES[1] Ahn H.-S., Chen Y., and Moore K. L. (2007), Iterative learning
control: Brief survey and categorization. IEEE Transactions on
Systems, Man, and Cybernetics-Part C: Applications and Reviews
37(6): 1099–1121[2] Geng Z., Carroll R., and Xie J. (1990), Two-dimensional modeland algorithm analysis for a class of iterative learning control
systems. International Journal of Control 52(4): 833–862[3] Kurek J. E. and Zaremba M. B. (1993), Iterative learning control
synthesis based on 2-D system theory. IEEE Transactions on
Automatic Control 38(1): 121–125[4] Saab S. S. (1995), A discrete-time learning control algorithm
for a class of linear time-invariant systems. IEEE Transactions on
Automatic Control 40(6): 1138–1142[5] Li X.-D., Chow T. W. S., and Ho J. K. L. (2005), 2-D systemtheory based iterative learning control for linear continuous systems
with time delays. IEEE Transactions on Circuits and Systems-I:
Regular Papers 52(7): 1421–1430[6] Meng D., Jia Y., Du J., et al (2009), Robust discrete-time iterative
learning control for nonlinear systems with varying initial state
shifts. IEEE Transactions on Automatic Control 54(11): 2626–2631[7] Meng D., Jia Y., Du J., et al (2009), Robust design of a classof time-delay iterative learning control systems with initial shifts.
IEEE Transactions on Circuits and Systems-I: Regular Papers
56(8): 1744–1757[8] Meng D. and Jia Y. (2010), Effects of initial input on stochastic
discrete-time iterative learning control systems. Proceedings of theChinese Control Conference 2193–2200[9] Hsiung K.-L. and Lee L. (1999), Lyapunov inequality andbounded real lemma for discrete-time descriptor systems. IEE
Proceedings-Control Theory and Applications 146(4): 327–331

113

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Distributed Robust Consensus Control of Uncertain Multi-Agent Systems
Yang Liu and Yingmin Jia
The Seventh Research Division and the Department of Systems and Control,
Beihang University (BUAA), Beijing 100191, China(E-mail: ylbuaa@163.com, ymjia@buaa.edu.cn)

Abstract: This paper is devoted to the robust consensus control of multi-agent systems with model parameter uncertainties and external disturbances for networks with switching topology. In particular, a sufficient condition for theconsensus performance with a given H∞ disturbance attenuation level is established for the multi-agent system governed
by general linear differential equations, and meanwhile the unknown feedback matrix of the proposed distributed state
feedback protocol is determined. The condition is given in terms of linear matrix inequalities (LMIs) and can be easily
verified. A numerical example is included to validate the theoretical results.
Keywords: Consensus, Robust H∞ control, Multi-agent systems, External disturbances, Model uncertainties

I. INTRODUCTION
Recently, some researchers have studied the consensus
problem of multi-agent systems with various external
disturbances and random communication noises [1]-[6].
However, unavoidable model and parameter uncertainties in the agents’ dynamics have not been considered in
the existing literature. This motivates us to investigate
the consensus problem for multi-agent systems with bothmodel uncertainties and external disturbances.
In this paper, we study the consensus control for
networks of multiple agents modeled by general highdimensional linear differential equations with bothmodel uncertainties and external disturbances, and propose a distributed protocol with an undetermined state
feedback matrix. In order to use the existing robust
H∞ theory of linear systems, a controlled output isdefined to reformulate the consensus control problem
as a robust H∞ control problem, and a series of modeltransformations are conducted to convert the original
singular closed-loop system to be an equivalent stabilized reduced-order one. Then, sufficient conditionsin terms of LMIs are derived to ensure the consensus
performance with a given H∞ index for the disturbed
multi-agent system without and with model uncertainties
respectively, and the feedback matrix of the proposed
protocol is determined accordingly.
II. PROBLEM REFORMULATION AND
PROTOCOL DESIGN
A. Problem statement and preliminaries
Consider a multi-agent system consisting of n identical agents with the ith one modeled byẋi (t) = Axi (t) + B1 ωi (t) + B2ui (t),(1)

where xi (t) ∈ Rm is the state, ui (t) ∈ Rm2 is the control
input or protocol, and ωi (t) ∈ Rm1 is the external dis-

©ISAROB 2011turbance that belongs to L2 [0, ∞), the space of squareintegrable vector functions over [0, ∞). If system matrices A, B1 , B2 are uncertain, they are assumed to takethe following forms:
A = A0+∆A(t), B1 = B10+∆B1 (t), B2 = B20+∆B2 (t), (2)where A0 , B10 , B20 are constant matrices, and ∆A(t),∆B1 (t), ∆B2 (t) are time-varying uncertain matrices satisfying[∆A(t) ∆B1 (t) ∆B2 (t)] = EΣ(t)[F1 F2 F3 ].

(3)

In (3), E and Fi (i = 1, 2, 3) are constant matrices ofappropriate dimensions, and Σ(t) is an unknown timevarying matrix that satisfies ΣT (t)Σ(t) ≤ I. It is also
assumed that (A0 , B20 ) is stabilized. A protocol ui (t) is
said to asymptotically solve the consensus problem, if
and only if the states of agents satisfy
lim [xi (t) − x j (t)] = 0, ∀i, j ∈ {1, · · · , n} , N .

t→∞Undirected graphs are used to model the interaction topologies among agents. Let G = (V , E , A ) bean undirected weighted graph of order n with the setof nodes V = {v1 , · · · , vn }, the set of undirected edges
E ⊆ V ×V , and a symmetric adjacency matrix A = [ai j ]with weighting factors ai j ≥ 0. It is stipulated that theadjacency elements associated with edges are positive,i.e., (vi , v j ) or (v j , vi ) ∈ E if and only if ai j = a ji >0. In graph G , node vi represents the ith agent, andedge (vi , v j ) represents that information is exchanged
between agents i and j. Then the set of neighbors ofvi is denoted by Ni = {v j ∈ V : (vi , v j ) ∈ E }. The
Laplacian of a weighted graph G is defined as L =D − A, where diagonal matrix D = diag{d1 , · · · , dn } isnamed the degree matrix of G , whose diagonal elements
are di = ∑nj=1 ai j . To describe the variable topologies,
a piecewise-constant switching signal function σ (t) :114

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

[0, ∞) 7→ {1, · · · , M} , M is defined, where M ∈ Z+
denotes the total number of all possible undirectedinteraction graphs. The interaction graph at time instantt is denoted by Gσ (t) , and the corresponding Laplacianis Lσ (t) . In this paper, the switching graphs Gσ (t) are
assumed to be always connected for all σ (t) ∈ M .
B. Problem reformulation
Define controlled output functionszi (t) = xi (t) −1 n∑ x j (t), i = 1, · · · , n
n j=1

(4)

and L1σ (t) is positive definite since the graph Gσ (t) isconnected. For the convenience of discussion, denote
U = [U1 U2 ] with U2 = √1n being its last column. Let
 T  1 (U1 ⊗Im )x̄(t)x̂ (t)
x̂(t) =(U T ⊗Im )x̄(t) =, 2
T(U2 ⊗Im )x̄(t)x̂ (t)
  1  T
(U ⊗Im1 )ω (t)ω̂ (t), 2(9)ω̂(t) =(U T ⊗Im1 )ω (t) = 1Tω̂ (t)(U2 ⊗Im1 )ω (t) T  1 (U1 ⊗Im )z(t)ẑ (t)ẑ(t) =(U T ⊗Im )z(t) =, 2,
T(U2 ⊗Im )z(t)ẑ (t)
whereto reformulate the consensus control problem of multiagent system (1) as the following H∞ control problem:ẋ(t) =(In ⊗ A)x(t)+(In ⊗ B1)ω (t)+(In ⊗ B2)u(t)z(t) =(Lc ⊗ Im )x(t),Therefore, the objective is to design a distributed protocol ui (t) (i ∈ N ) such thatkz(t)k2<γ
k06=ω (t)∈L2 [0,∞) ω (t)k2

ν ∈R

sup

holds, where γ > 0 is a given H∞ performance index.
C. Protocol design and model transformation

Using the neighbors’ local information, the distributed
protocol of agent i is designed asui (t) = K ∑ ai j (t)[xi (t)−x j (t)],j∈Ni (t)(6)

where Ni (t) is the neighbor set of agent i at time instantt, ai j (t) are adjacency elements of the interaction graphGσ (t) , and K ∈ Rm2 ×m is an undetermined feedback
matrix. Substituting protocol (6) into the system (5)results in the following closed-loop systemẋ(t) = (In ⊗A+Lσ (t) ⊗B2 K)x(t)+(In ⊗B1)ω (t)z(t) = (Lc ⊗Im )x(t),(7)

where Lσ (t) is the Laplacian matrix of graph Gσ (t) .
Note that the symmetric matrix Lσ (t) has a zero
eigenvalue. Thus, the state matrix of system (7) can
not be robust stable if A0 is unstable. In order toapply the H∞ theory, we first convert the system (7) tobe an equivalent reduced-order one that is completely
stabilized, by a series of model transformations. This
will be presented in the following.
By Lemmas 1 and 5 of [4], there exists an orthogonal
matrix U ∈ Rn×n such that




L
0
I
0
U TLcU= n−1 ,L̄c ,U TLσ (t)U= 1σ (t) ,L̄σ (t) , (8)0 0
0 0©ISAROB 2011n1
⊗ ( ∑ x j (t)).
nj=1From (7)-(10), we have˙ = (L̄c ⊗A+L̄c L̄σ (t) ⊗B2 K)x̂(t)+(L̄c ⊗B1)ω̂ (t)x̂(t)ẑ(t) = (L̄c ⊗ Im )x̂(t)(5)

where x(t) = [xT1 (t) · · · xTn (t)]T ∈ Rmn , ω (t) =[ω1T (t) · · · ωnT (t)]T ∈ Rm1 n , u(t) = [uT1 (t) · · · uTn (t)]T ∈Rm2 n , z(t) = [zT1 (t) · · · zTn (t)]T ∈ Rmn , and Lc = [Lci j ] ∈Rn×n is defined by(
n−1
n , i= j.
Lci j =
− 1n , i 6= jkTzω (s)k∞ =sup σ̄ (Tzω (jν ))=x̄(t) = x(t) −

(10)

(11)

that can be divided into the following two independent
subsystems:x̂˙1(t)=(In−1⊗A+L1σ (t)⊗B2 K)x̂1(t)+(In−1⊗B1)ω̂ 1(t),Hσ (t) x̂1 (t)+Gω̂ 1(t)(12)

ẑ1(t) =x̂1(t)and x̂˙2 (t) = 0, ẑ2 (t) = 0. Then by the definition of H∞
norm, it can be proved that kTzω (s)k∞ = kTẑω̂ (s)k∞ =kTẑ1 ω̂ 1 (s)k∞ . In addition, the state matrix In−1 ⊗ A +
L1σ (t) ⊗ B2 K can be robust stable by designing an appropriate matrix K, since (A0 , B20 ) is assumed to be stabilized and matrix L1σ (t) is positive definite. Therefore,
we can analyze the H∞ performance of the stabilized
reduced-order system (12) instead of (11). To summarize, the consensus performance of the closed-loop
multi-agent system (7) is achieved with H∞ disturbanceattenuation index γ , if the system (12) is asymptoticallystable and satisfies the H∞ level γ .
III. CONDITIONS FOR ROBUST H∞CONSENSUS
In this section, the robust H∞ performance of switchedsystem (12) is analyzed, and sufficient conditions arederived to ensure the desired consensus performance ofmulti-agent system (1) under the protocol (6). First, weconsider the multi-agent system (1) by neglecting matrix
uncertainties in (2), i.e., matrices A, B1 , B2 are known
constants. Denote λσ (t)i as the ith real positive eigenvalue of matrix Lσ (t) , i = 1, · · · , n − 1. Let σ∗ (t)i∗ andσ ∗ (t)i∗ be the subscripts associated with the minimum
and the maximum nonzero eigenvalues of all Laplacianmatrices Lσ (t) , respectively.
The following lemma is derived from Lemma 3.2 of[5] and Schur Complement Formula:
Lemma 1: For a given index γ > 0, the switchedsystem (12) is asymptotically stable and satisfies
kTẑ1 ω̂ 1 (s)k∞ < γ , if there exists a positive definite matrix

115

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

P ∈ R(n−1)m×(n−1)m such thatHσT(t) P + PHσ (t) + γ −2 PGGT P + I < 0(13)

holds for ∀σ (t) ∈ M .
Theorem 1: Under protocol (6), the multi-agent system (1) achieves consensus with a given H∞ disturbanceattenuation index γ , if there exist P ∈ Rm×m > 0 and
Q ∈ Rm2 ×m such that the linear matrix inequality (LMI) T
PA +AP+λσ (t)i QTBT2+λσ (t)i B2 Q+γ −2B1BT1 P<0 (14)P−I

and σ ∗ (t)i∗ . If the above

is satisfied for σ (t)i = σ∗ (t)i∗
two LMIs are feasible, then the feedback matrix of theconsensus protocol is K = QP−1 .
Proof : By Lemma 1, the system (12) is asymptoticallystable and satisfies kTẑ1 ω̂ 1 (s)k∞ < γ , if there exists a
positive definite matrix P ∈ R(n−1)m×(n−1)m satisfying(13). Particularly, take P = In−1 ⊗ X with X ∈ Rm×m > 0.
Since L1σ (t) is positive definite, there exists an orthogonal matrix U1σ (t) such that U1Tσ (t) L1σ (t)U1σ (t) =diag{λσ (t)1 , · · · , λσ (t)(n−1)}. Let Ū1σ (t) = U1σ (t) ⊗Im . According to the proof of Theorem 3.3 in [5], pre- andpost-multiplying the matrix inequality (13) with Ū1Tσ (t)and Ū1σ (t) , respectively, can yield a group of matrix
inequalities

X A+ATX+λσ (t)i XB2K+λσ (t)i K TBT2X+γ −2XB1BT1X+I<0(15)

σ (t) = 1, · · · , M, i = 1, · · · , n − 1, which are equivalentto (13). Due to Schur Complement Formula, inequality(15) is also equivalent to
(A+λσ (t)i B2 K)TX+X(A+λσ (t)iB2 K)+γ −2XB1BT1X I
<0.
I
−I
(16)
Then pre- and post-multiplying the inequality (16) withdiag{X −1 , I} yields (14) with P = X −1 and Q = KP. To
summarize, the multi-agent system (1) achieves consensus with a given H∞ disturbance attenuation index γ , if
the LMI (14) holds for ∀σ (t)i.
Due to the convex property of LMIs, if (14) holds
when λσ (t)i takes its extreme values λσ∗ (t)i∗ and λσ ∗ (t)i∗ ,then the LMI (14) holds for ∀σ (t)i, and the desiredconsensus performance is guaranteed. Further, if theabove condition is satisfied, then from Q = KP, it
is obtained that the feedback matrix of the proposed
consensus protocol is K = QP−1 .
Based on the previous development, the consensus
condition is now given for the multi-agent system (1)with model uncertainties (2). To achieve this, we adopt
the following lemma.
Lemma 2 [7]: Given symmetric matrices X,Y, Z ∈Rn×n satisfying X ≥ 0, Y < 0, Z ≥ 0, if for any nonzero
vector ζ ∈ Rn , (ζ TY ζ )2 − 4ζ T X ζ ζ T Z ζ > 0 holds, thenthere exists a scalar λ > 0 such that λ 2 X + λ Y + Z < 0.
Theorem 2: Under protocol (6), the multi-agent
system (1) with model uncertainties (2) can achieve©ISAROB 2011consensus with a given H∞ disturbance attenuation index
γ , if for a scalar λ > 0, there exist P ∈ Rm×m > 0 and
Q ∈ Rm2 ×m such that the LMIΨσ (t)i
B10 P λ1 (F1 P+λσ (t)i F3 Q)T
1 T


BT10
−γ −2 I 0

<0
λ F2


P
0
−I
0
1
1
0
−I
λ (F1 P+λσ (t)i F3 Q)λ F2Ψσ (t)i =PAT0+A0 P+λσ (t)i QT BT20+λσ (t)i B20 Q+λ 2 EE T
(17)

is satisfied for σ (t)i = σ∗ (t)i∗ and σ ∗ (t)i∗ . If the above
two LMIs are feasible, then the feedback matrix of theconsensus protocol is K = QP−1 .
Proof : Due to Schur Complement Formula, the matrix
inequality (14) in Theorem 1 is equivalent to T
PA +AP+λσ (t)i QT BT2+λσ (t)i B2 Q B1 P
Ξσ (t)i =
BT1−γ 2 I 0 <0.
P
0 −I

By the definition of negative definite matrices, Ξσ (t)i < 0
if and only if ξ T Ξσ (t)i ξ < 0 holds for any nonzero
vector ξ . Hence, to ensure the desired robust H∞ consensus performance, we only need to find conditions forξ T Ξσ (t)i ξ < 0 in the presence of uncertainties (2).
From (2) and (3), it is obtained that Ξσ (t)i = Γσ (t)i +∆Γσ (t)i , where T
PA0 +A0 P+λσ (t)i QT BT20+λσ (t)i B20 Q B10 P
Γσ (t)i =
BT10−γ 2 I 0 
P
0 −I


∆Γ11
EΣ(t)F2 0∆Γσ (t)i =F2T Σ(t)T E T
0
0
0
0
0
with ∆Γ11 = PF1T Σ(t)T E T +EΣ(t)F1 P+λσ (t)iQT F3T Σ(t)T
E T +λσ (t)i EΣ(t)F3 Q. That is, model uncertainties are decoupled from the determined constant system matrices.
Let ξ = [ξ1T ξ2T ξ3T ]T be a nonzero vector. The inequalityξ T Ξσ (t)i ξ < 0 holds if and only ifξ T Ξσ (t)i ξ=ξ T Γσ (t)i ξ +2ξ1TEΣ(t)[(F1P+λσ (t)i F3 Q)ξ1+F2 ξ2 ]<0(18)

is satisfied for any ΣT (t)Σ(t) ≤ I. Actually, if one takes
Σ(t) =(E T ξ1 )[(F1 P + λσ (t)i F3 Q)ξ1 + F2ξ2 ]T, (19)||E T ξ1 ||2 ||(F1 P + λσ (t)iF3 Q)ξ1 + F2 ξ2 ||2then ξ T Ξσ (t)i ξ reaches its maximum value. Therefore,(18) holds for any Σ(t) satisfying ΣT (t)Σ(t) ≤ I if andonly if it holds when Σ(t) is taken as (19). Instituting(19) into (18) leads toqp(20)
ξ T Γσ (t)i ξ + 2 ξ T X ξ ξ T Zσ (t)i ξ < 0,
where

116



EE T

X= 0
0


0 0
0 0≥00 0

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Z11
(F1 P+λσ (t)i F3 Q)TF2 0
Zσ (t)i =F2T (F1 P + λσ (t)i F3 Q)
F2T F20≥0000
Z11 =(F1 P + λσ (t)i F3 Q)T (F1 P + λσ (t)i F3 Q).

(21)
Obviously, (20) is satisfied if and only if Γσ (t)i < 0 and(ξ T Γσ (t)i ξ )2 − 4ξ TX ξ ξ TZσ (t)i ξ > 0.

(22)

V. CONCLUSIONSThis paper has addressed the consensus control problem for switching networks of autonomous agents withboth model uncertainties and external disturbances byrobust H∞ theory. Time delays arising in the informationexchange among agents are not considered in this paper,and this will be a topic of future research.
1

Using Lemma 2, we obtain that (22) holds if and only
if there exists a scalar λ > 0 satisfying Γσ (t)i + λ 2 X +λ −2 Zσ (t)i < 0. Inserting (21) into the above inequality
results in

3

x1,1(t)0.8

x2,1(t)0.6

x3,1(t)0.4

x4,1(t)

S=



λ ET
0
−1
λ (F1 P + λσ (t)i F3 Q) λ −1 F2

x4,2(t)

xi,1

0

1
0.5

−0.2
0

−0.4

−0.5
−1

−0.8
−1

0

2

4

6

8

−1.5

10

0

2

4

6

t

8

10

t

Fig.2. Left: The first element of states. Right: The second element of states.

.
14

which is further equivalent to (17). Combining with
Theorem 1, we know that if for a scalar λ > 0, there
exist P > 0 and Q such that the LMI (17) holds for
σ (t)i = σ∗ (t)i∗ and σ ∗ (t)i∗ , then the multi-agent system(1) with model uncertainties (2) achieves consensus with
H∞ index γ . Further, if the two LMIs are feasible, then
K = QP−1 is obtained.

IV. SIMULATION RESULTS
Consider a network of four agents, and the matricesin (2) and (3) are given by


 


0 −1
0
1 0
, B10 =
, B20 =
,
A0 =
2 1
1
0 2




0
0
sin(10t) 0E=, Σ(t) =,0.8 0.8
0
0






1.2 0
0.5
0.8 0
F1 =
, F2 =, F3 =
.
0 1.2
0.5
0 0.8
The external disturbance is assumed to be band-limited
white noise. The H∞ performance index is chosen as γ =1. For simplicity, the interaction graphs are constrained
to be within the set shown in Fig. 1, whose nonzero
weighting factors are all 1.

Fig.1. Undirected interaction graphs.

Fig. 2 depicts the state trajectories of four agentsxi (t) = [xi,1 (t) xi,2 (t)]T under the proposed protocol (6).
Fig. 3 gives the energy relationship between the controlled output z(t) and the external disturbance ω (t).
Obviously, the consensus is achieved with H∞ disturbance attenuation index 1.

||z(t)||2||ω(t)||2

12

According to Schur Complement Formula, the matrix
inequality (23) becomesΓσ (t)i ST
< 0,
S
−I

©ISAROB 2011

x3,2(t)

1.5

0.2

(23)
0
0

x2,2(t)2−0.6

Γσ (t)i + STS < 0,x1,2(t)

2.5

xi,2



Energy

10

8

6

4

2

0

0

2

4

6

8

10

t

Fig.3. Energy trajectories of z(t) and ω (t).

ACKNOWLEDGMENTS
This work was supported by the Fundamental Research Funds for the Central Universities, the NSFC
(60727002, 60774003, 60921001, 90916024), the MOE(20030006003), the COSTIND (A2120061303), the National 973 Program (2005CB321902).
REFERENCES[1] Xiao L., Boyd S. and Kim S. (2007), Distributed
average consensus with least-mean-square deviation.
Journal of Parallel and Distributed Computing 67(1):33–46[2] Li T. and Zhang J. (2009), Mean square averageconsensus under measurement noises and fixed topologies: Necessary and sufficient conditions. Automatica
45(8): 1929–1936[3] Li Z., Duan Z. and Huang L. (2009), H∞ controlof networked multi-agent systems. Journal of Systems
Science and Complexity 22: 35–48[4] Lin P., Jia Y. and Li L. (2008), Distributed robust H∞
consensus control in directed networks of agents with
time-delay. Systems and Control Letters 57: 643–653[5] Liu Y. and Jia Y. (2010), H∞ consensus control
of multi-agent systems with switching topology: A dynamic output feedback protocol. International Journal ofControl 83(3): 527–537[6] Liu Y. and Jia Y. (2010), Consensus problem of highorder multi-agent systems with external disturbances: An
H∞ analysis approach. International Journal of Robustand Nonlinear Control 20(14): 1579–1593[7] Jia Y. (2007), Robust H∞ Control (in Chinese).
Beijing: Science Press
117

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Switching synchronization in a heterogeneous agent network
Lei Wang, Yang Liu and Qi-ye Zhang
Laboratory of Mathematics, Information and Behavior of the Ministry of Education, Department of Systems andControl, Beihang University, Beijing 100191, P.R.China.
(Tel:010-82317934)

Abstract: Motivated by the fact that many real-world networks exhibit a mixture feature of time-invariant and timevarying topologies, we propose a heterogeneous agent network as a simple representation. The presented network
consists of two types of nodes: fixed agents and mobile agents, where the connections between fixed agents areconstant, while the mobile agents, abstracted as random walkers in plane, interact with the neighboring agents. Under
the assumption of fast-switching constraint, we further explore synchronized behavior in the heterogeneous network.
The theoretical and numerical results show that the mobile agent density determines synchronization of the consideredheterogeneous network. In particular, compared with the network constructed by the fixed agents, synchronizability isenhanced and a global synchronization appears by introducing a proper mobile agent density.
Keywords:Synchronization, switching topology, mobile agent networks.

I. INTRODUCTION
Synchronization in large-scale networks of coupledchaotic oscillators has been intensively investigated inrecent years. It has been demonstrated that two or morechaotic oscillators can synchronize by mutual couplings
among them, thus a particular interest in this concern
is how the network topology influences the propensityof the coupled individuals to synchronize [1-2]. The
master stability function (MSF) appraoch relates thestability of the fully synchronized state to the spectral
properties of the underlying topological structure, andfurther provides a framework of analyzing the stabilityof synchronous state of large populations of identicaloscillators [3]. Synchronizability of such a networkis then explored based on the notions of MSF andsynchronized region.
So far, most investigations have been established on
static networks, partially because of a successful performance of the MSF approach in dealing with synchronization problems of static networks. Having examineda variety of network topologies in reality, the classicalstatic network is very restrictive and only reflects a few
practical situations. Then the case of connections which
do evolve in time is more realistic to depict complex
networks, and various synchronization results have been
deduced for complex networks with switching topology[4-7]. Of particular interest is synchronization of a setof mobile agents. The mobile agent network, indeed,
can be used to explore many problems such as clock
synchronization in mobile robots [8], swarming animals
or the appearance of synchronized bulk oscillations [9],consensus problem in multi-agent systems [10] and soon.
None of two cases above mentioned, i.e. static network model and mobile agent network model, however,©ISAROB 2011

also seems to be an adequate description of many
relevant phenomena. For instance, in social networks,
lobby groups go about inducing voters whose attitudes
are already interacted by a fixed relationship to electa candidate or to give up an initial view, where thelobbies can be depicted by mobile agents, and static
topology seems to be more suitable to characterize interactions between voters [11]; in communication systems,
a mobile wireless network attaches the physical networkby clock synchronization so that data is transmitted
and processed [12]; and in volleyball, the libero as amobile agent influences the whole team cohesion, whileother players share relatively fixed connections. Roughly
speaking, there are two types of nodes in all systems
above mentioned, and the network corresponding to
the system can be decomposed into a (relative) static
subnetwork and a switching subnetwork due to theheterogeneity of nodes. Then there appears a question:
Is synchronization of the heterogeneous network easierto achieve or not under the existence of mobile agents.
There is no doubt, lobbies in social networks, mobile
wireless sensors, or volleyball libero player, seem towork as the role of pinned nodes — guiding theirneighbor nodes towards the desired objective — insynchronizing a complex network of coupled systems
through pinning. This paper is an attempt to explore
synchronized behavior based on a heterogeneous agent
network model. In this paper, we present a heterogeneous agent network model to characterize a mixture
feature of real-world network. Under fast-switching constraints, we investigate the synchronization problem of
the heterogeneous agent network. Particularly, we focus
on the effect of mobile agents to synchronization of thestatic network, which provides an insight into regulatory
mechanisms and design of complex systems.
118

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

II. A HETEROGENEOUS AGENT NETWORKMODEL
Generally, a complex network consisting of l linearlyand diffusively coupled nodes is described bylẋi = f(xi ) − σ ∑ Gsij Hx j , i = 1, 2, · · · , l,

(1)

j=1

where xi = (xi1 , xi2 , · · · , xin )T ∈ Rn is the state vector ofnode i, f(·) : Rn → Rn is a smooth vector-valued function,
governing the dynamics of each isolated node, σ > 0 is
the overall coupling strength, H ∈ Rn×n is the inner linking matrix, and the coupling matrix Gs = (Gsij ) ∈ Rl×l
is a zero-row sum matrix, describing network topology.
If network (1) is connected in the sense of having
no isolated clusters and edges signify the bidirectional
ability, then Gs is symmetric and all its eigenvalues areranked as 0 = µ1 < µ2 ≤ · · · ≤ µl , where eigenratio Rs =µl /µ2 is used to measure network synchronizability.
To obtain a heterogeneous agent network, we firstassign the l nodes in network (1) to be fixed agents.
For simplicity, we denote by Nl the set of fixed agents,
and all agents in Nl are uniformly distributed in a twodimensional space of size L with periodic boundary conditions . Moreover, we introduce m mobile agents to theplane, each of which is considered as a random walker
whose position and orientation are updated according to½
yi (t + ∆t) = yi (t) + vi (t)∆t,(2)
θi (t + ∆t) = ηi (t + ∆t),where i ∈ Nm , Nm is the set of mobile agents, yi (t)is the position of agent i in the plane at time t,
ηi (t), i ∈ Nm are independent random variables chosen
at each time unit with uniform probability from theinterval [−π , π ], vi (t) is the velocity of agent i, and∆t is the time unit. In the following, assume that thetime unit is sufficiently small so that fast-switching
synchronization is guaranteed. Similar to Ref.[5], eachagent of the heterogeneous network is associated witha chaotic oscillator whose state variable is characterizedby xi ∈ Rn . Then agent i evolves according to ẋi = f(xi ).
Without lack of generality, we consider the case ofRössler oscillators, where the¡ state ¢dynamics of eachagent is given
by
ẋ
=
−
xi2 + xi3 , ẋi2 = xi1 + axi2 ,i1¡¢
ẋi3 = b+xi3 xi1 −c with xi = (xi1 , xi2 , xi2 )T , and a = 0.2,
b = 0.2, c = 7.
It is obvious that, the heterogeneous agent networkof order N can be conveniently described by graph
G = {N , ε }, where N = Nl ∪ Nm is the node set(representing N agents) and ε ⊂ N × N is the edge setof the graph, which is defined as: Each mobile agent,i ∈ Nm , interacts at a given time with only those agents
located within a neighborhood of an interaction radiusaccording to the rule of moving neighborhood network.
In detail, agents i and j are said to be adjacent if andonly if|yi (t) − y j (t)| < r, ∀i ∈ Nm , j ∈ N©ISAROB 2011(3)

at time t, where r is a parameter that defines the sizeof a neighborhood, | · | refers to an induced norm.
For any two fixed agents, denoted by i, j ∈ Nl , theconnection between them is a constant, i.e., Gi j = Gsij .
In other words, the constant matrix Gs describes the
topology of network Gl = {Nl , ε }. Hence, we construct
a heterogeneous agent network by combining fixed andmobile agents, chaotic oscillators and their dynamicallaws, where the heterogeneous couplings include thetime-invariant connections between nodes in Nl and theswitching connections due to the moving of agents.
Based on above assumptions, the heterogeneous network can be mathematically formulated as:Nẋi = f(xi ) − σ ∑ Gi j (t)Hx j , i ∈ N ,

(4)

j=1

where the entries of the coupling matrix G(t) =(Gi j (t)) ∈ RN×N are as follows: for non-diagonal entries,
Gi j (t) = Gsij if i, j ∈ Nl , and Gi j (t) = G ji (t) = −1 ifagent i ∈ Nm or agent j ∈ Nm are adjacent at time t; andthe diagonal entries satisfy Gii (t) = − ∑Nj=1, j6=i Gi j (t).
Thus there exists a completely synchronized state in
network (4), i.e., x1 (t) = x2 (t) = · · · = xN (t) = s(t),which is a solution of an isolated node ṡ = f(s).
III. ANALYSIS OF SWITCHINGSYNCHRONIZATIONThis section investigates the synchronized behaviorof the heterogeneous network under the constraint of
fast-switching. As shown in Ref.[4], Stilwell et al. consider a switching network topology of coupled chaotic
oscillators and provide a fast-switching synchronization
criterion. Following this result, we will show that synchronization of network (4) can be also assessed by aparticular static network.
We first give an average of G(t) for network (4). Consider an infinite sequence of contiguous time intervals[tk ,tk+1 ), k = 0, 1, · · · , with t0 = 0 and tk+1 − tk = ∆t.
It is easy to see that G(t) is a constant matrix for anyt ∈ [tk ,tk+1 ) and k = 0, 1, · · · . For simplicity, let Gk be theconstant coupling matrix at k-th interval [tk ,tk+1 ), thenwe derive the average of coupling matrix G(t) satisfyingoG = ∑ pi Gi ,

(5)

i=1

where pi is the probability that topological configuration
i occurs, o is the number of possible configurations.
Recalling the evolution of network (4), we learn thatthe connections between any fixed agents are timeinvariant, i.e., Gi j = Gi j (t) = Gsij , ∀i, j ∈ Nl . And foriother cases, Gi j = ∑m
i=1 pi Gi j , i ∈ Nm or j ∈ Nm . Therefore, we write down the non-diagonal entries of G for
network (4):
½ s
Gi j ,
if i, j ∈ Nl
Gi j =(6)−p,otherwise
119

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

where p = π r2 /L2 is the probability that two agents areneighbors, Il is an l × l identity matrix. By elementarytransformation, we calculate the N eigenvalues of theaverage Laplacian G as½¾
λi = 0, µ j + mp, pN, · · · , pN , j = 2, · · · , l .
(7)| {z }m

Note that G is a symmetric constant matrix, thensynchronization of switching network (4) can be investigated by the network readingNẋi = f(xi ) − σ ∑ Gi j Hx j .

(8)

j=1

¡ TLetT ei be Tthe
¢T variation on the i-th node and e =e1 , e2 , · · · , eN
be the collection of variations.
Then£linearizing
network
(8)
at
x
=
s
yields
ė
=
I
⊗
J
(s) −
i
N
f
¤
σ G ⊗ H e, where J f is the Jacobian of the function
f evaluated at s(t), and ⊗ stands for the Kronecker
product. It is easy to verify that the linear stability of thesynchronized state s(t) for network (8) can be studied
by diagonalizing the variational equations of network (8)into N blocks of the form

ξ̇i = (J f − σ λi H)ξi , i = 1, · · · , N,

(9)where ξi = (U ⊗ In )ei ∈ Rn , U ∈ RN×N is a unitary matrix such that U T GU = diag(λ1 , · · · , λN ). Furthermore,
the synchronized state s(t) is stable if the Lyapunov
exponents for the N blocks are negative i.e., Γ(σ µi ) < 0
for i = 2, · · · , N. For the coupled Rössler oscillator with
H = diag(1, 0, 0), there is a single interval (γ1 , γ2 ), in
which the largest Lyapunov exponent is negative, where
γ1 and γ2 are constant. Therefore, synchronization of
network (4) can be guaranteed byγ1
γ2< λ2 < λN < ,(10)σσwhere λ2 and λN are the second smallest and largest
eigenvalues of matrix G, respectively. Based on Eq.(7)and Eq.(10), we thus derive the synchronization condition for the heterogeneous network.
IV. DISCUSSIONS AND NUMERICALSIMULATIONSρl = l/L2 is the fixed agent density and ρm = m/L2
is the mobile agent density. Namely, a smaller ρmprobably means the heterogeneous network (4) is harderto achieve synchronization from the point of view of
the interval width in Eq.(10), while synchronization isprobably easier to realize by assigning a larger mobile
agent density.
Though a larger mobile agent density ρm means abetter synchronizability of network (4), it is likely to leadto the largest eigenvalue of G over the upper bound inEq.(10). And synchronization is lost with a large mobile
agent density for a particular heterogeneous network.
According to Eq.(10), an upper bound of ρm is given byγ2µlρm < ρmu =− max{ρl , }.
(12)
σκ
κ
An obvious result is that, no matter what value of ρmis, the heterogeneous network (4) is not synchronizable
about synchronized state in the condition of µl > γ2 /σ ,where the expression µl > γ2 /σ implies a nonsynchronized motion of static network (1). It is not difficult to
see the existence of mobile agents fails to synchronize
the considered heterogeneous network. As a result, wefavor the introduction of mobile agents for those static
networks (1) whose eigencoupling σ µl is located in thenegative region of the MSF. In the following, we always
assume that σ µl < γ2 holds.
Similarly, we derive a lower bound of mobile agent
density from Eq.(10), i.e.,
γ1µ2ρm > ρml =− min{ρl , }.
(13)
σκ
κ
To validate our theoretical findings, we consider thestatic network Gs to be the case of a Barabási-Albert(BA) network [13], where the parameters of BA model
are given by m0 = m = 3, and the degree distribution follows a power law. Fig.1 reports a numerical simulation,
where synchronization error δ x(t) = (∑Ni=2 ||xi −x1 ||)/N.
As explained in Fig.1, the considered network achieves
synchronization again when ρm > 0.2. Also notice that asynchronized motion disappears as ρm > 0.5 due to thebounded synchronization region.

The existence of mobile agents affects the eigenvalues of G, which further plays an important role insynchronizing network (4). It is noted that Eq.(10) is
fulfilled for some values of σ when the eigenratio R
satisfies the following inequality R ≡ λλN < γγ21 . We then2characterize, similarly to the definition in static network,
the synchronizability of network (4) with R.
Compared with the static network (1), the heterogeneous network (4) shows a better synchronizability if
R < Rs . By solving this inequality, we derive ρm > ρmc ,where ρmc is a critical value of ρm satisfying

ρmc =©ISAROB 2011

Rs
µ2
1
µ2
· max{0,− 1, s −
},
s
R −1
κρl
R
κρl

2
0.4
1.5

<δ x>

0.2
1
0

0

0.5

1

0.5

0

(11)

0

0.1

0.2

0.3
ρm

0.4

0.5

0.6

Fig.1. Synchronization index < δ x > vs mobile agent density ρm .

120

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

which its two end-points are determined by the fixed
agent density and the static topology. It is worth noting
that, compared with the static network, synchronizability
can be enhanced when a proper density of mobile agentsis introduced to the heterogeneous network. All these results may provide some insights for the future theoreticalinvestigations and practical engineering designs.

1412

δ x(t)

10
8
6
4

VI. ACKNOWLEDGMENTS

2
0

0

200

400

600

800

1000

t

This work is supported by the Fundamental ResearchFunds for the Central Universities, the National Natural
Science Foundation of China No. 61004106.

Fig.2. Synchronization index < δ x > vs mobile agent density ρm .

Under the other case σ µ2 < γ1 , it has been shownby MSF that static network (1) cannot synchronize.
However, synchronization can be easily realized byintroducing some, even only one mobile agent to network (1) according to Eq.(13). A numerical example isgiven in Fig.2 to validate the analytical result, whereµ2 is assigned to be zero. It is obvious that Gl is an
unconnected graph, then a global synchronization of
static network (1) cannot be accessed due to isolated
clusters in Gl . We observe from Fig.2 that adding several
mobile agents (in simulations, five mobile agents are
introduced to network at t = 400s) can guarantee network synchronization. The role of mobile agents works
as a bridge which creates connections among different
isolated clusters.
From above discussions, there does exist a boundedregion of mobile agent density: synchronization of network (4) is ensured if and only if ρm ∈ (ρml , ρmu ).
For a particular static network (1) with given Gs andρm , a too large or a too small mobile agent density
will prevent heterogeneous network (4) from achieving
synchronization.
V. CONCLUSIONSIn this paper, we propose a heterogenous agent network to capture a mixture feature of time-invariant
and time-varying topologies existing in many real-world
complex network. The heterogeneous network consistsof a certain number of mobile agents and fixed agents,
each of which is equipped with a chaotic oscillator ina planar space. In particular, the connections between
fixed agents are assigned to be time-invariant, and themobile agents, abstracted as random walkers, interact
with the neighboring agents. Then the heterogenous
agent network can be simply regarded as a mixture of astatic subnetwork and a switching subnetwork. Under
the constraint of fast-switching, we theoretically andnumerically show that synchronization of the heterogeneous network depends on the mobile agent density, thefixed agent density and the spectrum of time-invariant
subnetwork. For a given heterogeneous network, synchronization motion can be established if mobile agent
density of the network lies in an bounded interval, in©ISAROB 2011

REFERENCES[1] S. Boccaletti, V. Latora, Y. Moreno, M. Chavez
and D.-U. Hwang, “Complex networks: Structure anddynamics,” Phys. Rep., Vol. 424, pp. 175-308, 2006.
[2] A. Arenas, A. Dı́az-Guilera, J. Kurths, Y. Moreno
and C. S. Zhou, “Synchronization in complex networks,”
Phys. Rep., Vol. 469, pp. 93-153, 2008.
[3] L. M. Pecora and T. L. Carroll, “Master stability
functions for synchronized coupled sytems,” Phys. Rev.
Lett. Vol. 80, pp. 2109, 1998.
[4] D. J. Stilwell, E. M. Bollt and D. G. Roberson,“Sufficient conditions for fast switching synchronizationin time varying network topologies,” SIAM J. Appl. Dyn.
Syst., Vol. 5, pp. 140-156, 2006.
[5] M. Frasca, A. Buscarino, A. Rizzo, L. Fortuna,and S. Boccaletti, “Synchronization of moving chaotic
agents,” Phys. Rev. Lett., Vol. 100, pp. 044102, 2008.
[6] J. Zhao, D. J. Hill, T. Liu,“ Synchronization ofcomplex dynamical networks with switching topology:
a switched system point of view,” Automaica, Vol. 45,
pp. 2502-2511, 2009 .
[7] L. Wang, H. Shi, Y. X. Sun, “Power adaptation fora mobile agent network,” Europhys. Lett., Vol. 90, pp.
1001, 2010.
[8] A. Buscarino, L. Fortuna, M. Frasca, and A. Rizzo,“Dynamical Network Interactions In. Distributed Control Of Robots,” Chaos, Vol. 16, pp. 015116, 2006.
[9] S. DanóS, P. G. Sórensen, and F. Hynne, “Sustained
oscillations in living cells,” Nature, Vol. 402, pp. 320322, 1999.
[10] Y. Hong, L. Gao, D. Cheng, and J. Hu, “Lyapunovbased approach to multiagent systems with switching
jointly connected interconnection,” IEEE Trans. on Automatic Control, Vol. 52, pp. 943-948, 2007.
[11] F. Amblard and G. Deffuant, “The role of network
topology on extremism propagation with the relative
agreement opinion dynamics,” Physica A, Vol. 343, pp.
725-738, 2004.
[12] H. Karl, and A. Willig, Protocols and Architectures
for Wireless Sensor Networks John Wiley & Sons, Inc,New York, 2005.
[13] Barabási A L, and Albert R, “Emergence of scaling
in random networks,” Science, Vol. 286, pp. 509-512,
1999.
121

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Model Matching Adaptive Control of Time Delay Systems with
Unknown Relative Degree
Haixia Su and Yingmin Jia
The Seventh Research Division and the Department of Systems and Control, Beihang University (BUAA),
Beijing 100191, China(Tel : +86-10-82317661; Fax : +86-10-82316100)(haixsu@yahoo.cn, ymjia@buaa.edu.cn)

Abstract: This paper considers the adaptive control problem of time delay systems with unknown relative degree
based on model matching technique. For single-input single-output (SISO) systems, the only known knowledge of therelative degree is the upper bound of it. An adaptive control scheme is designed so that all signals in the close-loopsystems are bounded and the tacking error can converge to zero. A simulation example is included to illustrate theproposed adaptive control scheme.
Keywords: Model matching control; adaptive control; time delay; unknown relative degree

matching control of multiple-output-delay systems wasgiven in [4]. However, only unknown parameter
uncertainty was considered in [3-4]. In this paper, we
will consider the adaptive control of time delay systems
with unknown relative degree.
A strict assumption is that the relative degree isexactly known in the adaptive control literature. This
assumption was relaxed in [7] for plants with relative
degree n* satisfying 1  n*  3 . A new model reference
adaptive control (MRAC) scheme was proposed in [8].
Where the control required only the known upper boundof the relative degree, this result was obtained at theexpense of additional complexity in the control andadaptive laws. However, to the best of our knowledge,
there is few results about time delay systems withunknown relative degree yet. In this paper, a class of
SISO delay systems with unknown relative degree isconsidered. An adaptive control scheme is designed
using the model matching technique.
This paper is organized as follows. Section 2 isproblem statement. In section 3, an adaptive control
scheme is designed for SISO delay systems and thestability analysis is completed. A simulation example isgiven in section 4 to illustrate the designed scheme. The
last section is a conclusion of this paper.

I. INTRODUCTION
Time delay exists in many industrial control systems
such as chemical process systems, hydraulicallyactuated systems and combustion systems. Researches
have paid much attention to the control of time delaysystems since last century. Stability analysis andcontroller design for delay systems are more difficult
than delay-free systems. Many methods have been
proposed to deal with time delay systems. The known
smith predictor proposed in [1] could cancel the timedelay from the characteristic equation of the closed-loopsystems. The finite spectrum assignment method in [6]could assign the eigenvalues of the closed-loop plant atarbitrary prescribed place of the complex plane.
However, the two methods are difficult to be applied toadaptive control.
Model matching technique can be easily used toadaptive control scheme design. This technique is todesign a controller so that the transfer function of theclosed-loop plant coincides exactly with the transfer
function of the reference model. Controller design for
linear systems based on model matching technique canbe found in the book [5]. This method was used tocontroller design for SISO delay systems in [2]. Thenthe result was extended to adaptive control in [3]. For
multivariable delay systems, a general solution of model

II. PROBLEM STATEMENT

This work was supported by the NSFC (60727002,
60774003, 60921001, 90916024), the MOD (20030006003), the COSTIND (A2120061303), the National 973
program (2005CB321902).

©ISAROB 2011

Consider the SISO linear time-invariant time delaysystems
gr ( s )  Ls
(1)y( s) e u ( s)p( s )

122

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

g
g 
0

Where y(t) and u(t) are the output and the input,
respectively. g is the gain, L is the known time delay.
r ( s ) and p ( s ) are monic polynomials with degree mand n respectively, denote [ r ( s )]  m, [ p ( s )]  n ,where
m and n are unknown constants. The following
assumptions are made for the plant (1).
Assumption 1: r(s) is a Hurwitz polynomial.
Assumption 2: The upper bound n of the unknown
degree n of p(s) is known, i.e. n  n .
*Assumption 3:The relative degree n satisfies
*
*
*
*
*
1  nl  n  nu , where nl and nu are known
constants.
The reference model is chosen to be(2)
g d rd ( s )  Ls

yd ( s ) 

pd ( s )*r * ( s) p( s)   ( s) n  nl  n i e zi L
(7)
 
1 g
r * ( s) p( s)i 1 s  zi
Remark 1: In order to employ the precompensator
In (7)r * ( s) p( s )  gr ( s ) p* ( s)(8)r * ( s) p(s)should satisfy
[r * ( s )]  [ p ( s )]  [ r ( s )]  [ p* ( s )] (9)i.e.
[ r * ( s )]  [ p* ( s )]  [ p ( s)]  [r ( s)](10) n  n*e  ( s) n  n*l

Therefore, the monic stable polynomial r * ( s) should
be chosen as [ r * ( s )]  n  nl* . In this paper, wechoose [r * ( s )]  n  nl* .
Define a polynomial equation by(11)
k ( s ) p ( s )  gh( s ) r ( s )  gr * ( s ) p ( s )   ( s )where k(s) and h(s) are unknown polynomials.
Theorem 1. There are solutions k(s) and h(s)for the polynomial equation (11) with degrees[k ( s )]  n  nl*  1 and [ h( s )]  n  1 .
Proof. From the equation (7), gr * ( s ) p ( s )   ( s )is of degree at most n  nl*  n  1 . It is know that

III. ADAPTIVE CONTROL DESIGNFirst, we design the model matching controller
structure for the systems (1). The controller design
procedure is different from that the known relative
degree case.
Choose monic stable polynomials r * ( s ) and p* ( s ) ,[ p* ( s )]  n , [r*(s)]  n  nl* . Then the reference model

there are unique polynomial solutions k(s) and h(s).
k(s) of degree at most n  nl*  1 , polynomial h(s)of degree at most n-1. However, the degree n ofp ( s ) is not known, the only knowledge of it isn  n . Therefore, h(s) is of degree at most n  1 .
The proof is completed.
Using the equations (5) and (7), we have thefollowing integral

can be rewritten asr * ( s )  Ls g d rd ( s ) p* ( s )ep* ( s )pd ( s ) r * ( s )

(3)

L

g d rd ( s ) p* ( s )pd ( s )r * ( s )(4)n  nl*  n



i 1

i

s  zi

1 g

i 1



s

i
s  zin  nl*  nu (s)  i 1

i e Lzis  ziu ( s )e  Ls

gr ( s ) p* ( s ) ( s)u ( s )  gu ( s )  *u ( s )e  Ls*
r (s) p(s)r (s) p(s) gu ( s )e  Ls
Using the equation (11), the above equation (12)can be rewritten as(5)

where z i are roots of r * ( s ) for k  1, 2, , n  nl*

n nl* n   e u(s)e d
0

and roots of p(s) for k  n  nl*  1, , n  nl*  n ,Lrespectively. Obviously, we have

©ISAROB 2011

i 1

 

 zi

i

n  nl*  nObviously,  is a realizable dynamical input signal.
When both r * ( s ) and p(s) have single or disti
nct roots, writer * ( s) p( s)  gr ( s) p* ( s)r * ( s) p( s)n  nl*  n   e u ( s )e d
0

Denote(6)nl*  n*  nu*

Define the polynomial  ( s ) satisfying the equation

Where pd  s  is a stable polynomial, its degree is n ,the relative degree of the reference model satisfies
[ pd ( s)]  [rd ( s )]  nu* . The reference input  (t ) is a
uniform bounded piecewise continuous signal. The
objective is to design an adaptive control scheme so thatall signals in the closed-loop systems are bounded andthe plant output tracks the reference model output as
close as possible for any given reference input.

yd ( s ) 

n*  nl*

123

i1

 zi

i

s

(2)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

The system output is equal to the reference model
output. Therefore, the controller in (18) is the desired controller. In time domain, the controller canbe rewritten as
1 k ( p)h( p)gr ( p)u(t )  { *u(t  L)  *y(t )  *u(t )g r ( p)r ( p)r ( p)

gr ( s ) p * ( s )k (s)u (s)  *u ( s ) e  Ls*
r (s) p(s)
r (s)
(13)
h(s)
 *
y ( s )  gu ( s )r (s)Another delay compensator is needed to design thecontroller. Choose any monic stable polynomialr ( s ) with degree [r ( s )]  [r * ( s )]  n  nl* , then

 ( p)

we have*n  nlgr ( s) * g   ir ( s)i 1 s  zi



(14)

n  nl* e zi L * g   ir ( s)i 1 s  zin  nl*

 e
L

i 1

 zi

i

(15)



*

respectively, where the coefficients of the polynomi
als are unknown constants. Define parameter vector1  [ k n  n * 1 ,  , k 0 ,  , h n 1 ,  , h0 ,lg(23)
q n  n * ,  , q 0 , g , g ,1]T ，

0n  nl*  e  u ( s)e d
 zi

l

s

i

i 1

 

L

 gu ( s )e  Ls   ( s )Substituting the controller in the equation (18) into
the equation (17), we can obtaingr ( s ) p* ( s )(19)u (s)  r * (s) p(s)It is noting that the polynomials p* ( s ) and r * ( s ) arechosen to be stable polynomials. Therefore, theabove equation (19) can be further rewritten as©ISAROB 2011

(24)

(25)

where ˆ(t ), ˆ (t ,  ) are the estimates of the realparameters  ,  ( ) , respectively. Define the tracking error by(26)e  t   y  t   yd  t sgr ( s)  Lsr (s)e u( s)  * e Lsp( s)p ( s)*0u (t )  ˆT (t ) (t )   ˆ (t ,  )u (t   )d

i

*

i 1

*

  e  u (s)e d  gu (s)
i 1

i

1
pn nl1y(t),
u(t  L),,  * u(t  L),**r ( p)r ( p)r ( p)r ( p)
 u(t ), * u(t )  u(t  L), (t )]Tr ( p)
Then the controller (22) can be represented as

*

L

i

pn nl 11
pn 1u(t  L),，* u(t  L), *y(t ),,r ( p)r ( p)r ( p)

(t)  [ *n  nl  n0
 (s) * u ( s )e  Ls     i e  zi u ( s )e s d (18)
L
r (s)
i 1
 zi



n  n l* i e  z    i e  z

and the signal vector

 gu ( s )  gu ( s )  gu ( s )e  Ls
Therefore, we can choose the controller u as
1 k ( s)h( s )
gr ( s )u ( s )  { * u ( s )e  Ls  *y(s)  *u (s)g r (s)
r (s)
r (s)

0

n  n l*  n
i 1

(17)

h( s )
gr ( s ) (s)y(s)  *u ( s )  * u ( s )e  Ls *r ( s)
r (s)
r (s)



*

l

h( s )  hn 1s n 1  hn  2 s n  2    h0

(16)

gr ( s ) p* ( s )k (s)u ( s )  * u ( s )e  Ls *r ( s) p( s)r ( s)n  nl*

*
l

*
l

l

L

i 1

*
l

k ( s )  kn  n* 1s n  nl 1  k n  n*  2 s n  nl  2    k0，(22)

i

L

i 1

*
l

 gu ( s )e  Ls
Combining the two integrals in the equation (13)and (16) yields

 i e z u ( s)e s d  

 zi

i

 ( s )  qn  n s n  n  qn  n 1s n  n 1    q0，

u ( s )e s dn  nl*  n  e  u(t   )d  gu(t )[ h( s )]  n  1 . Thus, they can be written as

gr ( s ) (s)u ( s )  gu ( s )  * u ( s )e  Ls
 *
r (s)
r ( s)

0n  nl*(21)

 gu(t  L)   (t )}where p is the differential operator in time domain.
Because [ ( s )]  n  nl* , [ k ( s )]  n  nl*  1 , andUsing the equations (14) and (15), we can obtain
another integral
0

0

L

Define the polynomial  ( s ) by

 ( s)n nl*  nu(t  L)    i e zi u(t   )d *Lr ( p)i 1
0

Theorem 2. The tracking error can be represent
ed by the following equationr* ( p)  L Te(t )  g *q { (t )(t )(27)
p ( p)0  (t , )u(t   )d}Lwhere  (t )  ˆ(t )   ,  (t ,  )  ˆ (t ,  )   ( ) . q  Ldenotes a time delay operator, q  Lu (t )  u (t  L) .

(20)

124

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

In order to design the adaptive law of thecontroller parameters, define a signal by0  T (t ) (t )   (t ,  )u (t   )dWhen the reference input is the unit step signal, use theadaptive control scheme designed in this paper and theparameters       2 , the simulation result isgiven in Figure 1. Figure 1 shows that the tracking error
converges to zero. The designed adaptive control
scheme achieves the control objective.





L

(28)
0
r ( p)  L T
 *q { (t ) (t )    (t ,  )u (t   )d }Lp ( p)
wherer * ( p)r * ( p)(29)
 (t )  *  (t  L), u (t )  * u(t  L)p ( p)p ( p)
The augmented error is defined by* (t )  e(t )  gˆ (t ) (t )



0.08
e
0.06
0.04
0.02

0
 g T (t ) (t )    (t ,  )u (t   )d
L



0

(30)

−0.02
−0.04

 g (t ) (t )−0.06

Define a signal by−0.08

 (t )  [ (t ), sup u (t   ), (t )]

T

(31)

−0.1

0

200

400

600

800

1000

 L   0

Choose the following adaptive law
 (t )gˆ ( t )  ,1‖  ( t ‖) 2(32) (t )
ˆ ( t )   ,21‖  ( t ‖)u (t   )ˆ ( t ,  )   ,  L    0,1‖  ( t ‖) 2where  ,  ,  are positive constant parameters to
be chosen.
Theorem 3: The adaptive control scheme consi
sts of the controller (25) and the adaptive law (32)designed for the plant (1), it can guarantee that all signals in the closed-loop plant are bounded and t
he tracking error converges to zero.
Proof. Without loss of generality, g is assumed
to be positive. Consider a Lyapunov function
1 2
g T 
g 0 2
V (t ) g (t ) 
 (t ) (t )    (t ,  )d (33)222  LThe proof procedure is similar to the proof of thetheorem 7 in [5]. Therefore, it is not detailed here.

Figure 1. Tracking error of the system.

V. CONCLUSION
This paper design adaptive control schemes fordelay systems with unknown relative degree. The
schemes are obtained at the expense of updating moreparameters than the case of known relative degree.

REFERENCES[1] Smith O (1957), Closer control of loops with deadtime. Chemical Engineering Progress 53(5): 217-219[2] Ichikawa K (1985), Frequency-domain pole assig
nment and exact model matching for delay systems.
International Journal of Control 41(4): 1015-1024[3] Ichikawa K (1986), Adaptive control of delaysystem. International Journal of Control 43(6): 165
3-1659[4] Jia YM (2009), General solution to diagonal model matching control of multiple-output-delay syste
msand its applications in adaptive scheme. Progress
inNatural Science 19: 79-90[5] Ichikawa K (1985), Control system design based on exact model matching techniques. Lecture Not
es in Control and Information Sciences, Berlin: Springer.
[6] Manitius AZ, Olbrot AW (1979), Finite spectrum assignment problem for systems with delays.IEE
E Transactions on Automatic Control, 24:541-553.
[7] Morse AS (1987), A 4(n+1)-dimensional model
reference adaptive stabilizer for any relative degree
one or two, minimum phase system of dimensiionn or less', Automatica 23(1): 123-125.
[8] Tao G, Ioannou PA (1993), Model reference ad
aptive control for plants with unknown relative degree', IEEE Transactions on Automatic Control, 38,(6): 976-982.

IV. SIMULATION EXAMPLE
Consider a SISO time delay system in the form of(1), the known knowledge is n  5 , nl*  1 , andnu*  3 . The reference model is chosen to be1t (s) e 3 s (34)s 5  9 s 4  31s 3  51s 2  40 s  12dChoose the monic stable polynomials
p* ( s)  s 5  10s 4  38s 3  68s 2  57 s  18,
r * ( s )  s 4  10 s 3  35s 2  50 s  24,(35)

r ( s )  s  3.6 s  3.85s  1.35 s  0.1
4

3

2

When the plant model isy( s) ©ISAROB 2011

2( s 2  4.5s  2)(36)
e 3 s u ( s )32s  8.5s  21.5s  21.5s  7.5
4

3

125

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Robust Active Suspension Control of Vehicles
with Measurement Noises
Katsuhiro Okumura ∗ , Hideki Wada ∗∗ , Yuichiro Taira ∗∗∗ and Masahiro Oya ∗∗∗∗∗

Fukuoka Industrial Technology Center, 3-6-1 Norimatsu, Yahatanishi-ku, Kitakyushu-shi, Fukuoka, Japan(Tel: +81-93-691-0260; Fax: +81-93-691-0252) (E-mail: kokumura@fitc.pref.fukuoka.jp)∗∗
Shin-Nippon Nondestructive Inspection Co., Japan, 4-10-13, Ibori, Kokurakita-ku, Kitakyushu-shi, Fukuoka, Japan∗∗∗
National Fisheries University, 2-7-1, Nagatahonmachi, Shimonoseki-shi, Yamaguchi, Japan∗∗∗∗
Kyushu Institute of Technology, 1-1 Sensui-cho, Tobata-ku, Kitakyushu-shi, Fukuoka, Japan
Abstract: In this paper, we propose a robust ride comfort control scheme for vehicles without using measurements oftire deflections. To realize good ride comfort without using measurements of the tire deflections, we propose using an
estimator for the acceleration of the road disturbance, the derivative of the suspension strokes and the derivatives ofthe displacements from the road disturbance to the vehicle body. Using the estimates, we can design a combined ideal
vehicle. Then, a tracking controller is designed so that the real vehicle can track the motion of the combined ideal vehicle.
Moreover, by carrying out numerical simulations, the influence of measurement disturbances on control performance will
be investigated. As a result, it is shown that the proposed ride comfort control scheme is effective even in the presence ofmeasurement disturbances.
Keywords: Vehicle, Ride comfort, Active suspension, Estimator, Robust Tracking Control, Ideal Model

I. INTRODUCTION

fr

Recently, in order to achieve good ride comfort andgood handling qualities, a large number of control
schemes using active suspensions have been proposed in[1]-[4]. In conventional schemes [1]-[4], the active suspensions are controlled so that the suspension strokes lie
within an admitted range and the handling quality does
not become worse. As a result, the ride comfort will be
best at only one specified location on the vehicle body.
In cases when the specified location has to be moved,
too much time is needed to redesign an active suspension controller. To struggle with this problem, the authors
have proposed some active suspension control schemes[5]−[7]. In vehicle systems using controllers proposed in[5]-[7], the following good properties exist: (1) The ride
comfort at a specified location will be best. (2) The best
location can be easily moved by setting only one design
parameter without redesigning the suspension controller.
However, in the proposed schemes [5]-[7], it is assumed
that the tire deflections can be measured. Since road surfaces are uneven, and using noncontact sensors such as
laser position sensors, it is difficult to measure the tire
deflections with high accuracy.
To overcome this problem, the authors have proposeda ride comfort control scheme without using the measurements of tire deflections [8],[9] . To realize good ride comfort without using measurements of the tire deflections,the signals are measured such as the vertical acceleration on the vehicle body, the displacement and the velocity of the suspension stroke, the vertical accelerationof the unsprung mass and the force added to the vehicle body. In addition, we propose using an estimator forthe acceleration of the road disturbance and the deriva-©ISAROB 2011ff

Fig. 1. Two wheels model.
tives of the displacements from the road disturbances tothe vehicle body. In the papers [8],[9], however, the influence of measurement disturbances on control performance has not been considered. Moreover, it is assumed
that the derivative of the suspension strokes can be rigorously measured.
In this paper, we propose an estimator for the acceleration of the road disturbance, the derivative of the suspension strokes and the derivatives of the displacements fromthe road disturbances to the vehicle body. A robust active
suspension controller with the estimated signals is proposed. Moreover, carrying out numerical simulations, theinfluence of measurement disturbances on control performance is investigated. As a result, it is shown that the
proposed ride comfort controller is very effective even in
the presence of measurement disturbances.
II. VEHICLE MODEL
The two wheels model is shown in Fig. 1. The explanation for parameters is shown in Table 1. It is assumed
that the pitching angle is small, and then, the dynamic

126

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

equation of vehicles is given as follows.

ẍz (t) = d(t) ¡ H ẅ(t)ẍu (t) = ¡Ku xu (t)¡Mu−1 f (t)¡ ẅ(t)



−1
T
xz (t) = H [zf (t) ¡ wf (t), zr (t) ¡ wr (t)] Txu (t) = [zuf (t) ¡ wf (t), zur (t) ¡ wr (t)](1)
d(t) = M −1 H T f (t)
f (t) = [ff (t), fr (t)]T= ¡C ẋs (t) ¡ Kxs (t) + u(t)xs (t) = Hxz (t) ¡ xu (t)u(t) = [uf (t), ur (t)]T , w(t) = [wf (t), wr (t)]T 
M = (ThT )−1 diag[m, ic ]Th−1(2)
Mu = diag[muf , mur ], K = diag[kf , kr ]
C = diag[cf , cr ], K·u = Mu−1 diag[k
,
k
]

uf
ur

¸
·
¸
1 a
0 1 


Th = I2 ¡ Dh, H =
,D=

1 ¡a
0 0
−1

The control objective is to develop an active suspension controller so that the ride comfort at a specified location becomes best. In general, humans feel uncomfortable in the vertical oscillation with the frequency about 1
Hz. Therefore, to achieve the control objective, we develop a controller so that the vertical acceleration can be
reduced to a small value at any specified location ` on
the vehicle body. To meet the objective, the following assumptions are made for actual vehicles.
A1 The accelerations z̈f (t), z̈r (t), z̈uf (t) and z̈ur (t) aremeasured.
A2 The force f (t) = [ff (t), fr (t)]T added to the sprungmass are measured.
A3 Suspension displacement xs (t) is measured.
A4 Vehicle parameters are known except for the lengtha, the front and the rear tire stiffness kuf , kur and the tiremass muf , mur .
A5 The second and third derivation of the road disturbance w(t) are bounded.

Combined Ideal Vehicle&&w
Real Vehiclexx = [ x Tz (t ), xuT (t ), x& Tz (t ), x& uT (t ), d T (t )]T

u

−

+

~
x

Tracking Controller

Fig. 2 Configuration of active suspension control system.
Moreover, it is easy from (1), (2) to ascertain that thefollowing signals are also available.
µ
¶ 

−1
−1
p1 (t) = Ku ẍs (t) ¡ (Hd(t) + Mu f (t)) Fig. 2 shows the configuration of the vehicle active
suspension system proposed in [7]. The combined ideal
model shown in Fig. 2 has good properties. Namely, 1)
The ride comfort at a specified location becomes best, 2)
The best location can be easily moved by setting only one
design parameter without redesigning the combined ideal
vehicle. If the real vehicle can track the motion of the designed combined ideal vehicle, the control objective canbe achieved. The active suspension control system in Fig.
2 is designed so that the real vehicle can track the motion of the combined ideal vehicle. To achieve the control
objective, the signals of road disturbance ẅ(t) and statevariables xz (t), ẋz (t), xu (t), ẋu (t) d(t) are required in
the controller proposed in [7].
According to the assumption A1, it can be seen thatthe following signals are available.

©ISAROB 2011xmTT
xm = [ x Tzm (t ), xum(t ), x& Tzm (t ), x&um(t ), d mT (t )]T

III. ACTIVE SUSPENSION CONTROLLER

ẍs (t) = [z̈f (t), z̈r (t)]T ¡ [z̈uf (t), z̈ur (t)]T
Hd(t) = [z̈f (t), z̈r (t)]T

Table 1 Notation of vehicle model.
center and center of gravity of vehicle bodyvertical displacement at CG and pitchingvertical displacement of vehicle body at
positions on front and rear wheel axle
zuf , zur vertical displacement of front and rear unsprung mass
wf , wr
vertical displacement of road disturbanceadded to front and rear wheelv
longitudinal velocity of vehiclem, ic
sprung mass and moment of inertia of vehicle bodyahalf of vehicle body lengthh, `distances from C to CG and from C to P
muf,mur front and rear unsprung mass
kf , kr
front and rear suspension stiffness
cf , cr
front and rear suspension damping rate
kuf , kur front and rear tire spring stiffness
ff , fr
front and rear force added to sprung massuf , u r
front and rear active suspension control
force
C, CG
zcg , θzf , zr= xu (t)p2 (t) = xs (t) + p1 (t) = Hxz (t)If an estimator for ẋs (t) and ẋz (t) are developed, then, the signal ẋu (t) becomes also available.
Therefore, we will develop an estimator for ẋs (t), ẋz (t) and ẅ(t).
Consider the
new state η(t) = [η 1 (t)T , η 2 (t)T , η 3 (t)T ]T =[ẋs (t)T , H ẋz (t)T , ẅ(t)T ]T . Then, the estimator for thestate η(t) is proposed asb 1 (t) = αxs (t) + ζ 1 (t),η(5)ζ̇ 1 (t) = ẍs (t) ¡ αb
η 1 (t),ζ 1 (0) = ¡α(p2 (0) ¡ p1 (0))
b 2 (t) = 2αp2 (t) + ζ 2 (t),η(6)ζ̇ 2 (t) = ¡2αb
η 2 (t) + Hd(t) ¡ η 3 (t),
ζ 2 (0) = 2αp2 (0)

¾
(3)

127

(4)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

0.05

0.1

xm

Combined Ideal Vehicle[m/s2][N]

Robust active suspension controller

0

0

n(t )

&&
w
u

+

y
Real Vehicle+y (t ) = [ &z& f ( t ), &z&r (t ),&z&uf ( t ), &z&ur ( t ), f f (t ), f r ( t ) ]T

&ˆ& (t )w
Estimator

x̂

+

−

-0.1
0

1

2[s]

(a) nf1(t), wn =100[Hz]xˆ (t ) = [ x z (t ), x u (t ),xˆ& z (t ), xˆ& u (t ) , f (t )]T

-0.05
0
1
2[s]
(c) nz1(t), wn =100[Hz][N][m/s2]
0.05

0.1

Tracking Controller

0Fig. 3 Configuration of active suspension control systemusing proposed estimator.
b 3 (t) = ¡3α2 p2 (t) + ζ 3 (t), ηb 2 (t),ζ̇ 3 (t) = 3α2 ηζ 3 (0) = 3α2 p2 (0)

0

-0.1
0

1
2[s]
(b) nf1(t), wn =1000[Hz]

-0.05
0
1
2[s]
(d) nz1(t), wn =1000[Hz]Fig. 4 Disturbance nf 1 (t), nz1 (t) added to measurements ff , ẍz (t).

(7)

[m]
where η̂ i , i = 1, 2, 3 are estimated signals for η i , i =1, 2, 3, and α is a positive deign parameter introduced to
improve performance of the proposed estimator. Differentiating the first equation in (5)(6)(7), we obtain the following equation.
ḃ 1 (t) = αeη
η 1 (t) + ẍs (t)ḃ 2 (t) = 2αeb 3 (t)ηη 2 (t) + Hd(t) ¡ ηḃ 3 (t) = ¡3α2 ηe 2 (t)η¡αė =  O2
η(t)
O2

O2
¡2α
3α2

0.05

0
(8)

0




O2
0
e (t) +  0  ẅ(t) (9)¡I2  ηO2
1

Where η̃ = η ¡ η̂ and η˜i = η i ¡ η̂ i , i = 1, 2, 3 . It
is assumed that the proposed controller starts in the situation where the oscillation of the vehicle body dose notoccur. According to the assumption A5, for the proposed
estimator(5)(6)(7), the following theorem holds.
e 1 (t), ηe 2 (t), ηe 3 (t), there exTheorem 1: For estimation η
ist bounded positive constants ρEi , i = 1, 2 independentof the design parameter α such thatke
η 1 (t)k2 = 0keη 2 (t)k2 ≤ ρE1 α−4
.
(10)

ke
η 3 (t)k2 ≤ ρE2 α−2It can be concluded from the theorem 1 that the estimated errors decrease as the design parameter α increases. In case when the design parameter α is set tobe large enough, we can obtain the signals H ẋz (t) andẋs (t) with enough accuracy. Then, the signal ẋu (t) =H ẋz (t) ¡ ẋs (t) becomes also available. Namely, from(3), (4) and theorem 1, it can be concluded that the all
required signals are available. Configuration of the active
suspension control system using the proposed estimatoris shown in Fig. 3.

©ISAROB 2011

128

1
2[s]
wf
Fig. 5. Road disturbance wf (t).
IV. NUMERICAL SIMULATION RESULTS
To investigate the influences of measurement disturbances, numerical simulations are carrying out. The frequencies of measurement disturbances are set as 100[Hz]and 1000[Hz]. The measurement disturbance nf 1 (t)added to a signal of force sensor ff (t) are shown in Figs.
4 (a), (b), and the measurement disturbance nz1 (t) added
to a signal of acceleration sensor (cT ẍz (t), c = [1, 0]T )are shown in Figs. 4 (c), (d). The maximum valuesof measurement disturbances of force sensors and acceleration sensors are set as 0.12[N] and 0.05[m/s2 ], respectively. For the other sensors, similar measurement
disturbances are added. The values of vehicle parameters are shown in Table 2. The design parameter α forthe proposed estimator is set as α = 2000. The design
parameters for the tracking controller shown in Fig. 2
are set as a large value so that the tracking error between the actual vehicle and the ideal vehicle model
can become small. And the vehicle velocity is set as
v = 100 £ 1000/3600[m/s]. Road disturbance wf (t) =wr (t ¡ L), L = 2a/v is shown in Fig. 5.
Fig. 6 shows the responses of the vertical accelerationof the controlled vehicle shown in Fig. 3 at the specified location l = ¡1.0(lp = ¡0.7). The thin line shows
the responses of the acceleration of the controlled vehicle
without measurement disturbances. The thick lines showThe Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

m
h
kf
cf
muf
kuf

Table 2 Nominal values of parameters.
781
kg
ic
990
kgm2
0.04
m
a
1.38
m
27160
N/m
kr
29420
N/m
4000
Ns/m
cr
2500
Ns/m
69
kg
mur
96
kg
229000 N/m
kur 255000 N/m[m/s2]
4
2
0-2

the responses of the controlled vehicle with measurement
disturbances, and the dashed lines show the responses ofthe passive vehicle. It is seen from Fig. 6 that the acceleration responses do not vary even in the presence of themeasurement disturbances.

-4
-6
0

0.5

1

1.5

2 [s]

(a ) &z&l (t )l = −1.0 (l p = −0.7)

V. CONCLUSION
We have proposed the active suspension control
scheme in which tire deflections are not required. The
proposed suspension controller has a good property thatthe location where the ride comfort becomes best can beeasily moved by setting only one design parameter lp . It
has been shown by carrying out numerical simulations
that in the closed loop system using the proposed ride
comfort control scheme, the influence of measurement
disturbances is analyzed by using numerical simulations.
As a result, it is shown that the proposed ride comfort
control scheme is very effective even in the presence ofmeasurement disturbances.

[m/s2]
4
2
0
-2
-4
-6

REFERENCES
[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

©ISAROB 20110

Hrovat, D., Survey of Advanced Suspension Developments andRelated Optimal Control Application, Automatica, Vol.33-10,
pp.1781-1817, (1997).
Y. Zhang, and A.G. Alleyne, A New Approach to Half-Car Active Suspension Control, Proc. of the American Control Conference, pp. 3762-3767.(2003).
Thompson, A.G., and Davis, B.R., RMS Values of Force,
Stroke and Tire Deflection in a Half-Car Model with Preview Controlled Active Suspension, Vehicle System Dynamics,
Vol.39-3, pp.245-253. (2003).
J.-S. Lin and C.-J. Huang, Nonlinear backstepping active suspension design applied to a half-car model, Vehicle System Dynamics, Vol. 42, No. 6, pp. 373-393, (2004).
M. Oya, H. Harada, Y. Araki, An Active Suspension Controller
Achieving the Best Ride Comfort at Any Specified Location on
A Vehicle, Journal of System Design and Dynamics, pp.245256, (2007).
H. Okuda, Y. Tsuchida, M. Oya, Q. Wang, and K. Okumura,
Robust Active Suspension Controller Achieving Good Ride
Comfort, Proc. of SICE Annual Conference 2007, September
17-20, Kagawa, Japan, pp. 1305-1310, (2007).
M. Oya, Y. Tsuchida, Q. Wang, Y. Taira, Adaptive Active Suspension Controller Achieving the Best Ride Comfort at Any
Specified Location on Vehicles with Parameter Uncertainties,
International Journal of Advanced Mechatronic Systems, Vol.
1, No. 2, pp. 125-136, (2008).
Katsuhiro Okumura, Masahiro Oya, Masashi Nagae, Hidetaka
Ota and Hideki Wada, Active Suspension Control Scheme forvehicles without Measurements of Tire Deflection, Proc. of the
8th IEEE International Symposium on Computational Intelligence in Robotics and Automation, Daejoen, Koria, December
15-18, pp. 153-158, (2009)
Katsuhiro Okumura, Masahiro Oya and Hideki Wada, Robust
Ride Comfort Control of vehicles without Measurements ofTire Deflection, Artificial Life and Robotics, Volume 15, Number 2, pp. 133-137, (2010)

0.5

1

1.5

2 [s]

(b) &z&l (t )l = −1.0 (l p = −0.7), wn = 100[Hz][m/s2]
4
2
0
-2
-4
-6
0

0.5

1

1.5

2 [s]

(c) &z&l (t )l = −1.0 (l p = −0.7), wn = 1000[Hz]Fig. 6 Responses of the vertical acceleration of the vehicle at the specified location l = ¡1.0(lp = ¡0.7)

129

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A Method to Improve Stability of
Adaptive Steering Driver-Vehicle Systems
Shingo Tamaru, Jinxin Zhuo, Qiang Wang, Masahiro Oya
1-1 Sensui-cho, Tobata-ku, Kitakyusyu-shi, Fukuoka, Japan(Tel : 093-884-3103)( Email: i344209s@tobata.isc.kyutech.ac.jp)

Abstract: We have developed an adaptive steering controller achieving good tracking performance. However, we havedesigned an ideal vehicle model in disregard of the variations of the driver properties. For the variations of
the driver properties, if an adequate ideal vehicle model can be designed, the better handling stability of theadaptive driver-combined-vehicle systems can be realized. In this paper, we propose a scheme to design an ideal
vehicle model adequate for the variations of the driver properties. Finally, it is shown by carrying out numericalsimulations that the designed ideal vehicle model is very effective.
Keywords: Steering Control, Adaptive Control, Model Following Control

vp

I. INTRODUCTIONlt

To improve handling stability of combined vehicles,
the three states, the combination angle, the lateralvelocity and yaw rate of tractor, must be controlled withthree independent control inputs. Recently, theresearches of steer-by-wire, are conducted actively [1] .
Using the technology of steer-by-wire, the three steering
angles such as the steering angle of the trailer, the front
and rear steering angle of the tractor can be used as theindependent control inputs possibly. Based on thenotion, control schemes have been proposed [ 2, 3] inwhich three independent steering control inputs are used.
However, these proposed controllers require all accurate
knowledge of combined vehicle parameter.
To straggle with the problem stated above, theauthors have proposed robust steering control
schemes [ 4, 5] . In the schemes, an ideal vehicle model isdesigned, and then, the steering controller is developed
so that the actual vehicle tracks the ideal vehicle modeleven if the large parameter variation occurs in thevehicle dynamics. In the proposed schemes, however,an ideal vehicle model is designed in disregard of thevariations of the driver properties. Therefore, theauthors have proposed a design scheme for ideal vehicle
models [ 6, 7 ] adequate for the variations of the deriverproperties. In the scheme, several ideal vehicle models
are designed against the variation of the driverproperties and the ideal vehicle model has to be changed
according to the variation. For this reason, the scheme
proposed in [6,7] require accurate information of thevariation of the driver properties. In addition, there
exists the problem that the construction of controller
becomes too complex.
In this paper, we propose a new design scheme for
an ideal vehicle model. A design method based on a costfunction is proposed and only one ideal vehicle model is©ISAROB 2011

εc

P C.G.

δt

ε&2
C.G.

Tractor

δr

d2

dp
lr

ε&1

δf
vx

h
lf

Trailer

Fig.1 Tractor-semitrailer model
designed. In the designed ideal vehicle model, the roughinformation of the variation of the driver properties isonly required. Carrying out numerical simulations, forthe variation of the driver properties, it is shown that theadaptive steering controller using the proposed ideal
vehicle model has good effectiveness.

Ⅱ. INTRODUCTION
In this paper, a simplified bicycle model of tractorsemitrailers shown in Figure 1 is used to design anadaptive steering controller. In Figure 1, the point C.G.
is the center of gravity, and the point P is the reference
point. Definitions for parameters of combined vehicles
are shown in Table 1.
The following assumptions are made to develop acontroller for the combined vehicles shown in Fig.1.
A1 Lateral velocity of tractor v p (t ) , yaw rate oftractor γ 1 (t ) , combination yaw rate ε&c (t ) andcombination angle ε c (t ) can be measured.
A2 The length d p , l f , l r and the length of trailer lt are
known. The other parameters include uncertainties.
A3 Longitudinal velocity v x is a bounded constant andavailable.
The dynamic equation of combined vehicle can be
described as follows.

130

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011y

Table 1 Notation of tractor-semitrailer modelvx
longitudinal velocity of tractorv p , vc
lateral velocity of tractor at P and C.G.

ε1 , ε 2εcγ1
δ f ,δr ,δtyaw angles of tractor and trailer

m1 , m2
J z1 , J z 2mass of the tractor and trailer

1P

T p y& r~yTarget lanex

Fig.2 Driver-Vehicle-System
Therefore, the equation of the driver steering angle
δ c (t ) is given by(5)
T s δ&&c (t ) + δ&c (t ) = − g p ~y& (t ) − g i ~y (t )
where Ts is a steering time constant and denotes thereaction time delay of drives and mechanical systems,
g p , g i is the steering gain. In this paper, we add an
integrator to the driver model of [8] so that y r (t ) canconverge to zero when the curvature of target lane is notzero.
2. The Form of Ideal Vehicle ModelIn the following explanation, s is Laplace variable,
the symbols L, L −1 denote Laplace transform andinverse Laplace transform.
Based on the proposed ideal vehicle model proposedin [5], the form of ideal vehicle modes is proposed asg ω2sv pd (t ) = L −1  v n L[v x δ c (t )]
 AM ( s )

2


g
(
s
+
1
)
ω
η
γ
n
γ 1d (t ) = L −1 L[v xδ c (t )]
 AM ( s )(6)tractorlength of trailer
distances from the connector to P and C.G.

distance from P to C.G.
hc f , cr , ct cornering stiffness

z&(t ) = Az z (t ) + B z (u(t ) − bε c (t ))

[

]

(1)

[

]

z(t) = H vp (t) γ1(t) ε&c (t) , u(t) = δ f (t) δr (t) δt (t)T

T

Az = −v x Dz − v Bz , Bz = QK , b = [0 0 1]
−1
x

T

[

Q = H Tp (T T M cT ) −1 H p , K = diag c f
m1 + m2
M c =  − m2 d 3
 m2 d 2

− m2 d 3
J z 3 + m2 d 32
− J z 2 − m2 d 2 d 3

cr

ct

( 2)

]

m2 d 2
− J z 2 − m2 d 2 d 3 
J z 2 + m2 d 22 

0 1 0 
1 h 0
T −1
Dz = H 0 0 0 ( H p ) , T = 0 1 0
0 0 00 0 1
1
1
1


T
H p = l f − l r − (d p + lt )
 0

0
lt
T
p

(3)

AM ( s ) = s 2 + 2ω nζs + ω n2
gv = gv

g p s + g i Tp s + 1
g p s + g i Tp s + 1, gγ = gγ (vx )ˆgˆ p s + gˆ i T s + 1
gˆ p s + gˆ i Tˆ s + 1
pWhere d1 = d p + h, d 3 = d1 + d 2 and J z 3 = J z1 + J z 2 .
M c is a positive definite matrix.
The nominal values of combined vehicle are shownin Table 2.

L[ε cd (t )] =ωn2AM (s)

L[δ ci (t )], ε ci (t ) =p

2d p vxγ 1d (t )vx2 − (d pγ 1d (t ))2( 7)

where ω n , ζ ,η , g v , g γ (v x ), g p , g i , T p are positive design
parameters to be determined later. v pd (t ) is the ideal
lateral velocity of the tractor, γ 1d (t ) is the ideal
combination yaw angle, and ε cd (t ) is the ideal
combination yaw angle. In (6), gˆ p , gˆ i , Tˆp are theestimated values of the steering gains g p , g i and thepreview time constant T p . In the case of a constantdriver steering angle, the desired lateral velocity v pd (t )of the tractor converges to zero and the desired yaw
rate γ 1d (t ) of the tractor tends to a constant value
lim γ 1d (t ) = γ 1d . Then, the tractor becomes tangent tot →∞the circular lane with the radius v x / γ 1d and travels.
In the actual combined vehicles using the adaptive
steering controller, the property of the actual vehicles
becomes as same as that of the ideal vehicle model. In
this case, the following property can be obtained.

Ⅲ. IDEAL VEHICLE MODEL
1. Driver ModelIn this paper, a linear preview driver model [8] isemployed to describe the properties of drivers. In Fig.2,
the solid lines represent the state of the combined
vehicles, and the dashed lines represent the ideal state of
the combined vehicles tracking the target lane. y r (t ) is
the lateral distance between P and the target lane,
ε r (t ) = ε 1 (t ) − ε d (t ) is the relative yaw angle between
the vehicle and the target lane. ε d (t ) is the yaw angle ofthe ideal state, ρ (t ) is the curvature of target lane.
The predicted deviation ~y r (t ) is defined by~(4)y (t ) = y r (t ) + T p y& r (t )where T p is the preview time constant, v x is thelongitudinal velocity of the combined vehicles.

©ISAROB 2011

εrsteering angles

distances from P to front and rear wheel axle of

T
p

C.G.

yr

yaw rate of tractor

inertialt
d p , d2

Preview Point

combination angle

tractor's moment of inertia and trailer's moment ofl f , lr

ρ

131

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 2
m1
lf

1060
1.13
2.27
2.26

dp
d2
cf
m2

29700
436

lt

2.48

Nominal values of parameters
kg
J z1
1507
m
lr
1.34
m
h
0
m

kgm
m

(b) The evaluation function is minimized preferably inlane change and cornering maneuver.
Step 5 If the designed ideal vehicle model does nothave good handling stability, we return to Step 3~4.
As result, g v ,η , ω n are set as g v = 0.25,η = 0.39,
ω n = 5.4 . In the following part, using numericalsimulations, the property of designed the ideal vehicle
model is shown. In the lane change maneuver, thecurvature of target lane is ρ (t ) = 0[m] . The initial valueof y r (t ) is set as y r (0) = −3.0[m] . In the cornering
maneuver, the curvature of target lane isρ (t ) = 1 / 500[m] . The initial value of y r (t ) is set
as y r (0) = −1.0[m] . Until now, from many research
results, the value of Ts is obtained as 0.05 ~ 0.4[s] . For
this reason, in the following numerical simulations, wecheck controlled performance in 0.05 ≤ Ts ≤ 0.4[s] .
At first, the effectiveness of the proposed ideal
vehicle model is shown in the lane change maneuver. In
the case of varying v x and Ts , the variation of J is shownin Figure 3. Fig.3 (a) shows the values of J for thepassive combined vehicle, Fig.3 (b) shows the valuesof J for the adaptive steering driver vehicle systemusing the proposed ideal vehicle model. Hereafter, it iscalled the adaptive vehicle simply. In Fig.3, in casewhere J exceeds 80, the values of J are shown as 80. As
shown Fig.3 (a), there exists a region in which J
exceed 80. In the region, the handling performance ofthe passive vehicle becomes unstable. However, as
shown in Fig.3 (b), for the adaptive vehicle, the valuesof J are less than 70 and good handling performancecan be achieved.
Next, the effectiveness of the proposed ideal vehicle
model is shown in the cornering maneuver. In the caseof varying v x and Ts , the variation of J is shown in
Figure 4. Fig.4 (a) shows the values of J for the passive
combined vehicle, Fig.4 (b) shows the values of J for
the adaptive vehicle. In Fig.4, in case where J exceeds
35, the values of J are shown as 35. As shown Fig.4 (a),there exists a region in which J exceeds 35. In theregion, the handling performance of the passive vehicle
becomes unstable. However, as shown in Fig.4 (b), forthe adaptive vehicle, the values of J are less than 21 andgood handling performance can be achieved.
Moreover, we show the effectiveness of the
proposed ideal vehicle in the presence of the estimatederrors in gˆ p , gˆ i , Tˆp . In case where the values of g p / gˆ p
and g i / gˆ i are not equal to one, we can obtain the similargraphs to Fig.3 (b) and Fig.4 (b). For each values ofg p / gˆ p and g i / gˆ i , the maximum value of Fig.3 (b) isplotted in Fig.5 (a) and the maximum value of Fig.4 (b)is plotted in Fig.5 (b). Similarly, for each values ofT p / Tˆp , the maximum values of Fig.3 (b), Fig.4 (b) areplotted in Fig.6.
As shown in Fig.5, in the region of
0.6 ≤ g p / gˆ p ≤ 1.4 and 0.8 ≤ g i / gˆ i ≤ 1.6 , the values of
J became less than 75 in the lane change maneuver and

2

m

N/rad
kg

cr
J z2

41460
805

N/rad
kgm 2

m

ct

100000N/rad− s(Ts s + 1) AM (s)vx2
ρ ( s)s (Ts s + 1) AM (s) + C(s)vxωn2 BM (s)s2 (Ts s + 1) AM (s) + D(s)Tpvxωn2 BM (s)+ 3
Yr (0)s (Ts s + 1) AM (s) + C(s)vxωn2 BM (s)Yr (s) =3

B M ( s ) = g v s 2 + v x g γ ηs + v x g γ(8)g p s + gi Tp s + 1
C ( s ) = ( g p s + g i )(T p s + 1)gˆ p s + gˆ i Tˆp s + 1

D(s) = ( g p s + g i )g p s + gi Tp s + 1
gˆ p s + gˆ i Tˆp s + 1

From (8), it can be seen easily that if the estimatedvalues gˆ p , gˆ i , Tˆp are almost equal to the parameters
g p , g i , T p of actual vehicle, the variation of the property
shown in (8) becomes small.
3. Determination of Design Parameters
In the passive combined vehicles, the nominal
values of vehicle parameters (see Table 2) are used. In
the following explanation, the determination method ofdesign parameters ω n , ζ ,η , g v , g γ (v x ), g p , g i , T p isshown.
Step 1 In order that oscillation cannot occur in thedesired trajectories of a ideal vehicle model, ζ is set
as ζ = 1 .
Step 2 Using the average values of the driverparameters, the driver parameters g p , g i , T p are set as
g p = 0.008, gi = 0.0012, Tp = 2.2 . In addition, it is assumed
that g p = g p = gˆ p , g i = g i = gˆ i , T p = T p = Tˆp .
Step 3 For a constant longitudinal velocity v x , we can2
obtain the yaw rate gain as g γ (v x ) = 7.59×1032.06+ 2×.10.
24×v x ( t ) 2
In order to make it easy for the ideal vehicle model todrive a corner, the design parameter g γ (v x ) is
determined as
1.8 × 3.06 ×10 2
g γ (v x ) =(9)7.59 × 10 2 + 2.24 × v x (t ) 2
Step 4 To design a vehicle model with good handlingstability, an evaluation function is introduced by using
the trail and error approach.
J =t∫ (2 y (t ) + 2 y (t ) + 4 &y& (t ) + 4ε&& (t ) )dt
0

2

r

2

rt

+ 3 max &y&r (t ) + 3 max ε&&c (t )

2

r

2

c

(10)

Where T is the simulation time. Based on the evaluation
function, the remaining design parameters g v ,η , ω n aredetermined so that the following conditions can besatisfied.
(a) The imaginary part of the eigenvalues of BM ( s) is
minimized preferably.

©ISAROB 2011132

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

J

J

J

v x [km/h ]

gi
gˆ i

v x [km/h ]Ts [ s ]gˆ p(a) Lane Change

Fig.3 Variation of J for various Ts
and v x (lane change)

J(b) Cornering

Fig.5 Variation maximum value of J
for various g p and g iJJv x [km/h ]v x [km/h ]Ts [ s](a) passive

Tp
Tˆp

Ts [s ](b) ideal vehicle model(a) Lane Change

TpTˆp(b) Cornering

Fig.6 Variation maximum value of J for various T p

Fig.4 Variation of J for various Ts
and v x (cornering)

1996, pp.395-408.
(３) R. Hayama, S. Maeda, S. Kawahara, S.Nakano and
H. Kumamoto, Bilateral Control of Human
Machine Interface for Steer-By-Wire System,
Proceedings of International Symposium on
Advanced Vehicle Control, Kobe, 2008, pp.752757.
(４) Qiang Wang, Masahiro Oya, and Toshihiro
Kobayashi, Robust Integral Control to Improve
Stability of Tractor-Semitrailer Combination
Vehicles, Proc. 2005 International Symposium on
Advanced Control of Industrial Processes, Seoul,
Korea, (2005), pp. 121-126.
(５) Q.Wang, M.Oya, N.Takagi, Y.Taira and H.Ota,
Adaptive Steering Controller to Improve Handling
Stability for Driver-Combined-Vehicle System,
Proc. of the 8th IEEE International Symposium on
Computational Intelligence in Robotics andAutomation, Daejoen, Koria, December 15-18,(2009), pp.409-414.
(６) Qiang Wang, Masahiro Oya, Natsuki Takagi, On
Design of Ideal Vehicle Models in Adaptive
Steering Driver-Vehicle Systems, Proc. of theSecond International Conference of Modeling
Identification and Control, Okayama, Japan, July
17-19, pp.203-208, 2010.
(７) Qiang Wang, Singo Tamaru, Hideki Wada,
Masahiro Oya, One Design Method for Ideal
Vehicle Models In Adaptive Driver-Combined
Vehicle Systems, Proc. of the SICE Annual
Conference 2010, Taipei, Taiwan, August 18-21,
pp.156-161, 2010.
(８) J.Ishio, H.Ichikawa, Y.Kano, M.Abe, Vehiclehandling quality evaliation through model-based
driver steering behaviour, Vehicle System
Dynamics, vol.46, 2008, pp.549-560.

the values of J became less than 30 in the cornering
maneuver. As shown in Fig.6, in the region of
0.7 ≤ T p / Tˆp ≤ 1.3 , the values of J became less than 75
in the lane change maneuver and the values of J becameless than 25 in the cornering maneuver. From the facts,
it can be concluded that the proposed ideal vehicle
model is very effective when the estimated values of thedriverparametersarewithintheregions
0.6 ≤ g p / gˆ p ≤ 1.4,0.8 ≤ g i / gˆ i ≤ 1.6 and 0.7 ≤ T p / Tˆp ≤ 1.3 .
Namely, it can be seen that accurate values of the driverparameters are not required.

IV. CONCLUSION
In this paper, we propose a design method to design
not several ideal vehicle models but only one ideal
vehicle model against variations of the driver properties.
Carrying out numerical simulations, it is shown that thehandling performance in the adaptive driver vehicle
system is far better than that of passive combined
vehicle. Moreover, it is also shown that in the design
scheme of an ideal vehicle model, rough information of
the driver properties are only required to maintain good
handling performance.

REFERENCES(１) P. Pisu, A. Serrani, S. You, and L. Jalics, Adaptive
Threshold Based Diagnostics for Steer-by-wire
Systems, Trans. ASME. Journal of Dynamic
Systems, Measurement, and Control, vol.128,(2006), pp.428-435.
(２) B.A. Jujnovich, A.M.C. Odhams, R.L. Roebuckand D. Cebon, Active Rear Steering Control of a
Tractor-Semitrailer, Proceedings of International
Symposium on Advanced Vehicle Control, Aachen,©ISAROB 2011

gp

gˆ p

(b) ideal vehicle model

J

gi
gˆ i

gp

Ts [s ](a) passiveJ

133

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Robust Controller for Underwater Vehicle-Manipulator Systems
Including Thruster Dynamics
Yuichiro Taira∗ , Junpei Sugino† , Natsuki Takagi‡ , and Masahiro Oya†∗

Dept. of Ocean Mechanical Engineering, National Fisheries University, Shimonoseki 759-6595, Japan(taira@fish-u.ac.jp)†
Dept. of Mechanical and Control Engineering, Kyushu Institute of Technology, Kitakyushu 804-8550, Japan(oya@cntl.kyutech.ac.jp)‡
Dept. of Mechanical Engineering, Miyakonojo National College of Technology, Miyakonojo 885-8567, Japan(takagi@cc.miyakonojo-nct.ac.jp)

Abstract: This paper deals with a control scheme for underwater vehicle-manipulator systems with the
dynamics of thrusters in the presence of uncertainties in system parameters. We have developed an adaptive controller that overcomes thruster nonlinearities, which cause an uncontrollable system. However, thestructure of the adaptive controller is very complex due to the regressors of dynamic system models andparameter estimators. In this paper we develop a robust controller whose structure is much simpler thanthat of the adaptive controller.
Keywords: Underwater vehicle-manipulator system, thruster dynamics, robust control.
ics is expressed as

I. INTRODUCTION[]
R̄(ϕ)f¯b (t)M (ϕ)ẍ(t) + f (ϕ, u) = J(ϕ)−Tτm (t)

An autonomous underwater vehicle with manipulators, referred to as underwater vehicle-manipulator
system (UVMS), is expected to play an important rolein ocean development [1]. Adaptive or robust control schemes for UVMSs have recently been developedin the presence of not only hydrodynamic forces acting on the UVMS and the dynamic coupling between
the vehicle and the manipulator but also uncertainties in system parameters [1–5]. In a general type of
UVMS, the vehicle is propelled by marine thrusters,
whereas the manipulator is driven by electrical motors. Despite such a diﬀerent actuator system, theexisting control schemes in [1–5] were designed basedon the dynamic system models without the thruster
dynamics to obtain a simply-structured controller. In
each control scheme, furthermore, a high gain control
system is constructed in order to achieve good control performance. However, the vehicle propelled bymarine thrusters generally has a considerably slower
time response than the manipulator driven by electrical motors [4], and hence the high gains may excite
the ignored thruster dynamics, which degrades control
performance and may even cause instability.
In order to overcome the problem, the authors havedeveloped an adaptive controller for UVMSs with thethruster dynamics [6]. Since the slow thruster dynamics was taken into consideration in the developmentof controller, control performance can be improved by
the adaptive controller with high gains. However, thestructure of the adaptive controller is very complex,
compared with that of a normal robust controller, due
to the regressors of the dynamic system models andthe parameter estimators. In this paper we develop arobust controller whose structure is much simpler thanthat of the adaptive controller proposed in [6].

M (ϕ) = J(ϕ)−T M̄ (ϕ)J(ϕ)−1 ∈ RDn×Dn
f (·) = J(ϕ)−T [f¯(ϕ, u)
˙ u)u(t)] ∈ RDn −M̄ (ϕ)J(ϕ)−1 J(ϕ,(2)

where the explanation of the main symbols is shownin Table 1. Most of the controllers reported in theliterature of UVMS control are designed for the model(1) without the dynamics of f¯b (t). In our controller
design, the following dynamic model for thrusters isused [7]:}f¯b (t) = K̄D(v)v(t)11(3)v̇(t) = − AD(v)v(t) + Bτb (t)22
D(v) = diag{|v1 |, . . . , |vDm |} ∈ RDm×Dm(4)
Table 1. Symbols in the models (1) to (4)
Da
Dn
x(t)

Number of dimension Dp + 2Do
Number of dimension 2(Dp + Do) = 2Dm
Signal composed of vehicle’s and manipulator end-effector’s positions and orientations (∈ RDn )ϕ(t)Signal composed of vehicle’s orientation and manipulator’s joint angles (∈ RDa )u(t)Signal composed of vehicle’s translational velocity andϕ̇(t) (∈ RDn )f¯b (t) Thrust forces produced by thruster’s propellers(∈ RDm )τm (t) Joint torques of manipulator (∈ RDm )J(ϕ) Jacobian matrix in the equation ẋ(t) = J(ϕ)u(t)(∈ RDn×Dn )R̄(ϕ) Transformation matrix from f¯b (t) to force and torqueconcerning inertial coordinate system (∈ RDm×Dm )M̄ (ϕ) Inertia matrix (∈ RDn×Dn )f¯(·)
Signal composed of centrifugal, Coriolis, gravitationaland buoyant forces, fluid drag and bounded disturbances (∈ RDn )v(t)Shaft velocities of thruster’s propellers (∈ RDm )τb (t) Shaft torques of thruster’s propellers (∈ RDm )A, B, Diagonal matrices composed of thruster’s system paK̄rameters (∈ RDm×Dm )II. UVMS MODEL
Consider an underwater vehicle equipped with a
Dm link manipulator with revolute joints. Without
loss of generality, we assume that Dm = Dp + Do,
where Dp and Do are the numbers of translationaland rotational dimensions, respectively. As in [1, 2],
the mathematical model without the thruster dynam-©ISAROB 2011

(1)



134

fbi (t) [N]The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 201110

x r (t)(Vehicle control)0䯷UVMS model x (t)(Manipulator control)

x (t)

τm(t)

-10
-0.5

0
0.5
vi (t) [krad/s]
Fig. 1. Nonlinearity in thruster dynamics

z (t)Step 1x r (t)UVMS modelτb(t)Step 2

Robustcontroller Iz(t)Thrustermodel

where the explanation of the main symbols is shownin Table 1, and vi (t) is the ith element of v(t). Fig. 1,
where f¯bi (t) is the ith element of f¯b (t), shows theschematic representation of the ﬁrst equation of(3). As shown in Fig. 1, the thruster model (3) has
the dead-zone-like nonlinearity that f¯bi (t) = 0 anddf¯bi (t)/dt = 0 when vi (t) = 0.
In this paper, the backstepping control technology
is used to develop a robust controller. To this end,
the state v(t) has to be replaced by the new onez(t) = D(v)v(t) ∈ RDm . Combining (1) and (3), andrewriting the signal D(v)v(t) as z(t), we obtain the
new representation][z(t) 
−T
M (ϕ)ẍ(t) + f (ϕ, u) = J(ϕ) R(ϕ)K τ (t)(5)
mż(t) = −AD(v)z(t) + BD(v)τb (t)[][]R̄(ϕ) 0K̄ 0R(ϕ) =, K=∈ RDn×Dn
(6)
0 Im
0 Im

τm(t)

Robustcontroller II 䱊䯷zn (t)

䯹zr (t)

䯷䯹x (t) x䱊(t)Robustcontroller I

Fig. 2. Controller design procedure
It follows from the property P2 and the assumptions
A1 and A2 that J(ϕ) and R(ϕ) are known matrices, and hence ẋ(t) is available by using the equationẋ(t) = J(ϕ)u(t).
In the following subsections we develop a controllerthat achieves the control objective by using a two-stepbackstepping procedure, as shown in Fig. 2. The ﬁrst
step is the design of a robust controller with the inputs
z(t) and τm (t), called robust controller I in this paper.
The second step is the design of a robust controller
with the input τb (t), called robust controller II in thispaper. In this step we ﬁrst replace z(t) determined in
the ﬁrst step by the desired trajectory zr (t), and thendesign the control input τb (t) for the second equationof (5) so that z(t) tracks zr (t).
1. Robust Controller I
According to the design procedure shown in Fig. 2,
we make the following assumption in the design of robust controller I:
A5) The control inputs are z(t) for vehicle control andτm (t) for manipulator control.
In order to achieve the aforementioned control objective, we use the tracking errors

where Im ∈ RDm×Dm is an identity matrix.
The model (5) has the following properties useful forour controller development [1, 7]:
P1) The diagonal elements of A, B and K̄ are positive
constants, and there exists a positive constant cB suchthat cB ∥ȳ∥2 ≤ ȳ T B ȳ for any ȳ ∈ RDm .
P2) Each of J(ϕ) and R(ϕ) is composed of the kinematic parameters (e.g., length) and the functions ofϕ(t). In addition, if each of J(ϕ) and R(ϕ) has a fullrank, then there exists a positive constant cRKJ suchthat cRKJ ∥x̄∥2 ≤ x̄T J(ϕ)−T R(ϕ)KR(ϕ)T J(ϕ)−1 x̄ forany x̄ ∈ RDn .
P3) If J(ϕ) has a full rank, then M (ϕ) is symmetricand positive deﬁnite, and there exists a positive constant cM 1 such that ∥M (ϕ)∥ ≤ cM 1 .

˙s̃(t) = x̃(t)+ αx̃(t), x̃(t) = x(t) − xr (t)(7)

where α > 0 is a design parameter. Using the ﬁrst
equations of (5) and (7), we have the error models

[
]
1

z(t)
−T
˙

M (ϕ)s̃(t) = J(ϕ) R(ϕ)K τ (t) − Ṁ (·)s̃(t)
m
2(8)+fx (t) − x̃(t)



˙
x̃(t)
= −αx̃(t) + s̃(t)˙fx (t) = −f (·) + M (ϕ)[αx̃(t)− ẍr (t)]III. CONTROLLER DESIGN
The control objective is to develop a controller sothat all signals in the closed loop system are bounded
and the state x(t) tracks the desired trajectory xr (t)under the condition that the dynamic and hydrodynamic parameters (e.g., mass and a drag coeﬃcient)
are unknown constants.
In order to meet the objective, we make the following assumptions about the model (5) and the reference
model (i.e., the desired trajectory xr (t)):A1) The signals ϕ(t), x(t), u(t) and v(t) are available.
A2) The kinematic parameters in (5) are known constants.
A3) Each of the matrices J(ϕ) and R(ϕ) in (5) has afull rank.
A4) The desired trajectory xr (t) and the derivativesẋr (t) and ẍr (t) exist and are bounded.

©ISAROB 2011䯹

䱊

1
+ Ṁ (·)s̃(t) + x̃(t) ∈ RDn
2(9)

and fx (t) has the following property useful for ourcontroller development:
P4) There exists a positive constant cx such that∥fx (t)∥ ≤ cx ωx (t)(10)ωx (t) = 1 + α + α2+(1 + α2 )∥x̃(t)∥2 + ∥u(t)∥2 ∈ R.

135

(11)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

It is noteworthy that the coeﬃcient matrix BL(v) of
the input τb (t) in the error model (17) has a full rank
for all v(t), and hence the error model is controllable
in spite of the thruster nonlinearities.
The robust control law for the error model (17) subject to the assumptions A1 to A4 and A6 is given by

The robust control law for the error models (8)subject to the assumptions A1 to A5 is given by
[
]
z(t)
T
−1
(12)
τ (t) = −µx (t)R(ϕ) J(ϕ) s̃(t)mµx (t) = α + βx ϵ2 + βx ωx (t)2 ∈ R

(13)

(21)

µz (t) = α + βz ωz (t) ∈ R

(22)

2

where βx , ϵ > 0 are design parameters. It is shownthat the robust controller (12) guarantees an ultimateboundedness of the tracking error x̃(t).
2. Robust Controller II
According to the design procedure shown in Fig. 2,
we make the following assumption instead of the assumption A5 in the design of robust controller II:
A6) The control inputs are τb (t) for vehicle control
and τm (t) for manipulator control.
As shown in Fig. 2, we ﬁrst replace the input z(t)in (12) by the desired trajectory zr (t), i.e.,[]zr (t)= −µx (t)R(ϕ)T J(ϕ)−1 s̃(t)(14)τm (t)

where βz > 0 is a design parameter.
In addition to the error z̃(t), we use the errors s̃(t)and x̃(t), introduced for the design of robust controller
I, to guarantee the stability of the overall closed loop
system. The error models (8) concerning s̃(t) and x̃(t)need to be modiﬁed because z(t) in the input (12) for
robust controller I is replaced by zr (t). Using (15), we
rewrite (8) as

[
]
1

zr (t)−T˙
M (ϕ)s̃(t) = J(ϕ) R(ϕ)K τ (t) − Ṁ (·)s̃(t)



m
2
−T
¯
(23)
+J(ϕ) R(ϕ)K I z̃(t) + fx (t) − x̃(t)¯−2ϵJ(ϕ)−T R(ϕ)K Il(v)



˙
x̃(t)
= −αx̃(t) + s̃(t)and then design robust controller II by using the tracking error of z(t). When we choose the error as the normal one z̃n (t) = z(t) − zr (t), then the error model iswritten as z̃˙ n (t) = −AD(v)z(t) − żr (t) + BD(v)τb (t).
This model has a situation where the system is uncontrollable due to lack of the rank of BD(v) whensome of vi (t) equal zero. This situation is caused by
the thruster nonlinearities shown in Fig. 1. In order
to avoid the situation, we propose the following error
instead of the normal one z̃n (t):For robust controller I and II, the following theorem
holds:
Theorem 1 Consider the robust controller (14) and(21) for the error models (17) and (23) subject to theassumptions A1 to A4 and A6. This controller guarantees that the signals x(t), ẋ(t), u(t), v(t), z(t), zr (t),τm (t) and τb (t) in the closed loop system are bounded,and that the tracking error x̃(t) satisﬁes the inequality
ρ2(24)∥x̃(t)∥2 ≤ ρ1 e−γαt +
αβwhere ρ1 and ρ2 are positive constants, β = min{βx ,βz }, and γ = min{2 cRKJ /cM 1 , 2, 2 cB }.
The inequality (24) in Theorem 1 means that an ultimate bound of x̃(t) can be arbitrarily reduced by increasing the design parameters α, βx and βz . It should
be noted that a high gain controller can be constructed
for the model (5), since the slow thruster dynamics istaken into consideration in the design of controller.
Proof: We ﬁrst choose the positive deﬁnite function]
1[
V (t) = s̃(t)T M (ϕ)s̃(t) + x̃(t)T x̃(t) + z̃(t)T z̃(t) (25)2
and then the time derivative of V (t) along the solutionsof (17) and (23) is given byγρ2V̇ (t) ≤ −γαV (t) +
.
(26)
2β
In the derivation of (26), we use the control law (14)and (21), the inequalities in the properties P1 to P3
and the inequalities
ρ3 
−βx ωx (t)2 s̃(t)T H(ϕ)s̃(t) + s̃(t)T fx (t) ≤


2 βx



−βx ϵ2 s̃(t)T H(ϕ)s̃(t)
ρ4
(27)
T
−T
¯
−2 ϵ s̃(t) J(ϕ) R(ϕ)K Il(v) ≤

2 βx


ρ5


−βz ωz (t)2 z̃(t)T B z̃(t) + z̃(t)T fz (t) ≤
2 βzz̃(t) = z(t) − zr (t) + 2 ϵ l(v)
(15)

Dm
l(v) = {Im −{E(v)} v̄(v) ∈ R }
E(v) = diag e−|v1 | , . . . , e−|vDm | ∈ RDm×Dm (16)Tv̄(v) = [sgn(v1 ), . . . , sgn(vDm )] ∈ RDm .
It should be noted that the signal l(v) is bounded forall v(t). As a result of adding the term l(v), the error
model of z̃(t) is expressed as˙z̃(t)= BL(v)τb (t) − I¯ T KR(ϕ)T J(ϕ)−1 s̃(t)+fz (t)
(17)

Dm×Dm
L(v)[= D(v)
+
ϵE(v)
∈
R

]



Im
Dn×Dm
¯
I= 0 ∈R(18)
fz (t) = −AL(v)z(t) + I¯ T KR(ϕ)T J(ϕ)−1 s̃(t) −żr (t) ∈ RDm
and fz (t) has the following property useful for ourcontroller development:
P5) There exists a positive constant cz such that∥fz (t)∥ ≤ cz ωz (t)(19)
ωz (t) = wz2 (t) + wz3 (t)∥s̃(t)∥ ∈ R
wz1 (t) = 1[ + ∥u(t)∥2 + ∥z(t)∥] + ∥τm (t)∥˙
wz2 (t) = wz1 (t) + α∥x̃(t)∥ µx (t) + ∥L(v)z(t)∥ (20)˙wz3 (t) = 1 + (1[+ α2 )βx ωx (t)∥x̃(t)∥∥x̃(t)∥]+ µx (t) + βx ωx (t)wz1 (t) ∥u(t)∥.

©ISAROB 2011τb (t) = −µz (t)L(v)−1 z̃(t)H(ϕ) = J(ϕ)−T R(ϕ)KR(ϕ)T J(ϕ)−1 ∈ RDn×Dn (28)136The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

0.81[m]0.3[m] 0.3[m]Vehicle0.42[m]0.3

28.32[kg]0.2[m]
0.25[m]Joint 2

0.25[m](Front view)

0.12[m]4.25[kg]

0.12[m]1.23[kg]

Y [m]0Thruster
Joint 1
Manipulator-0.3(Side view)-0.3

where ρ3 , ρ4 and ρ5 are positive constants. From Lemma 3.2.4 in [8], (26) can be rewritten as
ρ2
.
(29)
V (t) ≤ e−γαt V (0) +
2 αβ

0
X [m]

0 [s]

0.3

Fig. 4. Robot motion for α = 104

×10 -2
α=8

䱊

x (t)

It follows from (29) that s̃(t), x̃(t), z̃(t) ∈ L∞ . Using(7), (11), (13) to (16), (20) to (22), the assumption
A4 and the inequalities in APPENDIX, we can easily prove that x(t), ẋ(t), u(t), v(t), z(t), zr (t), τm (t)and τb (t) ∈ L∞ . Moreover, we can directly derive theinequality (24) from (29), and the proof is complete.

α=9

α = 10

2

20
10
15
t [s]
Fig. 5. Tracking error x̃ for α = 8, 9, 10
0

IV. SIMULATION EXAMPLE

5APPENDIX
Inequalities: The following inequalities, where c∗ is apositive constant, are used for the design of controller
in this paper:˙(i) ∥J(ϕ)∥ ≤ cJ1 , ∥J(·)∥≤ cJ2 ∥u(t)∥, ∥R(ϕ)K∥ ≤cRK , ∥R(ϕ)∥ ≤ cR1 , ∥Ṙ(·)∥ ≤ cR2 ∥u(t)∥, ∥u̇(t)∥ ≤cu wz1 (t)(ii) ∥Ṁ (·)∥ ≤ cM 2 ∥u(t)∥, ∥M (ϕ)−1 ∥ ≤ cM 3 , ∥f (·)∥ ≤cf (1 + ∥u(t)∥2 ), ∥J(ϕ)−1 ∥ ≤ cJ3 (if J(ϕ) has a fullrank)

REFERENCES[1] Antonelli G (2003), Underwater robots: motion and force
control of vehicle-manipulator systems. Springer-Verlag[2] Fossen TI (1991), Adaptive macro-micro control of nonlinear underwater robotic systems. Proc Int Conf Advanced
Robotics, pp 1569-1572[3] Sarkar N, Yuh J, Podder TK (1999), Adaptive control ofunderwater vehicle-manipulator systems subject to joint
limits. Proc IEEE/RSJ Int Conf Intelligent Robots andSystems, pp 142-147[4] Canudas de Wit C, Diaz EO, Perrier M (2000), Nonlinear
control of an underwater vehicle/manipulator with composite dynamics. IEEE Trans Control Systems Technology
8(6): 948-960[5] Yatoh T, Sagara S, Tamura M (2008), Digital type disturbance compensation control of a floating underwater robot
with 2 link manipulator. Artificial Life and Robotics 13(1):377-381[6] Taira Y, Oya M, Sagara S (2010), An adaptive controller forunderwater vehicle-manipulator systems including thruster
dynamics. Proc Int Conf Modelling, Identification and Control, pp 185-190[7] Yoerger DR, Cooke JG, Slotine JJE (1990), The influence of thruster dynamics on underwater vehicle behavior
and their incorporation into control system design. IEEE J
Oceanic Engineering 15(3): 167-178[8] Ioannou PA, Sun J (1996), Robust adaptive control. Prentice Hall, p 75

V. CONCLUSION
In this paper we developed a robust controller forunderwater vehicle-manipulator systems with thruster
dynamics. In the controller development we presented
a new tracking error model that overcomes uncontrollability caused by the thruster dynamics. It is, furthermore, shown that all signals in the closed loop
system are bounded, and that an ultimate bound of
the tracking error can be reduced by increasing controller’s design parameters.

©ISAROB 2011

20 [s]
Trajectory ofend-effector position

Fig. 3. UVMS for numerical simulation

In order to conﬁrm the usefulness of our robust controller (14) and (21), we performed numerical simulation. Typical simulation results are presented in thispaper. The UVMS simulated here was an underwater
vehicle with a two-link manipulator, as shown in Fig. 3.
The values of these system parameters were the sameas those used in the reference [6]. In this ﬁgure, only
the values of the main parameters are shown. Except
for α, the controller design parameters were chosen as
βx = 0.003, βz = 0.001, ϵ = 1. Each of the desired trajectories of the vehicle’s position and the manipulator
end-eﬀector’s position is set up along a straight path.
Each of the velocities is given by a ﬁltered trapezoidal
function. On the other hand, the desired trajectory of
the vehicle’s orientation is selected to remain at theinitial value.
Fig. 4 shows the simulation result for α = 10. It canbe seen from this ﬁgure that x(t) (the vehicle’s positionand orientation and the manipulator end-eﬀector’s position) tracks the desired trajectory xr (t) in spite of thenonlinearities of thruster dynamics and the uncertainties of system parameters.
The simulation where α is selected as various values
was carried out. In this paper, the tracking errors for
α = 8, 9, 10 are shown in Fig. 5. As shown in Fig. 5,
the control performance is improved by increasing thedesign parameter α.

Trajectory ofvehicle position

137

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Digital Adaptive Control of a Winged Rocket
Applicable to Abort Flight
Tomoaki Shimozawa
Shinichi Sagara
Department of Control Engineering, Kyushu Institute of Technology
Tobata, Kitakyushu 804-8550, Japan
E-mail: sagara@cntl.kyutech.ac.jp
Abstract: Since reusable launch vehicles (RLVs) have wide range flight conditions, the values of parameters ofthe dynamic equation are not constant. Then some adaptive control methods for the RLVs have been proposedand digital control systems are suited for digital computers. However, the control performance decreases when
the nonlinearity strengthens though a linear adaptive control has an excellent performance when the nonlinearityof the controlled system can be disregarded. In this paper, we apply a digital adaptive feedback linearization
control method with time-scale separation to a winged rocket in the abort flight. The simulation results show
the eﬀectiveness of the control systems.
Keywords: Adaptive Control, Flight Control

Nomenclature
δa , δe , δrp, q, rα, βφ, θL∗ , M∗ , N∗
Iij
VTAS

1

large to guarantee control performance above a certain level in all flight conditions. Second, since thegain scheduling control method can only correspondence to the known change; the control performanceof the method becomes worse for the unpredictablechange in the flight condition such as the abort flight.
For the change of the flight conditions, adaptive
control methods using approximated linear dynamic
equation of the RLVs have been researched [2]— [5].
However, the control performance decreases when
the nonlinearity strengthens though a linear adaptive control has an excellent performance when the
nonlinearity of the controlled system can be disregarded. Then, the feedback linearization method fordeleting the nonlinear term by the state feedback forthe nonlinear equation of motion of the RLV havebeen researched [6]. In addition, the method for dividing time-scale by a fast motion and a slow motion
is researched for the simplification of the structureof the control system [7]. However, an internal aerodynamic parameter is fixed in their control methods.
In this paper, we apply a digital adaptive feedback
linearization control method with time-scale separation to a winged rocket [8] in the abort flight, and thesimulations are done to validate the eﬀectiveness ofthe control systems in wide range flight conditions.
The simulation results show that the control system
has a good control performance.

=aileron, elevator and rudderdeflection angles=rotatonal rates=angle of attack, sideslip angle=bank angle, pitch angle=aerodynamic rolling, pitching
and yawing moments=moment/product of inertia=true airspeed

Introduction

In late years space development is performed lively
all over the world, and the research of space transportation systems to enable them is performed. Especially, Reusable Launch Vehicles (RLVs) are expected for the space transportation systems because
the RLVs are the low-cost and highly reliable transport system instead of the conventional disposablerockets.
Since the RLVs have wide range flight conditions
to the space from the ground, the values of parameters of the dynamic equation of the RLVs are not constant. Therefore, gain scheduling control method [1]that is linearized at a series of design points to be
decided by the velocity and the altitude of the RLV
and whose parameters of the controller are switched
for the change of flight conditions has been applied
for the RLVs. However, the gain scheduling control
method has drawbacks for the RLVs. First, when the
air traﬃc window expands, the number of required
gains to be designed and the schedule becomes very

©ISAROB 20112

Model of winged rocket

Fig. 1 shows an outline of a winged rocket [8] andthe parameters are shown in Table 1. It has two
elevons and two rudders as aerodynamic control surfaces. Since the configuration of the winged rocket

138

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

On the other hand, nonlinear equation of slow
states motion is expressed as follows [9]:ẏ(t) = D(y)x(t) + Eh(t) + g(t)(3)

where,
y(t) = [α β φ]T
·
¸T
sin α
cos α
1
h(t) =
V cos β VTAS cos β VTAS
⎡TAS⎤
− cos α tan β
1− sin α tan β
sin α0− cos α ⎦ ,D(y) = ⎣1
sin φ tan θ
cos φ tan θ⎡ X Z⎤−m m 0
Y ⎦
E=⎣ 0
,
0 m
0
0 0

Fig. 1 Winged Rocket
Table 1 Winged rocket parameters
Body length
Mass
Body outer diameter
Wing area
Wing span
Mean aerodynamic chord
Position of center of gravity

Parameter
2.5[m]
241[kg]0.57[m]1.05 [m2 ]1.8[m]
0.67[m]65[%]

and g(t) is the vector consisting of the gravitational
forces. Similar for fast states motion, ẏ is discretizaed for forward diﬀerence by sampling period
T , Eq. (3) is expressed as follows:y(k + 1) = y(k) + DT (y)x(k) + ET h(k) + T g(k).
(4)

shown in Fig. 1 is similar for general airplanes, thenonlinear equation of fast states motion is expressed
as follows [9]:ẋ(t) = A(x)x(t) + B(x)u(t) + C(x)w(t)3

The design method in this paper is a digital adaptive feedback linearization control method with timescale separation. To derive the input u to control the
output y, this method is separated two time-scale.
In the fast time-scale, control surface deflection command u = [δac δec δrc ]T is derived from rotational
rate command xc = [pc qc rc ]T . In the slow timescale, rotational rate command xc is derived from
output command yc = [αc βc φc ]T .

(1)

where
x(t) = [p q r]T , u(t) = [δac δec δrc ]T ,w(t) = [pq qr rp r2 − p2 α β]T ,
⎡
⎤
Lp
0
Lr
A(x) = Id ⎣ 0 Mq 0 ⎦ ,
0 Nr
Np
⎡
⎤
Lδa
0
Lδr
Mδ e
0 ⎦,
B(x) = Id ⎣ 0
0
Nδr
Nδa

3.1

Inner loop for the fast states

Here, control input to Eq. (2) is defined asu(k) = BT−1 {xd (k + 1) − AT (x)x(k) − CT (x)w(k)− Λ(xd (k) − x(k))}C(x) = Id⎤⎡
Ixz
Iyy −Izz
000 Lβ0Izz −Ixx −Ixz Mα 0 ⎦
·⎣ 0
0
0
0 Nβ
Ixx −Iyy −Izz

(5)where, Λ =diag{λi } (i = p, q, r) is the gain matrix
and xd is the output of the reference model. From
Eqs. (2) and (5), the output error ef (k) = xd (k) −x(k) isand Id is inertia matrix. When ẋ is discretizaed forforward diﬀerence by sampling period T , Eq. (1) isexpressed as follows:ef (k + 1) = Λef (k).

(6)

And if λi is selected to satisfy 0 < λi < 1, the output
error ef (k) tends to zero as k tends to infinity. In acase of unknown parameters, we introduce the matrices ÂT (x), B̂T (x) and ĈT (x) those coeﬃcients are
estimated values of the coeﬃcients in AT (x), BT (x)and CT (x), respectively. The coeﬃcients in ÂT (x),B̂T (x) and ĈT (x) are estimated by using adaptivealgorithms [11].

x(k + 1) = (I + T A(x))x(k)+ T B(x)u(k) + T C(x)w(k)= AT (x)x(k) + BT (x)u(k) + CT (x)w(k)(2)
where, I is identity matrix.

©ISAROB 2011

Control system

139

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig. 2

3.2

Fig. 3

Input

Outer loop for the slow states

the winged rocket changes the thrust power from3000[N] into 0[N] for changing the route. That is
an abort flight. At the start of the simulation, thealtitude of the winged rocket is 3200[m], downrange
is 3000[m], crossrange is 0[m], the velocity is 80[m/s]and the pitch angle is 70[deg]. At the target point,
downrange and crossrange are 0[m], and altitude is
200[m]. The aerodynamic coeﬃcient of the winged
rocket uses the value obtained from the wind tunnel
examination result. The actuators of elevons is usedthat the attenuation coeﬃcient ζ = 0.7 and the natural frequency ωn = 72[rad/s]. The sampling periodof the control system is T = 0.01[s]. The gain of
fast states are λp = 0.95, λq = 0.90 and λr = 0.90.
The gain of slow states are pα = 0.90, pβ = 0.92 andpφ = 0.95. A constant trace algorithm [11] is utilized
for parameter estimation.
Figs. 2 and 3 show the input and the output, respectively. In Fig. 3, the dotted lines show the output of the reference model. From Fig. 2, we cansee that the input commands to the actuators is almost smooth. And from Fig. 3, it can be seen thatthe angle of attack and the bank angle follow to thereference output well, and the side-slip angle is suppressed to 0.5[deg] or less. Next, Fig. 4 is flight trajectory. From Fig. 4, we can see that the winged

From Eq. 4, the rotational rate command is obtainedby following equations:xc (k) =DT−1 {yd (k + 1) − y(k) − ET (y)h(k) − T g(k)(7)
−P (yd (k) − y(k))}
where, P = diag{pi } (i = α, β, φ) is gain matrix
and yd is output of the reference model. From Eq. 3
and 7, output error es (k) = yd (k) − y(k) is
es (k + 1) = P es (k).

(8)

And if pi is selected to satisfy 0 < pi < 1 , the output
error es (k) tends to zero as k tends to infinity.
In this paper, unknown parameters in Eq. 7 uses
the fixed values obtained from the wind tunnel examination result.

4

Numerical simulation

To validate the adaptive control system described,
computer simulations for a 6-DOF nonlinear winged
rocket model [10] considering the atmospheric fluctuation are performed. simulation condition is follows. When the winged rocket is climbing, a trouble is happened at about 4000[m] in altitude. Then

©ISAROB 2011

Output140

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig. 4

Trajectoryrocket arrives at the neighborhood the target point.
From the simulation result, it can be confirmedthat the applied control method is eﬀective for theflight control of the winged rocket.

5

Control for a Tailless Advanced Fighter Aircraft”, Proceedings of AIAA Guidance, Navigation and Control Conference, pp. 75—87, 1998.
[5] Eric N. Johnson, Anthony J. Calise, “PseudoControl Hedging:A New Method for Adaptive
Control”, Advances in Navigation Guidance andControl Technology Workshop, November, 1-2,
2000.
[6] Eric N. Johnson, Anthony J. Calise, “A Six Degree of Freedom Adaptive Flight Control Architecture for Trajectory Following”, Proceedingsof the AIAA Guidance, Navigation, and Control Conference, 2002.
[7] P. K. Menon, V. R. Iragavarapu, andE. J. Ohlmeyer, “Nonlinear Missile Autopilot
Design Using Time-Scale Separation”, AIAA
paper, 97-3765-CP, pp. 1791-1803, 1997.
[8] K. Yonemoto, T. Shidooka, K. Okuda, “Development and Flight Test of Winged Rocket”, The
27th International Symposium on Space Technology and Science, 2009.
[9] B. Etkin, L. D. Reid, Dynamics of Flight: Stability and Control, New York: John Wiley &Sons, Inc , 1996.
[10] HIMES Research Group, “Conceptual Designof HIMES (Winged Test Rocket)”, Institute ofSpace and Astronautical Science, Ministry of
Education, March 19, 1987 (in Japanese).
[11] G. C. Goodwin, R. H. Middleton, Digital
Control and Estimation, Prentice-Hall International, p. 379, 1990.

Conclusion

In this paper, we apply a digital adaptive feedback
linearization control method with time-scale separation to a winged rocket in the abort flight. From
the numerical simulation, we showed that the control system of the winged rocket has a good control
performance.

References[1] T. Tsukamoto, H. Suzuki, T Ninomiya, “Guidance and Control Law Design for High Speed
Flight Demonstration Phase II”,JAXA Research and Development Report, JAXA-RR-04006, 2004 (in Japanese).
[2] Eric N. Johnson, Anthony J. Calise, “Limited
Authority Adaptive Flight Control for Reusable
Launch Vehicles”, Journal of Guidance, Control, and Dynamics, vol. 26, no. 6, pp. 906—913,
2003.
[3] M. Morimoto, K. Uchiyama, Y. Shimada,
A. Abe, “Adaptive Attitude Control with Reduced Number of Estimated Parameters for Automatic Landing System”, International Conference on Control, Automation and Systems
2007, pp. 2865—2870, 2007.
[4] J. Brinker. and K. Wise, “Reconfigurable Flight©ISAROB 2011141

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011                                
    !"" #! $% & & &'


 

 



  "                        

    !         "          %  &

 #



    
 $

%   




'     ( !   !    )             

 "  

 

      *+,-.

 (  0 

/ 

  "        %            )  (0) 

 "  *1! 2.        %  &  



 $



 "



 (0)
%

  '    /!        

%   3          










          !      

 



      "  !

 #

% "

       "   !       %  &  " #

    











 4  (0) 5            





 $

     ¿ ¿     ¾    ! !"    $ !"    & !"  ' ('''  !"  ('''  $"  ('''  &"  ('''    ¾ ) *Æ* !") *Æ* $") *Æ* &" "     

%   '           #   

6 % +# 

   

  '     7  


!

5

©ISAROB 2011 *1.

( 0      4     

(0) 

 





6 % +

  )    142

 
 
  
# 

%

%
%
 
 # 
  # 








'     





 #
%
%
 








The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Tx =[x 0 x e ]
T

xd

T

νd

Eq. 3

αd

Eq. 2

ν =[ ν 0T ν eT ]6 % '"   #ζ =[ ν 0T φ T ]

T

UVMS

T

xe

φ ν 0 νe " (0)  

 (0)  "    

x0

u

Eq. 1

)8%

" % 4


φ x0 xeq =[ x 0T φ T ] T





 

 "



% "



  !      

 


 

¼

      
9

 9





+

:







:

+









:

 : +

 :

 9



6 % ;

'



+ :



+

Base controller

;



¼


 " <4 +

 ¼  
9 *

.



 

   

 > 







%   


%



 "

  %

 ¼  
¼

=

9 *=

 " 
!



.



   % " !
%





 " )     "









 %!

  *.

"    !







 > 

4

   6 <4 '  ;

 9



 "



!

% "



!

 9







! 

"

   "  



!



 

6 % ' 

6 % ; 

8%

% 

 " ;?56 

% %  



 

 

  "    " ©ISAROB 20119  % 





 "

   %   #



 (0)  

"









5

 



> 

"

#


  "  

 

      

 



%         4         "      $   

!       %    % 
 > 

%



8%   !      

 %  "    " "

  

 

8

   "      A !

    

 

 " 0    "     %        %    



 

  ;?56   "       4 % %    

 

!

#

    % 

% 

!

 "







   

 





! +' % "      
# " 





.

 "     

9



9 *

)8%

4

 

 "  %   

6 % @

9

 9+!



Manipulator controller   %  !

   !

 "9  % 

 "  

Robot dynamics
and RAC law

RC servomotor





RS232C

Micro
computer
(SH2)

RS485





 

RS232C

Transducer  


Voltage

RS485
!

!

!

UVMS

RC servomotor Potentiometer




¼ ¼   ¼ ¼

 





¼
 ½


     
 ¼  







!

  )  "      












    #%  "
   >    143 

%       %    !  %     

 6 % @  % 

 




     




       



 ++-'BB*.







  

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

LED
Cable
LED
LED
Base
Thruster
CCD
cameraLink 1
AC
servo
motor

Joint 1

Link 2
Joint 2
Test Tank

CCD
cameraXY-Tracker

Motor driver
Thruster driverCounterboard

6 % -



D/A
board

)8%

Side view of UVMS

Micro
ComputerComputer "      

 *+. / 



 

$


!      "   

 %





 "



         






"

  
%   



+FF+



 "# E!

   " 

% 

 



       " 









!   "J 

!  '+; , ';'! +FF1

 



      





E!

 <

! D#  % )  "     !

'-! A ;!  ;FF , @+;! 'BBB
*@. A   # I J! D)    J %  )  " 0  !

    "





! I 0 *;. K 0  "

  

       " 6 % 2" "   )

! D<   

 8%!    " 6 % 2      %


 

 

   )  "    0H "
!

 C! A ;!  ;;F , ;2B!

    " 6 % 1    "6

  E!

!

       




 < 

6 % 2   (

*'. #   G 

"



! D0 )   )  "       "  

 



 

6 % 6 % 1 

MasterController      >

 ?% 5 

 '1! A '!  ''C , ';F! 'BB+
*-. K

 6 % C

 3     0  !

E!

   !

!

 %!

"

 ++F@,+'B1! 'BB;
*1.    !      !

 5   "     " " ©ISAROB 2011 

%

  

 #

           %  & 

!D? %

E!

  

 "

!

% '    

144

(0)

 

)  %       +B! A '!  +B1 , +++! 'BB1 

!D? %

5 "    E!

        "  % )8%

*2. 



%

(0) 



? 





       +-! A ;!  '2B , '2@! 'B+B

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011     

 "     



 3    "      

6 % 2#

 



Seesaw motion 

 "  Horizontal Motion 
6 % 1©ISAROB 2011

Vertical motion "  (  

6 % C



145

5

 "



 



The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Survey of AI-Based Systems for Human Awareness Promotion
in Meta-Cognition
Kiyota Hashimoto*, Kazuhisa Seta*, Hiroshi Tsuji*, and Kazuhiro Takeuchi**Osaka Prefecture Univ*.: 1-1, Gakuen-cho, Naka-ku, Sakai, 599-8531, Japan
Osaka Electro-Communication Univ**., 18-8, Hatsu-machi, Neyagawa, 572-8530, Japan(Tel : 81-72-252-1161; Fax : 81-72-254-9944)(hash@lc.osakafu-u.ac.jp)
Abstract: More and more AI-based systems are being developed and used for human awareness promotion. Humanawareness promotion is important in various fields like those of learning and problem-solving where participants are
expected to be aware of the changing contextual information of themselves and the environments around them toperform better. For a better computational supports of such promotions, AI-based approaches with particular reference
to the mechanism of human meta-cognition seem to be plausible. In this paper, as an introduction to the session
dedicated to this issue, we give a short survey on the definition of awareness and key factors of AI-based approaches tohuman awareness promotion.
Keywords: AI-based system, human awareness, awareness promotion, meta-cognition
though not limited to these.
Gutwin and Greenberg [1], for example, discuss
human awareness as a situational one with whichhumans are aware of situational changes related to thetask they are doing. They identified the following fourbasic characteristics of human awareness, as distinct
from other kinds of knowing, according to [2],[3],[4].
1. Awareness is knowledge about the state of someenvironment, a setting bounded in time andspace. For example, the environment might be
the airspace that an air traffic controller is
responsible for, and their knowledge might
include aircraft headings, altitudes, andseparation, and whether these factors imply asafe or unsafe situation.
2. Environments change over time, so awareness isknowledge that must be maintained and kept upto-date. Environments may change at different
rates, but in all cases a person must continuallygather new information and update what they
already know.
3. People interact with the environment, and themaintenance of awareness is accomplishedthrough this interaction. People gather
information from the environment throughsensory perception, and actively explore theirsurroundings based on the information that theypick up.
4. Awareness is almost always part of some otheractivity. That is, maintaining awareness is rarely
the primary goal of the activity: the goal is tocomplete some task in the environment. For

I. INTRODUCTION
When we do something, we do not only do the task
but perform a variety of mental activities related to thetask. One of the most important performances among
them is awareness. We humans are always aware of, andsometimes miss, something that may be related to thetask we are doing. Then what and how are we aware of,and how can computational supports promote andfacilitate human awareness?
In this paper, we will review the concept of humanawareness and some key factors of AI-based
applications for human awareness promotion.

II. AWARENESS AND METACOGNITION
Although we too often take human awareness for
granted, the nature and characteristics of humanawareness is not apparently obvious. In fact, researchers
have discussed human awareness differently fromdifferent perspectives.
Most broadly, awareness is the mental state or abilityof a person to perceive, to feel, or to be conscious ofanything, from events that are happening, to thesituation around him, to the conditions of himself. In
this sense, awareness may not imply understanding
what it is and what it means. From the viewpoint ofcomputational support of the awareness, however, it isusually assumed that awareness implies understanding.
More specifically, awareness may be classified into
several kinds: situational awareness, workspaceawareness, knowledge awareness and self-awareness,©ISAROB 2011

146

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

example, the air traffic controller’s task is to
move aircraft through a region efficiently andsafely, and although awareness may affectsuccess, it is not the primary intent. ( [1] pp.8-9)
Here, importantly, Gutwin and Greenberg clearly statethat awareness is something related to the surrounding
environment during the course of performing a task.
Then they distinguish situational awareness fromworkspace awareness. Workspace awareness, theyargue, is “the up-to-the-moment understanding ofanother person’s interaction with the shared workspace([1] p.10),” according to [2]. Note that their concept ofworkspace awareness is solely related to the people in
the workspace and how they interact with it.
On the other hand, Ogata, Matsuura, and Yano focus
on knowledge awareness in their seminal works [5]-[10].
They follow the definition of awareness by Dourish andBellottie [11]: “understanding of the activities of others,
which provides a context for your own activity,” anddefine knowledge awareness as awareness of the use ofknowledge. Knowledge awareness, they argue, “gives
each learner information about other learners’ activities
in a shared knowledge space.”[10]Both Gutwin and Greenberg and Ogata et al. focus on
the possibility of computational support for a successful
group dynamics and the sharing of knowledge in thegroup. Thus computational systems to support
awareness along these lines, including many studies on
computer-supported
collaborative
learning/work(CSCL/CSCW), are inevitably designed to facilitateawareness of others’ activities.
However, awareness can also be viewed as a mental
activity of a single individual. In this context, awarenessis captured as the monitoring of object-level in themetacognitive process, as shown in Fig. 1, and as thediscovery of something useful for the task and/or thedevelopment of metacognitive ability. From this viewpoint, not only the awareness of what a learner is doing
and the environment around him but also the awarenessof his own mental state and process is focused.

This concept of awareness is heavily related to theconcept of self-regulated learning [13],[14]. According
to the idea of self-regulated learning, learning is
considered to be guided by metacognition, cognition inlearning processes, and motivation, as shown in Fig. 2.
Metacognition
Knowledge and self-awareness the learner has tomonitor his/her understanding and cognitive
processes

Cognition in Learning Processes-Goal settings-Situation understanding-Choice and execution of learning
strategies suitable to the targetedlearning object-Evaluation of the progress

Motivation
The will to learn and students’ confidence intheir ability to organize tasks and makejudgments in executing the necessary course ofaction to achieve explicit types of outcomesFig. 2 Simple model of self-regulated learning
Here, what learners are supposed to learn is what abetter learning is, as well as the learning objects. Thus,
computational systems to support awareness along thisline focus on facilitating or promoting awareness of
self-condition compared to the desirable condition.
There are numerous attempts along this line, including[15],[16],[17],[18].
Before leaving this section, note that these twodifferent lines do not contradict each other. Rather, selfawareness can be boosted with the help of awareness ofthe others around him. Thus, the latter often focus on
the cooperative or collaborative efforts in the group fora better self-awareness.

Meta-level
Control

Monitoring

Flow of InformationObject-level
Fig. 1 Theoretical Model of the Mechanism of
Metacognitive Process (Nelson and Narens 1990)©ISAROB 2011

III. KEYS TO HUMAN AWARENESS PROMOTION
For promoting human awareness, whether it be

147

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Thus it has to be investigated what a better, effective
way is to extract or mine the relevant, useful
information, and to present it to learners.

shared awareness in a group or self-awareness, there
seem to be some key factors that must be considered:personalization or adaptivity, inferential model,
effective information extraction, and explicitization or
manifestation, though not limited to these. All thesefactors are expected to be supported effectively by AIbased systems.

4. Explicitization or manifestation
As we can walk without being consciously aware of
inner muscle movements, we often do something
without being aware of our relevant mental processes.
However, the concept of self-regulated learning assumes
that the more conscious, the better.. Sharing knowledge
requires each to be conscious of what they know. Thus
awareness, be it shared in the group or noticed by asingle individual, involve the process of making the
unknown or the hidden explicit to cognition. So
computational support systems for awareness promotion
has to do with effective explicitization of unknown or
hidden information by persuading learners to turn theireyes to them.

1. Personalization or adaptivity
Naturally, humans differ from one another, not onlyin their preferences, characteristics, and socio-cultural
backgrounds but also in their maturity and ability. Thus,
any computational support system for promoting
awareness has to adapt to individual learners in order tobridge the gap among different types of learners. In
order to achieve this kind of adaptivity, the presentationof the domain knowledge and/or the learner’s learning
condition should be tailored in complexity andgranularity according to his changing profiles. Formerly,
this was achieved by preparing different profilecorresponding templates the choice of which was doneby the learner himself. However, as expected,
dependence on the leaner’s choice do not always reflect
the actual condition of the learner and the more detailed,the harder the choice will. This naturally requires thesystem to be driven by a inferential modeling in order tobe intelligent enough.

VI. CONCLUDING REMARKSThis paper attempted a short, fragmental survey ofawareness promotion, roughly consisting of review of
the concept of awareness and key factors related toawareness promotion. To see more instances ofcomputational support systems for awareness promotion,[26],[27] are a good starting point, together with theproceedings of annual ICWL, CSCWD, ICCE, andother related conferences and workshops.

2. Data-driven inferential model
If voluntary choices of preferences, conditions, andothers, are not informed of by the learner himself,
computational support systems have to obtain relevant
information otherwise. For example, most web-basedlearner support systems make use of various records ofuser actions as well as conscious inputs by the users. All
these data, however, speak nothing by themselves. A
rather detailed mental/learning process model is needed.
The model to be adopted may vary among systems, but
due to the large number of possible complex
combinations of the acquired data, more than a simple
matching method will be needed, and AI-based methods
have been utilized. In particular, more and more studies
are pursuing the utilization of ontology as a descriptiveengine for inferential reasoning. As for mental models
in detail, see [19]-[25], for example.

REFERENCES[1] Gutwin, C. and S. Greenberg (1999) A Frameworkof awareness for small groups in shared-workplacegroupware, Technical Report 99-1, Dept. ofComputer Science, Univ. of Saskatchewan,
Canada.Mourlas, C., N.
[2] Adams, M., Y. Tenney, and R. Pew (1995) Situation
awareness and the cognitive management ofcomplex systems, Human Factor, 37(1), 85-104.
[3] Endsley, M. (1995) Toward a theory of situation
awareness in dynamic systems, Human Factors,
37(1), 32-64.
[4] Norman, D. (1993) Things that make us smart,
Reading, MA: Addison-Wesley.
[5] Matsuura, N. T. Hidaka, K. Okada, and Y.
Matsushita, (1995) VENUS: An informal
communication environment supporting interest
awareness, Trans. Information Processing Societyof Japan, 36-6, 1322-1341.
[6] Ogata, H. K. Matsuura, and Y. Yano (1996)
Knowledge awareness: bridging between shared

3. Effective information extraction and mining
Modern computational support system for awarenesspromotion naturally have to deal with a great quantity ofdata both on learning objects and learner monitoring.

©ISAROB 2011

148

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011[19] Dunlosky, J. and J. Metcalfe (2009) Metacognition,
Los Angeles: Sage.
[20] Flavell, J. H. (1979) Metacognition and cognitive
monitoring: a new area of cognitive-developmental
inquiry, American Psychologist 34, 906-911.
[21] Fox, E. and M. Riconscente (2008) Metacognitionand self-regulation in James, Piaget, and Vygotsky,
Educational Psychological Review 20, pp.373-389.
[22] Hacker, D.J. and J. Dunlosky (eds.) (2009)
Handbook of Metacognition in Education, New
York: Routledge.
[23] Hacker, D.J., J. Dunlosky, and A.C. Grasser (eds.)(1998) Metacognition in Educational Theory andPractice, New Jersey: Lawrence Elbaum
Associates.
[24] Perfect, T.J. and B.L. Schwartz (eds.) (2002)
Applied Metacognition, Cambridge: Cambride
Univ. Pr.
[25] Nonaka, I. and H. Takeuchi (1995) The knowledge
creating company. How Japanese companies
create the dynamics of innovation, New York:
Oxford University Press.
[26] Pahl, C. (2008) Architecture Solutions for Elearning Systems, Idea Group Reference.
[27] Tsianos, and P. Germanakos (2009) Cognitive andEmotional Processes in Web-Based Education:
Integrating Human Factors and Personalization,
Hershey, NY: Information Science Reference

knowledge space and collaboration in Sharlok,
Proc. of Educational Telecommunications ’96,
232-237.
[7] Ogata, H., K. Matsuura, and Y. Yano (1997)
Awareness filtering: toward efficient collaborative
learning, Proc. of Artificial Intelligence in
Education, 207-214.
[8] Ogata, H. and Y. Yano (1998) Knowledge
awareness: bridging learners in a collaborative
learning environment, International Journal of
Educational Telecommunications, AACE, 4-2/3,
219-236.
[9] Ogata, H., K. Imai, K. Matsuura, and Y. Yano (1999)
Knowledge awareness map for open-ended andcollaborative learning on World Wide Web, Proc.
of ICCE ’99, Vol.1, 319-326.
[10] Ogata, H., Matsuura, K. and Yano, Y. (2003)
Supporting Awareness in a CSCL Environment,International Journal of Cognitive Processing
Vol.2-3, 1-15.
[11] Dourish, P. and V. Bellotti (1992) Awareness andcoordination in shared workspaces, Proc. ofComputer Supported Cooperative Work ’92, 107114.
[12] Nelson, T.O. and L. Narens (1990) Metamemory: A
theoretical framework and new findings, The
Psychology of Learning and Motivation, 26: 125173.
[13] Zimmerman, B. J. and Martinez-Pons, M. (1990)
Student differences in self-regulated learning:
Relating grade, sex, and giftedness to self-efficacy
and strategy use, Journal of Educational
Psychology, 82, 51-59.
[14] Zimmerman, B. J., & Risemberg, R. (1997). Selfregulatory dimensions of academic learning andmotivation. In G. D. Phye (Ed.), Handbook ofacademic learning: Construction of knowledge, pp.
105-125, New York: Academic Press.
[15] Seta, K., K. Tachibana and M. Umano (2003) Basic
consideration on reflection in problem-solving
oriented learning, Proc. of ICCE 03, 160-168.
[16] Seta, K., K. Tachibana, M. Umano and M. Ikeda(2005) Human Factor Modeling for Developmentof Learning Systems Facilitating Meta-Cognition,
in C.-K. Looi, D. Jonassen, M. Ikeda (Eds):
Towards Sustainable and Scalable Educational
Innovations Informed by the Learning Sciences,
Frontiers in Artificial Intelligence and Applications,Vol. 133, pp. 396-403, IOS Press[17] Seta, K. and M. Ikeda (2008) Conceptualizations
for Designing a Learning System to Facilitate
Metacognitive Learning, Proc. of World
Conferenceon
Educational
Multimedia,
Hypermedia & Telecommunication, 2134-2143.
[18] Nakagawa, Y., H. Tsuji, K. Seta and K. Hashimoto(2010) Prototype of Collaborative Communication
Portfolio for Better Awareness through SelfReviews, Proc. of International Symposium on
Aware Computing, 220-226.

©ISAROB 2011

149

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A Multilingual Problem-Based Learning Environment for
Awareness Promotion
R. Taguchi, Katsuko T. Nakahira, H. Kanematsu*, and Y. Fukumura
Nagaoka University of Technology *Suzuka Technical College(Tel : 81-258-47-9847)(s095146@stn.nagaokaut.ac.jp)

Abstract: Traditionally, Problem Based Learning (PBL) has attracted attention as a method for training in engineering
design skills. However, PBL is constrained in that students have to gather in one location, so cooperation between
several institutions is difficult. Constructing a multilingual PBL environment in a virtual space on the Web (SecondLife) is one solution to this problem. In this paper, a summary is provided of one such multilingual PBL that has been
constructed.
Keywords: Less than 6 words.

into the variations in the various parameters and therelationships between phenomena. Using a virtual spacecan provide learners with experiences not possible intraditional e-learning or face-to-face lessons, making it
possible to promote new discoveries by the learners.
For this paper, we investigated the requirements for
multilingual PBL in a virtual space and implemented
them.

I. INTRODUCTION
Traditionally, Problem Based Learning (PBL) has
attracted attention as a method for training inengineering design skills. PBL is a learning process bywhich learners communicate with each other about acertain issue and discuss methods of solution. It is often
employed in fields such as medicine and engineering
where importance is placed on solving technicalproblems on site. However, when carrying out PBL, thelearners need to gather in one location and work
together, making cooperation between widely scatteredinstitutions difficult. Furthermore, due to theglobalization of recent years, the nationalities ofstudents are becoming diverse and the language barrier
this causes is also a problem.
Use of a virtual space connected to via a network andequipped with multilingual functionality is one answer
to this problem. By using it, students are not restrictedby location and, as well as being able to communicate
synchronously and remotely with other users, it ispossible to promote constructive PBL amongst students
of diverse nationalities. Also, by providing materials
that exploit the characteristic features of the virtual
space, the effectiveness of learning can be further
enhanced. There are many reported cases, such, forexample, as teaching materials which enablevisualisation of a miniaturised world, or learning
through Virtual Reality (VR) (XXX[1]); medical
teaching materials using 3D computer graphics(XXX[2]); or, by adding simulation functionality inaddition to these, a system which can provide insight©ISAROB 2011

II. DEVELOPPING MULTILINGUAL PBL
SUPPORT SYSTEM
An overview of the multilingual PBL support
environment and a configuration diagram of themultilingual PB L support system constructed areshown in Figures 1. In this paper, Second Life (SL) [4]
was used as the virtual space. SL is a virtual space
operated by the US company, Linden Lab. As well as
being able to freely construct environments such asclassrooms within the region possessed, by using thededicated language, Linden Script (LSL), functionality
such as HTTP communication with an external server
can be implemented. The multilingual PBL support
system used mainly this mechanism to implement
various features. PHP+PostgreSQL and Ajax were used
for system development. The multilingual PBL support
system is composed of a client that the students operate
and a university server that is able to connect to Lang,
the Language Grid(LanG)[5] server that is required forthe multilingual functionality provided within SL. Also,
the SOAP protocol was used to communicate between
the university server and the LanG server. In the section
below, the multilingual communication functionality,

150

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

START（１）HTML+JavaScript

Whiteboard

User draws

The drawing data is transmitted
to the data management server

The data of drawing is accumulated
in the data base（２）PHP

END

Figure 1 System View of Second Life

Figure 2. Flowchart of writing on whiteboard
START

START
The drawing data is requested （１）HTML+JavaScriptto the data management server

Sensor set（1）LSL

User's location data is collected
and it transmits to the server

The data base is checked, and it is （２）PHP

3 seconds confirmed whether there is an update
sleeping
Presence of updateNO

５ seconds
sleeping

YES（２）PHP

The data of location is
accumulated in the data baseResponse is returnedWhiteboard ismade the latest

Sensor is ON（３）HTML+JavaScript

YES
NO

YES

Whether the user
is looking or not

NO

NO

YES

END

Figure 3. Flowchart for whiteboard update

END

Figure 4. Process for gathering recorded learner behaviour
service infrastructure enabling sharing of translation
services and dictionaries over the internet. Currently,
English, Japanese, Chinese, Korean, etc., are supported,but, by adding dictionaries, support of other languages
is possible. For this paper, only Japanese, Chinese andKorean are used, but in the future, 11 languages will be
provided in LanG starting with Malay, which is handled
in the dictionary for engineering training at thisuniversity, and this will enable contributions to be madeto engineering education.

whiteboard system and learner monitoring functionality
are explained.
1.

Improving Multilingual Communication Function
The most difficult point in multilingual
communication is the language used whencommunicating. Usually, in international conferences
and the like, English is used as a common language tocommunicate; however this can be a high hurdle,
particularly for students carrying out multilingual
communication for the first time. Machine translation isprovided within SL as a HUD [6]; however, astranslation is supported only between Japanese andEnglish, it cannot be used to interact with users whose
mother tongue is a language besides these two. For thispaper, the text-based multilingual communication
functionality developed by Yoshino and Ikenobu [7]was used so that this language barrier did not inhibit
lively discussion. This is achieved by connecting thechat feature implemented in SL with LanG. LanG is the©ISAROB 2011

The fixed time passes

2.

Common White Board
Even with the multilingual communication
functionality included, it is perfectly conceivable thatstudents are not able to develop the discussion in theway they expected. In that kind of case, using diagrams
to supplement text broadens the possibilities forstudents to communicate their ideas in greater depth.
Consequently, a shared whiteboard was implemented,
on which the students could freely write. As part of151

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

functionality was implemented that records and enables
browsing of learners’ behaviour.
After the user chat and behavioural records arecollected using LSL within the metaverse, they are sent
to the university server after passing through the SL
server and are stored in the database. The flowchart forthis functionality is shown in Figure 4.
Monitoring is not always ongoing: information
gathering begins when the instructor turns on the sensor
switch in the virtual space. If the switch is turned off ora set amount of time passes, there is a mechanism tohalt information gathering.
It is possible to browse the gathered information on
an administrator Web page. Figure 5 shows theadministrator Web interface which enables browsing of
the students’ position information and the most recent
conversations.
It is possible to browse the students’ positional
information and the most recent conversations at thetime and date indicated by the upper timer. As well as
being able to operate the timer manually using a button,
it is also possible to advance it or move it backwards
automatically. Also, the position information alone or
the conversation information alone can be browsed. In
addition, the whiteboard drawing history can also be
displayed. This functionality is implemented in Ajax,
and a request is sent to the server when the timer isoperated, and information is acquired at the indicatedtime and date.

Figure 5. Learner monitoring screen
understanding student behaviour, the shared whiteboard
saves the drawing history, and a feature enablingbrowsing on an administrator Web page was
implemented. Further, from this record of drawings,
instructors should be able to comprehend more precisely
such things as the content of the discussion and thestudents’ levels of understanding. The processing flowof the shared whiteboard from drawing to recording is
as shown in Figures 2 and 3.
Figure 2 shows the processing that occurs
when a user writes something down on the whiteboard
and the data written down is sent to the server andregistered in the table that records the state of thewhiteboard in the database. In addition, Figure 3 shows
the processing when the whiteboard is checked for
updates and updated to the latest state when someone
has written something down. The whiteboard continues
to function as long as the user displays it, but it isdevised so as not to return a response until it has been
updated, thus easing the communication load. Throughthese features, the students are able to share the same
whiteboard, and are also able to keep the recorded
drawings. This whiteboard is not only viewable fromwithin SL, but also from a web browser.

III. THE RELATION OF LEARNER
MONITORING AND MULTILINGUALPBL SUPPORT SYSTEM
Based on the above elements, it was possible to carryout PBL practice in the metaverse. Here we consider the
relationship between the activities of learners that wereacquired from the metaverse and the awarenesstriggered in the instructors.
The state of the individual learners in the metaverse
classroom can be confirmed through 3 steps:
confirmation of spatial participation status by checking
the learner’s avatar’s position, confirmation of the statusof the student’s avatar’s utterances, and confirmation of
the content of the student’s avatar’s utterances. Thisconfirmation is also carried out in the real world, but inthe real world it is not possible to confirm down to thedetailed content of the utterances. On the teacher’s side,
by using the learners, it is possible to provide guidance

3. Learner Monitoring Function
In PBL where the students are the main actors, when
student evaluation is carried out, understanding thestudents’ conversations and behaviour during the lesson
is particularly important. Remarks by the students
during the lesson are recorded, if able to be monitored,
and the level of student contribution to the discussion
can be understood. If information about the students’positions and directions in the metaverse can berecorded, it can be used to identify such things as thelevel of interest students have towards their studies, andwhich students did not take part in any discussion.
Therefore, in order to achieve this, learner monitoring©ISAROB 2011152

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

functionality, construction of a shared whiteboard, and alearner monitoring system were developed as a support
system to carry out livelier multilingual PBL. In
addition to providing the above environment, byenabling the PBL to take place in the metaverse, there
was mutual understanding of differences in ways ofthinking and values, and it was possible to foster
communication ability to overcome differences andcooperate together to accomplish a project.

to stimulate utterances from learners' avatars that havebeen unable to contribute to group chat.
Next, the content of the group debate within the
metaverse classroom can be confirmed, right down to
the quality of conversation, by carrying out abibliometric analysis of the chat between students inreal time (Nakahira et. al., [8]).
Furthermore, in a PBL among people who are
multilingual, i.e., possess diverse values, remarks about
a given problem can also be expected to differ fromthose in a discussion involving only people of identicalbackgrounds. As stated by Kido[9], to carry out
intercultural collaboration, besides language issues it isnecessary to understand the differences and similarities
due to differences in ways of thinking and values.
Conversely, through learners’ experiencing differencesin values and ways of thinking, it is possible to obtainan extraordinary chance, even if it is just in the
metaverse. It is conceivable that intellectual stimulation
coming from the extraordinary will be a trigger to open
new perspectives not only for the learners but also for
the instructors.

REFERENCES[1] Kimihiko Shirato, Hitoshi Sasaki, and Makoto
Takeya,(1998), The Distance Education System using
Virtual Reality Technology(in Japanese), the Technical
Report of IEICE, ET 98(144):73-78.
[2] Masayuki Kawamoto et. al.(1995), Application of
3D Computer-Graphics to Dental Education: Part. 1
Development of the Trial Teaching Aids for Bevel
Preparation of Metal Inlay Cavity(in Japanese), Journalof Japanese Association for Dental Education,
11(1):101-107.
[3] Satoru Fujii, Toru Watanabe, Sanshiro Sakai, andTadanori Mizuno(2003), A System for Learning
Ecology having local materials database constructedwith XML(in Japanese), Technical Report of IPSJ,
2003(52):25-30.
[4] http://jp.secondlife.com/[5] http://langrid.nict.go.jp/jp/[6] http://experience.co.jp/gallery/index.html[7] T. Yoshino and K. Ikenobu(2008), Multilingual
Chat Communication Experiment on 3D Online VirtualSpace(in Japanese), The Special Interest Group Notes ofInformation Processing Society of Japan 2008(91):125130[8] Katsuko T. Nakahira, Rodrigo N. R., Ryosuke
Taguchi, Hideyuki Kanematsu, and Yoshimi
Fukumura,(2010), Design of a Multilinguistic Problem
Based Learning learning Environment in the Metaverse,
the Proceedings of ISAC2010, in press.
[9] T. Kido(1999), Information Sharing for Diversity
Collaboration(in Japanese), The Special Interest Group
Notes of Information Processing Society of Japan,
47:25–30.

However, from the educational point of view, there
are limits to the intercultural collaboration possible withsimple conversations alone. In particular, thinking about
the elements of intercultural collaboration incorporated
into the training of engineers, what is required is not
only an understanding of the other party’s culture, butalso an understanding of the differences between one’s
own and the other party’s way of thinking and values,
and to develop the communicative ability to overcome
differences and cooperate to accomplish a project. In
that sense, in setting the scene, just setting up simple
conversations is not sufficient. As with PBL, it isimportant that students be given some kind of problem
by which they can gain experience in the process ofworking together to solve problems.

IV. CONCLUSION
For this paper, a multilingual PBL learning
environment was set up and implemented in the
metaverse, and an environment was provided to makerecords of various learning activities. Chiefly, 1) aclassroom was constructed to acquire records oflearning activities in the metaverse, and 2)
improvementsinmultilingual
communication©ISAROB 2011

153

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

An effective visualization of style inconsistencies for interactive text editing
Kazuki Shimamura

Kazuhiro Takeuchi

Kiyota Hashimoto

Department of Engineering Informatics, Department of Engineering Informatics,
School of Humanities
Osaka Electro-Communication University Osaka Electro-Communication University and Social Sciences,
18-8, Hatsu-cho, Neyagawa,
18-8, Hatsu-cho, Neyagawa,
Osaka Prefecture University572-8530, Japan
572-8530, Japan
1-1, Gakuen-cho, Naka-ku, Sakai,
Email: shimason.1988@takelab.jp
Email: kazuh@takelab.jp599-8531, Japan
Email: hash@lc.osakafu-u.ac.jp
Abstract : Any authors, particularly learners, have much difficulty choosing the proper style for their particular writing anddetecting style inconsistencies. We propose a new system that allows users to revise documents through human-systeminteraction. Although many systems for text writing have been proposed, most of the works focused mainly on automatedtechniques that detect human errors in texts. In contrast to those works, our study focuses on the visualization of multi-levelstyle inconsistencies in texts to promote authors' awareness. In order to evaluate and visualize the differences in styles, we
propose a model to compute the style similarity between a part and some genres. The similarity function that we propose isbased on a model in which sentences are regarded as sequences of functional expressions. Applying the function, we develop atryout system that shows which parts are inconsistent with the other parts from various viewpoints. Through interactions
between users and the system, the user can repeat revising the text until the text maintains consistencies in various levels.
Much has to be done towards a practically effective system, but our system helps to point out undesirable text should beconscious of stylistic differences in writing text.
Keywords : writing support, style consistency, corpora-based approach, awareness promoting visualization

I. INTRODUCTION

II. Environment for Text Editing

We can easily distinguish a newspaper article and anewspaper editorial, or a textbook for graduate students
and one for undergraduate or high school students, notjust in terms of their contents but in terms of their style,though few of us can always make clear our criteria forthis kind of distinction. How to narrate, or style, ishighly related to the targeted audience and the author’s
communicative purpose of the text, and stylistic
consistency is required in a text, though deliberateinconsistencies bring extra literary effects. [1]Several studies have tackled with style; particularly it
has been pointed out that basic stylistic consistency isheld by the restrictive use of functional expressions
particularly in the case of Japanese and other languages
that have a rich variety of stylistic grammar forms. For
example, the following pair has the same meaning ('This
is a book' in English) but differs in the style:

Stylistic consistency does not only depend on the innerconsistency in the text, but also on the appropriate
choice of style for the textual purpose. In other words,
the targeted audience and the author’s communicative
purpose determine the desirable style; then the author,
with his/her limited reading experiences, attempts tokeep stylistic consistency: i.e., he or she tries to use as
many appropriate stylistic features as possible and tries
not to use inappropriate stylistic features. As afundamental dimension of the features, it is necessary
for him/her to be aware of the following two
viewpoints.

•
•

•
•

For dealing with the first consistency in computer
systems, many previous works have been developed.
And now some of the modules that detect errors (andinconsistencies) of sentences are provided through
WEB-API services in the Internet. The followings are
an example of the list which we can use through of thesuch WEB-APIs. These kinds of the features, which
detects sentence-level consistencies such as spelling
errors and simple syntactic errors, have been
implemented in some word processors such as
Microsoft Word.

Kore-wa hon-desu.
Kore-wa hon-da.

Both 'desu' and 'da' are auxiliary copular verbs and thedifference is up to politeness, which in turn should bedetermined according to the targeted audience and theauthor’s communicative purpose. So the mixed use of'desu' and 'wa' causes undesirable stylistic inconsistency
and thus should be avoided. Particularly for languages
like Japanese, which has a rich set of style-sensitive
functional expressions, the mere detection of themisuses of such functional expressions is useful to somedegree for style consistency.

©ISAROB 2011

intra-sentential consistency
passage-level consistencya) Spelling errorsb) Inappropriate use of Synonymsc) Inappropriate use of Kanji-charactersd) Inconsistency of Proper Nouns

154

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011e) Inconsistency of Okuriganaf) Use of Double Negationsg) Use of Redundant Expressionsh) Inappropriate use of Abbreviations

atomic expressions frequently used in Wikipedia but notfrequently used in 2ch, (b) atomic expression frequently
used in both data, and (c) atomic expressions notfrequently used in Wikipedia but frequently used in 2ch.
This classification means that the class (b) consists ofrather neutral atomic expressions, whose use may wellnot characterize a text as either of the two, while theclass (a) and (c) are to be considered to be stylisticfeatures for Wikipedia and 2ch, respectively.
With these classes representing the resemblance of agiven text T with the model set of texts of the targeted
genre, we have been able to visualize the textual
characteristics on this aspect[3] . Suppose that thefunction freq(a, X) is the frequency of occurrence of
atomic expression a in the corpus X. For computing thestylistic similarity o f a to the style of Wikipedia, we
compare freq(a, Wikipedia) and freq(a, Y), in which Y
is another corpus(for example, corpus of BBS articles).
This stylistic similarity of atomic expressions whose
the size is three or four characters shown in Fig.1, is too
short to represent the passage-level consistency. For the
problem, we have to extend the function freq (a, X) to afunction freq(w, X) for computing the appearances ofdesirable expressions for the targeted subgenre X, where
w is a function that returns the set of expressions
included from certain span in the text. With thisfunction, we compute the style similarity of any span in
the text to a certain subgenre.

On the other hand, some discussions still remain
concerning visualization of the inconsistency from thesecond viewpoint (passage-level consistency). With thisin mind, we have been investigated on the visualizationof passage-level inconsistency based on a set of textual
corpora that consists of two or more subgenres andextract stylistic features of each subgenre. These setsroughly correspond to our reading experiences but thelarger size is naturally expected to contribute to a better
detection of stylistic features.
For the first approximation, let us consider that stylisticfeatures of a subgenre are based on the use of preferable
expressions for the subgenre and the lack of undesirable
expressions for it. The definition of ’expression’ mayvary, but it should be noted here that a large number ofwords, including misspelled ones.
A scatter diagram between these two subgenres isshown in Fig.1, where samples of atomic expressions
are distributed on the corresponding two axis. The
determination of the proper (range of) n is a heuristic
issue, and we first adopt the range of n as two to four,
mainly because most of the Japanese words consist of
one or two characters. To conduct the first experiment,
we used a dumped file of Japanese Wikipedia[2] and asample of 2ch BBS (http://www.2ch.net/) as sample
contrastive data set, the sizes of which are shown in
Table 1.

III. Inconsistency Analysis Based on
Templates
As we have summarized in Section II, there areexpressions which are successfully used in identifying
the genre of a text. Those n-grams are often parts ofcontent words and perhaps they would be better to findfrequently occurring words. Content words are definedas the words which are not functional expressions. In
contrast with them, functional expressions are those thathave little lexical meaning or have ambiguous meaning.
For example, dictionaries always define the specificmeanings of content words, but only describe thegeneral usages of function words. Instead of thecomplexities to describe the meaning of function words,
these words serve to express grammatical structure of asentence or clause and specify the attitude or mood of
the writer. In order to extend a simple consistency
analysis based on content words to adopt more various
degree of consistency, we adopt a method based on
occurrence of function words.
One of the problems when we examine the occurrenceof functional expressions is that the use of them strongly
depends on the own style of the writers. If we adopt a
corpus in which sentences are corrected from a freeInternet forum to which a lot of writers can freely post
their articles, the use of functional expressions must
quite vary. In other words, it is difficult to collect a large
corpus in which the use of functional expressions is
expected to be consistent.
In order to overcome the problem of the shortage, we

Fig.1. Scatter Diagram of 2- to 4-gram expression

Data
Wikipedia
2ch

Table1 Data SET
Number of characters161,223,892
108,031,243

The frequency of each atomic expression in Wikipediaand 2ch classifies them roughly into three classes: (a)©ISAROB 2011

155

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

prepare templates that are sequence patterns of thefunctional expressions. We extract these templates fromstudents’ essays that we assigned them to write theessays referring to some Wikipedia articles. The purposeof this setting is that the students will use their familiar
functional expressions even if they refer to the articles(Wikipedia articles) in which the consistency of the useof functional expressions is maintained.
The process to extract templates is conducted based oncomparison between the sentences which the student
w r i t e s and their corresponding sentences in the
Wikipedia articles. The measures for the comparison iscomputed based on how many content words arecommon between the sentences. For example, each pairof sentences in (s1) and (s2) is compared because of
more than half content words are commonly used
between them.

the text.
{s1,s2,...,sn}.each do |w|
y = freq(x, Wikipeida) - freq(w, 2ch)
if y > 0 thenprint “+”else if y < 0 thenprint “-”else
print “*”endif
endFig.2. Algorithm for Evaluating Passage-Level
Consistency

IV. Proposal for Interactive Editing(s1)(Wikipedia)テキストをデータ構造に変換する。
(Student)テキストにデータ構造は変換されます。
(Template)○にSystems for interactive text editing are designed toallow one to use pre- and post-processing of subparts ofthe text when he/she is imputing or editing it. A system
we propose here has the following features:○は変換されます。

(s2)(Wikipedia)北米は家庭に約2台のパソコンがある。
(Student)北米は家庭のパソコンがあります。

(Template) ○は ○の○があります。

••

As a result of comparing process between them, thewords which are anything other than the functional
expressions in (s1) and (s2) are replaced with meta word'○', and we obtain a ordered sequence of functional
expressions like {'に','は変換されます'} and {'は','の','があ り ま す'}. In this paper, we refer to the sequences as“templates.” Because each template mainly shows akind of features of the syntactic structure of thesentence, the occurrences of infrequent content words inthe sentence are ignored with the computations which
adopt the templates.
By using the collection of these templates, we extend
the function appropriately, which we have mentioned in
Section II. For the purpose of evaluating the passagelevel consistency, the first argument w in the function
freq(w,X) to be extended. In our proposal, our algorithm
to evaluate the consistencies from the viewpoint of thepassage-level processes sentence by sentence. Thisevaluation is conducted by the algorithm shown in
Fig.2.
Regarding a target text as the set of sentences like {s1,s2, ..., sn}, each sentence is evaluated whether it holds
the expected style by applying the function freq (w, X),
where X is the corpus in which texts are assumed to be
written in the expected style. Note that the algorithm
needs one or more corpora to compare the stylesimilarities of the target sentences to the various styles.
As outputs of the algorithm, either of three characters{'+', '*', '-'} is to output for each unit by the printcommand. This output does not need to be displayed
directly, it means that we have to devise an effective
way to promote users' awareness of the inconsistency in©ISAROB 2011detecting the errors in the inputted phrases andsuggest the exact parts to correct
visualizing the multi-level consistencies (fromintra-sentential and passage-level viewpoints)

W e designed our system to be run on the server
computer and to be used through web browsers. In theinternet, some of the useful modules which analyze
some errors of intra-sentential inconsistencies areavailable for such kind of text processing services which
are used from Web browsers. The algorithm that we
described in Section III is implemented as a module likethem and our system is developed combining thesevarious modules.
Interactive input systems are familiar with Japanesepeople because these systems have been developed tohelp to input Japanese characters. Intelligent interactive
text editing is also investigated a s applications of themachine translation. For example, some works focus oninteractive text pre-editing to improve the output qualityof machine translation[3][4], in which the user can inputo r edit the source-language sentence interactively
referring to the obtained sentence by translating it. Like
the works, in which the users edit his/her writing text
according to the messages from the system, our systemprovides an interface that allows t h e user to check
his/her writing text from two viewpoints (passage-levelconsistency and intra-sentence consistency).
Fig.3 shows the screenshot of our interface, in which
there are two big input boxes on the upper side of thepage and under these boxes there are two areas forinteractive editing to the inputted text.
An original text is inputted in the left hand side of the
big input boxes and then the user pushes the checkbutton to edit the inputted text. The box on the righthand side displays the log for the editing history.

156

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

On the other hand, the user can concentrates oncorrecting the intra-sentential errors in the edit area B.
In the edit area, some parts of the sentences are replacedt o t h e input boxes, to which the system promote theusers’ awareness of errors and mistakes that are detectedfrom intra-sentential viewpoint to correct them. For
each box, the reason why the corresponding part should
be correct is also informed.
In this way, the user can be aware of the changes of theconsistencies in the passage-level as soon as he/sherevises the errors in area B. On the other hand, he/she
rewrites a whole sentence in the edit area A to hold thepassage-level consistency of the context, the systeminteractively evaluate the intra-sentential consistenciesof the corresponding parts.

V. CONCLUSION

Fig.3. Screenshot of Our System

In this paper, we proposed a system for interactive text
editing that allows one to check the multi-levelconsistencies in his/her writing text. In the system, weused the sequences of functional expressions to computeone of the consistencies from the passage-levelviewpoint. In order to promote awareness on the multilevel consistencies of the writing text, we combined itwith some algorithms that previous works proposed.
Much has to be done towards a practically effective
tool, but any advanced tool to help to detect undesirable
expression should be conscious of stylistic differencesamong subgenres and our proposed methods arefundamentally effective.

The interactive edit area is divided into the two areas asshown in Fig. 4 and Fig. 5.

ACKNOWLEDGMENT
This work was partly supported by KAKENHI(21500113).
Fig.4.Area A

REFERENCES[1] Schiffrin, D., D. Tannen, and H.E. Hamilton (2001)
The Handbook of Discourse Analysis, Blackwell[2] Japanese Wikipedia dumped file.
http://download.wikimedia.org/jawiki/
20071013/jawiki-20071013-pages-articles.xml.bz2[3] K. Hashimoto, K. Takeuchi, H. Ando (2010) A
Corpora-based detection of stylistic inconsistencies oftext in the targeted subgenre, AROB15, pp.110-113[4] K. Uchimoto, N. Hayashida, T. Ishida and H. Isahara(2005) Automatic Rating of Machine Translatability, In
Proceedings of 10th Machine Translation Summit, pp.
235-242[5] N. Hayashida, T. Ishida (2005) Performance
Prediction of Supporting Self-Initiated Repair byTranslation Agents (in Japanese)
The transactions of the IEICE. D-I J88-D-I(9),pp.14591466

Fig.5.Area B
We designed that each sentence is displayed with theinformation of its style i n the edit area A. Under eachsentence box, the template which the sentence contains
is colored with the color which represents the stylesimilarity for user to check the consistency from thepassage-level viewpoint. The computation of the stylesimilarity follows the algorithm described in Section III.

©ISAROB 2011157

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

An Intelligent Meta-Learning Support System Through Presentation
Kazuhisa Seta
Graduate School of Science, Osaka Prefecture University
1-1,Gakuen-cho, Naka-ku, Sakai, Osaka, 599-8531, Japan(Tel : 81-72-254-9691; Fax : 81-72-254-9691)(seta@mi.s.osakafu-u.ac.jp)

Abstract: As described in this paper, we propose a presentation-based meta-learning scheme. Firstly, we present support
functions that we embed into the system. Secondly, we conduct experiments to verify the meaningfulness of ourlearning scheme, which suggests the system can stimulate learners’ reflection on their learning processes. Furthermore,it can stimulate learners’ meta-learning communications. Results show that users tightened their criteria to evaluate
their own learning processes and understanding states. It is useful for learners to facilitate change in their learning
processes.
Keywords: meta-learning, meta-learning communication, presentation-based learning

I. INTRODUCTION

II. Embedding Support Functions to
Facilitate Meta-learning Communication

Research into computer-supported systems toenhance meta-cognitive skill is investigated by many
researchers [2, 3, 4, 5]. Results show, particularly in the
educational psychology field, that an emphasis on metacognition must accompany domain-specific instructionin each of the disciplines, but not generic instruction ina general context, because the type of monitoring that isrequired will vary [1]. In a history course, for example,
a student might be asking herself in an internal selfconversation, “who wrote this document, and how doesthat knowledge affect the interpretation of events,”
whereas in a physics course, the student might bemonitoring her understanding of the underlying physicalprinciple at work [1].
Our research goal is the enhancement of metalearning through stimulation of learners’ reflections on
their own learning processes. To achieve this goal, we
assign a task to make a presentation material on a
specific pre-learned topic for other learners whose
academic abilities are similar to those of the learner.
Collaborative learners with no regulation, however,might stray in undesired directions: in the case of ourpresentation-based learning, for instance, they tend to
discuss illustrations of the slides, the impact of thepresentation, and so on.
We therefore propose a support system thatfacilitates meta-learning communication by providing
learners with viewpoints to discuss their learning
methods.

©ISAROB 2011

In our research, we developed a presentation-based
meta-learning scheme whereby learners can specificallyexamine learning on their own learning processes.
Learners in our learning scheme perform learning byfollowing three steps.
i. Learning specific domain contents through selfstudy or attending lectures until they think they
have understood themii. Making comprehensive presentation materials toteach other learners who have the same academic
leveliii. Collaborative learning using presentation materials
In the following, we explain support functions
embedded into the system at (ii) and (iii) phases tofacilitate meta-learning, although phase (i) is beyond
our support.
1. Intention Structure Reflecting Learning Contexts
To encourage meaningful meta-communication
among learning partners, each learner must (A) becomeaware of performing meta-learning and (B) shareindividual learning contexts. In our learning system,
providing a representation to describe their intention of
the presentation, intention structures and guidance
function according to them play roles of enhancing theirawareness at the presentation design phase.
At the presentation design phase, we make learners
construct intention structures to be aware of learning
skill acquisition. Giving appropriate instructions

158

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

learning goal “make the learners understand DP
using Abstract Factory pattern as an example?”
Choose “embed into presentation” by right-mouseclicking if you think you need to do so.
(2) “Do you have sufficient understanding of theseteaching activities? Check the items you had
already understood.”□ Make the learner understand the meaningfulnessof the fact that each DP has its own name.
□ Make the learner understand the advantages ofobject-oriented programming by combining its
general theories with concrete examples in the
Design Patterns.
□ … (Required learning activities defined inlearning skill ontology are listed)
The learner is required to examine the importance ofeach learning activity for constructing comprehensive
presentation materials: the learner judges whether thelearner’s presentation is valid or not and whether eachlearning activity should be included in the learner’s
presentation. This guidance is a stimulation to facilitate
the learner’s reflection on personal learning processes.
The fact that the learner did not embed listed
learning activities is interpreted as follows: (a) the
learner has no learning activities as domain-specificlearning operators in his own consciousness, (therefore
the learner cannot perform them) or (b) the learner does
not understand the importance of the learning activities
even if they have and they had performed their learning
processes. The learner’s checking activity in query (2) isinterpreted as a declaration of whether the learner has
them as learning operators.

according to learners’ learning contexts is significant tofacilitate their learning skill acquisition processes.
In our task setting of making truly comprehensive
presentation materials for use by those who have thesame academic level with the presenter, we adopt anassumption that intention structures of presentationreflect learners’ learning contexts in their learning.
In the intention structure (Fig. 1. (iii)), each node
represents an educational goal and terms to represent
them are provided from the system to represent thelearners’ educational goals.
2. Guidance Function to Enhance Meta-Cognitive
Awareness
Guidance information to facilitate the learner’s
reflection on personal learning processes is provided
when the learner intends to move to the subsequentcollaborative learning phase. It represents queries on
domain-specific learning activity based on the learner’s
intention structure. The teacher giving a presentation
subjects also constructs an intention structures andindicates required learning (teaching) activities on themthat should be embedded into learners’ intention
structures. The system cannot understand the contents oflearners’ presentation written in natural language.
However, it can process intention structures by referring
learning skill ontology. Therefore, if learners did notembed them, then the system provides queries byreferring domain-specific learning skill ontology andthe teacher’s intention structure as follows:(1) “Do the following learning activities need to be
included in your presentation to achieve the(i) Presentationmaterials(ii) Interaction
History(iii) Intention Structure(Intention of the Presentation)

Window for the personalpresentation
Window for the learning
partners’ presentation

embedded(iv) Video Chat(v) Text Chat(vi) Viewpoint Function

Each learner uses
windows of two kinds

Figure 1. CSCL Environment to Facilitate Meta-Learning Communication.

©ISAROB 2011

159

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

For (a), the learner must perform the learning
activities spontaneously or must be taught from thelearning partners at the collaborative learning phase. For(b), the learner must encourage internal selfconversation to consider the importance of eachlearning activity.
The guidance function plays a role of building afoundation to encourage meta-learning communications
among learning partners by stimulating their awarenessin meta-learning before starting collaborative learning.

example.” as important. It is an important learning
activity in the software development domain and you
embedded it into your presentation. On the other hand,
your learning partner judged it as not important. Explain
why you think this learning activity is important.”
Collaborative learners can discuss their domainspecific teaching methods by referring to the viewpoints
for meta-learning communication.

3. Viewpoint Function to Stimulate Meaningful
Learning Communications
Figure 1 portrays a screen image at the collaborative
learning phase. The window comprises six panes: thepresentation pane (Fig. 1 (i)), interaction history pane
(Fig. 1 (ii)), intention structure pane (Fig. 1 (iii)), video
chatting pane (Fig. 1 (iv)), text chatting pane (Fig. 1(v))
and discussion viewpoint pane (Fig. 1 (vi)). The system
is implemented in Visual Basic and Java, functioning
cooperatively with Power Point (Microsoft Corp.).
The system in the collaborative learning phaseprovides support of two kinds to facilitate learners’
learning skill acquisitions (acquiring learning operatorsand tightening evaluation criteria) as follows.
(1) Support to share learning (teaching) contexts oflearning partners by referring to presentationmaterials with intention structures.
(2) Facilitate meaningful discussions to encourage
their reflections on their own learning processes by
providing discussion viewpoints.
As described in this paper, we particularly examine
the topics on the viewpoint function.
Thinking processes related to one’s own learning
processes are quite tacit. Therefore it is not easy toexternalize and to discuss learners’ thinking processes(while teaching processes reflecting their learning
processes are externalized as intention structure).
Ordinary learners with no support tend to discuss theappearance of illustrations, animations, and so on.
To eliminate the problem, our system provides
viewpoints to discuss their teaching and learning
methods based on the interaction history between thelearner and the system at the presentation design phase.
As shown in Fig. 1 (vi), the system provides eachlearner with respective viewpoints to discuss as follows:“You judged the learning activity “Make the learnerunderstand the significance of the fact that an interface
specifies the name of each method by taking an

We conducted an experiment to verify themeaningfulness of our learning scheme and usefulnessof support functions embedded into the system. We
specifically examine the issues of whether the system
can encourage meta-learning communications. The
outline of the experimentation is described below.
9 Subjects: 16 graduate students participated. They
had completed software engineering (UML) andobject-oriented (Java) programming courses when
they were undergraduate students. They weredivided into two groups at random: eight students
were in the experimental group (ExpG) using thesystem; eight were in the control group (CtlG).
9 Presentation topic: Make presentation materials
explaining the merits of building design patterns
by taking the abstract factory pattern as an
example.
9 Flow of the experiments: Continuous 7 days
lecture (90 min lecture each day) without
weekend:
Table 1 presents results of questionnaires after theircollaborative learning. Questionnaire items 1 is related
to the usefulness of the presentation-based learning
scheme and 2–3 are related to learning effects from theviewpoint of meta-learning.
Regarding item 1, participants in both ExpG and
CtlG gave quite high marks, which suggests thepresentation based meta-learning stimulates learners’
reflection on their learning processes.
It is expected that learners will execute better
learning processes using the acquired domain-specificlearning activities and tightened evaluation criteria if thelearners’ meta-learning processes are performed
successfully. Items 2–6 inquired the about learners’
consciousness of them. Both groups gave high marks toeach item. However, CtlG gave higher marks than ExpG
for the acquisition of domain-specific learning activities(items 3 and 5), whereas ExpG gave higher marks than

©ISAROB 2011

III. Experimentation

160

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 1. Results of Questionnaire after the Collaborative Learning Phase
ExpG
Mean

Questionnaire Items
1
2
3
4
5
6

Do you think the collaborative learning after making your presentatio
n materials enhanced your reflection on your own learning processes?
Do you think collaborative learning changed your criteria to evaluate
your understanding of DP?
Do you think you could acquire learning methods using collaborative
learning?
Do you think your learning processes for other DPs will change after
performing this presentation-based learning?
Do you think you could acquire learning methods by performing thispresentation-based learning?
Do you think your consciousness of learning will change by performi
ng this presentation-based learning?

CtlG
Mean

SD

4.375 0.267 4.375 1.982
2.875 1.553 3.375 1.982
3.375 0.839 3.625 1.41
3.75

1.071 3.5

1.428

3.625 0.553 4.125 0.982
4.1

0.238 3.875 0.982

learning processes and understanding states. It ismeaningful for the learner to change their learning
processes. We also evaluated their learning outcomes ofdomain dependent knowledge: it suggests participantsin ExpG could get higher mark than ones in CtlG. We
will carefully address the issues of this in future works.

CtlG for items related to the consciousness of changesof their own future learning processes (items 4 and 6).
Those responses seem to be mutually contradictory.
However, they are not so by the followinginterpretation: learners in ExpG had tightened theirlearning criteria to evaluate their learning processes andunderstanding states; thereby, they also strictly
evaluated their meta-learning processes. The results of
the average time ratio of meta-learning communication
support this. However, the fact that participants in ExpG
gave low marks related to item 2 suggests that they
were unable to perform all meta-learning processes bythemselves even though they were able to understandthe importance of meta-learning. They might beconscious of the functions. Actually, we do not embed
the functions that support performance of learning
activities acquired by meta-learning processes evenwhen the system triggers learning activities. On theother hand, participants in CtlG spent less time formeta-learning communications, suggesting that thelearners’ evaluation criteria had not been tightened
through their communications. Consequently, theirevaluation results for these items were more tolerant.

REFERENCES[1] Bransford, J., Brown, A., & Cocking, R. (2000).
Brain, Mind, Experience, and School, In How
People Learn. Washington, DC: National Academy
Press.
[2] Flavell, J. H. (1976). Metacognitive aspects ofproblem solving, In L. Resnick (Ed): The Nature ofIntelligence. (pp. 231-235) Hillsdale, NJ:
Lawrence Erlbaum Associates.
[3] Kayashima, M., Inaba, A., & Mizoguchi, R. (2005).
What Do You Mean by to Help Learning of
Metacognition?, Proc. 12th Artificial Intelligencein Education (AIED2005), pp. 346-353,
Amsterdam, The Netherlands.
[4] Maeno, H., Seta, K., & Ikeda, M. (2010).
Development of Meta-Learning Support System
based on Model based Approach. Proc. 10th
IASTED International Conference on Artificial
Intelligence and Applications (AIA2010), (pp.442449).
[5] Noguchi, D., Seta, K., Fujiwara, M. and Ikeda,
M. (2010): Presentation Based Learning Support
Systemto
Facilitate
Meta-Learning
Communications, Proc. of 18th International
Conference on Computers in Education (ICCE),(to appear),IV. CONCLUDING REMARKSAs described in this paper, we present discussion ofa presentation-based meta-learning scheme. We
introduced our presentation based meta-learning scheme.
Then, we conducted an experiment to verify themeaningfulness of our learning scheme and usefulnessof support functions embedded into the system, whichsuggests that the system was able to stimulate learners’
reflection on their learning processes. It stimulateslearners’ meta-learning communications. Consequently,
they tightened their criteria to evaluate their own©ISAROB 2011

SD

161

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A Task Ontology Construction for Presentation Skills
Kiyota Hashimoto* and Kazuhiro Takeuchi**Osaka Prefecture Univ*.: 1-1, Gakuen-cho, Naka-ku, Sakai, 599-8531, Japan
Osaka Electro-Communication Univ**., 18-8, Hatsu-machi, Neyagawa, 572-8530, Japan(Tel : 81-72-252-1161; Fax : 81-72-254-9944)(hash@lc.osakafu-u.ac.jp)
Abstract: Presentation is an integrated art of communication in which both linguistic and paralinguistic skills are
employed, and deliberate preparation is necessary. We’re developing a comprehensive learner support system for tohelp learn presentation skills. This paper reports our development of task ontology for presentation skills as one of thefoundational components of our system. It consists of three sub-ontologies: presentation strategy ontology, rhetoricalstructure ontology, and lexical ontology. They are used for background inferences and evaluations of learners’
presentations. With a prototypical application of them shows the employment of task ontologies is effective forintelligent learning support systems.
Keywords: Task ontology, Presentation strategy, Rhetorical Structure Theory, rhetorical structure ontology, learner’s
presentation

I. INTRODUCTION
Presentation preparation- Decisions on style factors- Oral and slide contents and style- Preparation for discussion- Presentation exercises

It is more and more important for any one of us tomake a good presentation, particularly that in English,in various situations. Oral presentation is one of themostsophisticatedcommunicative
activities;
deliberately designed to be presented to specificaudience with slides to deliver information and attempt
to make a persuasion. Not only the linguisticorganization but also the paralinguistic effects like bodylanguage and eye contact are utilized, and the presenter
should have a deep understanding not only of what heor she delivers but of how the audience will react. Mostof the people who have to make presentations havesome difficulties preparing and making them, andcomputational learning support is desirable. From thisviewpoint, we are developing a comprehensive online
learning support system for presentation [1], and are
constructing a multimedia learner corpus of learners’
presentations [2]. The former contains Presentation
Organizer which can be used to construct oral
manuscripts and corresponding slides simultaneously,
and the latter contains orally spoken texts and slides tobe annotated. Both require a fine-grained way ofdescribing and representing the text organization, or
rhetorical structure of presentation.Thus, we havedesigned and constructed a couple of ontologies related
to presentations. In this paper, we report two of them,Presentation Strategy Ontology and Rhetorical Structure
Ontology.

©ISAROB 2011

Presentation Delivery- Presentation itself- Onsite adjustments

Fig.1 Presentation process

II. PRESENTATION STRATEGY ONTOLOGY
The process of preparing and delivering apresentation is roughly done in a way as shown in Fig. 1.
As this simple process tells, the success or failure of apresentation is mostly decided by the preparation phase.
Thus, any talk ontology for presentation has to focus on
the preparation phase.
Presentation, whether it is oral or written, isdelivered for a particular communicative purpose,
which is to determine what and how to be delivered—content and style. A communicative purpose can beanalyzed into different aspects. Let us call these aspects
style factors. As each style factor affects what and how
to be delivered, they have to be deliberately considered.
Numerous books on presentation preparation emphasize
the consideration of style factors, though each calls stylefactors differently, and we classify them as in Table 1.

162

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Factors
Topical(Genre)Media

Targetedreader/audience
PurposeIntention
Time

Examplesinorganic chemistry
English literature...
(Note that the topical factor can be verymuch specific.)
newspaper
textbook
oral presentation...
laypeople
professionals...
persuasion
explanation...
formalinformal or friendly...
the length of presentation
Table 1 Style Factors

Presentation

Style Factors

Property
is-a relation
Fig.3 Brief Overview of Presentation Strategy Ontology

III. RHETORICAL STRUCTURE ONTOLOGY
1. Rhetorical Structure Theory and Ontology
Rhetorical structure has long been investigated inliterature and linguistics, and computational linguists
have pursued the possibility of applying it to natural
language generation and automatic summarization inparticular [3]. A number of theories and frameworks
have been proposed [4], of which Rhetorical Structure
Theory, originally proposed by Mann and Thompson[5],[6],[7], is the most popular.
In the RST framework, the discourse structure of atext is considered to be coherent and thus capable ofbeing be basically represented as a single-rooted treewith a set of directed graphs. Each directed graph islabeled for its rhetorical relation. Originally 23 relations
were proposed [8], but the exact number proposed
varies among researchers.
Various types of domain ontology have been
investigated in order to make structured knowledge
machine-readable, and instances of rhetorical structure
ontology have also been proposed [8]-[11]. In particular,
Rahal et al.[11] share a similar perspective with us inthat their system is intended for supporting collaborative
writing. None of them, however, can deal withpresentations successfully.

Consistency
Lexical Choices
Functionwords

Grammatical Choices
Average sentence lengths

2. Reconsideration of RST
A. EDUAs shown in the previous section, RST is basicallyclause-based: its EDU is assumed to be clausal. A
number of researchers have repeatedly reinvestigatedthe validity of this approach both from theoretical andCoherence
Logical Organization

Fig.2 Linguistic Style Elements©ISAROB 2011

Slides

TopicalWith the decisions on style factors, oral and slide
contents are to made not only on what to be delivered
but also how to deliver it. Preparation for discussion is
also affected. In other words, all the other aspects ofpresentation preparation and delivery are decided along
the line of the choices of the appropriate style factors
for the presentation. Particularly, all the linguistic styleelements (See Fig.2) which manifest the choices areheavily affected. Note that the logical organization in
Fig.2 is more concerned with the other ontology,
Rhetorical Structure Ontology, which we will discuss inthe next section.
With these consideration in mind, we have devised a
prototype of Presentation Strategy Ontology as in Fig. 3

Contentwords

Speech163

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011at the graph.” This type of instruction is to beconsidered supra-textual in that it points out someelement outside the orally spoken content, but it still has
its nucleus adjacent to it, since such instructions arenaturally give in elaborating an EDU. So supra-textual
cues are to be incorporated into rhetorical relations.
D. Revised RST for representing the rhetorical structure of presentation
Based on the discussions in the previous sections,
we defined our revised RST as shown in Table 2.

practical perspectives, but nearly all of them stillassume clause as EDU [3],[4],[8]. However, Kibble
pointed out, in investigating a small corpus ofpharmaceutical leaflets, that at least a nominalized nounphrase, particularly a gerundive one, plays the same role
as a clause[12], and a given relation between two
conceptual units can be embodied in several different
ways. So we define the EDU as a “semantically” clausal
segment. “Semantically” clausal segment means asegment in which a subject-predicate relation iscontained, whether explicitly or implicitly.
B. Surface or Deep?: Potential Textual Inconsistencies and Imperfections
Rhetorical structure deals with the detailedorganization of the content to be written or told. In thatsense, a given rhetorical structure is assumed torepresent the deep structure of the content, on the naïve
assumption that the deep structure is properly mapped
onto the surface structure, the text. This cannot beassumed, however, in the case of learner’s presentations.
Thus, in order to reveal textual inconsistencies andimperfections, it is important to annotate a rhetoricalstructure mainly based on surface logical cues.
C. Supra-textual Cues
One of the biggest differences between written texts
and oral presentations is the existence of interactions
between the text itself and the outer world. A presenter
frequently gives instructions to the audience to draw
their attention to some part of the slide, such as “Look3 Rhetorical Structure Ontology for Presentation
Our revised RST discussed above has to beformalized for machine-readability, and one of theplausible implementations is to make a rhetorical
ontology. We are now implementing a prototypicalrhetorical ontology for presentations, PRESONTO,
using OWL 2.0. A key feature is to annotate rhetoricalstructures of the oral part and the slide part of apresentation in the same way. The underlying
assumption is that a span, maximally of a slide,necessarily corresponds to a span of oral speech, andthis correspondence guarantees the correspondence
between oral speech and slides. Furthermore, it ispossible, with this correspondence, even non-verbal
elements like graphs and pictures to be properly
annotated in the same way. PRESONTO is still
prototypical, but its overview is shown in Fig. 4.

EDU: Semantically clausal segments
A rhetorical relation is primarily based on the surface structure and is assigned 0 or 1 according to the logical validity.
The Set of Rhetorical Relations for our purpose
Name
Description
Orderof( * in the Name indicates that the item is newly introduced here)
N&S
BackgroundS before N
Satellite provides background information to the nucleus
Contrast
Applies to two nuclei that contrast each otherElaboration
N before S
Satellite elaborates the information in the nucleus*Elaborative
N before S
Satellite exemplifies the information in the nucleus
ExampleEnablement
N before S
Information in the satellite enables the audience to perform the action in the
N before S
nucleus
Satellite provides evidence to the statement in the nucleus
Evidence
Satellite justifies the nucleus
Justify
Satellite motivates the reader to perform the action in the nucleus
Motivation
Listed nuclei
List
S before N
Multiple nuclei that follow each other in sequence
Sequence
Satellite is the problem; Nucleus provides the solution.
Solutionhood
N before S
Short summary or paraphrase of the previous span
Summary*Supra-textual Cue Cue for an action on the part of the audience.
Any other orphaned nucleus, to be connected to the nearest dominating nucleus*Orphaned
Table 2 Our Revised RST©ISAROB 2011164

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Correspond

Presentation
Oral SpeechPresentation
SlidesBackground

hasRoot
Span

hasContents
hasN

EDU
hasContents

Contrast

Rhetorical
Relations
Segment

hasSattelites
hasValidityValue

Property

1 or 0Orphanedis-a relation
Fig.4 Overview of PRESONTO, a Rhetorical Ontology for Presentations

Springer Verlag: Berlin, (available from http://ceurws.org/Vol-523Groza.pdf)[5] Mann, W.C. and S.A. Thompson (1987) Rhetorical
Structure Theory: A Theory of Text Organization,
ISI/RS-87-190.
[6] Mann, W.C. and S.A. Thompson (1988) Rhetorical
Structure Theory: Toward a Functional Theory of
Text Organization, Text 8(3), 241-281.
[7] Taboada, M. and W.C. Mann (2006b) Applicationsof Rhetorical Structure Theory, Discourse Studies
8(4), 567-588.
[8] Nicholas, N. (1994) Problems in the Application of
Rhetorical Structure Theory to Text Generation,
Unpublished M.Eng. Sc. Thesis, University of
Melbourne.
[9] Harris, R. and C. DiMarco (2009) Constructing a
Rhetorical Figuration Ontology, Proc. of Symposium
on Persuasive Technology and Digital Behaviour
Intervention, Convention of the Society for the Studyof Artificial Intelligence and Simulation of
Behaviour (AISB), April 2009, Edinburgh, Scotland.
[10] Pinto, H.S., S. Staab, and C. Tempich (2004)
DILIGENT: Towards a fine-grained methodology
for DIstributed, Loosely-controlled and evolvInG
Enginnering of oNTologies, Proc. of ECAI 2004,
393-398.
[11] Rahhal, C., et al. (2007) OntoReST: A RST-based
Ontology for Maintaining Semantic Consistency inCollaborative Writing, Rapport de recherche n 6384.
[12] Rodger Kibble (1999) Nominalisation andrhetorical structure, Proc. of ESSLLI Formal
Grammar 1999, pp.49-60.

VI. CONCLUSION
In this paper, we reported our ontologies forpresentation description. They are expected to serveboth as a descriptive tool and as a hinting tool to support
learners’ preparation of presentations. We are currently
refining our related modules of learner support systemfor presentation to utilize these ontologies.

ACKNOWLEDGMENT
This study is supported by KAKENHI (22520576).

REFERENCES[1] Hashimoto, K. and K. Takeuchi (2010) Prototypical
Development of Awareness Promoting Learning
Support System of Basic Presentation, Proceedingsof ISAC 2010.
[2] Hashimoto, K. and K. Takeuchi (2010) Multimedia
Learner Corpus of Foreigner’s Basic Presentation in
English with Evaluations, Proc. of International
Conference of Educational and Information
Technology 2010, Volume 2, 469-473.
[3] Forsbom, E. (2005) Rhetorical Structure Theory in
Natural Language Generation, (http://stp.lingfil.uu.
se/~evafo/gslt/nlg/assignment.pdf)[4] Groza, T.; Handschuh, S.; Clark, T.; Buckingham
Shum, S. and de Waard, A. (2009). A Short Surveyof Discourse Representation Models, Proc. of 8thInternational Semantic Web Conference, Workshop
on Semantic Web Applications in Scientific
Discourse. Lecture Notes in Computer Science,©ISAROB 2011165

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Country Domain Governance: An analysis byDatamining of Country Domains
Katsuko T. Nakahira, Hiroyuki Namba, Minehiro Takeshita, Shigeaki Kodama, and Yoshiki Mikami
Nagaoka University of Technology, Nagaoka, Japan(Tel : +81-258-47-9847)(Email katsuko@vos.nagaokaut.ac.jp)

Abstract: Along with the expansion of opportunities to use the internet, the system of managing it has become a serious problem. Even from the global perspective, questions concerning internet governance have been ra
ised at various places, starting with the WSIS (World Summit on the Information Society). However, the main discussion has been from the viewpoint of administrators and service providers, placing emphasis on ensuri
ng safety and equal opportunities of use, while the discussion from the user’s point of view is limited. This is thought to be due to the acquisition of statistical data which the discussion is based on being largely deriv
ed from statistics related to the physical means of communication, and to the difficulty of comprehending alluser activity in the internet space. In this paper, in response to the challenge of internet governance, we propose the country domain governance index. This is an index for comprehensive discussion of governance of in
ternet resources from the point of view of both administrators and users by carrying out data mining by regi
on by merging various pieces of Web information obtained from results of observations in the Language Observatory with various statistical data.
Keywords: Less than 6 words.
combining the results of these observations and existingstatistical data, it is possible to obtain a large amount of
information such as the number of Web pages per capita
and the number of Web pages by language, the ratio of
native language pages, the degree of mixing-up ofcharacter codes, and basic research concerning theinfrastructure.Some of the results have been publishedby Nakahira and Mikami[5], but this does not extend to
the interpretation of the results obtained nor to thedevelopment of an index to systematically handle them.
In this paper, we consider country domain governancebased on a distribution/growth chart of outdegree,which is one of the data mined LOP observation results,
and on an investigation of numbers of links.

I. INTRODUCTION
Along with the progress of the IT revolution thatstarted with the use of the internet, it has been pointed
out since the 1990s that a gap in enjoyment of thebenefits continues to exist between countries[1]. Thisgap is termed the “digital divide,” but the digital divide
is classified into several layers according to the objectof research. For example, in [2] the difference in the“place” in which the IT technology is used is identified.
Among these, in particular, the digital divide between
countries caused by economic disparities and various
other gaps, starting with the language barrier (educationgap), are addressed, and the need for the elimination of
the digital divide is discussed. However, these are allbased on statistical values related to the physical meansof communication, and do not afford a view of the big
picture of the overall disparity.
Mikami et. al. established the Language Observatory(LOP)[3] with the purpose of carrying out observationsof the digital divide between languages on the internet.
LOP combines Web crawling technology and alanguage identification engine[4] to retrieve up to 109
multi-lingual Web pages, and automatically retrieve data
included in a Web page such as URL information, tags,
the main body, and Web server response time. By©ISAROB 2011

II. COUNTRY DOM AIN GOV ERNANCE
Country domain governance (CDG) indicatesappropriate governance of a country code domain, but
in this paper, it indicates in particular the domain
administration policy. The creation of an index isrequired for investigating CDG. We set up what serves
as a CDG index as in Table 1. By carrying out generalmining of widely available statistical data and Web
crawling data gathered in LOP, these indices can beshown for the first time.

166

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 1. CDG Index
Level

Title
Accessible
andAffordable

Accessible/
Affordable
SystemContent

Linguistic
Diversity
Secure andTrusted
Accessible/
Affordable

Index
Relative Price of DNs to Monthly Income/the Global
average price, Number of DNs (published) per Population/
GDP, Ratio of Servers Located Outside of the jurisdiction
Availability of IDN Service in local language,
Number of Active IDNs

IDN service
Security, Stability,
and Resiliency
Openness of
Network

Linguistic
Diversity

Locallanguage use

Secure andTrusted

Trustable content

Ratio of DNs whose Registrant is Anonymous, …(other)
Ratio of the number of total outgoing links to the numberof Outgoing links to news Media
Number of Local language pages per population, Ratio of
total pages to pages written in Local Languages, Linguistic
Diversity measured by Lieberman Index/Entropy
Availability of Dispute Resolution Process, …(other)(Fukuda, [8]).
If this property is used, the state of the Web spacethat produced the outdegree distribution can be
determined simply by looking at the outdegree
distribution. Namely, if the outdegree distribution looks
like white noise then the Web space is undeveloped, andif it is a power series, the Web space is developing withmutual links. However, with the increase inautomatically created sites, as more in-depth services
are being provided in the Web space with the formationof portal sites, shopping sites and the like, the Web can
be said to be in the mature stage.
These kind of properties are ascertained and theoutdegree distribution is produced for each country code
top level domain (ccTLD), and the distribution
indicated for each state of the Web space is theIII. WEB LINK ANALYSIS AND
OUTDEGREE GROWTH CHART
On the other hand, for an investigation carried out of
the usual Web space, outdegree distribution is used.
Outdegree distribution plots the number of Web pages
taking each outdegree value on the vertical axis against
the outdegree value on the horizontal axis. The
outdegree distribution follows a power law as indicatedoriginally by Broder et. al.[7],but in cases where there
are a lot of automatic Web page creation such as withshopping sites, which are coming to form themainstream these days, or CMS, an outdegree
distribution is shown in the form of a power law with aδ function superimposed, because things with particularoutdegree values are distributed in greater numbers
1000000lb

10000010000
1000

1000000

1000000mm

100000
10000
1000

10000
1000

100

100

100

10

10

10

1

1
1

10

100

1000

(a) Type 0: low growth

kz

10000
1000

10

100

1000

10000

(b) Type I: domestic growth only

1000000
100000

1

1

10000

1000000
10000
1000
100

10

10

1

10

100

1000

10000

(c) Type II:major offshore growthjp100000100

la100000

1

1
1

10

100

1000

10000

(d) Type III(a): balanced growth

1

10

100

1000

10000

(e) Type III(b) : beyond maturity

Figure 1. Example of an outdegree maturity diagram. The horizontal axis represents the number of links,
and the vertical axis the number of pages which have that many links. Black squares represent domestic s
ervers, and white squares the offshore servers.

©ISAROB 2011

167

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

outdegree growth chart. An example is shown in Figure
1.We have roughly divided them into 5 grades from type
0 to type III(b).

VI. EXAMPLE OF OBSERVEDCDG I NDEX
Next, we show several actually observed Web spaceexamples based on the CDG Index.

1. Openness of Network
Openness of the network is determined by verifying
whether or not the environment allows users within thedomain to freely view content outside the domain. Care
1000000

1000000

mv

100000
10000
1000
100

10000
1000
100

10

10

1

A

pk

100000

1
1

10

100

1000

10000

1

10

100

1000

10000

(B)
1000000

my100000
10000
1000
100
10
1
1

10

100

1000

10000

Figure 2. Ratio of the number of total outgoing links to the number of Outgoing links to news media and outdegree diagrams.

group B

group A

2. Relationship between Indicators
With the CDG Maturity Index, it is also possible tointegrate each of the indicators and analyse them.
An example of this is shown in Figure 3. Figure 3
indicates the percentage of overseas installation of Web
servers in each ccTLD and the percentage of nonofficial language usage in the regions belonging to the
ccTLD. This corresponds to visualisation in an identicalspace of the index and indicated “percentage of servers
located outside of the jurisdiction” in Accessibility andAffordable,and the index and indicated “ratio of total
pages to pages written in local languages” in Local
Language Use.
Figure 3 divides the whole region into 4 groups andgives each group a meaning. In group A, the domestic

group D

group C

Figure 3. Estimated actual usage of ccTLD©ISAROB 2011

must be taken over the fact that the guarantee of
openness is roughly divided into (1) access restrictions
due to censorship, and so on, and (2) social barriers to
implementation, for example undeveloped infrastructure.
Case (1) is connected to the restriction of the user’s freeuse of the internet, and case (2) is seen as an
infrastructure development problem, in advance ofdomain management problems. Whichever the case,investigation of the openness of the network can beanalysed through investigating the number of Web
pages belonging to the relevant ccTLD, and the link
structure.
Several results of this are shown. In Figure 2, thenumber of links to the global news media (GNM) site ofeach ccTLD to which Web pages belong are graphedaccording to the physical location of the Web server onwhich the Web page resides. In producing this data,
the UbiCrawler[6] that LOP uses was run in December,2009 on approximately 2.3×107 Web pages used in 42
countries across Asia.
In Figure 2, the average number of links to the GNM
per server from Web pages on domestic servers on thehorizontal axis, are plotted against equivalent data foroffshore servers on the vertical axis. Therefore, thepoint at the top left of chart A indicates that the links to
the GNM are concentrated on offshore servers, and thepoint at the bottom right of B indicates that the links to
the GNM are concentrated on domestic hosts. The 4
countries fj, np, pk, mv illustrate these possibilities forA, and the 2 countries lk and my for B. The outdegree
distribution corresponding to these ccTLDs is also
shown in Figure 2.

168

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

VI. CONCLUSION

infrastructure is basically stable, and a country isenvisaged in which installation of a Web server is also
possible if desired. For this reason, the ratio for overseas
installation of Web servers is low, and also, as it is easy
to make use of the internet in the mother tongue, usageof Web pages in the official language or mother tongue
is common. In group B, although the domestic
infrastructure is stable, for some reason such as users
being restricted to overseas or to a minority living inthat country, usage of Web pages in a language besides
the official language or mother tongue is common.
Group C represents cases where the domestic
infrastructure is too unstable to be able to set up a Web
server within the country, or where the domain for a
local server is too expensive to purchase. Even in thiskind of country, if mobile communication is developed,
Web pages can be placed on overseas servers, and it isconceivable that Web pages in the country’s official
language or mother tongue could be used. Group D
represents countries that do not even consider web page
usage within the country, but want to profit by sellingdomains, or where a minority is able to obtain the
country’s ccTLD and make use of it.
Through these considerations, by looking in detail at
the relationship between indicators, we believe that we
can more precisely understand the reality of the digital
divide using the CDG Maturity Index.

In this paper, the CDG index is proposed as anobservation index by which to home in on the reality ofcountry domain governance. This allows indexing of thethe reality of the digital divide in a form that is closer tothe conditions of use to be undergone, along withcarrying out data mining by region in a form in which
the various pieces of Web information obtained from theLanguage Observatory observation results are mergedwith the various statistical data. Several observation
examples of this have been provided here. In the future,
we plan to continue observation of Web space based on
this index.

REFERENCES[1] Ohashi Ikuo,Toward the construction of a global
information society: international trends and issues
surrounding the digital divide,and creation of intell
ectual property 2009(3):6-36.
[2] NTT C&C Foundation, the Digital Divide(in Ja
panese), NTT Foundation book: 2002.
[3] Y. Mikami et. al., The Language Observatory
Project: Proceedings of the 14th International World
Wide Web Conference 2005 :990-991[4] S. T. Nandasara et. al., An Analysis of Asian
Language Web Pages: The International Journal on
Advances in ICT for Emerging Regions 2008 01 (01) :12 - 23[5] K. T. Nakahira and Y. Mikami, Measuring Language Diversity in Cyberspce: International Conferen
ce on Linguistic and Cultural Diversity in Cyberspa
ce, Yakutsk, Russia, July 2-4, 2008.
[6] Boldi, P. et. al., UbiCrawler: a scalable full Dis
tributed web crawler: Software-Practice & Experience, 34(8):711-726.
[7] Broder, A. et. al., Graph Structure in the Web,
Computer Networks: The International Journal ofComputer and Telecommunications Networking AR
chive 33: 1-6[8] Kensuke Fukuda (2004), Analysis of Statistical
Properties of WWW (in Japanese), IPSJ SIG Technical Report, 2004(136): 17-22.
[9] Y. Arai et. al., A survey on language distrib

V. ANA LYSIS T OOL F OR CD G
Through the analysis carried out so far, by using the
CDG index, we have shown that we have approachedmore closely the realities of the management policy foreach ccTLD and the realities of the digital divide.
However, to gather the data on which these analyses
are based, appropriate analysis tools are required.
Accordingly, we have added modules to and carried out
preparation for the stable operation of the analysis tool(iGalaksy) currently developed by Arai et. al. [9].
iGalaksy consists mainly of an information input part(including crawling data), an information management
part (CDG Index analysis), and an output part, and in
one crawl it is possible to gather roughly 107-108 Web
pages (approximately 10GB of crawling data). If thesedata were, for example, gathered every quarter, the data
size would be so huge as to require a database withspecial data striping. We have implemented this using
pgpool-II[10].

©ISAROB 2011

ution on the internet : The 8th Proceedings ofForum on Information Technology 2008-4:529532.
[10] http://pgpool.projects.postgresql.org/

169

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Extraction and Comparison of Tourism Information on the Web
Xiaobin Wu1), Sachio Hirokawa2), Chengjiu Yin2), Tetsuya Nakatoh2), Yoshiyuki Tabata2)
1)

Graduate School of Information Science and Electrical Engineering,2)
Research Institute for Information Technology, Kyushu University
Hakozaki 6101,Fukuoka,
8128581,JAPAN
2ie10056y@s.kyushu-u.ac.jp, {hirokawa,yin,nakatoh,tabata}@cc.kyushu-u.ac.jp
Abstract: The number of tourists to Japan from foreign countries is drastically increased in recent years. However, there
is a scene where the traveler is made uneasy by differences between the word, the custom and the culture in travelingabroad. We are aiming at the construction the horizontal search engine intended for tourism information in the Kyushu
region as a test case of a special vertical search engine. As the first step for the research, we extracted 312 events from apublic tourism portal site and compared the ranking of each event in the site with that in the general search engine.
We confirmed a weak correlation of the ranking. Moreover, we found that the big difference of rankings for the eventswith strong regionality.The number of tourists to Japan from foreign countries is drastically increased in recent years.
However, there is a scene where the traveler is made uneasy by differences between the word, the custom and theculture in traveling abroad. We are aiming at the construction the horizontal search engine intended for tourism
information in the Kyushu region as a test case of a special vertical search engine. As the first step for these theresearch, we extracted 312 events from a public tourism portal site and compared the ranking of each event in the sitewith that in the general search engine. We analyzed the correlation of the rankings and confirmed a weak correlation. In
addition, it was also confirmed that there was a large gap for the events with strong regionality.
Keywords: Search engine, information extraction, rank correlation coefficient ,data collection, area tourism
language, in a custom, or in culture. Of course, guidebooks have been used as helpful tools. Information on
the Internet is much useful and reliable with respect tonewest and up-to-date tourism information. This paperchooses touristic event information in Kyushu area inJapan, as the target of the analysis.
We collected the tourism information from the portalsite of "Kyushu Tourism Promotion Organization." We
analyzed the ranking of each event in the portal and theranking of that in the general search engine Google. To
evaluate the ranking, we used the estimated number of
Web pages that match the name of the events which we
send as a query.

I. INTRODUCTION
According to the development of the Internet, many
information came to exist on WWW. By using ageneral search engine, we can get Web pages whichcontain the keywords we send to the search engine asquery. However, it is not easy to reach the information
unless we know appropriate keywords. Even if weknow some of suitable keywords, the target pages, weare searching for, might be buried under the major
pages displayed with higher ranked position of thesearch results.
One of the reasons of this problem is in that thecoverage of general search engine is too wide and too
general for particular purpose. We are studying thesearch engine focused on specific fields and objects on
WWW. By limiting the search targets, we can expect torealize a search engine excellent in precision and recall
for the targets. The present paper focuses on the
tourism information on WWW, and compares thefeature of information found in a tourism portal site andin a general search engine.
The number of foreign tourists visiting Japan are
increasing continuously in recent years. Thanks to thesimplification of the visa to the well-to-do populationof China, more and more Chinese tourists are expected
to come to Japan. However, there are many scenes
where a traveler feels uneasy by the difference in©ISAROB 2011

Fig.1. The information of Welcomekyushu

II. Collection of Tourism Event Information
The event information on the Kyushu area waschosen as tourism information dealt with by this170

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

number of the obtained search results was made into theamount of information on a general Web page. Next, thesearch range of Google was limited towww.welcomekyushu.jp using the "site:" operator. The
number of the search results obtained fromwelcomekyushu was made into the amount of
information of a tourism special site. The number of theinformation in a general Web page was from 29 to
241,000,000 by the event name. by the event name, The
number of the information on a tourism special site wasfrom 2 to 603 similarly. Fig. 2 plotted the number ofsearch results in descending order of it.

research. We prepare the list of events as follows.
Firstly, we obtained the 906 information from the "event
list of Kyushu and Okinawa" in the web site of the"Kyushu Tourism Promotion Organization", which werefer as "welcomekyushu" in the sequel of the present
paper. This list contains not only the names of event but
also the names of food, such as "colander tofu", and thenames of places, such as "alpine rose park". We
removed these non-event names and compiled 316
event names. Some of the events are held every yearwith the name of the year, such as "X festival 2009" and"X festival 2010". The list of 312 events was obtainedby removing these duplication. Some of the list is
shown in Table 1.

III. Analysis by Rank Correlation
1. Kendall tau rank correlation coefficient
We compared event information with Kendall tau
rank correlation coefficient [1].A computational
procedure consists of four following Steps.The paper
title text is 15 point bold font in Times New Roman.
Step 1. The number of measured value is set to n
about two variables X and Y.That is, there are measured
values of Xk and Yk (k= 1 .. n).
Step 2. Ranking is attached to an ascending order at
Variable X and Variable Y. When there is equal order,
average of the ranks is given.They are arranged in an
ascending order about Variable X.
Step 3. The number of the equal order in Variable X
and Variable Y is set to nx and ny.The size of equal orderis set to ti and tj (i=1,2,...,nx; j=1,2,...,ny).Then, Tx and Ty
can be found as follows.

Table 1. 20 Events of Event Name list of Kyushu
十五夜ソラヨイ, 鏡山スカイスポーツフェステ
ィバル, 串木野浜競馬, 「小城」ホタルの里ウ
ォーク, とす弥生まつり, べっぷ鶴見岳一気登山大会, 宮崎神宮大祭, 美山窯元祭り, 小倉祗園太鼓, 四十九所神社「やぶさめ祭り」, 白地楽, 幸若舞, うすき竹宵, 筑前いいづか雛のまつり, 牛深ハイヤ祭り, 武雄の荒踊り, 門司み
なと祭, 鹿児島カップ火山めぐりヨットレー
ス, 南大隅町ねじめドラゴンボートフェスティ
バル, 山鹿灯籠浪漫・百華百彩
Next, in order to compare the amount of information
which exists in the general Web page on WWW, and theamount of information which exists in a tourism special
site, the number of the search results with the event
names was obtained. Google was chosen as a generalsearch engine and the Kyushu Tourism Promotion
Organization was chosen as a tourism special site. First,
each event name was searched with Google and theStep 4. Number in the case of being Yi < Yj about Yi(i=1, ..., n-1) and Yj (j=i+1, i+2, ..., n) is set to Pi.
Number in the case of being Yi > Yj about Yi and Yj is
set to Qi. ∑Pi is the number of times whose direction of
the ranking of two variables corresponds. ∑Qi is thenumber of times whose direction of the ranking of two
variables corresponds with an opposite direction. At thistime, a rank correlation coefficient is calculated for by
the following formula.

Fig.2.The Number of Search Results of Each Event©ISAROB 2011

171

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

contained a name of a place."Hosenji firefly festival"
and the "Sakurajima enjoying -the-evening-cool tour
ship" (Fig.3) are samples of these names.
4)The area Y contains more events than that ofZ.The region Z displays the regional strength of the site.

2. Examination
To compare the ranking of each event from 312
events, in the portal site "welcomekyushu" and with thatin general search engine, we used the estimated numberof Web pages that contain the name of the event. The
obtained rank correlation coefficient was 0.20744. The
correlation coefficient is not so strong to confirm a clear
correlation. However, the p-value we obtained is4.616*10-8, which is less than the required significancelevel 5% (α= 0.05). Thus, we can reject the null
hypothesis at the given level of significance. Eventually,
we see that the correlation is true.

IV. Events with Ranking Gaps
In Fig.3, each point represents to an event, where thex-axis is the ranking of the event in Google, and the yaxis is the ranking of the event in welcomekyushu.
There is a large gap between the ranking by Google
and by welcomekyushu,when an event is displayed in
distance from the line A, which represents the strong
correlation.By analyzing those events, we observed theFig.3 Ranking of Events
following four facts.
1)The points in Domain X support the correlation of
We compared the ranking of the number of thethe two rankings.The more points are in X, the higher issearch results for each event name, in order to confirm
the correlation coefficient.We can find out weakthe difference between the amount of information in acorrelation from Fig. 3.
general Web page, and the amount of information of a2) The points in Domain Y have a high ranking intourism special site. Table 2 shows the list of events
Google, and a low ranking in welcomekyushu.The
with high ranking with Google search engine. Table 3
names of these events seems to appear anywhere allshows the list of events with high ranking inover Japan. "Ohito Kabuki" and "Amazake
welcomekyushu. From these two tables, we can say that
Festival"(Fig.2) are samples of such names.
regional events are much easier to find by
3)The points in Domain Z have a low ranking inwelcomekyushu than by Google.
Google, and a high ranking in Welcomekyushu. Those
events are peculiar to Kyushu area.Many event titles
Table 2. 10 Events with High Ranking in Google, Low Ranking in WelcomeKyushu
Ranking
1

Rk-Rg*
63.79

Rg*
1.21

Rg
5

Rk
G
65 8710000

K
5

2
3
4
5
6
7
8
9
10

62.42
62.14
62.11
61.38
61.18
60.87
60.35
59.97
59.94

4.58
3.86
2.89
3.62
4.82
3.13
2.65
6.03
5.06

19
16
12
15
20
13
11
25
21

67
66
65
65
66
64
63
66
65

3
4
5
5
4
6
7
4
5

©ISAROB 2011

8580009350001240000
9650008360001180000
1400000
625000819000

172

Event
竹ん芸
おしろい祭
みそ五郎まつり甘酒まつり
鮒市
八天神社例大祭
大人歌舞伎
出の山ホタル恋まつり
サン・サン・さんわフェスティバル
くも合戦

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Ranking
303
304
305
306
307
308
309
310
311
312

Table 3. 10 Events with High Ranking in WelcomeKyushu, Low Ranking in Google
Rg*
Rg
Rk
G
K
Event
32.07 133
9 25700
93 皿山まつり
23.07
24.37
44.37 184
20 10200
65 早岐茶市
24.94
55.94 232
31 397047 長串山つつじまつり
25.42
29.42 122
4 32700
112 薩摩のひなまつり
27.94
28.94 120
1 36300
603 山鹿温泉祭
32.40
50.40 209
18 6100
67 仙酔峡つつじ祭り
32.71
48.71 202
16 6790
72 筑後吉井おひなさまめぐり
34.30
46.30 192
12 8980
83 宝泉寺ホタル祭り
36.92
42.92 178
6 10900
101 桜島納涼観光船
41.12
51.12 212
10 595090 人吉球磨は、ひなまつり

Rg*-RkEach item shown in Table 2 and Table 3 is described as below
Rg*: normalization ranking of Google search results
Rg: ranking of Google search results
Rk: ranking of Welcomekyushu search results
G : number of Google search Results
K : number of Welcomekyushu search results
events, locations, shrines and souvenirs. We used 312
events for comparison of portal and general searchengine. We extracted them from the lists in the portal
site. Therefore, the development of the technique of thelist discovery is an important problem. It is also
necessary to devise a method to identify the same eventswith different names. As the next step of the research,
we are considering to analyze other regional portal sites
to compare with each other.

V. Related Work
The tourism industry is one of the industries that
suffered big influence of internet. Sightseeing
information was unavailable only in a special travel
agent before. Now, everyone can easily obtain it thanks
to the internet. We can find sightseeing information on
Web in (a) tourism portal sites, (b) general web pages,and (c) blog sites.
There are several systems and researches intended foreach targets. [2] proposes a recommendation and aclustering system and shows their effectiveness fortourism portals.[6]proposes a natural language interface
for tourism search engine. [5] analyses the patterns inHTML documents that characterize the occurrences ofNEs(Named Entity), such as the name of the locationand the name of the touristic events. [3] studies the cluewords that can be used to extract tourism related NES.
[4] reports the characteristic keywords that distinguishtourism blogs from other general blogs.
We are aiming at the construction of the vertical searchengine for tourism information. The difference of thecharacteristic of (a) and (b) was examined in the present
paper.
VI. CONCLUSION
This paper analyzed tourism information available
on a public tourism portal site that covers 7 prefecturesin Kyushu area. 312 events were extracted from the site
and were used to compare the portal site with a generalsearch engine. We analyzed the correlation of the
rankings of each event in the portal site and in thegeneral search engine. It turned out that there exists a
weak correlation between the rankings. It is also
confirmed that there is a large gap for the events withstrong regionality, which indicates a characteristic of theportal site. One of the lessons we learned is theusefulness of the lists of names, such as the names of©ISAROB 2011

REFERENCES[1] H. Abdi, Kendall rank correlation, In N.J. Salkind(Ed.): Encyclopedia of Measurement and Statistics.
Thousand Oaks (CA): Sage,2007.
[2] S. Esparcia, V. Sanchez-Anguix, E. Argente, A.
Garcia-Fornes, V. Julian, Integrating Information
Extraction Agents into a Tourism Recommender System,
Proc. HAIS2010, Springer LNAI 6077, pp.193-200,
2010.
[3] Q. Hao, R. Cai, Ch.Wang, R. Xiao, J.-M. Yang, Y.
Pang, L. Zhang, Equip Tourist with Knowledge Minedfrom Travelogues, Proc. WWW2010, pp.401-410, 2010.
[4] A. Ishino, H. Nanba, H. Gaguma, T.Ozaki, D.
Kobayashi, T. Takezawa, Automatic Compilation ofTravel Information from Automatically Identified Travel
Blogs(in Japanese), IEICE Tech Report, WI2-2009,
pp.19-23, 2009.
[5] I. Kinjo, A. Ohuchi, Web data analysis for Hokkaido
tourism information (in Japanese) IEICE Tech. Report,
DE2001-07,pp.99-104,2001.
[6] J. M. Ruiz-Martinez, D. Castellanos-Nieves, R.
Valencia-Garcia, J. T. Fernandez-Brieis, F. GarciaSanchez, P. J. Vivancos-Vincente, J. S. Castejon-Garrido,
J. B. Camon, R. Martinez-Bejar, Accessing Touristic
Knowledge Bases through a Natural Language Interface,Proc. PKAW2008, Springer LNAI 5465, pp.147-160,
2009.

173

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

An Analysis of Firm’ Capacity in Mazda’s Keiretsu
R. Takida1, T. Ito1, S. Matsuno1, K. Voges2, Y. Ishida3, and M. Sakamoto4

1 Dept. of Business Administration, Ube National College of Technology, Yamaguchi, Japan(Tel : 81-836-35-7115; Fax : 81-836-35-7115)(ito@ube-k.ac.jp, matsuno@ube-k.ac.jp)
2 Dept. of Management, University of Canterbury, New Zealand(Tel :64-3364-2987; Fax :-)(kevin.voges@canterbury.ac.nz)
3 Dept. of Knowledge-Based Information Engineering, Toyohashi University of Technology, Japan(Tel :81-532-44-6872; Fax :-)(ishida@tutkie.tut.ac.jp)
4 Dept. of Computer Science and Systems Engineering, University of Miyazaki, Japan(Tel : 81-985-58-7392; Fax : 81-985-58-7392)(sakamoto@cs.miyazaki-u.ac.jp)
Abstract: Capacity is defined as the power resulting from the specific position in network organizations in thispaper. Thus, it becomes one of the important issues to measure firm’s capacity. In this paper, we review the relevant
studies of network organizations, and focus our study on Yokokai, the Mazda’s Keiretsu. We propose a new approachto calculate firm’s capacity. The capacity is divided into two categories, take-in capacity and take-out capacity in thispaper. The relationship between the two capacities is called capacity difference. The relationship between capacity
difference and corporate performance has been analyzed in order to discover the determinants of corporate performa
nce in network organizations. Therefore, this paper provides a new perspective to find the determinants of thesuccessful corporate management.
Keywords: capacity, capacity difference, corporate performance, Keiretsu

I. INTRODUCTION

II. BACKGROUND

As a special type of corporate group, Keiretsu is oneof the well known organization forms widely today. It is
an important factor for successful Japanese companies
because Keiretsu is considered as the sources of the
Kaizen and technical innovations. Many studies about
the Keiretsu research have been published recently. In
this paper, we propose a new structural index, capacity,in order to discover the rational relationship between
capacity and corporate performance. The main
contributions of this paper are: 1) the new quantitative
concept of capacity is proposed; 2) the validity ofcapacity is proved; 3) rational relationship between
capacity and corporate performance is analyzed.
Therefore, this paper provides a new perspective to findthe determinants of the corporate performance.
This paper is organized as follows. In Section 2, we
briefly review some relevant researches of quantitative
approaches of Keiretsu. Section 3 introduces thecapacity model. Section 4 shows the results, anddiscusses the implications of the results. Finally in
Section 5 we conclude by a summary of this paper.

Relationship is one of the important factors instructural analysis. Most researchers use strong tie
and/or weak tie to describe the different strength of therelationship.
However, quantitative approaches are needed to
discover the determinants of corporate performance.
Dyer H. J. analyzed the relationship among firms basedon distances between their locations and the frequency
of face-to-face communication among engineers in theautomobile industry [1].
High correlation relationship between degree andcorporate performance has been found [2]. Interorganizational relationships in the Keiretsu have been
analyzed with quantitative analysis tools such as
CONCOR and other statistical methods [3]. Moreover,Fukuoka et al. reported a new finding in relationships
between firms in the Keiretsu of Nissan from theviewpoints of transaction and cross shareholdings [4].
Recently T. Ito begins to apply graph theory to networkorganization analysis, and clarifies some characteristics
such as centrality, size of network [5-7]. Like otherindices, such as centrality and density, capacity is also

©ISAROB 2011174

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Then the value of capacity of nodes can be expressed asfollows.

one of the important indices of the analysis of thecorporate performance. To the best of our knowledge
and investigation, no study has examined capacity froman organizational network vantage point; therefore thisstudy attempts to shed light on one of the mostadvanced quantitative analysis using data gathered from
Mazda’s Yokokai Keiretsu.

⎛ A⎞
C (r ) ij = ∑ ⎜ ⎟i =1 ⎝ L ⎠r

Basically Capacity means power pertaining to, or
resulting from, the possession of strength, wealth. It
shows one kind of possible powers of being or of doing.
In this paper, capacity is defined as the power resulting
from the specific position in network organizations.

Then the terminal capacity of the row will becalculated as follows.
C ( pi , r ) = C (r )i1 + C (r )i 2 + C (r )i 3 + ⋅ ⋅ ⋅ + C (r ) in1.

Outline of the capacity model
Generally a graph consists of a set of nodes and a setof arcs. Two nodes are connected if a path between
these two nodes. Path is one of the important concepts
in graph theory. According to graph theory, a path is asequence of nodes such that the nodes and the arc are
adjacent. A walk is a sequence of nodes and arcs suchthat the nodes and arcs are adjacent. The difference
between path and walk is that a path is a walk that does
not include any node twice, except that it its first node
might be the same as its last. For digraphs, walks cantravel arcs only in the direction of the arrows.
The length of walk is formed by a sequence of thenumber of arcs such that any two successive arcs in thesequence share a node. Basically the node’s capacity isdetermined by the length of walk and the number ofwalks.
Suppose S(r)ij is the summation of the length ofwalks from node pi to pj when the length of the walkequals to r.

(3)

When the r equals to infinite, the total capacity of C
is calculated as follows.

A

(4)
−1

C = C (∞) = ( I − ( )) − IηThe summationof row in matrix C means take-out capacity, and thesummation of column means take-in capacity.
In real society, the node generally means theindividual and/or firm, and the arc means therelationship. In transaction network, walk can be
explained as one power to begin transaction with othercompanies. Therefore, the volume of walk could be
explained as firm’s capacity in network organization. In
transaction matrix take-out capacity means the capacityof firm i that sell its parts to all other firms, and take-in
capacity means the capacity of firm j that purchases
parts from all other firms.
2. Data collection
In order to measure all firms’ capacity in Yokokai,transaction data in the Yokokai keiretsu have beencollected from our interviews and the publications of
the Japan Auto Parts Industries Association andAutomotive Parts Publishing Company [8].
In 2004, 177 parts suppliers are included in Yokokai.
72 parts suppliers and Mazda have reciprocaltransactional relationships, and 105 parts suppliers are
singletons. A singleton is an isolate company whose indegree and out-degree are both 0. In Yokokai’s case,(1)

A is a normalized adjacent matrix. The reachable
matrix, denoted by Ar, refers to the fact that node pi canreach node pj through the number of steps r. For
instance, A2 means that node pi can reach node pj
through 2 steps. The element of matrix A is the numberof the walks between node pi and pj.
The strength of the walk should be considered as theinverse of the length of the walk. In other words, thestrength will be weaker if the length of walk is longer.

©ISAROB 2011

(2)

where
C ….. capacity from node pi to pj
A ….. the number of walks in a given graph
L ….. parameter of the length from node pi to pj
r ….. length of the walk from node pi to pj

III. METHOD

S ( r ) ij = A + A 2 + A 3 + ⋅ ⋅ ⋅ + A ri

175

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

respectively in 2004, and 0.52 (p=0.00) and 0.49
(p=0.02) respectively in 2005. Therefore, it is clear thatcertain correlation between capacity difference andcorporate performance exits.

singleton means the firm which has no relationship withother firms.
The transactional relationships among thecompanies were identified through graph modeling. A
tie shows the percentage of the transaction between
each pair of firms. We collected directed and weighted
data to measure the capacity of each firm.

Table 1 Matrix of correlation coefficient between
capacity difference and corporate Performance
2004

IV. RESULTS AND DISCUSSIONS

1
Sales
33
0.64**
Profit0310.44**
Capacity
0.1
Difference
33

We developed a computer program and calculated thecapacity of each firm in Yokokai. The result of thecapacity of transactions is shown in Fig. 1.

131
0.88**
0
31

2005

1
42

1
37
0.53**
10.00
31
31
0.69** 0.42*
0
0.2
37
31

1
53

The scatter diagram between capacity difference andsales can be illustrated as Fig. 2.

Fig. 1 Yokokai’s capacity in 2004
As mentioned above, the capacity can be divided
into two categories, take-in capacity and take-out
capacity. The difference between those two capacities
means the difference between the purchase capacity andsell capacity. It is called capacity difference. The firm
has unbalanced issue if the absolute value of thecapacity difference is high.
In order to discover the rational relationship between
capacity and corporate performance, we collected thedata of corporate performance including sales and profit
from Mazda in fiscal years of 2004 and 2005, andcalculated correlation coefficient between the capacity
and corporate performance. Car maker Mazda is
excluded because this paper focused on the study ofparts suppliers. The result is shown as Table 1.
Table 1 shows us that a high correlation between
capacity difference and corporate performance.
Therefore, basically it is effective to improve corporate
performance if the parts supplier to find the way to cut
down the capacity difference.
In order to compare with Mazda’s results, wecollected transactional data from Nissan group. The
correlation coefficient between capacity difference andsales and profit is 0.44 (p=0.01) and 0.39 (p=0.02)©ISAROB 2011

Fig. 2 Scatter diagram of capacity difference and sales
The selling power is larger than its purchase power if
its capacity difference is larger than zero. Therefore,
these firms have good management skills if the value ofsales of these firms is high; otherwise the management
skills of these firms, such as No. 6; Sumino Kogyo Co.
Ltd., and No. 4; Keylex, should be improved. No. 64 is
Denso. Its capacity difference is -2.83, but its sales arehigh. It means Denso’s selling power is less than its
buying power because Denso is one of the main
subsidiaries in Toyota’s group. Denso purchases parts in
Mazda’s keiretsu and sells them in Toyota’s group. This

176

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011is the reason why Denso has negative value of capacity
difference but high sales.
The scatter diagram of take-out and sales, take-in andsales can be drawn as Fig. 3 and 4 respectively.

V. CONCLUSION AND FUTURE WORKSIn this paper, the new approach of capacity analysis
is proposed, and its validity is proved. We discovered
the rational relationship between capacity and corporate
performance using the example of Mazda’s Keiretsu.
However, much more works should be done to build up
rational relationship with other parts suppliers. For
instance, two indices are helpful to find the unbalanceissue in corporate management. One is the difference
between capacity difference and other corporate
performance, such as capitals and stock prices, andother is the difference between capacity difference andcentrality. Furthermore, the relationship between
capacity and density should be done in the near future.
Acknowledgment: This research was partially
supported by the Ministry of Education, Culture, Sports,
Science, and Technology, Grant-in-Aid for Exploratory
Research, 21510171, 2009.

Fig. 3 Scatter diagram of take-out capacity and sales

REFERENCES[1]Dyer H. J. (1996) “Specialized Supplier Networks as a
Source of Competitive Advantage: Evidence from the Auto
Industry”, Strategic Management Journal, Vol. 17, 271291[2]Ito T. (2004) Quantitative analysis of the firm’s
relationship in the Keiretsu of Toyota group, Proceedingsof the 2004 Information Resources Management
Association, International Conference on Innovations
Through Information Technology, pp.1078-1079, May 2326, 2004, New Orleans USA[3]Lincoln R. L. and Gerlach M.L. (2004) Japan’s
Network Economy Structure, Persistence, and Change,
Cambridge University Press[4]Fukuoka S., Ito T., Passerini K. and Sakamoto M.
(2006) An Analysis between Transaction and Cross
Shareholdings in the Keiretsu of Nissan, Managing
Information in the Digital Economy Issues & Solutions,163-169, IBIMA International Conference, Bonn Germany[5] T. Ito, K. Passerini, M. Sakamoto (2008) Structure
Analysis of Keiretsu of Toyota, Encyclopedia of
Networked and Virtual Organizations, pp.1542-1548, Idea
Group Publishing[6]T. Ito, C. Medlin, K. Passerini, M. Sakamoto (2009)
Influence Trust and Trade in the Keiretsu of Toyota: A
Centrality Analysis, Trust, Globalisation and Market
Expansion, Chapter 8, pp.101-118, Nova Science[7]Ito T., Matsuno S., Xia Z., Sakamoto M., and Rajiv
Mehta (2010) An Analysis of Interactive Influence in
Mazda's Yokokai Keiretsu, Artificial Life and Robotics,
Volume 15, Number 3, 249-252, Springer Japan[8]JAPIA&APPC (2005) Japanese Automotive Parts
Industry, Automotive Parts Publishing Company,(Japanese Edition)

Fig. 4 Scatter diagram of take-in capacity and sales
In Fig. 3, No. 6 is Sumino Kogyo Co. Ltd., No.8 isJapan Climate Systems Corporation, and No. 4 isKeylex. They have high value of take-out capacity. Itmeans that these firms have strong power to sell theirparts in Mazda’s keiretsu. But their sales are low.
Therefore, the selling policy of these firms’ should be
adjusted.
In Fig. 4, No. 64 is Denso. Denso has an extremely
high value of take-in capacity. It means that Denso
purchases parts in this group, and maintains its good
corporate performance because Denso has strong
selling power in its own market.

©ISAROB 2011

177

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

An analysis of structure importance in Mazda’s Keiretsu
S. Tagawa1, R. Takida1, T. Ito1, R. Mehta2, H. Hasama3, and M. Sakamoto4
1 Dept. of Business Administration, Ube National College of Technology, Yamaguchi, Japan(Tel : 81-836-35-7115; Fax : 81-836-35-7115)(niki@ube-k.ac.jp, ito@ube-k.ac.jp)
2 School of Management, New Jersey Institute of Technology, USA(Tel :1-973-596-6419; Fax :1-973-596-3074)(Mehta@njit.edu)
3 Dept. of System Management, Fukuoka Institute of Technology, Japan(fit-doctor_mas@kmd.biglobe.ne.jp)
4 Dept. of Computer Science and Systems Engineering, University of Miyazaki, Japan(Tel : 81-985-58-7392; Fax : 81-985-58-7392)(sakamoto@cs.miyazaki-u.ac.jp)
Abstract: Centrality is one of the effective indices to measure organizational structure. Freeman once proposed a set ofcentrality indices including degree, closeness, and betweenness. However, Ito discussed the implications of centrality,and found that differences exist even when the centrality is same. In this paper, the authors collected the data oftransactions and cross shareholdings from Mazda’s Keiretsu Yokokai, and calculated the structural importance based on
the new method, so-called SNW model. Furthermore, the authors discussed the implication based on the results ofcorrelation coefficient between the SNW results and corporate performance such as sales and profits. This paper
provides a new perspective to discover the structural importance of the network organizations.
Keywords: degree, betweenness, closeness, transaction, cross shareholdings, Keiretsu
coefficient between the new findings of the results of
the SNW and corporate performance such as sales andprofits are analyzed. Therefore, this paper provides a
new perspective to discover the structural importance ofthe network organization.
This paper is organized as follows. In Section 2, therelevant literature of organizational structure isreviewed briefly. In section 3, the authors introduced
some basic concepts, and explained the SNW model.
Section 4 shows the measurement results, and discusses
the implications of the new findings. Finally in Section
5 we conclude by a summary of this paper.

I. INTRODUCTION
According to the conventional organization theories,
corporate organizations can be basically categorized
into at least three types. The first one is line
organizations, the second is line-and-staff organizations,and the third is functional organizations. Other types
such as matrix organizations and project-teamorganizations have been developed recently. As a new
type of successful organizations, Davidow and Malone
suggested a new idea of virtual organization in the
beginning of 1900’s [1]. Another typical successfulorganization is considered as the Keiretsu in Japan [2].
The authors reviewed the previous research andcompleted a comparative study of the virtualorganizations and Keiretsu theoretically. The authors
found many common characteristics between them. One
is that they both can be defined as a networkorganization with mutual transactions and shared
information. The common issue in these different
organizations is how to discover the effective structure.
Centrality is one of the effective indices to measureorganizational structure.
The main contributions of this paper are: 1) the newquantitative analysis, so-called SNW model, isdiscussed; 2) centrality index of all firms including in
Mazda’s group Yokokai are calculated; 3) correlation©ISAROB 2011

II. BACKGROUND
Most of the research of network organizations canbe characterized by qualitative approach andquantitative approach. One of the most important issues
in this field is to discover what kind of relationship isthe most effective for corporate management in the IT
age today. To determine the relationship in the networkorganizations, many different models have beendeveloped. A classic but typical method is called
Freeman model proposed by Freeman in 1979 [3]. It
includes a set of centrality index to calculate the
network organizations from the viewpoint of degree,
closeness and betweenness. Many new measurements

178

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

have been discovered on the basis of Freeman model
later. For instance, Tyler et al. proposed a new method
applying a betweenness algorithm to identify
community within a network [4]. For calculating therelationship of cross shareholdings between firms in the
Keiretsu of Toyota, as a revised Freeman model, Itointroduced directed and valued connection lines into
Freeman model, and developed new computer program
to calculate the centrality index using the data of
Toyota’s Keiretsu. Strong correlation among out-degree,
betweenness and corporate performance has been found[5]. Centrality index is a useful tool for the structure
analysis of the network organizations. The structure
analysis could be carried out from many different
viewpoints, such as position, size in any networkorganizations. It is obvious that importance analysis ofeach firm would be one of the most important steps forthe structure analysis.

2

The SNW analysis
In order to discuss the implication of the Freeman
model, following example was given [7].
1

3

4

5

Figure 1 A 5-Node Network
According to Freeman model, the degree index ofnode 1, 2 and 4 are 2. It is obvious that their structural
importance is quite different. The degree index of theentire network is 0.417. But the degree index of the subnetwork excluded node 1, 2 and 4 are 0.33, 1 and 0.33
respectively. Node 2 and 3 are more important because
the degree index of sub-network excluded node 3 is lessthan the degree of the entire network.
Therefore, a third method called the SNW model is
proposed in 2005 [7]. The SNW model is expressed asfollows.

II. METHOD
Freeman proposed centrality of each node fromthree viewpoints of degree, betweenness and closeness[3]. Degree index is calculated as follows.
1

DegreeFreeman defines a degree of node pk is the numberof node connect with it directly. Therefore, Freeman
calculated it with the measurement developed byNieminen [6]. The centrality index of degree of node k
can be defined in the following way.

−I D ( pk ) = CD − CD ( pk ) …… (2)wheren∑[max C ( p ) − C ( p )]n

C D ( pk ) = ∑ a( pi , pk ) …… (1)C D = i =1

i =1

where
a(pi, pk)=1 if and only if pi and pk connected by a linea(pi, pk)=0 otherwise

D

i

D

i

n 2 − 3n + 2

Node k is called strengthening node if the centrality
index of sub-network excluded node k is less than thatof the entire network. In this case, ID (pk) is larger thanzero. The opposite is called weakening node when
ID (pk) is less than zero. Node k is called neutral node if
the centrality index excluded sub-network of node k isequivalent to that of the entire network. In this case,ID (pk) is equivalent to zero.

The number of nodes adjacent to a given node in asymmetric network is the degree of that node. For
asymmetric network the in-degree of a node pk is thenumber of ties received by pk and the out-degree is thenumber of ties initiated from pk.
The degree means the proportion of other nodes
that are adjacent to pk and is viewed as important index
of its potential communication activity.

©ISAROB 20112

III. MEASUREMENT
To avoid conceptual confusions, some importantconcepts of network organization are given as below.

179

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

cross shareholdings is 230. It includes 177 parts
suppliers, 11 carmakers, 42 banks and financial
institutions. The total number of the firms in the
network of transactions is 188, including 177 parts
suppliers and 11 carmakers.
The numbers of singletons in the networks oftransactions and cross shareholdings are 99 and 135
respectively. Therefore, there are 11 car makers and 77
parts suppliers which have reciprocal transaction
relationships, 87 companies including Mazda, 51
suppliers and 35 banks and financial institutions havereciprocal capital relationship in Yokokai.
The network of transactions and cross shareholdingsof Mazda can be illustrated as Figure 2.

1

Keiretsu and network
The definition of network organization means agroup of nodes, such as persons or firms, having certain
relationships among themselves under the condition of
having common purpose and willingness to participate.
In accordance with this definition, Keiretsu could beinterpreted as one kind of network organization in
management science.
A graph consists of a set of points and a set of lines
connecting pairs of point. The point, which composes a
network, is called node, and the line, which connects
any two nodes directly, is called an edge in graph theory.
A graph is a model with an undirected dichotomous
relation. In other words, a tie is present or absent
between each pair of nodes. The data consists of valued
and directed connections in which the strength orintensity of each tie is recorded. Therefore, a graphcould be considered as one specific type of networkorganizations. In network organization, generally thefirm should be the node, and the edge should be thedifferent types of relationships such as transactional
relationship and friend relationship.
2

Data collection
Data of transactions and cross shareholdings in the
Keiretsu of Mazda are collected from the publications
and investigation by interviews [8].
The process of data-collection can be expressed asfollows.

Network of transactions

Step1: Determine the boundary of the network;
Step2: Collect the data of transactions and crossshareholdings; basically the data is percent value of thetransactions or stock holdings between two firms. And
input the data into a matrix table.
Step3: Remove singletons in the matrix table.
The data of each cell in the matrix meanstransactional relationship which the firm in column
accepts auto-components from other suppliers in row of
the transactions network, or capital relationships which
the firm in row invests in stocks to other firms incolumn in the network of cross shareholdings.
The authors removed the singleton in Keiretsu of
Mazda. Singleton means that the node has no any
relation with other nodes. They are isolated in the
network. The total number of the firms which holdcapital relationship with each other in the network of©ISAROB 2011

Network of cross shareholdings
Figure 2 Network of transactions and Cross
shareholdings of Mazda in 2004

IV. RESULTS AND DISCUSSIONS
It is difficult to determine which firm is more
important in Figure 2. To calculate the importance ofeach firm, two methods mentioned above are applied inthis paper.

180

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

The results of the first method can be expressed inTable 1.
Table1 Selected part of degree of each firm in Yokokai
Noout-degree(T)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
66
70
90.6
66.8
89
48.9
92
0
84
76.5
40
81.3
0
66.6

in-degree(T) out-degree（C) in-degree（C)
0
3
0
0
9.4
0
0
5.4
0
0
2.6
0
0
3.5
0
0
1.9
0
0
1.9
0
0
6.2
0
0
23.7
0
0
24.8
0
0
13.2
0
0
6.9
0
0
3.1
0
0
10.8
0
0
3.8
0
0
8
0
0
4.4
0
0
3.4
0
0
4.8
0
0
17.9
0
0
13.7
0
0
6.8
0
0
12.5
0
0
16
0
0
16.5
0
0
13
0
0
46.1
0
0
59.4
0
0
7.4
0
0
31.5
0
0
19.5
0
0
40.7
0
0
86.4
0
0
107.1
0
0
52.4
0
901.13
143.3
0
0
0
0
0
0
0
0
0
35
0
0
75
0
0
0
0
0
0
2
0
66.6
2
0
0
0
0
0
0
0
0
0
0
0
0
0
0
4.9
0
0
0
0
31.3

Figure 3 Correlation coefficients between degree andcorporate performance
In Figure 3, r1 and r2 means correlation coefficient
between profits and in-degree (T), profit and out-degree(C) respectively. And r3 and r4 is correlation coefficient
between sales and in-degree (T) and sales and outdegree (C) respectively. All of them are significant.
Basically the suppliers which have high value of outdegree (C) have strong control power for other suppliers.
And the suppliers which have high value of in-degree(T) should have specific skills of production andmanufacturing. From this result, higher degree, stronger
sales and profit is hold. Unfortunately, not only thevalue of all correlation coefficients between corporate
performance and degree out-degree (T) and in-degree(C) are very low, but also they are not significant.
Therefore, the authors calculated the structural
importance using the SNW model. The results areshown in Table 2.
The authors calculated the correlation coefficient
between the SNW and sales, profits. Not only the valueof the results between the SNW and sales are very low,
but also all of them are not significant. The results of the
SNW and profit can be illustrated as Figure 4.
In Figure 4, r5 and r6 means correlation coefficient
between out-degree (T) of the SNW and profit, andbetween in-degree (C) and profit respectively. All ofthem are significant. High value of out-degree meansstrong selling power. Therefore, stronger selling power,
more profit is hold. The suppliers which accept
investment higher should be considered as they are
under control of the investment companies. Therefore,
the prices of the orders comes from investmentcompanies may be very low. This is the considerable
reason to explain why correlation coefficient of r6 isnegative.

In Table 1, (T) and (C) means transaction and crossshare holding respectively. Generally the firm which has
high value of out-degree and in-degree of crossshareholdings, and the firm which has high value of outdegree and in-degree of transactions is an important
hypothesis in the network organization.
In order to identify the rational relationship between
degree and corporate performance, the authors
calculated the correlation coefficient among degree andsales, profits. It can be illustrated as Figure 3.

©ISAROB 2011

181

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 2 Structural importance of each firm in Yokokai
based on the SNW analysis
No

Out-degree(T) In-degree(T) Out-degree（C)In-degree（C)Criterion=0.54 Criterion=9.85 Criterion=2.99 Criterion=0.87

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

W
W
W
W
W
W
W
W
S
W
W
W
W
S
W

S
S
S
S
S
S
W
S
W
W
S
S
W
W
W

W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
S

W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
W
S
W
W
W
W
W
W
W
W
W
W
W

Figure 4 Correlation coefficients between the SNW andProfitpaper. Much more data are required for further analysis.
Correlation coefficient between sales and the results of
the SNW model is also needed to be analyzed more.
Moreover, the SNW is only applied into degree, more
analysis applying into other centrality indices such as
closeness and between need to be done in the near
future.
Acknowledgment: This research was partially
supported by the Ministry of Education, Culture, Sports,
Science, and Technology, Grant-in-Aid for Exploratory
Research, 21510171, 2009.

REFERENCES[1] Davidow, W. H. and Malone, M. S. (1992). The
Virtual Corporation, Harper Collins Publishers Inc.
[2] Imai, K. (1989). Firm’s group, In Imai K. and Komiya
R. (ed.) Japanese Firm, Tokyo University Publication(Japanese Edition).
[ 3 ] Freeman, L. C. (1978/1979). Centrality in Social
networks Conceptual clarification, Social Networks 1.
[4] Tyler J. R., Wilkinson D. M., and Huberman B. A.
(2003) Email as spectroscopy: Automated discovery ofcommunity structure within organizations, in Proceedingsof the International Conference on Communities andTechnology. (Netherlands), Kluwer Academic Publishers.
[5] Ito, T. (2004). Quantitative Analysis of a Firm’s
Relationship in the Keiretsu of Toyota group, Innovations
Through Information Technology , IRMA 2004
International Conference, New Orleans, USA, 1078-1079.
[6] Nieminen J. (1974) On centrality in a graph, Scandinavian Journal of Psychology 15, 1974, pp.322-336.
[7] Ito, T. and Sakamoto, M. (2005) Importance Analysis
of Each Firm in the Keiretsu of Toyota. Managing Modern
Organizations with Information Technology, IRMA 2005
International Conference, pp.930-933.
[8] JAPIA and APPC (2004) Japanese Automotive Parts
Industry,
Automotive Parts Publishing Company,(Japanese Edition).

IV. CONCLUSION AND FUTURE WORKS
The authors calculated the degree, one of thecentrality indices, and the structural importance using
the SNW model. In order to identify the determinants ofcorporate performance, correlation coefficients between
degree and sales and between degree and profit,
between the results of the SNW and sales and profit
have been measured. Degree and the SNW are both very
effective. And it is shown that these two methods arepractical for the empirical study in this paper.
Needless to say, some problems are still left. For
instance, the data collected is only one fiscal year in this©ISAROB 2011

182

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A centrality analysis between transactions and cross shareholdings in
Mazda’s Keiretsu
E. Niki1, R. Takida1, T Ito1, R. Mehta2, L. P. NG3, and M. Sakamoto 41 Dept. of Business Administration, Ube National College of Technology, Yamaguchi, Japan(Tel : 81-836 -35 -7115; Fax : 81-836 -35-7115)(niki@ube-k.ac.jp, ito@ube-k.ac.jp)
2 School of Management, New Jersey Institute of Technology, USA(Tel :1-973-596-6419; Fax :1-973-596 -3074)(Mehta@njit.edu)
3 Hoecheong Industries (Holding) Co., Ltd, China(Tel :86-595 -2208-5151; Fax :86-595 -2208-5188)(hcgroup@hoecheong.com)

4 Dept. of Computer Science and Systems Engineering, University of Miyazaki, Japan(Tel : 81 -985-58-7392; Fax : 81 -985 -58 -7392)(sakamoto@cs.miyazaki-u.ac.jp)
Abstract: Firm’s relationship is one of the important issues in the field of corporate management. The authors review
network literature related with the automotive industry, and focus their study on the firm’s relationship in the Keiretsu
of Mazda, Yokokai, in this paper. The authors collected the data of transactions, cross shareholdings, and sales from all
firms in Yokokai, and calculated the centrality index based on graph theory in order to discover the rational
relationship between transactions and sales m and between cross shareholdings and sales. Generally centrality
index are calculated based upon degree, betweenness and closeness indicators. Certa in co rrel atio n betw ee ntransactions and sales, and between cross shareholdings and sales have been found. Implications of these findings
are discussed. This paper provides a new perspective to find the determinants of the successful corporate management.
Keywords: degree, betweenness, closeness, transaction, cross shareholdings, the Keiretsu

research focuses on the firm’s relationship in thekeiretsu of Mazda.
The main contributions of this paper are: 1) To
uncover the correlation between transactions and sales,
and between cross shareholdings and sales, measures ofgraph theory such as centrality index, including degreeand betweenness are measured using computer software
called Ucinet. 2) Certain correlation between
transactions and cross shareholdings, and between
transactions and sales has been found. All of these new
findings have been discussed.
This paper is organized as follows. In Section 2 , weintroduce some basic concepts and explain the centrality
model including degree, closeness and betweenness.
Section 3 shows the measurement results, and discusses
the implications of the new findings. Finally in Section
4 we conclude by a summary of this paper.

I. INTRODUCTION
Firm’s relationship is one of the important issues in
the field of corporate management. The Keiretsu, as one
special type of group companies is known broadly today.
One of the Keiretsu’s characteristics is crossshareholdings. The purpose of cross shareholdings canbe expressed as follows [1].
The first is to secure long-term strong stockholders
for stabilization of corporate governance under the
regulation of financial policy. The second is to acquire
the latent profits of stock-holding brought by rising ofstock prices as a buffer to stabilize corporate
performance. And the third is to realize the highproductivity by attempting stabilization of the so -calledreciprocal business transactions.
However, only a few empirical studies to prove thecorrelations
betweentransactions
and
cross
shareholdings have been reported until now [2, 3 ]. And
further research is required in this area to improve
corporate management relationships and priorities.
Therefore, it is necessary to measure the quantitative
relationships among the firms, and verify the correlation
between transactions and cross shareholdings. This

©ISAROB 2011

II. M ETHOD
A graph consists of a set of points and a set of lines
connecting pairs of point. The point, which composes a
network, is called node, and the line, which connects
any two nodes directly, is called an edge in graph theory.

183

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

The shortest path linking a given pair of nodes is called
geodesic. A graph is a model with an undirecteddichotomous relation. In other words, a tie is present orabsent between each pair of nodes. The data consists ofvalued and directed relationship in which the strength orintensity of each tie is recorded. The authors collected
directed and valued data to compute the centrality ofeach firm.
1.

Betweenness is useful as an index of the potentialof the node for control of communication, and it is also
useful as an index of network structure.

1.3

The third is an index of closeness that is a distancefrom pk to all other node linking with pk directly or
indirectly. It expresses the distance conditions from anode pk to other nodes in a network. Closeness index
may be useful when measures based upon independence
or efficiency is desired.
Closeness index is excluded because all nodes in the
networks of transactions and cross shareholdings are notconnected with each other. Therefore the distanceamong firms is infinite. It could not be calculated
technically.

Outline of the centrality model

Freeman proposed centrality of each node fromthree viewpoints of degree, betweenness and closeness

[4].
1.1

DegreeFreeman defines a degree of node pk is the numberof node connect with it directly. Therefore, Freeman
calculated it using the measurement developed byNieminen [5]. The centrality index of degree of node k
can be defined in the following way.

2. Data collection
Data of transactions and cross shareholdings in the
Keiretsu of Mazda has been collected from thepublications and investigation by interviews [8].
The process of data-collection can be expressed asfollows.
Step 1: Determine the boundary of the network;
Step 2: Collect the data of transactions and crossshareholdings. Basically the data is a percent value ofthe transactions or stock holdings between two firms.
And input the data into a matrix table.
Step 3: Remove singletons in the matrix table.
The data of each cell in the matrix meanstransactional relationship which the firm in column
accepts auto -components from other suppliers in row of
the network of transactions, or capital relationships
which the firm in row invests in stocks to other firms incolumn of the network of cross shareholdings.
The total number of the firms which hold capital
relationship with each other in the network of crossshareholdings is 230. It includes 177 firms, 42 financial
institutions, and 11 carmakers. The total number of thefirms in the network of transactions is 188, including
177 firms and 11 carmakers.
Part of them is isolated in the network. It means thatthese companies have no any relation with othercompanies. They are called singletons. All of thesingletons are removed from the networks oftransactions and cross shareholdings because no
essential difference exists between the entire networkand the entire network excluded singletons. The
network of transactions and cross shareholdings of
Mazda can be illustrated as Figure 1.

n

where

CD ( pk ) = ∑ a( pi , p k ) …… (1)i =1a(pi, pk)=1 if and only if p i and pk connected by a linea(pi, pk)=0 otherwise
The number of nodes adjacent to a given node in asymmetric network is the degree of that actor. For
asymmetric network the in-degree of a node pk is thenumber of ties received by pk and the out-degree is thenumber of ties initiated from pk.
The degree means the proportion of other nodes
that are adjacent to pk and is viewed as important index

of its potential communication activity.
1.2

Betweenness
The index of betweenness is calculated as aprobability that node pk falls on a randomly selected
geodesic linking p i and p j. Freeman defines thecentrality index of betweenness of node k as follows [6].
n n g (p )nnijkCB ( pk ) = ∑∑= ∑∑bij ( pk ) …… (2)
gij
i
i
j
j

where
㨕? 㨖? 㨗㧧gij(pk): The number of geodesics linking p i and p j that
contains pk
gij: The geodesics linking p i and p j©ISAROB 2011

Closeness

184

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 1 Top ten of the centrality index in thetransactions network of Mazda in 2004
(1) Top ten of in-degree㪥㫆㩺㪥㪸㫄㪼㩷㫆㪽㩷㪽㫀㫉㫄㫊㪈 㪤㪸㫑㪻㪸㩷㪤㫆㫋㫆㫉㩷㪚㫆㫉㫇㫉㪸㫋㫀㫆㫅㪎㪇 㪛㪼㫅㫊㫆㩷㪚㫆㫉㫇㪅㪉㪌 㪡㪸㫋㪺㫆㩷㪣㫋㪻㪅㪋㪋 㪟㫀㫋㪸㪺㪿㫀㪃㩷㪣㫋㪻㪅㪌㪏 㪘㫀㫊㫀㫅㩷㪘㪠㪃㩷㪣㫋㪻㪉㪇 㪚㪸㫃㫊㫆㫅㫀㪺㩷㪢㪸㫅㫊㪼㫀㩷㪚㫆㫉㫇㪅㪌㪉 㪤㫀㫋㫊㫌㪹㫀㫊㪿㫀㩷㪜㫃㪼㪺㫋㫉㫀㪺㩷㪚㫆㫉㫇㫆㫉㪸㫋㫀㫆㫅㪈㪐 㪢㪸㫐㪸㪹㪸㩷㪠㫅㪻㫌㫊㫋㫉㫐㩷㪚㫆㪅㪃㩷㪣㫋㪻㪅㪉㪎 㪪㫋㪸㫅㫃㪼㫐㩷㪜㫃㪼㪺㫋㫉㫀㪺㩷㪚㫆㪅㪃㩷㪣㫋㪻㪅㪉㪉 㪢㫌㫉㪸㫉㫀㫆㫅㩷㪚㫆㪅㪃㩷㪣㫋㪻(2) Top Ten of Out-degree㪥㫆㩺㪥㪸㫄㪼㩷㫆㪽㩷㪽㫀㫉㫄㫊㪦㫌㫋㪄㪻㪼㪾㫉㪼㪼㪋㪍 㪫㪿㪼㩷㪝㫌㫉㫌㫂㪸㫎㪸㩷㪼㫃㪼㪺㫋㫉㫀㪺㩷㪚㫆㪅㪃㩷㪣㫋㪻㪅㪈㪇㪇㪉㪏 㪫㪸㫂㪸㫋㪸㩷㪚㫆㫉㫇㫆㫉㪸㫋㫀㫆㫅㪐㪐㪅㪐㪐㪋㪋 㪟㫀㫋㪸㪺㪿㫀㪃㩷㪣㫋㪻㪅㪐㪐㪅㪐㪍㪋㪊 㪧㫀㫆㫃㪸㫏㪅㩷㪠㫅㪺㪅㪐㪐㪅㪐㪍㪉㪐 㪫㫆㫂㫐㫆㩷㪩㫆㫂㫀㩷㪺㫆㪅㪃㩷㪣㫋㪻㪅㪐㪌㪅㪇㪋㪍㪈 㪠㫄㪸㫊㪼㫅㩷㪜㫃㪼㪺㫋㫉㫀㪺㩷㫀㫅㪻㫌㫊㫋㫉㫀㪸㫃㩷㪚㫆㪅㪃㩷㪣㫋㪻㪅㪐㪉㪅㪍㪏 㪡㪸㫇㪸㫅㩷㪚㫃㫀㫄㪸㫋㪼㩷㪪㫐㫊㫋㪼㫄㫊㩷㪚㫆㫉㫇㫆㫉㪸㫋㫀㫆㫅㪐㪉㪊㪋 㪥㫀㫊㫊㪿㫀㫅㪹㫆㩷㪠㫅㪻㫌㫊㫋㫉㫀㪼㫊㪃㩷㪠㫅㪺㪅㪐㪈㪅㪍㪊㪎㪈 㪫㫆㫂㪸㫀㩷㪩㫀㫂㪸㩷㪚㫆㪅㪃㩷㪣㫋㪻㪅㪐㪈㪅㪊㪋 㪢㪼㫐㫃㪼㫏㩷㪚㫆㫉㫇㫆㫉㪸㫋㫀㫆㫅㪅㪐㪇㪅㪍(3) Top Ten of Betweenness

Figure 1 Network of transactions (upper) and crossshareholdings (lower) of Mazda in 2004
In Figure 1, a dot represents a company and an edgemeans a relation of transactions or cross shar eholdings.
89 companies and 95 companies are included in the
networks of transactions and cross shareholdings
respectively.

㪥㫆㩺㪥㪸㫄㪼㩷㫆㪽㩷㪽㫀㫉㫄㫊㪙㪼㫋㫎㪼㪼㫅㫅㪼㫊㫊㪎㪇 㪛㪼㫅㫊㫆㩷㪚㫆㫉㫇㪅㪊㪌㪅㪌㪊㪏 㪡㪸㫇㪸㫅㩷㪙㫉㪸㫂㪼㩷㪠㫅㪻㫌㫊㫋㫉㫀㪸㫃㩷㪚㫆㪅㪃㩷㪣㫋㪻㪅㪉㪏㪉㪇 㪚㪸㫃㫊㫆㫅㫀㪺㩷㪢㪸㫅㫊㪼㫀㩷㪚㫆㫉㫇㪅㪈㪈㪍㪌 㪟㫀㫂㪸㫉㫀㩷㪪㪼㫀㫂㫆㩷㪚㫆㪅㪃㩷㪣㫋㪻㪅㪏㪋㪋 㪟㫀㫋㪸㪺㪿㫀㪃㩷㪣㫋㪻㪅㪍㪎㪊 㪫㫆㫐㫆㫋㪸㩷㪞㫆㫊㪼㫀㩷㪚㫆㪅㪃㩷㪣㫋㪻㪅㪍㪎㪉 㪫㫆㫐㫆㩷㪫㫀㫉㪼㩽㪩㫌㪹㪹㪼㫉㩷㪚㫆㪅㪃㩷㪣㫋㪻㪅㪋㪍㪈 㪠㫄㪸㫊㪼㫅㩷㪜㫃㪼㪺㫋㫉㫀㪺㩷㫀㫅㪻㫌㫊㫋㫉㫀㪸㫃㩷㪚㫆㪅㪃㩷㪣㫋㪻㪅㪉㪅㪌㪌㪎 㪘㫀㫊㪸㫅㩷㪠㫅㪻㫌㫊㫋㫉㫐㩷㪚㫆㪅㪃㩷㪣㫋㪻㩷㪟㫀㫉㫆㫊㪿㫀㫄㪸㩷㪪㪸㫃㪼㫊㪉㪈㪐 㪢㪸㫐㪸㪹㪸㩷㪠㫅㪻㫌㫊㫋㫉㫐㩷㪚㫆㪅㪃㩷㪣㫋㪻㪅㪉

III. M EASUREM ENT AND ANALYSISThe authors calculated the centrality including
degree and betweenness of each firm in the networks oftransactions and cross shareholdings of Mazda. The top
ten of each centrality index in 2004 are summarized inTable1.
It means the volume of the firm’s purchasing poweris big if the value of in-degree is high. If the value ofout-degree is high, it means the firm has strong selling
power in its network. And if the firm has high value of
betweenness, it means the firm purchases and sells its
parts simultaneously.
In Table 1, only Hitachi has high values of in-degree,
out-degree, and betweenness. It means Hitachi plays animportant role in Yokokai because it has extremely
power of centrality.
The second finding is that part of the firms such as
Denso and Calsonic Kansei Corp. are included in indegree and betweenness both. It means that certain
correlation between in-degree and betweenness exists inthe transaction network.

©ISAROB 2011㪠㫅㪄㪻㪼㪾㫉㪼㪼㪐㪇㪈㪅㪈㪊
㪉㪇㪌㪅㪉㪋
㪌㪋㪅㪎㪋
㪉㪉㪅㪈㪋
㪈㪌㪅㪊
㪈㪉㪅㪎
㪈㪈㪅㪌㪋
㪈㪈㪅㪈㪈
㪐㪅㪉
㪏㪅㪏

The third finding is that part of the firms are selling
their parts in the network of transactions because the top
ten firms in the group of in-degree is quite different withthe top ten firms in the group of ou t-degree.
In order to discover the relationship mentioned
above, the relationship between in-degree, out-degree,
betweenness and sales is required. The results areshown in Table 2.
Form Table 2, we can get three new findings asfollows.
The first is that the correlation coefficient between
out-degree and in-degree is -0.499 (p=0.01). It means
the more the firms sell, the less the firms purchase. In
other words, it means most of the parts sold out in the
network are produced out of this group. The second isthat certain correlation between transactions and cross

185

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 1 The correlation coefficient of centrality between network of transaction and cross shareholdings #㪦㫌㫋㪄㪻㪼㪾㫉㪼㪼 㪠㫅㪄㪻㪼㪾㫉㪼㪼 㪙㪼㫋㫎㪼㪼㫅㫅㪼㫊㫊 㪦㫌㫋㪄㪻㪼㪾㫉㪼㪼 㪠㫅㪄㪻㪼㪾㫉㪼㪼 㪙㪼㫋㫎㪼㪼㫅㫅㪼㫊㫊 㪪㪸㫃㪼㫊
䋨㪫㪀
䋨㪫㪀
䋨㪚㪀
䋨㪚㪀
䋨㪚㪀
䋨㪫㪀
㪈
㪦㫌㫋㪄㪻㪼㪾㫉㪼㪼㪄䋨㪫㪀㪏㪐㪄㪅㪋㪐㪐㪁㪁
㪈㪠㫅㪄㪻㪼㪾㫉㪼㪼
㪇
㪄
䋨㪫㪀
㪏㪐
㪏㪐
㪇㪅㪈㪉㪎
㪇㪅㪇㪉㪍
㪈
㪙㪼㫋㫎㪼㪼㫅㫅㪼㫊㫊
㪇㪅㪉㪊㪎
㪇㪅㪏㪈㪈
㪄
䋨㪫㪀
㪏㪐
㪏㪐
㪏㪐
㪄㪅㪊㪌㪋㪁㪁
㪅㪎㪉㪊㪁㪁
㪇㪅㪉㪉㪍㪈㪦㫌㫋㪄㪻㪼㪾㫉㪼㪼
㪇㪅㪇㪇㪍
㪇
㪇㪅㪇㪏㪍
㪄
䋨㪚㪀
㪌㪐
㪌㪐
㪌㪐
㪐㪌
㪅㪌㪋㪉㪁㪁
㪄㪅㪊㪏㪇㪁㪁
㪇㪅㪈㪈㪍㪄㪅㪉㪈㪏㪁
㪈㪠㫅㪄㪻㪼㪾㫉㪼㪼
㪇
㪇㪅㪇㪇㪊
㪇㪅㪊㪏㪊
㪇㪅㪇㪊㪋
㪄
䋨㪚㪀
㪌㪐
㪌㪐
㪌㪐
㪐㪌
㪐㪌
㪅㪋㪈㪋㪁㪁
㪇㪅㪇㪊㪄㪇㪅㪇㪈㪌㪅㪉㪋㪊㪁㪇㪅㪈㪈㪙㪼㫋㫎㪼㪼㫅㫅㪼㫊㫊
㪇㪅㪏㪉
㪇㪅㪐㪇㪎
㪇㪅㪇㪇㪈
㪇㪅㪇㪈㪏
㪇㪅㪊㪊㪎
㪄
䋨㪚㪀
㪌㪐
㪌㪐
㪌㪐
㪐㪌
㪐㪌
㪐㪌
㪅㪋㪋㪈㪁㪁
㪅㪊㪐㪉㪁㪁
㪅㪎㪉㪎㪁㪁
㪅㪎㪎㪇㪁㪁
㪄㪇㪅㪈㪐
㪄㪇㪅㪈㪈
㪈
㪪㪸㫃㪼㫊
㪇
㪄
㪇㪅㪈㪉
㪇
㪇㪅㪇㪇㪈
㪇
㪇㪅㪋㪌㪊
㪍㪏
㪍㪏
㪍㪏
㪋㪐
㪋㪐
㪋㪐
㪍㪏
# C means cross shareholdings network and T means transactions network. These data means correlation
ratio/probability/number of sample. **p<0.01 (two-sided); *p<0.05 (two-sided)shareholdings among out-degree, in-degree andbetweenness are significant. Especially the correlation
coefficient between out-degree (C) and in-degree (T)reach 0.723(p=0.01). It means the more investment onother suppliers the stronger purchasing power they have.
And from the fact that correlation coefficient between
in-degree (C) and out-degree (T), we can say the moreinvestment the firms accept the stronger selling power
the firms hold. The third is that almost of the centrality
indices have strong impacts on sales, one of theimportant indices of corporate performance.
One of the interest issues is that the correlation
coefficient between out-degree (T) and sales, between
in-degree (C) and sales are not significant. T he main
reason is considered as most of the parts are notpurchased by the firms in Mazda’s group.

Acknowledgment: This research was partially
supported by the Ministry of Education, Culture, Sports,
Science, and Technology, Grant-in-Aid for Exploratory
Research, 21510171, 2009.
REF ERENCES[ 1] Tsuru K. (1994) A system of Market Economy inJapan, Kodansha Ltd (Japanese Edition).
[2] Ito T and Sakamoto M. (2005) Importance analysis
of each firm in the Keiretsu of Toyota, Managing
Modern Organizations with Information Technology,
IRMA 2005 International Conference p p . 9 3 0 - 9 3 3 ,San Diego, USA.
[3] Fukuoka S, Ito T, Passerini K. and Sakamoto M. (2006)
An Analysis between Transaction and Cross Shareholdingsin the Keiretsu of Nissan, IBIMA International conference
on Managing Information in the Digital Economy: Issues& Solutions, pp. 163-169, Bonn Germany.
[4] Freeman L.C. (1979) Centrality in Social networks
Conceptual clarification, Social Networks 1, 1979,
pp.215-239.
[5] Nieminen J. (1974) On centrality in a graph, Scandi navian Journal of Psychology 15, 1974, pp.322-336.
[7] Freeman, L. C. (1977) A set of Measure of Centrality
Based on betweenness, Sociometry, 40, 1977, pp.35-41.
[8] JAPIA and APPC (2004) Japanese Automotive Parts
Industry, Automotive Parts Publishing Company,(Japanese Edition).

IV. CONCLUSION AND F UTURE WORKS
The authors calculated the centrality of network
index of Yokokai, and find strong correlation between

transactions and cross shareholdings. To discover theeffective network structure, more research such as
capacity analysis and comparative research of different
networks such as Toyota and Honda are required in thenear future.

©ISAROB 2011

186

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Bottom-Up Pyramid Cellular Acceptors with Four-Dimensional Layers
Yasuo Uchida1 , Takao Ito1 , Makoto Sakamoto2 , Takashi Ide2 , Kazuyuki Uchida2 ,Ryoju Katamune2 , Hiroshi Furutani2 ,Michio Kono2 ,Satoshi Ikeda2 ,and Tsunehiro Yoshinaga3
1

Ube National College of Technology 2 University of Miyazaki 3 Tokuyama College of Technology

Abstract

On the other hand, the question of whether processing
four-dimensional digital patterns is much diﬃcult
than two- or three-dimensional ones is of great interest from the theoretical and practical standpoints.
Thus, the study of four-dimensional automata as
the computasional model of four-dimensional patternprocessing has been meaningful. From this point ofview, we are interested in four-dimensional automata.
In this paper, we study about bottom-up pyramid cellular acceptors with four-dimensional layers,
and deal with the following problems (which is one ofthe open problems) : Does the class of sets acceptedby deterministic bottom-up pyramid cellular acceptors
with four-dimensional layers include the class of sets
accepted by deterministic four-dimensional finite automata [3-7]? This paper shows that the class of sets
accepted by four-dimensional finite automata is incomparable with the class of sets accepted by deterministicbottom-up pyramid cellular acceptors which operate
in time of order lower than the diameter of the input.

In 1967, M.Blum and C.Hewitt first proposed twodimentional automata as a computaional model of
two-dimentional pattern processing. Since then, many
researchers in this field have been investigating many
properties of two- or three-dimentional automata. In
1997, C.R.Dyer and A.Rosenfeld introduced an acceptor on a two-dimensional pattern (or tape), called
the pyramid cellular acceptor, and demonstrated thatmany useful recognition tasks are executed by pyramid cellular acceptors in time proportional to the logarithm of the diameter of the input. They also introduced a bottom-up pyramid cellular acceptor which
is a restricted version of the pyramid cellular acceptor, and proposed some interesting open problems
about bottom-up pyramid cellular acceptors. On
the other hand, we think that the study of fourdimensional automata has been meaningful as thecomputational model of four-dimensional infomation
processing such as computer animation, moving picture processing, and so forth[9]. In this paper,we investigate about bottom-up pyramid cellular accptors
with four-dimensional layers, and show their some accepting powers.

2

Let Σ be a finite set of symbols.
A threedimensional tape over Σ is a four-dimensional rectangular array of elements of Σ. The set of all the fourdimensional tapes over Σ is denoted by Σ(4) . Givena tape x ∈ Σ(4) , for each j (1 ≤ j ≤ 4), we let lj (x)be the length of x along the jth axis. The set of allx ∈ Σ(4) with l1 (x) = n1 , l2 (x) = n2 , l3 (x) = n3 , andl4 (x) = n4 is denoted by Σ(n1 ,n2 ,n3 ,n4 ) . When 1 ≤ ij
≤ lj (x) for each j (1 ≤ j ≤ 4), let x (i1 , i2 , i3 , i4 ),denote the symbol in x with coordinates (i1 , i2 , i3 , i4 ).
Furthermore, we define x [(i1 , i2 , i3 , i4 ), (i1 ’, i2 ’, i3 ’,i4 ’)], when i ≤ ij ≤ ij ’ ≤ lj (x) for each integer j (1 ≤j ≤ 4), as the four-dimensional input tape y satisfying
the following (i) and (ii) : (i) for each j (1 ≤ j ≤ 4),
lj (y) = ij ’ - ij + 1; (ii) for each r1 , r2 , r3 ,r4 (1 ≤ r1 ≤l1 (y), 1 ≤ r2 ≤ l2 (y), 1 ≤ r3 ≤ l3 (y),1 ≤ r4 ≤ l4 (y)),
y (r1 , r2 , r3 , r4 ) = x (r1 + i1 - 1, r2 + i2 - 1, r3 + i3
- 1, r4 + i4 - 1).
For each x ∈ Σ(n1 ,n2 ,n3 ,n4 ) and for each 1 ≤ i1 ≤
n1 , 1 ≤ i2 ≤ n2 , 1 ≤ i3 ≤ n3 , 1 ≤ i4 ≤ n4 , x[(i1 , 1,1, i4 ), (i1 , n2 , n3 , i4 )], x[(1, i2 , 1, i4 ), (n1 , i2 , n3 , i4 )],x[(1, 1, i3 , i4 ), (n1 , n2 , i3 , i4 )], x[(i1 , 1, i3 , i4 ), (i1 , n2 ,i3 , i4 )], and x[(1, i2 , i3 , i4 ), (n1 , i2 , i3 , i4 )] are called

Key Words : cellular automaton, diameter, finite automaton, four-dimension, pattern recognition.

1

Introduction

M.Blum and C.Hewitt first proposed twodimensional automata as a computational modelof two-dimensional pattern processing, and investigated their pattern recognition abilities [1]. Since
then, many researchers in this field have been investigating a lot of properties about automata on
a two- or three-dimensional tape. In [2], C.R.Dyer
and A.Rosenfeld introduced an acceptor on a twodimensional pattern (or tape), called the pyramid
cellular acceptor, and demonstrated that many usefulrecognition tasks are executed by the pyramid cellular
acceptors in time proportional to logarithm of thediameter of the input.
They also introduced abottom-up pyramid cellular acceptor, which is a restricted version of the pyramid cellular acceptor, andproposed some interesting open problems about it.

©ISAROB 2011

Definition

187

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

the i1 th (2-3) plane of each x, the i2 th (1-3) plane ofeach x, the i3 th (1-2) plane of each x, the i1 th row on
the i3 th (1-2) plane of each x, and the i2 th column on
the i3 th (1-2) plane of each x.

way that x (i1 , i2 , i3 , i4 ) is stored at the cell of the i1 th
row and the i2 th column on the i3 th plane of the i4 th
four-dimentional rectangular array, and the other cells
are initialized to a quiescent state qs (∈QN −QT −A).
As usual, we let δ (qs , qs , qs , qs , qs , qs , qs , qs , qs , qs , qs ,qs , qs , qs , qs , qs , qs ) = qs . The input is accepted if andonly if the root cell ever enters an accepting state. This
4-U P CA is called deterministic. A nondeterministicbottom-up pyramid cellular acceptor is defined as a 4QN
U P CA using δ : Q17
instead of the state tranN → 2
sition function of the deterministic 4-U P CA. Below,
we denote a deterministic 4-U P CA by 4-DU P CA,
and a nondeterministic 4-U P CA by 4-N U P CA. A
4-DU P CA (or 4-N U P CA) operates in time T (n) if
for every four-dimensional tape of size 2n × 2n × 2n
× 2n (n ≥ 0) it accepts the four-dimensional tape,then there is an accepting computation which uses
no more than time T (n). By 4-DU P CA (T (n)) [4N U P CA(T (n))] we denote a T (n) time-bounded 4DU P CA [4-N U P CA] which operates in time T (n).

We next give some basic concepts about bottom-up
pyramid cellular acceptors with four-dimensional layers [7]. A bottom-up pyramid cellular acceptor withf our-dimensional layers (3-U P CA) is a pyramidal
stack of four-dimensional arrays of cells in which thebottom four-dimensional layer has size 2n × 2n × 2n
× 2n (n ≥ 0), the next lowest 2n−1 × 2n−1 × 2n−1
× 2n−1 , and so forth, the (n + 1)st four-dimensional
layer consisting of a single cell, called the root. Eachcell is defined as an identical finite-state machine, M= (QN , QT , δ, A), where QN is a nonempty, finite
set of states, QT ⊆ QN is a finite set of input states,A ⊆ QN is the set of accepting states, and δ : Q17
N→ QN is the state transition f unction, mapping thecurrent states of M and its sixteen son cells in a 2× 2 × 2 × 2 block on the four-dimensional layer below into M ’s next state. Let c be some cell on the(i + 1)st four-dimensional layer, and let c (U N W P ),c (U N W F ), c (U SW P ), c (U SW F ), c (U SEP ), c
(U SEF ), c (U N EP ), c (U N EF ), c (DN W P ), c
(DN W F ), c (DSW P ), c (DSW F ), c (DSEP ), c
(DSEF ), c (DN EP ), and c (DN EF ) be sixteen son
cells (on the ith four-dimensional layer) of c, wherec(U N W F ) is c’s upper northwest son in the most future direction, c (DN W P ) is c’s lower northwest son
in the most past direction, etc. For example, if the coordinates of c on the (i + 1)st layer is (1, 1, 1, 1) ((2n ,2n , 2n , 2n )), the coordinates of sixteen son cells of con the ith layer c (U N W P ), c (U N W F ) ,c (U SW P ),c (U SW F ), c (U SEP ), c (U SEF ), c (U N EP ), c
(U N EF ), c (DN W P ), c (DN W F ), c (DSW P ), c
(DSW F ), c (DSEP ), c (DSEF ), c (DN EP ), and c(DN EF ) are (1, 1, 1, 1), (1, 1, 1, 2), (1, 2, 1, 1), (1,2, 1, 2), (1, 2, 2, 1), (1, 2, 2, 2), (1, 1, 2, 1), (1, 1, 2,
2), (2, 1, 1, 1), (2, 1, 1, 2), (2, 2, 1, 1), (2, 2, 1, 2), (2,2, 2, 1), (2, 2, 2, 2), (2, 1, 2, 1), (2, 1, 2, 2), ((2n −1, 2n − 1, 2n − 1, 2n − 1), (2n − 1, 2n − 1, 2n − 1,2n ), (2n − 1, 2n , 2n − 1, 2n − 1), (2n − 1, 2n , 2n −1, 2n ), (2n − 1, 2n , 2n , 2n − 1), (2n − 1, 2n , 2n , 2n ),(2n − 1, 2n − 1, 2n , 2n − 1), (2n − 1, 2n − 1, 2n , 2n ),(2n , 2n − 1, 2n − 1, 2n − 1), (2n , 2n − 1, 2n − 1, 2n ),(2n , 2n , 2n − 1, 2n − 1), (2n , 2n , 2n − 1, 2n ), (2n , 2n ,2n , 2n − 1), (2n , 2n , 2n , 2n ), (2n , 2n − 1, 2n , 2n −1), (2n , 2n − 1, 2n , 2n )),respectively. Then qc (t + 1)
= δ (qc (t), qc(U N W P ) (t), qc(U N W F ) (t), qc(U SW P ) (t),qc(U SW F ) (t), qc(U SEP ) (t), qc(U SEF ) (t), qc(U N EP ) (t),qc(U N EF ) (t), qc(DN W P ) (t), qc(DN W F ) (t), qc(DSW P )(t), qc(DSW F ) (t), qc(DSEP ) (t), qc(DSEF ) (t), qc(DN EP )(t)), qc(DN EF ) (t)), where for example qc (t) means thestate of c at time t. At time t = 0, the input tape(4)x ∈ QT [l1 (x) = l2 (x) = l3 (x) = l4 (x) = 2n , n ≥0] is stored as the initial states of the bottom fourdimensional layer, henceforth called the base, in such a©ISAROB 2011

We next introduce a four-dimensional finite automaton [8]. A f our-dimensional f inite automaton(4-F A) is a foure-dimensional Turing machine with noworkspace. A 4-F A M has a read-only four-dimensional tape with boundary symbols #’s, finite control,
and an input head. The input head can move in eight
direction − east, west, south, north, up, down, future,or past − unless it falls oﬀ the input tape. Formally,
M is defined by the 5-tuple M = (K, Σ ∪ {#}, δ,q0 , F ), where K is a finite set of states, Σ is a finite
set of input symbols, # is the boundary symbol (notin Σ), δ : K × (Σ ∪ {#}) → 2K×{E,W,S,N,U,D,F,P,H}
is the state transition f unction, where E, W , S, N ,U , D, F , P , and H represent the move directions of
the input head − east, west, south, north, up, down,
future, past, and no move, respectively, q0 ∈ K is
the intial state, and F ⊆ K is the set of accepting
states. The action of M is similar to that of the onedimensional (or two-dimensional) finite automaton [4],except that the input head of M can move in eight directions. That is, when an input tape x ∈ Σ(4) withboundary symbols is presented to M , M starts in its
initial state q0 with the input head on x (1, 1, 1, 1),
and determines the next state of the finite control andthe move direction of the input head, depending on
the present state of the finite control and the symbol
read by the input head. We say that M accepts thetape x if it eventually enters an accepting state. We
denote a deterministic 4-F A [nondeterministic 4-F A]
by 4-DF A [4-N F A].
We let each sidelength of each input tape of threeautomata, throughout this paper, be equivalent. We
denote the set of all four-dimensional tapes acceptedby M by T (M ). Define £ [4-DU P CA] = [T | T (M )is accepted by some 4-DU P CA M }. £ [4-N U P CA],£ [4-DF A], etc. are defined similarly.
Finally, we give definition of diameter. Given a

188

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

subset S of a tape x ∈ Σ(4) , we can define its extent
in a given direction θ as the length of its projection
on a plane in that direction. Here the length of a
projection is the distance between its farthest apartnonzero values. Thus the extent of S is the distance
between a pair of parallel planes perpendicular to θthat just bracket S. The diameter of S is defined as
its extent in any direction.

3

We consider the cases when the tapes in W (n)are presented to B. Let c be the cell which is situated at the first row, the first column, the firstplane, and the first three-dimensional rectangulararray in the nth layer (i.e., the layer just below
the root cell). (Note that there are eight cells in
the nth layer.) For each x in W (n) such thatx[(1, 1, 1, 1), (2n−1 , 2n−1 , 2n−1 , 2n−1 )] ∈ W 0 (n), andfor each r ≥ 1, let qr (x) be the state of c at time rwhen x is presented to B. Then the following proposition must hold.

Results

In this section, we show that the class of sets accepted by 4-DFA’s is imcomparable with the class ofsets accepted by 4-DUPCA’s which operate in time oforder lower than the diameter of the input. It has often
been noticed that we can easily get several propertiesof four-dimensional automata by directly applying theresults of one- or two-dimensional case, if each sidelength of each four-dimentional imput tape of these
automata is not equivalent. So we let each sidelengthof each input tape, throughout this paper, be equivalent in order to increase the theoretical interest.

Proposition 3.1. Let x, y be two diﬀerent tapes in
W (n) such that bothx[(1, 1, 1, 1), (2n−1 , 2n−1 , 2n−1 , 2n−1 )]andy[(1, 1, 1, 1), (2n−1 , 2n−1 , 2n−1 , 2n−1 )]are in W 0 (n) andx4[(1, 1, 1, 1), (2n−1 , 2n−1 , 2n−1 , 2n−1 )]6= y[(1, 1, 1, 1), (2n−1 , 2n−1 , 2n−1 , 2n−1 )].
Then, (q1 (x), q2 (x), . . . , qT (n) (x))= (q1 (y), q2 (y), . . . , qT (n) (y)).
[Proof : For suppose that (q1 (x), q2 (x), . . . , qT (n) (x))= (q1 (y), q2 (y), . . . , qT (n) (y)). We consider two tapesz, z 0 in W (n) such that

Lemma 3.1. Let T1 = { x ∈ { 0,1 }(4) | ∃ n ( n ≥1) [`1 (x) = `2 (x) = `3 (x) = `4 (x) = 2n ] and x (2n−1 ,2n−1 , 2n−1 , 2n−1 ) = 1 }. Then,(1) T1 (x) ∈/ £ [4-DFA], and(2) T1 (x) ∈ £ [4-DUPCA(n)].

(i) z [(1,1,1,1), (2n−1 , 2n−1 , 2n−1 , 2n−1 )]= x [(1,1,1,1), (2n−1 , 2n−1 , 2n−1 , 2n−1 )]andz 0 [(1,1,1,1), (2n−1 , 2n−1 , 2n−1 , 2n−1 )]= y [(1,1,1,1), (2n−1 , 2n−1 , 2n−1 , 2n−1 )],Proof : The Proof of (1) is similar to that of Theorem 3 in [7]. On the other hand, by using the same
technique as in the proof of Lemma 1 in [6], we can getPart (2) of the lemma.
(ii) the part of z except for z [(1, 1, 1, 1), (2n−1 , 2n−1 ,2n−1 , 2n−1 )] is identical with the part z 0 except
for z 0 [(1, 1, 1, 1), (2n−1 , 2n−1 , 2n−1 , 2n−1 )],Lemma 3.2. Let T2 = {x ∈ {0, 1}(4) | ∃n (n ≥ 1)[`1 (x) = `2 (x) = `3 (x) = `2 (x) = 2n ] and x [(1, 1, 1, 1),(2n , 2n , 2n , 1)] = x [(1, 1, 1, 2n ), (2n , 2n , 2n , 2n )] }. Let
T (n) be a time function such that limn→∞ [T (n)/22n ]= 0. Then,(1) T2 ∈ £[4-DFA],and(2) T2 ∈/ £[4-DUPCA(T (n))].

(iii) z[(1, 1, 1, 1), (2n , 2n , 2n , 1)]= z[(1, 1, 1, 2n ), (2n , 2n , 2n , 2n )].

and

By assumption, the root cell of B enters the samestates until time T (n), for the tapes z and z 0 . Since B
operate in time T (n) and z is in T2 , it follows that z 0 is
also accepted by B. This contradicts the fact that z 0 is
not in T2 .
]

Proof : It is obvious that there is a 4-DFA accepting
T2 , and so (1) of the lemma holds. Below, we prove (2).
Suppose that there is a 4-DUPCA B which accepts T2
and operates in time T (n), and that each cell of B has
k states. For each n ≥ 2, let
W (n) = {x ∈ {0, 1}(4) |`1 (x) = `2 (x) = `3 (x) =`4 (x) = 2n }, andLet t(n) be the number of diﬀerent sequences ofstates which c enters until time T (n). Clearly, t(n) ≤k T (n) . On the other hand (for any set S, let |S| de3(n−1)
note the number of elements of S.), |W 0 (n)| = 22.
2nSince limn→∞ T (n)/2 = 0 (by assumption of thelemma), it follows that |W 0 (n)| > t(n) for lange n.
Therefore, it follows that for large n there must exist
two diﬀerent tapes x, y in W (n) such that

W 0 (n) = {x ∈ {0, 1}(4) |`1 (x) = `2 (x) = `3 (x) = `4 (x)= 2n−1(i) both x[(1, 1, 1, 1), (2n−1 , 2n−1 , 2n−1 , 2n−1 )]and y[(1, 1, 1, 1), (2n−1 , 2n−1 , 2n−1 , 2n−1 )]and in W’(n),& x[(1, 1, 1, 1), (2n−1 , 2n−1 , , 2n−1 , 1)] ∈ {0, 1}(4)& x[(1, 1, 1, 2), (2n−1 , 2n−1 , 2n−1 , 2n−1 )] ∈ {0}(4) }.

(ii) x[(1, 1, 1, 1), (2n−1 , 2n−1 , 2n−1 , 2n−1 )]6= y[(1, 1, 1, 1), (2n−1 , 2n−1 , 2n−1 , 2n−1 )], and©ISAROB 2011189

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011(iii) (q1 (x), q2 (x), . . . , qT (n) (x))= (q1 (y), q2 (y), . . . , qT (n) (y)).

[6] K.Inoue and I.Takanami, A note on bottom-up
pyramid acceptors, Information Processing Letters Vol.8, No.1, pp.34-37, 1979.

This contradicts the above Proposition 3.1, andthus the Part (2) of the lemma holds.
[7] M.Sakamoto, S.Nogami, K.Inoue, and M.Kono,
A relationships between the accepting powers of three-dimensional finite automata andtime-bounded bottom-up pyramid cellular acceptors with three-dimensional layers, Trans. of
SCI(Japan), Vol.17, No.10, pp.451-458, 2004.

From Lemmas 3.1 and 3.2, we can get the following
theorem.
Theorem 3.1. Let T (n) be a time function such thatlimn→∞ [T (n)/22n ] = 0 and T (n) ≥ n(n ≥ 1). Then
£ [4-DFA] is imcomparable with £ [4-DUPCA(T (n))].

[8] H.Taniguchi, K.Inoue, and I.Takanami, A noteon three-dimensional finite automata, Information Siences, Vol.26, pp.65-85, 1982.

Corollary 3.1. £[4-DFA] is incomparable with £ [4DUPCA(n)], which is the class of sets accepted by 4DUPCA’s operating in real time.

[9] Y.Uchida,
T.Ito,
H.Okabe,M.Sakamoto,
H.Furutani, and M.Kono, Four-dimentional
multi-inkdot finite automata, WSEAS Trans. on
Computers, Issue 9, Vol.7, pp.1437 - 1446, 2008.

Corollary 3.2. £ [4-DFA] is incomparable with £[4NUPCA(n)].

4Conclusion

In this paper, we investigated the accepting powers of bottom-up pyramid cellular acceptors with fourdimensional layers, and showed that the class of sets
accepted by 4-DFA’s is incomparable with the classof sets accepted by 4-DUPCA’s which operate in timeof order lower than the diameter of the input. It is
still inknown whether the class of sets accepted by
4-DUPCA’s includes the class of sets accepted by 4DFA’s.

References[1] M.Blum and C.Hewitt, Automata on a twodimensional tape, in IEEE Symposium on Switching and Automata Theory, pp.155-160, 1967.
[2] C.R.Dyer and A.Rosenfeld, Cellular pyramids forimage analysis, Technical Report TR-544, Computer Science Center, University of Maryland,
1977.
[3] K.Inoue, and A.Nakamura, Some properties of
two-dimensional on-line tessellation acceptors,
Information Sciences, Vol.13, pp.95-121, 1977.
[4] K.Inoue, I.Takanami, and A.Nakamura, A noteon two-dimensional finite automata, Information
Processing Letters, Vol.7, No.1, p.49, 1978.
[5] K.Inoue and I.Takanami, Remaks on twodimensional finite automata with multiplication
and bottom-up pyramid acceptors, Technical Report IECE of Japan, No.AL77-61, 1978.

©ISAROB 2011190

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Cooperating Systems of Four-Dimensional Finite Automata
Yasuo UCHIDA1 , Takao ITO1 , Makoto SAKAMOTO2 , Kazuyuki UCHIDA2 , Takashi IDE2 ,Ryoju KATAMUNE2 , Hiroshi FURUTANI2 , Michio KONO2 and Tsunehiro YOSHINAGA3
1

Ube National College of Technology 2 University of Miyazaki 3 Tokuyama College of Technology

Abstract

way four-dimensional finite automata (CS-SV4-FA) which
is a restricted version of CS-4-FA’s, and mainly investigate its several properties as four-dimensional language acceptors. The seven-way four-dimensional finite automaton [7] is a four-dimensional finite automaton [1] whose
input head can move east, west, south, north, up, down,
or in the future, but not in the past. The paper has six
sections in addition to this Introduction. Section 2 contains some definitions and notions. Section 3 investigatesa relationship between seven-way four-dimensional simple multihead finite automata (SV4-SPMHFA’s) and CSSV4-FA’s. It is shown that SV4-SPMHFA’s and CS-SV4FA’s are equivalent in accepting power if each sidelengthof each four-dimensional input tape of these automata isequivalent. Section 4 investigates the difference between
the accepting powers of CS-SV4-FA’s and CS-4-FA’s, andshows that CS-SV4-FA’s are less powerful than CS-4-FA’s.
Section 5 investigates the difference between the accepting powers of deterministic and nondeterministic CS-SV4FA’s, and shows that deterministic CS-SV4-FA’s are lesspowerful than nondeterministic CS-SV4-FA’s. Section 6
concludes by giving some open problems. In this paper,
we let each sidelength of each input tape of these automata
be equivalent in order to increase the theoretical interest.

M.Blum and C.Hewitt first proposed two-dimensional
automata as a computational model of two-dimensional
pattern processing, and investigated their pattern recognition abilities in 1967. Since then, a lot of researchers inthis field have been investigating many properties about automata on a two- or three-dimensional tape. On the other
hand, the question of whether processing four-dimensional
digital patterns is much more difficult than two- or threedimensional ones is of great interest from the theoretical and practical standpoints. Thus, the study of fourdimensional automata as a computasional model of fourdimensional pattern processing has been meaningful. This
paper introduces a cooperating system of four-dimensional
finite automata as one model of four-dimensional automata. A cooperating system of four-dimensional finite
automata consists of a finite number of four-dimensional
finite automata and a four-dimensional input tape wherethese finite automata work independently (in parallel).
Those finite automata whose input heads scan the same cellof the input tape can communicate with each other, thatis, every finite automaton is allowed to know the internal
states of other finite automata on the same cell it is scanningat the moment. In this paper, we mainly investigate several
accepting powers of a cooperating system of seven-way
four-dimensional finite automata. The seven-way fourdimensional finite automaton is a four-dimensional finite
automaton whose input head can move east, west, south,
north, up, down, or in the future ,but not in the past on afour-dimensional input tape.
Key Words : computational complexity, cooperating system, finite automaton, four-dimension, multihead.

2∑
Definition 2.1. Let ∑be a finite set of symbols. A fourdimensional tape over∑ is a four-dimensional rectangulararray of elements
of
. The
∑
∑ set of all four-dimensional∑
tapes overis denoted by (4) . Given a tape x ∈ (4) ,for each integer j(1 ≤ j ≤ 4), we let lj (x) be the length∑(4)of x along the jth axis. The set of all x ∈withl1 (x) = n1 , l2 (x) = n2 , l3 (x) = n3 , and l4 (x) = n4 is∑(n1 ,n2 ,n3 ,n4 )denoted by. When 1 ≤ ij ≤ lj (x) for eachj(1 ≤ j ≤ 4), let x(i1 , i2 , i3 , i4 ) denote the symbol in x
with coordinates (i1 , i2 , i3 , i4 ). Furthermore, we define

1 Introduction
A cooperating system of four-dimensional finite automata (CS-4-FA) [2-4,8] consists of a finite number offour-dimensional finite automata and a four-dimensional
input tape where these finite automata work independently(in parallel). Those finite automata whose input heads scan
the same cell of the input tape can communicate with eachother, that is, every finite automaton is allowed to know theinternal states of other finite automata on the same cell it isscanning at the moment.
In this paper, we propose a cooperating system of seven-©ISAROB 2011

Preliminaries

x[(i1 , i2 , i3 , i4 ), (i′1 , i′2 , i′3 , i′4 )],when 1 ≤ ij ≤ i′j ≤ lj (x) for each integer j(1 ≤ j ≤ 4),
as the four-dimensional input tape y satisfying the follwing
conditions:(i) for each j(1 ≤ j ≤ 4), lj (y) = i′j − ij + 1;

191

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011(ii) for each r1 , r2 , r3 , r4 (1 ≤ r1 ≤ l1 (y), 1 ≤ r2 ≤ l2 (y),1 ≤ r3 ≤ l3 (y), 1 ≤ r4 ≤ l4 (y)), y(r1 , r2 , r3 , r4 )= x(r1 + i1 − 1, r2 + i2 − 1, r3 + i3 − 1, r4 + i4 − 1).
(Wecallx[(i1 , i2 , i3 , i4 ), (i′1 , i′2 , i′3 , i′4 )][(i1 , i2 , i3 , i4 ), (i′1 , i′2 , i′3 , i′4 )]-segment of x.)

5. q0i ∈ Qi is the initial state of FAi .
6. Fi ⊆ Qi is the set of accepting states of FAi .

theEvery automaton of M independently (in parallel)
works∑step by step on the same four-dimensional tape x
oversurrounded by boundary symbols ♯’s. Each stepis assumed to require exactly one time for its completion.
For each i(1 ≤ i ≤ k), let qi be the state of FAi at time‘t’. Then each FAi , enters the next state ‘pi ’ at time ‘t + 1’
according to the function

We recall a seven-way four-dimensional simple k-head
finite automaton (SV4-SPk-HFA)[5,6]. An SV4-SPk-HFA
M is a finite automaton with k read-only input heads operating on a four-dimensional input tape surrounded byboundary symbols ♯’s. The only one head(called the ‘reading’ head) of M is capable of distinguishing the symbols
in the input alphabet, and the other heads(called ‘counting’heads) of M can only detect whether they are on the boundary symbols or a symbol in the input alphabet. When an input tape x is a presented to M , M determines the next stateof the finite control, the next move direction (east, west,
south, north, up, down, future, past or no move) of each
input head, depending on the present state of the finite contorol, the symbol read by the reading head, and on whether
or not the symbol read by each counting head is boundary symbol. We say that M accepts x if M , when started
in its initial state with all its input heads on x(1, 1, 1, 1),
eventually halts in an accepting state with all its heads on
the bottom boundary symbols of x. As usual, we define
nondeterministic and dterministic SV4-SPk-HFA’s.
A seven-way four-dimensional sensing simple k-head finite automaton(SV4-SNSPk-HFA) is the same device asa SV4-SPk-HFA except that the former can detect coincidence of the input heads.
We denote a deterministic(nondeterministic) SV4-SPkHFA by SV4-SPk-HDFA(SV4-SPk-HNFA), and denotea deterministic (nondeterministic)SV4-SNSPk-HFA by
SV4-SNSPk-HDFA(SV4-SNSPk-HNFA).

′′
δi (x(α, β, γ, ρ), (q1′ , . . . , qi−1
, qi+1, . . . , qk′ ), qi ) =(pi , (d1 , d2 , d3 , d4 )),where x(α, β, γ, ρ) is the symbol read by the input head of
FAi at time ‘t’ and for each j ∈ {1, . . . , i−1, i+1, . . . , k},

qj′ =


qj ∈ Qj







ϕ

if the input heads of FAi and FAj ,are on the same input position at
the moment ‘t’;otherwise,

and moves 1st input head to x(α+d1 , β+d2 , γ+d3 , ρ+d4 )at time ‘t + 1’. We assume that the input head of FAi neverfalls off the tape beyond boundary symbols.
∑(4)
is presented to M , we say
When an input tape x ∈that M accepts the tape x if each automaton of M , whenstarted in its initial state with its input head on x(1, 1, 1, 1),
eventually enters an accepting state with its input head on
one of the bottom boundary symbols.

We now give formal definition of a cooperating systemof k four-dimensional deterministic finite automata (CS-4DFA(k)) as an acceptor.

We next introduce a cooperating system of k seven-way
four-dimensional deteministic finite automata (CS-SV4DFA(k)), with which we are mainly concerned in this paper.

Definition 2.2. A CS-4-DFA(k) is a k-tuple M = (FA1 ,FA2 , . . . , FAk ), k ≥ 1, such that for each 1 ≤ i ≤ k,∑
FAi = ( , Qi , Xi , δi , q0 i, Fi , ϕ, ♯),Definition 2.3. A CS-SV4-DFA(k) is a CS-4-DFA(k) M= (FA1 , FA2 ,. . . , FAk ) such that the input head of each FAi
can only move east, west, south, north, up, down, or in the
future, but not in the past.

where
∑
1.
is a finite set of input symbols.

To give the formal definition of a cooperating
system of k four-dimensional nondeterministic finite
automata (CS-4-NFA(k)) and a cooperating system ofk seven-way four-dimensional nondeterministic finite
automata (CS-SV4-NFA(k)) is left to the reader.
Foreach X ∈ {SV4-SPk-HDFA,SV4-SPk-HNFA,SV4SNSPk-HDFA,SV4-SNSPk-HNFA,CS-4-DFA(k),CS-4NFA(k),CS-SV4-DFA(k),CS-SV4-NFA(k)}, by X c wedenote an X which each sidelength of each input tape isequivalent; by L[X](L([X c ]) we denote the class of setsof input tapes accepted by X’s(X c ’s). We will focuse ourattention on the acceptors which each sidelength of each
input tape is equivalent.

2. Qi is a finite set of states.
3. Xi = (Q1 ∪ {ϕ}) × · · · × (Qi−1 ∪ {ϕ}) × (Qi+1 ∪{ϕ}) × · · · × (Qk ∪ {ϕ}), where ‘ϕ’ is a special state
not in (Q1 ∪ Q2 ∪ · · · ∪ Qk ).
∑4. δi : ( ∪{♯})×Xi ×Qi → Qi ×east(= (0, +1, 0, 0)),west(= (0, −1, 0, 0)),south(= (+1, 0, 0, 0)),north(= (−1, 0, 0, 0)),up(= (0, 0, −1, 0)),down(= (0, 0,+1, 0)),future(= (0, 0, 0, +1),past(= (0, 0, 0, −1)),no move(= (0, 0, 0, 0))is the next move∑ function,
where ‘♯’ is the boundary symbol not in .

©ISAROB 2011192

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

3 Relationship between
SV4-SPMHFA’s and CS-SV4-FA’s

the input heads of FA1 , FA2 , . . . , FAk leave theplane subsequently (i.e., FAd1 firstly moves its
input head in the future direction from the threedimensional rectangular array. FAd2 secondly
moves its input head in the future direction fromthe three-dimensional rectangular array, and soon.), and M ′ keeps the position where the input
head of each FAi , 1 ≤ i ≤ k, leaves the threedimensional rectangular array by the positionsof h1 , h2 , . . . , hk .

In this section, we establish a relation between the accepting powers of seven-way four-dimensional simple multihead finite automata and cooperating systems of sevenway four-dimensional finite sutomata over input tapes
which each sidelength is equivalent. This result will be
used in the latter sections.
Lemma 3.1. For any k ≥ 1 and X ∈ {N, D},(c) Furthermore, for each i(1 ≤ i ≤ k − 1), theinterval between the times at which FAdi andFAdi+1 move their input heads in the future
direction from the three-dimensional rectangular array is stored by a counter with O(n6k )space bound, which can be realized by usingh(2i−1)k−1 , h(2i−1)k−2 , . . . , h(2i−1)k , where nis the number of rows (or columns or planes or
three-dimensional rectangular array) of the input
tape.

L[SV4-SNSPk-HXFAc ]⊆L[CS-SV4-XFA(2k)c ]Proof. Let M be an SV4-SNSPk-HFAc . We will construct a CS-SV4-XFA(2k)c M ′ to simulate M . M ′ acts
as follws:
1. M ′ simulates the moves of the reading head of M andall the east, west, south, north up, or down moves ofcounting heads of M by using its (k + 1) finite automata.
2. M ′ simulates all the moves in the future direction ofcounting heads of M by making the down moves of
input heads of its other (k − 1) finite automata.

Note that M works in O(n6k ) time, that is, if an input
tape with n rows (or columns or planes) is accepted by M ,then it can be accepted by M in O(n6k ) time. Thus, it iseasy to verify that M ′ can simulate M .
□3. During the simulation, if M moves its reading headin the future direction, then M ′ makes all of input
heads of finite automata of M ′ move in the future direction so that all the automata of M ′ can keep their
input heads on the same three-dimensional rectangular array and can communicate with each other in thatthree-dimensional rectangular array.
It is easy to see that M ′ can simulate M .

From [5],it follows that ∪1≤k<∞ L[SV4-SPk-HXFAc ]= ∪1≤k<∞ L[SV4-SNSPk-HXFAc ] for any X ∈ {N, D}.
Combining this result with Lemmas 3.1 and 3.2, we havethe follwing thorem.
Theorem
3.1.
∪1≤k<∞ L[SV4-SPk-HXFAc ]= ∪1≤k<∞ L[CS-SV4-XFA(k)c ] for any X ∈ {N, D}.

□

Lemma 3.2. For any k ≥ 1 and any X ∈ {N, D},

Corollary 3.1. For any k ≥ 1, there is no CS-SV4-NFA(k)that accepts the set of connected patterns.

L[CS-SV4-XFA(k) ]⊆L[SV4-SNSP(2k − k + 1)HXFAc ].
c2

Remark 3.1. It is easy to see that for each k ≤ 1,
(1)four-dimensional sensing simple k head finite automata[5] are simulated by cooperating systems of (k + 1) fourdimensional finite automata, and (2) cooperating systemsof k four-dimensional finite automata are simulated byfour-dimensional sensing simple (k + 1) head finite automata.

Proof. Let M = (FA1 ,FA2 ,. . . ,FAk )be a CS-SV4-XFA(k)c .
We will construct an SV4-SNSP(2k 2 − k + 1)-HXFAc M ′to simulate M . Let R denote the reading head of M ′ , andh1 , h2 , . . . , h2k2 −k denote the 2k 2 − k counting heads ofM ′ . M ′ acts as follws:
1. M ′ stores the internal states of FA1 ,FA2 ,. . . ,FAk in its
finite contorol.

Remark 3.2. It is shown in [9] that (one-dimensional) oneway simple multihead finite automata snd cooperating systems of (one-dimensional) one-way deterministic finite automata are incomparable in accepting power. From thisfact, it follows that SV4-SPMHFA’s and CS-SV4-DFA’s
are incomparable in accepting power if the input tapes are
restricted to those x such that l4 (x) > l1 (x) = l2 (x) =l3 (x). We can also show that SV4-SPMHFA’s are more
powerful than CS-SV4-DFA’s if the input tapes are restricted to those x such that l4 (x) < l1 (x) = l2 (x) =l3 (x).

2. For each three-dimensional rectangular array of the
input tape:(a) M ′ simulates the east, west, south, north, up, or
down moves of input heads of FA1 ,FA2 ,. . . ,FAk
by using R and h1 , h2 , . . . , hk .
(b) M ′ stores in its finite control the internal stateof each FAi , 1 ≤ i ≤ k, when the input headof FAi leaves the three-dimensional rectangulararray and the order, (d1 , d2 , . . . , dk ), in which©ISAROB 2011193

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

4 Seven-way versus Eight-Way

L[CS-4-DFA(k)c ] ⊊ L[CS-4-NFA(k)c ] ?

In this section, we investigate the difference between theaccepting powers of CS-4-DFA(k)c ’s [CS-4-NFA(k)c ’s]
and CS-SV4-DFA(k)c ’s [CS-SV4-NFA(k)c ’s].

References

Theorem 4.1. For each X ∈ {N, D}, L [CS-4DFA(1)c ]− ∪1≤k<∞ L[CS-SV4-XFA(k)c ] ̸= ∅.

[1] M.Blum and C.Hewitt, Automata on a twodimensional tape, in IEEE Symp. Switching Automata
Theory,pp.155-160,1967.

Proof. Let T1 = {x ∈ {0, 1}(4) |(∃m ≥ 2)[l1 (x) = l2 (x)= l3 (x) = l4 (x) = m & x[(1, 1, 1, 1), (m, m, m, 1)] =x[(1, 1, 1, 2), (m, m, m, 2)]]}. Clearly, T1 ∈ L[CS-4DFA(1)c ]. From [5], it is easy to see that T1 is not in∪1≤k<∞ L[SV4-SPk-HNFAc ]. From this fact and Theorem 3.1, the theorem follows.
□[2] M.Blum and K.Kozen, On the power of the compass,in Processings of the 19th Annual Simp. on Foundations of Computer Science,1987.
[3] M.Blum and W.J.Sakoda, On the capability of finite
automata in 2 and 3 dimensional space, in Processings of the 18th Annual Simp. on Foundations of Computer Science pp.147-161,1977.

From Theorem 4.1, we can get the following corollary.

[4] A.Hemmerling, Normed two-plane trapes for finite
system of cooperating compass automata, in EIK23(8/9),pp.453-470,1978.

Corollary 4.1.
For each k ≥ 1 and X ∈{N, D}, (1)L[CS-SV4-XFA(k)c ]⊊ L[CS-4-XFA(k)c ],and (2) ∪1≤k<∞ L[CS-SV4-XFA(k)c ] ⊊ ∪1≤k<∞ L[CS4-XFA(k)c ].

[5] K.Inoue, I.Takanami, and H.Taniguchi, Three-way
two-dimensional simple multihead finite automata– hierarchical properties (in Japanese), IECE Jan.
Trans.(D),pp.65-72,1979.

5 Nondeterminism versus Determinism[6] K.Inoue, I.Takanami, and H.Taniguchi, Threeway two-dimensional simple multihead finite automata – closure properties (in Japanese),IECE Jan.
Trans.(D),pp.273-280,1979.

In this section, we investigate the difference between
the accpting powers of CS-SV4-NFA(k)c ’s and CS-SV4DFA(k)c ’s.

[7] M.Sakamoto, Three-dimensional alternating Turingmachines, Ph.D.Thesis, Yamaguchi University,1999.

Theorem 4.2. L[CS-SV4-NFA(1)c ] − ∪1≤k<∞ L[CSSV4-DFA(k)c ] ̸= ∅.

[8] M.sakamoto, S.Okatani, K.Kajisa, M.Fukuda,
T,Matsukawa, A.Taniue, T.Ito, H.Furutani, andM.Kono, Hierarchies based on the number ofcooperating systems of three-dimensional finite
automata, International Journal of AROB, Vol.4,
No.3, pp.425-428, 2009.

Proof. Let T2 = {x ∈ {0, 1}(4) |(∃m ≥ 2)[l1 (x) =l2 (x) = l3 (x) = l4 (x) = m]&∃i , ∃j (1 ≤ i ≤ m, 1 ≤j ≤ m, 1 ≤ k ≤ m)[x(i, j, k, 1) = x(i, j, k, 2) = 1] .
Clearly, T2 ∈ [CS-SV4-NFA(1)c ]. From [5], it is easy to
see that T2 is not in ∪1≤k<∞ L[SV4-SPk-HDFAc ]. From
this fact and Theorem 3.1, the theorem follows.
□[9] Y.Wang, K.Inoue, and I.Takanami, Some propertiesof cooperating systems of one-way finite automata(in Japanese), IECE Jan. Trans.(D-I) No.7,pp.391399,1992.

From Theorem 4.2, we get the following corollary.
Corollary 4.2.
For each k≥
1, (1)L[CSSV4-DFA(k)c ]⊊L[CS-SV4-NFA(k)c ], and (2)∪1≤k<∞ L[CS-SV4-DFA(k)c ] ⊊ ∪1≤k<∞ L[CS-SV4NFA(k)c ].

6 Conclusion
We conclude this paper by giving several open problems
except the open problem stated in the previous section.
In this paper, we introduced a cooperating system of fourdimensional finite automata, and investigated several basic
accepting powers. We conclude this paper by giving an
open problem as follows.
For each k ≥ 2,

©ISAROB 2011

194

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

DP method for Structural Change Detection as Optimal Stopping--- Verification and Extension ---*Tetsuo Hattori, *Katsunori Takeda, **Hiromichi Kawano, *** Tetsuya Izumi*Graduate School of Engineering, Kagawa University / 2217-20 Hayashi, Takamatsu City, Kagawa 761-0396, Japan** NTT Advanced Technology / Musashino-shi Nakamachi 19-18, Tokyo 180-0006, Japan*** Micro TechnicaCo., Ltd./ Yamagami BLD. 3-12-2 Higashi-ikebukuro, Toshima-ku, Tokyo 170-0013, Japan(Tel : 81087-864-2221; Fax : 81-087-864-2262)(hattori@eng.kagawa-u.ac.jp)
Abstract: Previously, we have formulated the structural change detection method in time series as an Optimal
Stopping Problem with an action cost, using the concept of DP (Dynamic Programming). Then we have proved atheorem that the solution satisfies an inequality. In this paper, we verify the solution by numerical computation andgives the extension of the method by clarifying the notions of estimated structural change point and detection time
point that the structure has changed so far.
Keywords: time series, structural change, optimal stopping problem
For example, a prediction expression is given in thefollowing equation as a function of time t, where yt, β1 ,β0 , ε mean the function value, two constantcoefficients, and error term, respectively.
y t = β1 ⋅ t + β 0 + ε(1)The error term ε is given as a random variable ofthe normal distribution of variance σ 2 and average of 0,i.e., ε ~N(0, σ 2 ). For a time series data based on theequation (1), we think of two situations: one is thesituation that the observed data goes out from thetolerance zone that means missing the range of±2σ from the predicted value. And, another is thesituation that the observed data goes in the zone. We callthe former situation “failing” (or “Out”) and the latter“hitting” (or “In”). We assume that the structure changes
when the failing occurs for continuing N times.
The evaluation function is given by (2) as the sum of
two kinds of cost: the damage caused by the failing (i.e.,
failing loss) and action cost to be taken after the change
detection.
Total_cost=cost(A)+cost(n)(2)
where cost(n) is the sum of the loss by continuing ntimes failing before the structural change detection, andcost(A) denotes the cost involved by the action after thechange detection. Then we have to find the number of N
that minimizes the expectation value of Total_cost,
assuming that the structural change occurs randomly.

I. INTRODUCTION
For ongoing time series analysis, three stages are
considered: prediction model construction, structuralchange detection (and/or disparity detection between themodel and observing data), and renewal of prediction
model. Especially in the second stage, it is important todetect the change point as quickly and also correctly as
possible, in order to renew the accurate prediction
model as soon as possible after the detection.
As the structural change detection, or change point
detection (CPD), some methods have been proposed[1]-[4]. The standard well known method is Chow Test
that is used in econometrics [2]. It does a statistical testby setting the hypothesis that the change has occurred attime t for all of data obtained so far.
Previously, we have formulated the structuralchange detection method in time series as an Optimal
Stopping Problem with an action cost, using the conceptof DP (Dynamic Programming) [5],[6]. Moreover, wehave shown a theorem [6]. This paper presents theverification of the theorem and shows the extension.

II. OPTIMAL STOPPING DP METHOD
1. Formulation ([5],[6])According to the previously presented description([5],[6]), we formulate the DP method for the change
point detection problem (CPD), using an evaluation
function that sums up the cost involved by prediction
error and action cost to be taken after the change
detection.

©ISAROB 2011

2. Structural change model ([5],[6])We can assume that the structural change is Poisson
occurrence of average λ, and that, once the change has
occurred during the observing period, the structure does

195

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

cost (A), respectively. Then, the evaluation function in(2) is denoted as the following equation (3).

not go back to the previous one. The reason why we set
such a model is that we focus on the detection of thefirst structural change in the sequential processing (orsequential test). The concept of the structural change
model is shown in Fig. 1.

λ1-λ

T = A+a⋅n(3)
We recursively define a function ET(n, N ) to obtain
the optimum number of times n that minimizes the
expectation value of the evaluation function of Equation(3), using the concept of DP (Dynamic Programming).
Let N be the optimum number. Let the function
EC(n, N ) be the expectation value of the evaluation
function at the time when the failing has occurred incontinuing n times, where n is less than or equal to N,i.e., 0 ≤ n≤ N .
Thus the function is recursively defined as follows.

1.0
Ec

E

Ec : State that the structural change occurred.
E : State that the structure is unchanged.

λ: Probability of the structural change occurrence.
(Poisson Process.)(if n = N ) ET ( n, N ) = A + a・N

Fig.1. Structural change model.

(4)
n

) a・n
(if n < N ) ET (n, N ) = P( Sn +1 | S ・Moreover, we introduce a more detailed model. Let
R be the probability of the failing when the structure isunchanged. Let Rc be the probability of the failing when
the structure change occurred. We consider that Rc is
greater than R, i.e., Rc>R. The detailed model for theState Ec and E are illustrated as similar probabilistic
finite state automatons in Fig.2 and Fig.3, respectively.

+ (1 − P( Sn +1 | S n )) ET (n + 1, N )where Sn means the state of failing in continuing n times,
Sn +1 the state of hitting at the (n+1) th observed data,
and P ( S n +1 | S n ) means the conditional probability
that the state Sn +1 occurs after the state Sn.
The first term in the right-hand side (RHS) of theequation (4) indicates the expectation value of theevaluation function at the time when hitting happens atthe (n+1)th data after the continuing n times failing. The
second term in the RHS of the equation (5) indicates the
expectation value of the evaluation function for the time
when failing happens at the (n+1)th data after
continuing n times failing.
Then, from the definition of the function ET ( n, N ) ,the goal is to find the N that minimizes ET (0, N ) ,because the N is the same as n that minimizes the
expectation value of the evaluation function of (4).

Ec
1-Rc

Rc

Rc
in

out

1-Rc

Fig.2. Internal model of the State E.
E

1-R

R

4. Minimization of the evaluation function
The analytical solution N that minimizes
EC ( 0, N ) can be deduced. The strict proof needs many
pages, then we show numerical solution.
The function ET (0, N ) is defined by recursive
expressions (4) and (5), then ET (0, N ) can becomputed by recursively. In the process of thiscomputation, P(Sn+1 | Sn ) can be calculated as follows.
Let Ecn be the event that the structural change

R

out

in
1 -R

Fig.3. Internal model of the State Ec.
3. Definition ([5],[6])Let the cost(n) be a ⋅ n as a linear function for n,where a is the loss caused by the failing in one time.
And for simplicity, let T and A denote the Total_cost and©ISAROB 2011(5)

occurs once during the period of observation incontinuing n times. Let P( Ecn | S n ) be the conditional
probability that the E cn happens under the condition

196

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011that failing has already occurred for continuing n times.
Based on the model in Fig.1-3,

Proof: Notice the equation (6).
P(Sn +1 | S n ) = (1 − R)(1 − P( Ecn | S n )) + (1 − Rc )P( Ecn | S n )The first term in the RHS of (6) shows the probability
that the hitting (“In”) occurs for the (n+1)-th timeobserved data when the structure is unchanged. The
second term shows the probability that the hitting
occurs for the (n+1)-th time observed data when the
structure changed.
From the equation (6), we haveP(Sn +1 | S n ) = (1 − R)(1 − P( Ecn | S n )) + (1 − Rc )P( Ecn | S n )(6)Subsequently, we show the Lemma 1 and Lemma 2.
Lemma 1: Let Ecn be the event that the structuralchange occurs once during the period of observation inncontinuing n times. Let P( Ecn | S ) be theconditional probability that the E cn happens under
the condition that failing (“Out”) occurs in continuing nntimes. P( Ecn | S ) is an increase function for n.

P(Sn+1 | S n ) = 1 − R + P(Ecn | S n )(R − Rc )n
By the aforementioned Lemma 1, P( Ecn | S ) is
an increase function, and R < Rc , therefore,
P(Sn +1 | S n ) is a decrease function for n.
Remark: Lemma 2 indicates that, if the number oftimes of continuous failing increases, the probability of
the fitting for the next observed data after thosecontinuous failing decreases. This is intuitively clear,because, by Lemma 1, the probability of the structuralchange increases if the number of times of thecontinuous failing increases.
By using the above Lemma 1 and 2, and the
reduction to absurdity, the following theorem holds [6],that gives the n that minimizes the expectation valueET(0,N).
Theorem [6].
The N that minimizes ET(0,N) is given as the largest
number n that satisfies the following Inequality (10).
a < ( A + a )・P( Sn | S n −1 )(10)where the number N+1 can also be the optimum one
that minimizes ET(0,N), i.e., ET(0,N) = ET(0,N+1),
only ifN

Proof: Let E be the event that there is no structuralchange. According to the Bayes’ theorem, we haveP( Ecn | S n ) =P( S n | Ecn ) P( Ecn )P( S n | Ecn ) P( Ecn ) + P( S n | E ) P( E )n −1∑ (1 − λ ) i R i λ R cn − i

=

i=0

n −1

∑ (1 − λ ) i R i λ R cn − i + (1 − λ ) n R n

i=0

1

=
1 + n −1

( 1 − λ)

n

R

=

n

1
1 + D (n )∑ ( 1 − λ) i R i λ R cn − i

i=0

(7)
where

(1 − λ) Rn

D( n ) = n −1

n∑(1 − λ) R λ R
i

.

n −i
c

i

i =0

The D(n) is also expressed as follows.

D(n ) =

⎛ R ⎞⎟⎟(1 − λ ) n ⎜⎜⎝ Rc ⎠
n −1

n

⎛ R ⎞
⎟⎟
⎝ Rc ⎠a = ( A + a )・P( S N +1 | S )i

λ ∑ (1 − λ ) i ⎜⎜
i =0

(9)

=

X

n

n −1

λ ∑ XiIII. VERIFICATION AND EXTENSION

i =0

(8)
R
.
where X = (1 － λ )Rc

1. Verification of the Theorem by numericalcomputation

Since 0 ≤ λ < 1 , 0 < 1 − λ ≤ 1 , and R c > R , then0<X<1. So, the D(n) becomes a monotonous decreasen
for n. Therefore, the probability P( Ecn | S ) is amonotonous increase function. This means that, if thenumber of the failing times n increases, the probability
that the structural change has occurred increases. This
meets our intuition clearly.
Lemma 2: The conditional probability P(Sn+1 | Sn ) is adecrease function for n.

By numerical computation, we evaluate the ET(0,N)
and the Inequality(10) under the same conditions. Fig.4(a) shows that the relation between the expectation
ET(0,N) and N, and Fig.4(b) shows the valuea − ( A + a)・P(SN +1 | S N ) . We can easily verify that the
number 3 minimize ET(0,N), and at the same time, isthe largest number satisfying Inequality (10). Fig.5 also
shows that the relation between the expectation ET(0,N)
and N by varying the A/a and fixed λ. It implies that as©ISAROB 2011

197

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

the action cost A is greater, the N that minimizes
ET(0,N) becomes greater.

ET (0,N)

2. Extension of the change point notion

λ=0.001 Rc=0.95, R=0.05●: A/a=18, ○: A/a=15,▲: A/a=10, △: A/a=3

We separate the notion of CPD into two that one is
the time point when the change has been detected so far
at the observing time and another is the estimatedchange point just the time when the change has occurred.
Then we define that, if the aforementioned detectedchange point is tc, then estimated change point exits
within a section [tc-N, tc]. We have also verified byexperimentation for ongoing real time series data, thatthe extended definition works very well.

Fig.5. Relation between the expectation ET(0,N) and Nwith Rc and λ fixed and varying A/a(18, 15, 10, 3).
Optimal N is 4,3,3,2, respectively, depending the orderof the above ratio of A/a.

IV. CONCLUSION
We have verified and shown by numericalcomputation, that the Theorem surely holds. And we
also have proposed the extended notion of change point
detection. We consider that the optimal stopping DP
method and the extension will be promising.

REFERENCES
ET(0,N)[1] R.Jana and S.Dey (2000), Change detection in
Teletraffic Models, IEEE Trans. Signal Processing,vol.48, No.3, pp.846-853.
[2] Chow,G.C. (1960), Tests of Equality Between Setsof Coefficients in Two Linear Regressions,
Econometrica, Vol.28, No.3, pp.591-605.
[3] S.MacDougall, A.K. Nandi and R.Chapman (1998),
Multisolution and hybrid Bayesian algorithms forautomatic detection of change points, Proc. of IEEE
Visual Image Signal Processing, vol.145, No.4, pp.280286.
[4]E.S.Page (1954), Continuous inspection schemes,
Biometrika, Vol.41, pp.100-115.
[5] Tetsuo Hattori, Katsunori Takeda, Izumi Tetsuya,
Hiromichi Kawano (2010): "Early Structural Change
Detection as an Optimal Stopping Problem (I) --Formulation Using Dynamic Programming with Action
Cost ----", Proc. of the 15th International Symposium on
Artificial Life and Robotics (AROB15th'10), ISBN 9784-9902880-4-4, pp.763-766.
[6] Hiromichi Kawano, Tetsuo Hattori, Katsunori
Takeda, Izumi Tetsuya (2010): "Early Structural
Change Detection as an Optimal Stopping Problem (II)--- Solution Theorem and its Proof Using Reduction toAbsurdity ----", Proc. of the 15th International
Symposium on Artificial Life and Robotics(AROB15th'10), ISBN 978-4-9902880-4-4, pp.767-772.

λ=0,01

Arg mini ET(0,N)=3
N
(a)

y

Optimal point (N=3)
N

y = a − ( A + a )・P( S N +1 | S N )(b)Fig.4. Evaluation of the expectation ET(0,N)
Inequality a < ( A + a)・P(Sn | S
Theorem [6].

©ISAROB 2011

and

n−1

) appeared in the

198

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Change Detection Experimentation for Multiple Regression Using ESPRT--- One Variation is Periodic Function --*Katsunori Takeda, *Tetsuo Hattori, **Hiromichi Kawano, ***Tetsuya Izumi*Graduate School of Engineering, Kagawa University / 2217-20 Hayashi, Takamatsu City, Kagawa 761-0396, Japan**NTT Advanced Technology/ Musashino-shi Nakamachi 19-18, Tokyo 180-0006, Japan***Micro Technica Co., Ltd. / Yamagami BLD. 3-12-2 Higashi-ikebukuro, Toshima-ku, Tokyo 170-0013, Japan(Tel : 81-87-864-2221; Fax : 81-87-864-2262)(hattori@eng.kagawa-u.ac.jp)
Abstract: Previously, we have proposed the application of sequential probability ratio test (SPRT) to the structuralchange detection of ongoing time series data. Moreover, we have also proposed the extended method of SPRT (ESPRT).
In this paper, we show experimental results by the Extended SPRT (ESPRT) and Chow Test when applying to timeseries data that are generated by a multiple regression model in the case where one explanatory variation is periodic
function (sine function). And we clarify the effectiveness of the ESPRT, in the sense of ability of early and correctchange detection and computational complexity.
Keywords: Time series, multiple linear regression, ESPRT (Extended Sequential Probability Ratio Test), Chow Test

I. INTRODUCTION

II. ESPRT AND CHOW TEST

Generally, we have three stages in predicting
ongoing time series data ([1], [2]). First, we have to finda prediction model that adequately represents thecharacteristics of the early time series data. Second, we
have to detect the structural change of the time series
data, as quickly and correctly as possible, when theestimated prediction model does not meet the data any
more ([3],[4]). Third, we have to reconstruct the next
prediction model as soon as possible after the change
detection.
For the second problem, we have already proposed
an application of SPRT (sequential probability ratio test)
that has been mainly used in the field of quality control[5], [6]. And we have presented the experimental resultsin comparison with Chow Test that is well-known
standard method for such structural change detection oftime series data ([6], [7]).
However, the experimentation has been done using
single regression model and has shown that the SPRT is
more effective than Chow Test. Since multiple
regression models are more generally used for timeseries data analysis than single regression one, we haveexamined by experimentation if the SPRT surpasses
Chow Test as well for the case of multiple regression
model data [8]. Moreover in the literature [8], we haveshown the extended SPRT (ESPRT) aiming at moreaccurate estimation of the change point.

©ISAROB 2011

1. SPRT ([8])
The sequential probability ratio test (SPRT) is used
for testing a null hypothesis H0 (e.g. the quality is underpre-specified limit 1%) against hypothesis H1 (e.g. thequality is over pre-specified limit 1%). And it is defined
as follows:
Let Z1, Z2, …Zi be respectively observed time series
data at each stage of successive events, the probability
ratio λ i is computed as follows.

λi =P( Z1 | H1 ) ⋅ P( Z2 | H1 ) ⋅ ⋅ ⋅ P( Zi | H1 )(1)
P( Z1 | H0 ) ⋅ P( Z2 | H0 ) ⋅ ⋅ ⋅ P( Zi | H0 )where P(Z | H0 ) denotes the distribution of Z if H0 istrue, and similarly, P(Z | H1 ) denotes the distribution of
Z if H1 is true.
Two positive constants C1 and C2 (C1 < C2) arechosen. If C1 < λi < C2, the experiment is continued bytaking an additional observation. If C2 < λi , the process
is terminated with the rejection of H0 (acceptance of H1).
If λi < C1, then terminate this process with theacceptance of H0.
1− β
β
(2)
C1 =
, C2 =1−ααwhere α means type I error (reject a true null hypothesis), and β means type II error (accept a null hypothesis
as true one when it is actually false).

199

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

2. Procedure of SPRT ([8])
The concrete procedure of structural change
detection is as follows (see Fig. 1):
Step1: Make a prediction expression and set thetolerance band (a) (e.g. a=2σs) that means
permissible error margin between the predicteddata and the observed one. (σs denotes astandard deviation in learning sample data at
early stage.)
Step2 ： Set up the null hypothesis H0 and alternative
hypothesis H1.
H0：Change has not occurred yet.
H1：Change has occurred.
Set the values α , β and compute C1 and C2 ,according to Equation (2). Initialize i = 0,λ0 = 1 .
Step3: Incrementing i (i = i+1), observe the following
data yi. Evaluate the error | εi | between the data
yi and the predicted value from theaforementioned prediction expression.
Step4: Judge as to whether the data yi goes in thetolerance band or not, i.e., the εi is less than (orequal to) the permissible error margin or not. If
it is Yes, then set λi = 1 and return to Step3.
Otherwise, advance to Step5.
Step5: Calculate the probability ratio λi , using thefollowing Equation (3) that is equivalent toEquation (1).

Here, we call Case a-c for each combination of(θ0,θ1), respectively, as follows:
Case a (θ0=0.1, θ1=0.9),
Case b (θ0=0.2, θ1=0.8),
Case c (θ0=0.3，θ1=0.7).
Start・Make a prediction expression
・Make a tolerance band (a) (e.g. a=2σs)

Step1

Step2

・Set the hypotheses H0 and H1 .
H0：Change has not occurred yet.
H1：Change has occurred.
・Set α , β and compute C1, C2・Initialize i = 0, λ0 = 1

Step3

・Increment i (i = i+1)
・Observe a new data yi
・Evaluate the | εi |Step4

| εi |≦a

P (ε i | H 1 )P (ε i | H 0 )

Step5

Yes

・Reject H0・Accept H1

・Calculate λi(3)λi > C 2

where, if the data yi goes in the tolerance band,( P(εi |H0 ), P(εi |H1 ) )= ( θ0, θ1 ), otherwise,( P(εi |H0 ), P(εi |H1 ) )= ( (1-θ0), (1-θ1) ).
Step6: Execution of testing.
(i) If the ratio λi is greater than C2 (= (1-β)/α ),dismiss the null hypothesis H0, and adopt thealternative hypothesis H1, and then End.
(ii) Otherwise, if the ratio λi is less than C1 (=β/(1-α) ), adopt the null hypothesis H0, anddismiss the alternative hypothesis H1, andthen set λi = 1 and return to Step3.
(iii) Otherwise (in the case where C1 ≤ λi ≤ C2 ),advance to Step7.
Step7: Observe the following data yi incrementing i.
Evaluate the error | εi | and judge whether thedata yi goes in the tolerance band, or not. Then,
return to Step5 (calculation of the ratio λi ).

No©ISAROB 2011

λi = 1No

Step6

λi = λi −1

Yes・Accept H0
・Reject H1
・ λi = 1

Yes

λi < C1

End

No

Step7

・Increment i (i = i+1)
・Observe a new data yi
・Evaluate the | εi |& Compare it with a

Fig.1. SPRT structural change detection [8].
3. Extended SPRT ([8])
The SPRT detects a change point at the time when theprobabilistic ratio λi is greater than C2 (=(1-β)/α). Then,the detected change point equals to the terminated time
point and its detection tends to be delayed from true200

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011⎛1 ⎞x1 = t , x 2 = sin ⎜ π t ⎟⎝8 ⎠change point. So, as an extension of SPRT, we define
the estimated change point that exists in the intersection[tc-M, tc], where tc means the aforementioned detectedchange point and M is the number of times when theobserved data continuously goes of tolerance zone until
the ratio λi > C2. The number M can be obtained from

Table 2. Parameters for generating time series data.
Equation (7)Equation (6)σ( t=70,71,…,100)( t=1,2,…,69)
5
y = x + 20x + 5 + ε
y = x + 10x + 5 + ε

the equation (4).

(

1

θ1 M
1− β
) > C2 (=
)
θ0
α

(8)

(4)

2

250

2

y = x1 +10x2 + 5 + ε

y = x1 + 20x2 + 5 + εyThen we have the following equation using Gauss
notation. So, the value of M depends on the parameters(see Table 1). That is, M=2 (Case a), M=3 (Case b),
M=4 (Case c).
⎤⎡1− β ⎥(5)M = ⎢ log θ 1
⎢
α ⎥
θ0
⎥
⎢

1

change point tc = 70

200

150

100

50

Table 1. Parameter values in SPRT and M.
α
β
θ0
θ1
M
2
0.05
0.1
0.9
0.05
0.2
0.8
3
0.3
0.7
4

0

100

30

40

50

60

70

80

90

100

tc
CT

90

Generally, in the experimentation for the case oftime series data based on multiple linear regression
model, the data is supposed to be generated by thefollowing equations.

a

b

c

80
70
60
50

(6)

40

y = a21 x1 + a22 x2 + b + ε (t *c ≤ t)(7)2
where ε~N(0,σ ), i.e., the error ε is subject to theNormal Distribution with the average 0 and thevariation σ2, and tc* means the change point. In addition,
we have set tc*=70.
Setting the coefficients of equation (6) and (7) asshown in the Table 2, we examine the case where30©ISAROB 201120

Fig.2 shows the example of the time series dataaccording to the equations in Table 2. Fig.3 illustrates
results in Chow Test and SPRT, where horizontal axis
shows observing time t (detection operation has started
from t=41). The vertical axis shows the detected change
point tc, whose value is the average of experimentation
results for 200 sets of generated time series data.

III. EXPERIMENTATION(t ≤ t *c )10

Fig.2. Example of the time series data generated byequations (6) and (7), where x1 and x2 are time functions
such as x1=t, and x2=sin( π t /8). (The true change point
tc=70.)

4. Chow Test ([6]-[8])The well known Chow Test checks if there are
significant differences or not, among residuals for three
Regression Lines, where regression Line 1 obtained
from the data before a change point tc, Line 2 from thedata after tc, and Line 3 from the whole data so far, bysetting up the change point hypothesis at each point in
the whole data (Fig.2).

y = a11 x1 + a12 x2 + b + εt0Case a (θ0=0.1, θ1=0.9)
Case b (θ0=0.2, θ1=0.8)
Case c (θ0=0.3，θ1=0.7)

20
10

t

0
40

50

60

70

80

90

100

Fig.3. Resultant relation between detected change point
tc and time point t, where CT means Chow Test and a-ccorresponds to each of Cases a-c, respectively.

201

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Case a, b, c, respectively. From Fig.4-6, we can see that
the ESPRT works better than SPRT and Chow Test.

frequency
20 0
18 0
16 0
14 0
12 0
100
80
60
40
20
0
t
40 45 50 55 60 65 70 75 80 85 90 95 100

IV. CONCLUSION
We have experimented the structural change point
detection by SPRT, ESPRT, and Chow Test for ongoingtime series data generated by multiple linear regression
model, where two variations are time functions and theone of two variation is periodic. From the results, weconsider that ESPRT works more effective than SPRT
and Chow Test in the sense of early detection, accuracy,
and computational cost.

Fig.4. Frequency of the detected change point tc by
SPRT in the Case a.
frequency
200
180
160
140
120
100
80
60
40
20
t
0
40 45 50 55 60 65 70 75 80 85 90 95 100

REFERENCES[1] C.G.E.P.Box and G.M.Jenkins (1976), Time Series
Analysis:Forecasting and Control, Prentice Hall.
[2] Peter J. Brockwell and Richard A. Davis (2003),
Introduction to Time Series and Forecasting, Springer;
2nd edition.
[3] C.Han, P.k.Willet and D.A.Abraham (1999), Somemethods to evaluate the performance of Page's test as
used to detect transient signals, IEEE Trans. Signal
processing, Vol.47, No.8, pp.2112-2127.
[4] S.D.Blostein (1991), Quickest detection of a timevarying change in distribution, IEEE Trans. Information
Theory, Vol.37, No.4, pp.1116-1122.
[5] A. Wald (1947), Sequential Analysis, John Wiley &Sons.
[6] Hiromichi Kawano, Tetsuo Hattori, Ken Nishimatsu(2008), Structural Change Point Detection Method ofTime Series Using Sequential Probability Ratio Test－
Comparison with Chow Test in the ability of early
detection－(in Japanese), IEEJ Trans. EIS, Vol.128，
No.4，pp.583-592．
[7] Chow, G. C. (1960), Tests of Equality Between Setsof Coefficients in Two Linear Regressions,
Econometrica, Vol.28, No.3, pp.591-605.
[8] Katsunori Takeda, Tetsuo Hattori, Izumi Tetsuya,
Hiromichi Kawano (2010), Extended SPRT for
Structural Change Detection of Time Series Based on
Multiple Regression Mode, Proceedings of the 15th
International Symposium on Artificial Life andRobotics (AROB 15th ‘10), pp.755-758, ISBN: 978-49902880-4-4, Oita, Japan, Feb..

Fig.5. Frequency of the detected change point tc by
SPRT in the Case b.
frequency
200
180
160
140
120
100
80
60
40
20
t
0
40 45 50 55 60 65 70 75 80 85 90 95 100

Fig.6. Frequency of the detected change point tc by
SPRT in the Case c.

Fig.3 shows that Chow Test outputs the change pointat the time when every data is observed after t=40. Thismeans that Chow Test tends to make false detection
before the true change point. And, the time when Chow
Test works well is long enough after the true change
point. Comparing with Chow Test, the SPRT works
better depending on the Cases a, b, c. In order tointerpret the results as ones by ESPRT, we only have toconsider [tc-M, tc], where M=2, 3, 4 corresponding to©ISAROB 2011202

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Continuous Change Point Detection for Time Series Images Using ESPRT*Katsunori Takeda, *Tetsuo Hattori, **Hiromichi Kawano, ***Tetsuya Izumi, ****Shinichi Masuda*Graduate School of Engineering, Kagawa University / 2217-20 Hayashi, Takamatsu City, Kagawa 761-0396, Japan**NTT Advanced Technology, / Musashino-shi Nakamachi 19-18, Tokyo 180-0006, Japan***Micro Technica Co., Ltd., / Yamagami BLD. 3-12-2 Higashi-ikebukuro, Toshima-ku, Tokyo 170-0013, Japan****C Micro Co., Ltd., 269-1 Hayashi-Cho, Takamatsu City, Kagawa 761-0396, Japan(Tel : 81-87-864-2221; Fax : 81-87-864-2262, hattori@eng.kagawa-u.ac.jp)
Abstract: In order to continuously detect the change point in ongoing time series data, we propose a method using
extended sequential probability ratio test (ESPRT). That is, it is a repetition of three stages, where the first stage isprediction model construction, and change point detection as the second stage, and reconstruction of prediction modelas the third stage. In this paper, we show experimental results by applying the method to time series data of image
storage, where the images are transmitted as compressed data (i.e. JPEG file) from remote monitoring camera systems.
Keywords: Continuous change detection, monitoring camera system, time series images, extended sequential
probability ratio test (ESPRT), security

I. INTRODUCTION

II. SPRT AND CHANGE DETECTION

Today, monitoring camera systems are widely usedin the society [1]. However, because there is a channel
capacity limit in the public communication line, remotemonitoring systems ordinarily use compressed data (e.g.
JPEG file) to send scene images if there happens
something abnormal or something changed. For thisreason, the camera system is required to become smart
enough to judge by itself as to whether somethingabnormal has occurred or not.
On the other hand, if something moves in the scene
image, the quantity of the compressed image tends to bedifferent from the steady situation so far. Based on thecharacteristics, in our previous paper, we have proposeda method for the change detection using applying theSequential Probability Ratio Test (SPRT) ([2], [3]) to thescene images of time series [6]. Moreover, in theliterature, we have experimented with the real timeseries images from a monitoring camera and havecompared the SPRT the method with the well-known
Chow Test [4].
However, we have just only presented the results offirst change detection. In fact, we need to continuously
deal with the change detection from the time series
images, and so that we have to uses the Extended SPRT(ESPRT), in order to know the change time point
correctly and reconstruct the next prediction model [5].
In this paper, we show the experimental results ofcontinuous change detection from real time series
images, and discuss the performance of the method.

For the change detection problem, we propose anapplication of Sequential Probability Ratio Test (SPRT)
that has been mainly used in the field of quality control.

©ISAROB 2011

1. SPRT ([2],[3])The SPRT is used for testing a null hypothesis H0(e.g. the quality is under pre-specified limit 1%) against
hypothesis H1 (e.g. the quality is over pre-specified limit1%). In addition, it is defined as follows.
At each stage of successive events Z1,Z2, …Zi thatare respectively corresponding to observed time series
data, the probability ratio λ i is computed.

λi =P( Z1 | H1 ) ⋅ P( Z2 | H1 ) ⋅ ⋅ ⋅ P( Zi | H1 )(1)
P( Z1 | H0 ) ⋅ P( Z2 | H0 ) ⋅ ⋅ ⋅ P( Zi | H0 )where P(Z | H0 ) denotes the distribution of Z when
H0 is true and P(Z | H1 ) denotes the distribution of Z
when H1 is true.
Two positive constants C1 and C2 (C1 < C2) arechosen. If C1 < λi < C2, the experiment is continued bytaking an additional observation. If C2 < λi , the process
is terminated with the rejection of H0 (acceptance of H1).
If λi < C1, the process is terminated with the acceptanceof H0.

C1 =

β
1−α

C2 =

1− β

α

(2)

where α means type I error (reject a true null
hypothesis), and β means type II error (accept a null
hypothesis as true one when it is actually false).

203

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011(ii) Otherwise, if the ratio λi is less than C1 (=β/(1-α) ), adopt the null hypothesis H0, anddismiss the alternative hypothesis H1, and thenset λi = 1 and return to Step3.
(iii) Otherwise (in the case where C1 ≤ λi ≤ C2 ),advance to Step7.
Step7: Observe the following data yi incrementing i.
Evaluate the error | εi | and judge whether thedata yi goes in the tolerance band, or not. Then,
return to Step5 (calculation of the ratio λi ).

2. Procedure
The concrete procedure of structural change
detection is as follows.
Step1: Make a prediction expression and set thetolerance band (a) (e.g. a=2σs) that means
permissible error margin between the predicteddata and the observed one (where σs means thestandard deviation).
Step2 ： Set up the null hypothesis H0 and alternative
hypothesis H1.
H0：Change has not occurred yet.
H1：Change has occurred.
Set the values α , β and compute C1 and C2 ,according to Equation (2). Initialize i = 0,λ0 = 1 .

3. Extended SPRT [5]We extend the definition of detected change point by
SPRT. As such extension, we adopt the number tc-M
where tc is ordinary aforementioned change point and M
is the number of times when the observed data
continuously goes of tolerance zone until the ratio λi >Remark: The statement of the null hypothesis H0,“Change has not occurred yet.”, means in statisticalsense. It means that the generation probability for thedata to go out from the tolerance band is less than (orequal to) θ0 (for instance, 1%). Similarly, the statementof the alternative hypothesis H1, “Change has occurred.”means that the generation probability for the data to go
out from the tolerance band is greater than (or equal to)θ1 (for instance, 99%). Additionally, we suppose that θ1
is considerably greater than θ0.

C2 The number M can be obtained from the equation
below.

(P (ε i | H 1 )P (ε i | H 0 )⎡⎤1− β ⎥⎢
M = log θ
⎢
α ⎥
θ
⎢
⎥
1

(5)

0

From some experimental results, we have found that,
if we adopt the interval [tc-M+1, tc] as the existing
range of true change point, the hitting (or correct
estimation) percentage will considerably increase.
Table 1. Parameter values in SPRT and M.
α
β
θ0
θ1
M
2
0.1
0.9
0.05
0.05
0.2
0.8
3
0.3
0.7
4

(3)

III. CONTINUOUS CHANGE DETECTION
FOR IMAGE SEQUENCE

where, if the data yi goes in the tolerance band,P(εi |H0 )= θ0 and P(εi |H1 )=θ1, otherwise,P(εi |H0 )=(1-θ0) and P(εi |H1 )=(1-θ1).
Step6: Execution of testing.
(i) If the ratio λi is greater than C2 (= (1-β)/α ),dismiss the null hypothesis H0, and adopt thealternative hypothesis H1, and then End.

©ISAROB 2011

(4)

Then we have the following equation using Gauss
notation. The value of M depends on the parameters (seeTable 1).

Step3: Incrementing i (i = i+1), observe the following
data yi. Evaluate the error | εi | between the data
yi and the predicted value from theaforementioned prediction expression.
Step4: Judge as to whether the data yi goes in thetolerance band or not, i.e., the εi is less than (orequal to) the permissible error margin or not. If it
is Yes, then set λi = 1 and return to Step3.
Otherwise, advance to Step5.
Step5: Calculate the probability ratio λi , using thefollowing Equation (3) that is equivalent toEquation (1)λi = λi −1

θ1 M
1− β
) > C2 (=)θ0α

In order to continuously detect the structural changeof the scene images from a monitoring camera, we apply
the ESPRT method to the time series of compressed
image data (JPEG file) quantity (Kbyte).

204

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig.1 and Fig.2 show two kinds of images when aperson has moved into and out of the scene at highspeed and low speed, respectively. Fig.3 and Fig.4 show
two graphs on time series data of JPEG file volumes,
corresponding to Fig.1 and Fig.2, respectively.
We assume that there are four change points in theboth of the two kinds of time series images. The firstchange point is the time when some person appears inthe scene image, and the second is when he stops
walking, the third when he begins to move again, andthe fourth when he moves out of scene.
The results of change detection by SPRT and Chow
Test are shown in the graphs in Fig.5 and Fig.6,
respectively, where the SPRT detects change points inthe case of condition 1(θ0=0.9, θ1=0.1). The number ofsample points for learning and analysis is five, where
those samples are used for deciding prediction model(regression line) in both cases.

If SPRT detects the change at tc, the ESPRT estimates
the true point as tc-1 under the condition 1.Then after
the first detection, next five points from tc-1 to tc+3 will
be used as learning samples for reconstructing the next
prediction model.
The experimental results show that ESPRT method
continuously detects the change point better than Chow
Test in the two kinds of time series images.

Fig.1. Example of time series images where someone
moves in, stops walking, moves again, and moves out athigh speed what we call “Fast-In Fast-Out Case”. (Timeis counted from the upper left to the lower right.)

Fig.2. Example of time series images where someone
moves in, stops walking, moves again, and moves out at
low speed what we call “Slow-In Slow-Out Case”.
(Time is counted from the upper left to the lower right.)©ISAROB 2011205

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

JPEG(Kbyte)

JPEG(Kbyte)
36 .0

35.5

35 .5

35.0

35 .0

34.5

34 .5

SPRT

SPRT

SPRT

SPRT

34.0

34 .0

33.5

33 .5
33.0

33 .0
32.5

32 .5

time

32.0
0

5

10

15

20

25

30

35

40

45

ChowTest
time

32.0
0

50

5

10 15 20 25 30 35 40 45 50 55 60 65

Fig.6. Change detection results by SPRT and Chow Testfor the time series data in Fig.4(Slow-In Slow-Out Case).

Fig.3. Time series data of JPEG file volumes in Fig.1(Fast-In Fast-Out Case).

IV. CONCLUSION
This paper has proposed the ESPRT method forcontinuous change detection and have shown its
effectiveness by experimental results that the method
has detected the four change points in two kinds of timeseries data, more correctly than Chow Test.

JPEG(Kbyte)
35 .5
35 .0
34 .5
34 .0
33 .5

REFERENCES

33 .0[1]Shinichi
Masuda,TetsuoHattori(2007),Development of a Wireless Remote Monitoring System
Utilizing Multiple Wireless Sensors (in Japanese), IEEJ
Trans. IA, Vol.127, No.6, pp.610-617.
[2] A. Wald(1947), Sequential Analysis, John Wiley &Sons.
[3]Hiromichi Kawano, Tetsuo Hattori, Ken Nishimatsu(2008), Structural Change Point Detection Method ofTime Series Using Sequential Probability Ratio Test－
Comparison with Chow Test in the ability of early
detection － (in Japanese), IEEJ Trans. EIS, Vol.128 ，No.4，pp.583-592．
[4]Chow,G.C. (1960), Tests of Equality Between Sets of
Coefficients in Two Linear Regressions, Econometrica,
Vol.28, No.3, pp.591-605.
[5] Katsunori Takeda, Tetsuo Hattori, Izumi Tetsuya,
Hiromichi Kawano (2010): "Extended SPRT for
Structural Change Detection of Time Series Based on
Multiple Regression Model", Proceedings of the 15th
International Symposium on Artificial Life and Robotics(AROB15th'10), ISBN 978-4-902880-4-4, pp.755-758.
[6] Katsunori Takeda, Tetsuo Hattori, Izumi Tetsuya,
Hiromichi Kawano, Shinichi Masuda (2010):"Application of SPRT to Image Data Sequence for
Remote Monitoring System", Proceedings of the 15th
International Symposium on Artificial Life and Robotics(AROB 15th'10), ISBN 978-4-9902880-4-4, pp.759-762.

32 .5

time

32.0
0

5

10 15 20 25 30 35 40 45 50 55 60 65

Fig.4. Time series data of JPEG file volumes in Fig.2(Slow-In Slow-Out Case).

JPEG(Kbyte)
36.0
35.5

SPRT

35.0

SPRT

SPRT

SPRT

35

45

34.5
34.0
33.5
33.0
32.5

ChowTest

time

32.0
0

5

10

15

20

25

30

40

50

Fig.5. Change detection results by SPRT and Chow Testfor the time series data in Fig.3 (Fast-In Fast-Out Case).

©ISAROB 2011

206

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

DP method for Structural Change Detection as Optimal Stopping---- Experimentation in Multiple Regression Model --*Hiromichi Kawano, **Tetsuo Hattori, **Katsunori Takeda, ***Tetsuya Izumi( *Musashino-shi Nakamachi 19-18, Tokyo 180-0006, JapanNTT Advanced Technology,**2217-20 Hayashi-Cho, Takamatsu City, Kagawa 761-0396, Japan
Graduate School of Engineering, Kagawa University,***Yamagami BLD. 3-12-2 Higashi-ikebukuro, Toshima-ku, Tokyo 170-0013, Japan
Micro Technica Co., Ltd. )Abstract: If a model begins to fail to prediction of the time series, we have to detect such a structural change (i.e.,
disparity between the prediction model and the data) quickly and correctly, and to renew the prediction model after thechange detection as soon as possible. In this paper, we formulate the structural change detection of time series as an
optimal stopping problem, using Dynamic Programming (DP). Moreover, we present experimental results of thechange point detection in multiple regression modeled time series data, comparing with SPRT and Chow Test.
Keywords: Time series analysis, Dynamic Programming, sequential probability ratio test (SPRT), Chow Test

I. INTRODUCTION

II. DP method, SPRT and Chow Test

Structural change detection for ongoing time series
data as early and correctly as possible is a very
important problem in a practical sense, especially in thefield of quality control and management of somethingthat depends on time. For example, early degradation
detection of the quality in communication system,
manufactures from production lines in a factory, and etc.,
are such kind of serious problems [1].
We have already proposed an application ofSequential Probabilistic Ratio Test (SPRT). Moreover,
we have proposed a Dynamic Programming (DP)method for the case where we have not only to detect
the change point, but also to take into account an action
cost after the detection. And we have presented theeffectiveness of the two methods in comparison with thewell-known Chow Test. Our experimentation has been
done just by using single regression model [2]. However,multiple regression models are more generally used fortime series data analysis than single regression one. And
also, the theory of Chow Test is based on the generalmultiple linear regression itself.
In this paper, we examine how the DP method,
SPRT and Chow Test work for the change detection ofmultipleregression
model
based data, by
experimentation.

1. DP method and SPRTFor simplicity, we explain the detection methods
using a linear single regression model as shown in Fig.1.
The concrete procedure of structural change detection is
as follows (see an example of time series data in Fig.1).
Step1: Make a prediction expression and set thetolerance band (a) (e.g. a=2σs) that means permissible
error margin between the predicted data and theobserved one.

©ISAROB 2011

2.DP method model ([2])
Step2：While monitoring the coming data, if thedata comes into a specified tolerance zone, then we callthe situation “in”, or “hitting”, otherwise “out” or“failing”. Based on this monitoring, we can judge thatthe structure of the time series data has changed, if the
failing occurs by continuing N times. This specified
tolerance is defined as, e.g., 2σ of the distribution error
as shown in the Fig.1.
We assume that the structural change is Poisson
occurrence of average, and that, once the change has
occurred during the observing period, the structure does
not go back to the previous one (Fig.2). The reason why
we set such a model is that we focus on the detection of
the first structural change in the sequential processing.

207

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

3. Definition of cost function
Let the cost (n) be a⋅ n an as a linear function for n,where "a" is the loss caused by the failing in one time.
And for simplicity, let T and A denote the Total
cost and Action cost. The Action cost means the costwhen some action has been taken at the time ofstructural change detection. Then, the evaluation
function is denoted as follows..

T = A+ a ⋅ n
We recursively define a function ET (n, N ) to obtain
the optimum number of times n that minimizes the
expectation value of the evaluation function of T, using
the concept of DP. Let N be the optimum number. Let
the function ET (n, N ) be the expectation value of the
Total cost at the time when the failing has occurred incontinuing n times, where n is less than or equal to N,i.e., 0 ≤ n ≤ N.
Then, the function is recursively given as in thefollowing.
( n = N ) ET ( n, N ) = A + a ⋅ N(1)n( n < N ) ET (n, N ) = P(Sn +1 | S ) ⋅ a ⋅ n+ (1 − P( Sn +1 | S n )) ⋅ ET (n + 1, N ) (2)where S n is the state of failing in continuing n times,
S n + 1 is the state of hitting for the (n+1) th timeobserved data, and P ( S n +1 | S n ) is the conditional
probability that the state S n + 1 occurs after the state S noccurred. Then, from the definition of the function
ET (n, N ) , the goal is to find the N that minimizes
ET ( 0, N ) , because the N is the same as n thatminimizes the expectation value (Eq.(2)).

Fig.1. Example of time series data where the change poi
nt tc* =70.

λ

1- λ

1.0

Ec

E

Ec : State that the structural change occurred.
E : State that the structure is unchanged.

λ : Probability of the structural change occurrence.
(Poisson Process.)(a) Structural change model

E

1-R

R
R
out

in

4. Optimal solution
For the aforementioned ET(0,N), the following
theorem holds, and gives the n that minimizes the
expectation value of the evaluation function of Eq. (2).
Theorem ([2]).
The N that minimizes ET(0,N) is given as the largest nu
mber n that satisfies the following Inequality (3).

1-R(b) Internal model of the State E.

a < ( A + a )・P( Sn | S n −1 )

Ec

1-Rc

Rc

5. Procedure of SPRT ([3])
The concrete procedure of structural change
detection is as follows (see Fig.1).
Step2: Set up the null hypothesis H0 and alternative
hypothesis H1. H0 means change has not occurred yet.
H1 means Change has occurred. Set the values α , βand compute C1 (= β/(1-α) )and C2 (= (1-β)/α ).
Initialize i = 0, λ0 = 1 .

Rc
in

out

1-Rc

(c) Internal model of the State Ec.
Fig.2. Structural change model of time series data.

©ISAROB 2011

(3)

208

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

III. Experimentation

Step3: Incrementing i (i = i+1), observe the following
data yi. Evaluate the error | εi | between the data yi andthe predicted value from the aforementioned prediction
expression.
Step4: Judge as to whether the data yi goes in thetolerance band or not, i.e., the εi is less than (or equal to)the permissible error margin or not. If it is Yes, then set
λi = 1 and return to Step3. Otherwise, advance to Step5.
Step5: Calculate the probability ratio λi , using thefollowing Equation (4) .
P (ε i | H 1 )(4)λi = λ i −1
P (ε i | H 0 )In our experimentation, time series data is generated
by the following equations.

y = a11 x1 + a12 x 2 + b + ε(t ≤ t *c )(5)y = a21 x1 + a22 x2 + b + ε(t *c ≤ t)(6)

where ε~N(0,σ2), i.e., the error ε is subject to theNormal Distribution with the average 0 and thevariation σ2, and tc* means the change point. In
addition, we have set tc*=70.
We have experimented with DP method and Chow
Test for the artificial data based on the above equations(5) and (6).

where, if the data yi goes in the tolerance band,P(εi |H0 )= θ0 and P(εi |H1 )=θ1, otherwise,P(εi |H0 )=(1-θ0) and P(εi |H1 )=(1-θ1).
Step6: Execution of testing.
(i) If the ratio λi is greater than C2 , dismiss the null
hypothesis H0, and adopt the alternative hypothesis H1,and then End.
(ii) Otherwise, if the ratio λi is less than C1 , adopt thenull hypothesis H0, and dismiss the alternative
hypothesis H1, and then set λi = 1 and return to Step3.
(iii) Otherwise (in the case where C1 ≤ λi ≤ C2 ),advance to Step7.
Step7: Observe the following data yi incrementing i.
Evaluate the error | εi | and judge whether the data yi
goes in the tolerance band, or not. Then, return to Step5(calculation of the ratio λi ).

1. Experimental conditions(i) Tolerant error: ± 2σ of the distribution on error ε.
(ii) The concrete values of parameters are shown in
Table 1. (Fig.4 shows an example of the graph ofgenerated time series data by the above equations. ).
(iii) Repetition times for making sets of data: M=100.
(iv) Parameters setting for detection:(a) SPRT: parameters are shown in Table 2(b) DP method: λ =0.01, A/a = 10, 20.
(c) Chow Test: significance level (α=0.05) for testing.
y = 3 x1 + 3 x 2 + 10 + ε

y = 2 x1 + 3 x 2 + 10 + εy180change point tc = 70

160

6.Chow Test ([3])
The well known Chow Test checks the significant
differences among residuals of three Regression Lines,
where regression Line 1 obtained from the data before achange point tc, Line 2 from the data after tc, and Line 3
from the whole data so far, by setting up hypothesis ofchange point at each point in the whole data.

140
120
100
80
60
40
20

t

0
0

y

Line 3 for the whole
d

20

30

40

50

60

70

80

90

100

Fig.4. Example of time series data in Data 1.
(True change point: t=70.)

Line 1 before

Table 1.Equations for generating time series

Line 2 after

Data No.

equation

equation(time t=1,2,…,69)(time t=70,71,…,100 )y = 2 x1 + 3 x2 + 10

y = 3 x1 + 3 x 2 + 10

52

y = 2.5 x1 + 3x 2 + 10

53

y = 2.5 x1 + 3x 2 + 10

1

1

Assumed change point tc
Fig.3. Conceptual image of Chow Test t©ISAROB 2011

10

209

σ

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Based on DP with Action Cost", Proc. of the 2004 IEEE
IRI, pp.402-407.
[3] Katsunori Takeda, Tetsuo Hattori, Izumi Tetsuya,
Hiromichi Kawano (2010), Extended SPRT for
Structural Change Detection of Time Series Based on
Multiple Regression Mode, Proc. of AROB15th‘10,
pp.755-758.

Table 2.Parameter values in SPRT
Case

Data No.

α

β

1-a
1-b
1-c
2-a
2-b
2-c
3-a
3-b
3-c

1

0.05

0.05

2

0.05

0.05

3

0.05

0.05

θ0

θ1
0.1
0.2
0.3
0.1
0.2
0.3
0.1
0.2
0.3

0.9
0.8
0.7
0.9
0.8
0.7
0.9
0.8
0.7

2. Results
The results are illustrated in Fig.5, where horizontal
axis shows observation time t (observation is started
after t=40) and vertical axis shows the time when thechange point was detected. Those results are based on
the average value of 100 times computation. It is
expected that the change point will be detected aroundthe time at t=70.
We have verified that the two methods (SPRT and
OS method) meet our intuition very well as follows.
(i) The graph of Chow test takes continuous value
for time t. This means that Chow test detects change atevery time when data is observed.
(ii) From Fig.5-Fig.7, even after the enough time
passes, Chow test cannot detect change point properly.
(iii) Both of SPRT and DP method can detect achange point more suitably than Chow Test.
(iv) Both of SPRT and DP method have a tendency
to detect a change point early when the σ is small. Thisis because, when the σ becomes smaller, the probability
that exceeds the tolerant interval (2σ) becomes larger,
i.e., the “failing” easily occurs.

1(Chow Test)

1(Chow Test)

1-a(SPRT)

1-b(SPRT)

1-c(SPRT)

A/a=10A/a=20100 tc
80
60
40
20
0
40

50

60

70

80

90

ｔ
100

(1) Case1
2(Chow Test)

2(Chow Test)

2-a(SPRT)

2-b(SPRT)

2-c(SPRT)

A/a=10A/a=20100 tc
80
60
40
20
0

IV. CONCLUSION

40

50

60

70

80

90

ｔ
100

(2) Case 2

We have made a comparison between both of SPRTand DP method and Chow Test using multiple
regression modeled time series data. We have found that
SPRT and DP method can detect the change point very
well for the real time ongoing data. Although the DP
method depends on the ratio A/a (A: Action cost, a: losscost), we can expect that it works well in a practicalsense. As further study, we investigate the relation
between the parameters of SPRT and OS method.

3(Chow Test)

3(Chow Test)

3-a(SPRT)

3-b(SPRT)

3-c(SPRT)

A/a=10A/a=20100 tc8060
40
20

REFERENCES

0[1] J.Chen and A.K.Gupta (2000), Parametric Statistical
Analysis, Birkhauser.
[2] Hiromichi Kawano, Tetsuo Hattori, Ken Nishimatsu(2004), "Structural Change Detection in Time Series©ISAROB 2011

40

50

60

70

80

90

ｔ
100

(3) Case 3
Fig.5. Relation between the observing time t anddetected change point tc.

210

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Comparison of Change Detection Methods for Ongoing Time Series
Data---- Extended SPRT, Chow Test, Extended DP ----

*Hiromichi Kawano, **Tetsuo Hattori, **Katsunori Takeda, ***Tetsuya Izumi( * NTT Advanced Technology/ Musashino-shi Nakamachi 19-18, Tokyo, Japan**Kagawa University/ 2217-20 Hayashi-Cho, Takamatsu City, Kagawa, Japan*** Micro Technica Co., Ltd./ Yamagami BLD. 3-12-2 Higashi-ikebukuro, Toshima-ku, Japan)(hattori@eng.kagawa-u.ac.jp)
Abstract: For change point detection of time series data, we have already proposed an application of Sequential Probab
ilistic Ratio Test (SPRT). In addition, we have proposed a Dynamic Programming (DP) method as well, for the case wh
ere we have not only to detect the change point, but also to take into account an action cost after the detection. In this paper, we show the effectiveness and differences of the two extended methods (ESPRT and EDP) in comparison with thewell-known Chow Test, by experimental results.
Keywords: Time series analysis, Dynamic Programming, sequential probability ratio test (SPRT), Chow Test
2. DP method model ([3])
The concrete procedure of structural change
detection is as follows (see an example of time series
data in Fig.1).
Step1: Make a prediction expression and set thetolerance band (a) (e.g. a=2σs) that means permissible
error margin between the predicted data and theobserved one.
Step2：While monitoring the coming data, if thedata comes into a specified tolerance zone, then we callthe situation “in”, or “hitting”, otherwise “out” or“failing”. Based on this monitoring, we can judge thatthe structure of the time series data has changed, if the
failing occurs by continuing N times. This specified
tolerance is defined as, e.g., 2σ of the distribution error
as shown in the Fig.1.

I. INTRODUCTION
Change point detection (CPD) problem in timeseries is to identify whether the generation structure ofmonitoring data has changed at some time point bysome reason, or not. We consider that the problem isvery important and that it can be applied to a wide range
of application fields. [1].
We have already proposed an application ofSequential Probabilistic Ratio Test (SPRT) and a
Dynamic Programming (DP) method for the case where
we have not only to detect the change point, but also totake into account an action cost after the detection. And
we have presented the effectiveness of the two methods
in comparison with the well-known Chow Test by using
multiple regression model [2]. We extend the definitionof the structural change point in the SPRT method(ESPRT), and show the improvement of the change
detection accuracy [2]. In this paper, we extend thedefinition of the structural change point in the DP
method (EDP) and we show the effectiveness anddifferences of the two extended methods (ESPRT and
EDP) in comparison with the well-known Chow Test,by experimental results.

II. DP method, SPRT and Chow Test
1. DP method and SPRTFor simplicity, we explain the detection methods
using a linear single regression model as shown in Fig.1.

©ISAROB 2011

Fig.1. Example of time series data where the change poi
nt tc* =70.

211

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

3. Cost function and optimal solution
Let the cost (n) be a⋅ n an as a linear function for n,where "a" is the loss caused by the failing in one time.
And for simplicity, let T and A denote the Total
cost and Action cost. The Action cost means the costwhen some action has been taken at the time ofstructural change detection. Then, the evaluation
function is denoted as follows.
(1)
T = A+ a ⋅ n
We recursively define a function ET (n, N ) to obtain
the optimum number of times n that minimizes the
expectation value of the evaluation function of T, using
the concept of DP. Let N be the optimum number. Let
the function ET (n, N ) be the expectation value of the
Total cost at the time when the failing has occurred incontinuing n times, where n is less than or equal to N,i.e., 0 ≤ n ≤ N.Then, the function is recursively given as
in the following.
( n = N ) ET ( n, N ) = A + a ⋅ N(2)expression.
Step4: Judge as to whether the data yi goes in thetolerance band or not, i.e., the εi is less than (or equal to)the permissible error margin or not. If it is Yes, then set
λi = 1 and return to Step3. Otherwise, advance to Step5.
Step5: Calculate the probability ratio λi , using thefollowing Equation (5) .
P (ε i | H 1 )(5)λi = λi −1
P (ε i | H 0 )where, if the data yi goes in the tolerance band,P(εi |H0 )= θ0 and P(εi |H1 )=θ1, otherwise,P(εi |H0 )=(1-θ0) and P(εi |H1 )=(1-θ1).
Step6: Execution of testing.
(i) If the ratio λi is greater than C2, dismiss the null
hypothesis H0, and adopt the alternative hypothesis H1,and then End.
(ii) Otherwise, if the ratio λi is less than C1, adopt thenull hypothesis H0, and dismiss the alternative
hypothesis H1, and then set λi = 1 and return to Step3.
(iii) Otherwise (in the case where C1 ≤ λi ≤ C2 ),advance to Step7.
Step7: Observe the following data yi incrementing i.
Evaluate the error | εi | and judge whether the data yi
goes in the tolerance band, or not. Then, return to Step5(calculation of the ratio λi ).

( n < N ) ET(n, N ) = P(Sn+1 | S n ) ⋅ a ⋅ n+ (1 − P(Sn+1 | S n ))⋅ ET (n + 1, N )(3)nwhere S is the state of failing in continuing n times,
S n + 1 is the state of hitting for the (n+1) th timeobserved data, and P ( S n +1 | S n ) is the conditional
probability that the state S n + 1 occurs after the state S noccurred.
For the aforementioned ET(0,N), the following
theorem holds, and gives the n that minimizes the
expectation value of the evaluation function of Eq. (1).

5.Chow Test ([2])
The well known Chow Test checks the significant
differences among residuals of three Regression Lines,
where regression Line 1 obtained from the data before achange point tc, Line 2 from the data after tc, and Line 3
from the whole data so far, by setting up hypothesis ofchange point at each point in the whole data.

Theorem ([3]).
The N that minimizes ET(0,N) is given as the largest
number n that satisfies the following Inequality (4).

a < ( A + a )・P( Sn | S n −1 )(4)y4. Procedure of SPRT ([2])
The concrete procedure of structural change
detection is as follows (see an example of time series
data in Fig.1).
Step2: Set up the null hypothesis H0 and alternative
hypothesis H1. H0 means change has not occurred yet.
H1 means Change has occurred. Set the values α , βand compute C1 (= β/(1-α) )and C2 (= (1-β)/α ),Initialize i = 0, λ0 = 1 .
Step3: Incrementing i (i = i+1), observe the following
data yi. Evaluate the error | εi | between the data yi andthe predicted value from the aforementioned prediction©ISAROB 2011

Line 3 for the whole data

Line 1 before tc
Line 2 after tc

Assumed change point tc
Fig.2. Conceptual image of Chow Test

212t

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

III. Extended DP and SPRTThen we have the equation (7) using Gauss notation.
So, the value of M depends on the parameters (Table 2).
That is, M=2 (case a), M=3 (case b), M=4 (case c).

1. Extended DP
The DP detects a change point at the time when the
expectation value of the evaluation function of Eq. (1) is
minimized. Then, the detected change point equals to
the terminated time point and its detection tends to bedelayed from true change point. Thus in this section, we
extend the definition of detected change point by DP
method. As such extension, we adopt the number tc-M
where tc is ordinary aforementioned change point andM is the number of times when the observed data
continuously goes of tolerance zone until the n thatminimize the expectation value. M is given by using
the posterior probability P(Ecn | Sn ) . According to the⎡⎤
1− β ⎥
M = ⎢ log θ
⎢
α ⎥
θ
⎢
⎥
1

0

IV. Experimentation
In our experimentation, time series data is generated
by the following equations.

Bayes’ theorem, the posterior probability P(Ecn | Sn ) is
given by the following (6).

P( Ecn | Sn ) =P( Sn ∩ Ecn )P( Sn ∩ Ecn ) ∪ P( Sn ∩ E n )(1 − λ ) n R n

=

1
(6)
1 + D (n )(time t=1,2,…,69)(time t=70,71,…,100 )y = 2 x1 + 3x2 + 10

y = 3 x1 + 3 x 2 + 10

5

y = 2.5 x1 + 3x 2 + 10

5

y = 2.5 x1 + 3x 2 + 10

1

Table 2 Parameter values in SPRT

∑ (1 − λ ) i R i λRcn − ii =0and λ means the probability of the structural change
occurrence and R is the probability of the failing when
the structure is unchanged, Rc is the probability of the
failing when the structural change occurred.

Case

Data No.

1-a
1-b
1-c
2-a
2-b
2-c
3-a
3-b
3-c

1

α
0.05

β
0.05

2

0.05

0.05

3

0.05

0.05

θ0

θ1
0.1
0.2
0.3
0.1
0.2
0.3
0.1
0.2
0.3

0.9
0.8
0.7
0.9
0.8
0.7
0.9
0.8
0.7

We have experimented with DP method and Chow
Test for the artificial data based on the above equations(8) and (9).

2.Etended SPRT([2])
The SPRT detects a change point at the time when
the probabilistic ratio λi is greater than C2 (=(1-β)/α).

1. Experimental conditions(i) Tolerant error: ± 2σ of the distribution on error ε.
(ii) The concrete values of parameters are shown in
Table 1.( Fig.4 shows an example of the graph ofgenerated time series data by the above equations. ).
(iii) Repetition times for making sets of data: M=100.
(iv) Parameters setting for detection:(a) SPRT: parameters are shown in Table 2(b) DP method: λ =0.01, A/a = 10, 20.
(c) Chow Test: significance level (α=0.05) for testing.

Then we adopt the number tc-M where tc is ordinary
aforementioned change point and M is the number oftimes when the observed data continuously goes oftolerance zone until the ratio λi > C2 The number M
can be obtained from the equation (7).

©ISAROB 2011σ.

(1 − λ ) n R n

θ1 M
1− β
)
)
> C 2 (=
θ0αequation

3

i=0

(

equation2∑ ( 1 − λ ) i R i λ R cn − iwhere D ( n ) = n −1

(9)

1

i=0

1 + n −1(t*c ≤ t)

Data No.

∑ (1 − λ ) i R i λ R cn − i + ( 1 − λ ) n R n1

y = a21 x1 + a22 x2 + b + ε

Table 1 Equations for generating time series

i=0

=

(8)

where ε~N(0,σ ), i.e., the error ε is subject to theNormal Distribution with the average 0 and thevariation σ2, and tc* means the change point. In
addition, we have set tc*=70.

∑ (1 − λ ) i R i λ R cn − i

n −1(t ≤ t *c )

2

n −1

=

y = a11 x1 + a12 x 2 + b + ε

(7)

213

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 3. Change detection point.
Data No.
123methodconditions
AverageStandard deviation
AverageStandard deviation
AverageStandard deviation

SPRT
θ0=0.1,θ1=0.9 θ0=0.2,θ1=0.8 θ0=0.3,θ1=0.7
69.25
74.50
76.54
6.35
4.73
4.97
75.39
83.84
86.88
11.70
9.22
7.98
68.85
71.88
73.53
6.02
1.90
1.70

DP
ChowTest
A/a=10 A/a=20 α=0.05
68.95
74.14
76.50
6.54
4.04
16.02
74.59
83.30
76.60
11.51
8.71
16.16
68.61
71.73
76.50
6.27
1.78
16.02

Table 4. Estimated change point.
methodData No.
conditions
Average1Standard deviation
Average2Standard deviation
Average3Standard deviation

SPRT
θ0=0.1,θ1=0.9 θ 0=0.2,θ1=0.8 θ 0=0.3,θ1=0.7
67.62
69.25
69.01
6.09
2.42
2.35
73.39
77.93
77.35
21.24
9.42
8.46
67.58
69.21
68.94
6.05
2.35
2.29

DP
ChowTest
A/a=10 A/a=20 α=0.05
67.95
72.14
58.21
6.54
4.04
20.38
73.71
81.24
57.09
11.42
8.72
20.95
67.61
69.37
58.35
6.27
5.36
20.38

Fig.3.Results for Power supply time series.
2. Results for artificial data
The results are shown in Table 3(Change detection
point) and Table 4(estimated change point using ESPRTand EDP). Those results are based on the average value
and standard deviation of 100 times computation. It is
expected that the change point will be detected aroundthe time at t=70.
Applying the extended definition, we obtain the
improvement of the change detection accuracy

IV. CONCLUSION
We have made a comparison between both of
ESPRT and EDP method and Chow Test using multiple
regression modeled time series data and real data. We
have found that ESPRT and EDP method works very
well. As further study, we investigate the relation
between the parameters of ESPRT and EDP method.

3. Results for real data
Results for the time series data of power generation
quantity in Japan are shown in Fig.3. In 1986
(Condition 1 denotes A/a=10, Condition 2 denotes
A/a=20). The “oil shock” happened in the worldwide
and oil prices suddenly rose. We can see that Chow Testcorrectly detects the change point of 1986, at 1990 and1991. Until then, it detects no change point. But after
1991, it outputs 1990 as change point. Both of ESPRTand EDP can estimate change point.

©ISAROB 2011

REFERENCES[1] J.Chen and A.K.Gupta (2000), Parametric Statistical
Analysis, Birkhauser.
[2] Katsunori Takeda, Tetsuo Hattori, Izumi Tetsuya,
Hiromichi Kawano (2010), Extended SPRT for
Structural Change Detection of Time Series Based on
Multiple Regression Mode, Proc. of AROB 15th ‘10,
pp.755-758.
[3] Hiromichi Kawano, Tetsuo Hattori, Ken Nishimatsu(2004), "Structural Change Detection in Time Series
Based on DP with Action Cost", Proc. of the 2004 IEEE
Intern’l Conf. on IRI, pp.402-407.

214

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Outdoor Autonomous Navigation Using SURF Features
M. Tabuse1, T. Kitaoka2 and D. Nakai3
1

Graduate School of Life and Environmental Science, Kyoto Prefectural University
1-5 Hangi-Cho Shimogamo, Sakyo-ku, Kyoto 606-8522, JAPANtabuse@kpu.ac.jp
2
Department of Informatics and Environmental Science, Kyoto Prefectural University
1-5 Hangi-Cho Shimogamo, Sakyo-ku, Kyoto 606-8522, JAPAN
3
Kyoto Prefectural Subaru High School
120 Mukaijima-Nishi-joke, Fushimi-ku, Kyoto 612-8156, JAPAN

Abstract: In this paper, we propose SURF feature based approach for outdoor autonomous navigation. In this approach,
we capture environmental images by using omni-directional camera and extract features of these images by using SURF.
We treat these features as landmarks and estimate robot self-location and direction of motion. SURF features areinvariant under scale change and rotation and robust under image noise, change in light condition and change viewpoint.
Therefore, SURF features are appropriate ones for robot self-location estimation and navigation.
Keywords: Mobile robot navigation, SURF feature, Robot self-location, Omni-directional camera

under scale change and rotation and robust under image
noise, change in light condition and change view point.
Furthermore, SURF calculation is several times fasterthan SIFT [2][3]. Therefore, SURF features areappropriate ones for robot self-location estimation andnavigation, because a mobile robot moves around andenvironmental images are captured from different
positions and directions.

I. INTRODUCTION
Recently, autonomous mobile robots are actively
researched. In order to work in the same environment as
people, the mobile robot must posses the ability to moveeverywhere. The most basic function for the mobile
robot is to be able to move to the destination
autonomously. Therefore, the robot must estimate selflocation and direction to the destination.
Many methods of autonomous navigation areproposed. Odometry is used in many mobile robots.
However, it has the cumulative error in the movement.
It is unsuitable to measures the movement of long
distance. Therefore, odometry is used with other sensors,e.g., GPS, visual sensor, laser range finder, gyroscope,
etc. GPS is a system that measures the position on theearth by using the space satellite. However, near a highbuilding the position is not always measured accurately
by GPS. Another navigation method is view based
approach. In view based approach, many images arememorized and self-location estimation is performed bytemplate matching. In general, image data is very large
amount of information and calculating cost of templatematching is large.
In this paper, we propose SURF feature [1] based
approach. In this approach, we capture environmental
images by using omni-directional camera and extract
features of these images by using SURF. We treat thesefeatures as landmarks and estimate robot self-locationand direction of motion. SURF features are invariant©ISAROB 2011

II. SYSTEM OVERVIEW
In our research, mobile robot navigation method
consists of two mode; teaching mode and navigation
mode. Fig.1 shows the flowchart of our method. In theteaching mode, the operator navigates the robot to a
destination. The robot memorizes a sequence ofenvironment using an omni-directional camera. In thenavigation mode, the robot captures omni-directional
environmental image, calculates SURF features in thisimage. Then the robot compares these features withSURF features of memorized images and estimates selflocation and direction of movement.

III. Self-Location Estimation
Firstly, we extract feature points in captured image
and in memorized images using SURF. Then we findfeature points in each memorized images matching tofeature points in capture image. We estimate that amemorized image which has maximum number ofmatching feature points is neighbor of capture image.

215

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Diagram of self-location estimation is shown in
Fig.2. At a starting of navigation, robot doesn’t know
where it is. Hence, we find matching feature points inall memorized images and estimate self-location. After
estimating the self-location, we find matching featurepoints in the image of self-location estimation and next
3 images.

in Fig.4. In each region, we calculate average angle,(front angle θ F , back angle θ B , left angle θ L , rightangle θ R ), respectively. In the case of going forward
shown in Fig.3 (a), θ L and θ R are large and similarangles. But, θ F and θ B are small. On the other hand,in the case of rotation shown in Fig.3 (b), θ F , θ B ,θ L and θ R are similar. Therefore, we define
movement angle Θ M and rotation angle Θ R as
follows:

ΘM =

ΘR =

θR −θL2

θF +θB
2

,

(1)

.

(2)

We determine movement of mobile robot using the
movement angle Θ M and the rotation angle Θ R of
the neighbor memorized image and the next memorized
image and correct the direction of mobile robot using
the rotation angle of the capture image and the neighbor
image. For more detail, see the next section.
(a)(b)
Fig.1. Flowchart of our method. (a) teaching mode. (b)navigation mode.

(a)(b)
Fig. 3. Movement of feature points. (a) going forward.
(b) turning left.

(a)(b)
Fig.2. Diagram of self-location estimation. (a) At firsttime, full searching. (b) After estimating the selflocation, searching the image of self-location estimationand next 3 images.

IV. Movement of Mobile Robot
In order to determine movement of mobile robot, we
calculate angles of feature points in the neighbormemorized image used in self-location estimation tomatching feature points in the next memorized image,and angles of feature points in capture image tomatching feature point in the neighbor memorized
image. Fig.3 shows feature points in an image and thematching feature points in the next image denoted bycircles and movement of feature points denoted by lines.
We divide omni-directional image in 4 regions, shown©ISAROB 2011

Fig. 4.

216

4 regions of omni-directional image.

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

V. Control System

V1 = f (1.85 + β ( Θ R + ∆Θ R )) ,(4)
where Θ M , Θ R and ∆Θ R denote movement

In our experiments, we use an electric wheelchair asa mobile robot. The wheelchair is controlled by joystick
controller. The joystick controller has two axes of
movement and outputs 2 channel voltages (V0 ,V1 ) .
When we move the joystick forward (back), the joystick
controller outputs V0 = 0 [V] and V1 = 1.85 [V]( V0 = 3.7 [V] and V1 = 1.85 [V] ) and the wheelchairgoes forward (back) at maximum speed, respectively.
When we move the joystick left (right), the joystick
controller outputs V0 = 1.85 [V] and V1 = 0 [V]( V0 = 1.85 [V] and V1 = 3.7 [V] ) and the wheelchairturns left (right) at maximum speed, respectively.
Controlling the wheelchair by PC, we setup AD/DA
converter in PC. We modify the wire lines connectingfrom the joystick controller to wheelchair main
controller, so that 2 channel outputs of the joystick
controller and of PC are switched with a switch (Fig. 5).

angle and rotation angle of feature points in theneighbor memorized image and the next memorized
image memorized image, and rotation angles of featurepoints of capture image and the neighbor memorized
image, respectively. α = 0.78 , β = 0.1552 ,determined by experiment, and0.0
f ( x) =  x
 3.7
Fig. 6.

( x < 0 .0 )( 0 .0 ≤ x ≤ 3 .7 )( x > 3.7 )2 channel outputs.

VI. EXPERIMENT

Fig. 5.

1. Mobile Robot
A mobile robot is an electric wheelchair (SUZUKI
MC2000). Omni-directional camera is setup at 150cm
in height (Fig.7).
Hardware and software of our mobile robot are as
follows:
Electric wheelchair : SUZUKI MC2000
Omni-directional camera :Camera : SONY DCR-HC 88
Omni-directional mirror : Vstone VS-C42N-TR
Note PC : Dell Latitude E6400(CPU : Core2Duo 2.66GHz, Memory : 2GB
OS : Windows XP)
AD/DA converter : CONTEC AD12-8(PM)C++ Compiler : Microsoft Visual C++ 6.0
Computer Vision Library : OpenCV 1.1preControl system.

Fig. 6 shows 2 channel outputs while the wheelchair
is moved by the joystick controller. In the area 1, twochannel outputs (0CH, 1CH) are ( V0 = 1.85 [V] ,V1 = 1.85 [V] ) and wheelchair is stopped state. In thearea 2, two channel outputs are ( V0 = 0 [V] ,V1 = 1.85 [V] ) and wheelchair goes forward at
maximum speed. In the area 3, two channel outputs are( V0 = 0.25 [V] , V1 = 3 [V] ) and wheelchair goesforward while turning right. In the area 4, two channel
outputs are ( V0 = 1.85 [V] , V1 = 0 [V] ) and wheelchairturns left.
Therefore, we define 2 channel outputs (V0 , V1 )as follows:

V0 = f (1.85 − αΘ M ) ,©ISAROB 2011

(3)

217

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig. 7.

#1

#60

#90

#140

#180

#240

Mobile robot
#308
#401
#424
Fig. 9.
Capture images in the navigation mode.

2. Teaching mode
In the teaching mode we control the wheelchairusing the joystick controller and teach a navigation
course. The course is about 150 m outdoor course on thecampus of Kyoto Prefectural University, where thewheelchair goes straight from the entrance of the libraryto the corner, turns to the right and goes straight to thenext corner, turns on the left and goes straight to thedestination (Fig.8). We memorize 282 omni-directional
images in this course.

Fig.10. Trajectories in teaching mode and navigation
mode.

VII. CONCLUSION
In this paper we present mobile robot navigation
using SURF features. The processing time of SURF isseveral times shorter than that of SIFT. Therefore, thenavigation speed of the mobile robot becomes near the
person walking speed.

Fig.8. Navigation course.

REFERENCES

3. Navigation mode
In the navigation mode, the mobile robot is moved
near the starting point and navigates to the destination
autonomously. Fig. 9 shows several captured images in
the navigation mode. Trajectories in the teaching andthe navigation mode are shown in Fig. 10. The mobile
robot navigates along the teaching course adjusting
direction of motion. Maximum error of navigation
trajectories is 3.3 m. Maximum speed is 4.5 km/h andaverage speed is 2.9 km/h.

©ISAROB 2011[1] H. Bay, A. Ess, T. Tuytelaars, L. van Gool,
Speeded-up Robust Features (SURF), Computer Vision
and Image Understanding (CVIU), Vol. 110, No.3, 2008,
pp. 346-359.
[2] D. G. Lowe, Object recognition from local scaleinvariant features, Proc. of IEEE International
Conference on Computer Vision (ICCV), 1999,
pp.1150-1157.
[3] D. G. Lowe, Distinctive image features from scaleinvariant keypoints, Proc. of International Journal of Co
mputer Vision (IJCV), Vol. 60, No. 2, 2004, pp. 91-110.

218

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Facial Expression Recognition of a Speaker Using Front-view Face Judgment,
Vowel Judgment, and Thermal Image Processing
T. Fujimura1, Y. Yoshitomi2, T. Asada2, and M. Tabuse2
1

Works Applications Co. Ltd., 1-12-32 Akasaka, Minato-ku, Tokyo 107-6019, Japan,
Graduate School of Life and Environmental Sciences Kyoto Prefectural University,
1-5 Nakaragi-cho, Shimogamo, Sakyo-ku, Kyoto 606-8522, Japan,
E-mail: yoshitomi@kpu.ac.jp

2

Abstract: For facial expression recognition, we previously selected three images: (i) just before speaking, and speaking(ii) the first vowel and (iii) the last vowel in an utterance. In this study, as a pre-processing module, we added a
judgment function to discriminate a front-view face for facial expression recognition. A frame of the front-view face ina dynamic image is selected by estimating the face direction. The judgment function measures four feature parameters
using thermal image processing and selects the thermal images that have all the values of the feature parameters within
limited ranges decided on the basis of training thermal images of front-view faces. As an initial investigation, we
adopted the utterance of the Japanese name “Taro,” which is semantically neutral. The mean judgment accuracy of thefront-view face was 99.3% for six subjects who changed their face direction freely. Using the proposed method, thefacial expressions of six subjects were discriminable with 87.7% accuracy when he or she exhibited one of theintentional facial expressions of “angry,” “happy,” “neutral,” “sad,” and “surprised.” We expect the proposed method tobe applicable for recognizing facial expressions in daily conversation.
Keywords: Facial expression recognition, Front-view face judgment, Speech recognition, Vowel judgment,
Thermal image processing
human feelings or inner mental states. Although
recognizing facial expressions has received considerable
attention in the field of computer vision research, themechanism of recognition still falls far short of humancapability, especially from the viewpoint of robustness
under widely varying lighting conditions. One of thereasons is that the nuances of shade, reflection, andlocal darkness influence the accuracy of facial
expression recognition through the inevitable change ofgray levels.
To avoid this problem and to develop a robust
method for facial expression recognition applicable
under widely varied lighting conditions, we did not use
a VR image, as would be expected. Instead, we used animage produced by infrared rays (IR), which describe
the thermal distribution of the face [1]–[9]. Although ahuman cannot detect IR, it is possible for a robot toprocess the information of the thermal images createdby IR. Therefore, as a new mode of robot vision,
thermal image processing is a practical method that is
viable under natural conditions.
The timing of recognizing facial expressions is also
important for a robot because the processing might betime-consuming. In a previous report, we adopted anutterance as the key to expressing human feelings ormental states because humans tend to say something
when expressing a feeling [5]–[9].

I. INTRODUCTION
To better integrate robots into our society, a robot
should be able to interact in a friendly manner withhumans. The goal of our research is to develop a robotthat can perceive human feelings and mental states.
The first stage is to develop a method for integrating
the information of human expression. The basic
information for integration is the visible ray (VR) image,thermal image, and voice. In this first stage, anautomatic, real-time, interactive system is not necessary.
It will be very difficult to equip a robot with a computer
which can process the information inputted to it with theefficiency of the human brain. Therefore, we chose toenable the robot to use a type of information, such as
thermal imaging, that the human brain cannot process.
Thermal imaging is a good example because it is
impossible for a human to perceive heat via the nakedeye.
The second stage is to develop an automatic, realtime, interactive system that has the information
integration of human expression as a processing
characteristic. The third stage is to develop a robot thathas a function developed from the synthesis of the firstand second stages for use in our daily lives.
The present investigation investigates the first stageof development, in which a robot can visually detect©ISAROB 2011219

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

III. PROPOSED METHOD

Our reported method [5]–[9] is applicable only to afront-view face. However, in daily conversations, we
often change our face direction, and therefore we needto select a frame of the front-view face in a dynamic
image. In this paper, we propose a pre-processing
module that has a judgment function to discriminate afront-view face. Using the module, a frame of the frontview face in a dynamic image is selected by estimating
the face direction. Consequently, by using the proposed
function for choosing the front-view face, facial
expressions are discriminable even when a personchanges his or her face direction freely.

In this study, as a pre-processing module, we add a
judgment function [12] of a front-view face to ourreported method for facial expression recognition.
Fig. 2 illustrates the flow chart of our method. We
have two modules in our system. The first is a module
for speech recognition and dynamic image analysis, andthe second is a module for learning and recognition. In
the module for learning and recognition, we embed themodule for front-view face judgment. The procedureincluding the proposed pre-processing module for frontview face judgment is explained in the following.

II. IMAGE ACQUISITION
The principle behind thermal image generation is the
Stefan–Boltzmann law, expressed as W  T 4 , where is emissivity,  is the Stefan–Boltzmann constant(=5.6705  10-12 W/cm2K4), and T is the temperature(K). For human skin,  is estimated as 0.98 to 0.99
[10], [11]. In this study, the approximate value of 1 wasused as  for human skin because the value of  foralmost all substances is lower than that of human skin[10]. Consequently, the human face region is easily
extracted from an image by using the value of 1 for 
when a range of skin temperatures is selected to produce
a thermal image [1]–[9], [12]. Fig. 1 shows examples offace images of a male, obtained by VR and IR. We canobtain a thermal image of the face without light, even atnight. In principle, the temperature measurement by IR
does not depend on skin color [11], darkness, or lighting
condition, and so the face region and its characteristics
are easily extracted from a thermal image.

Fig. 2. Flow chart of our method
1.

Front-view Face Judgment
We define the face rotation around the X, Y, and Z
axes, as demonstrated in Fig. 3. The training data is
calculated from the images of a front-view face. Afternormalizing the horizontal Feret's diameter of thesegmented face image, the centerline of the face region
in the limited region is drawn as the first standard line.
Each pixel point in the centerline has the same numberof pixels on both the left and the right sides of the point.
Then, a straight line as the second standard line is drawn

Fig. 1. Examples of a face image at night:(a) VR with lighting, (b) IR with lighting, (c) VR
without lighting, (d) IR without lighting [3]©ISAROB 2011220

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A. Parameter nod
The feature parameter nod is the value of thevertical Feret's diameter divided by the horizontal
Feret's diameter on the segmented face image. Thisparameter is used to evaluate the downward rotation
around the X axis (Fig. 4). Here, to eliminate theinfluence of the hair style and the throat region on thesegmented face image as much as possible, the area
having horizontal pixels less than an experimentally
decided threshold is ignored in the measurement of thevertical Feret's diameter on the segmented face image,and the area having vertical pixels less than anotherexperimentally decided threshold is also ignored in themeasurement of the horizontal Feret's diameter on thesegmented face image (Fig. 4). When a subject looks
downward, the feature parameter nod tends to besmaller than that of the front-view face of the subject.
B. Parameter up
The feature parameter up is the area ratio of theregion having a gray level of “0” in the rectangle
defined by the straight lines determined by themeasurements of the horizontal and vertical Feret’s
diameters to the rectangular area on the segmented face
image. This parameter is used to evaluate the upward
rotation around the X axis (Fig. 5). When a subject
looks upward, the feature parameter up tends to besmaller than that of the front-view face of the subject.
C. Parameter rotate
The feature parameter rotate is the area of the region
surrounded by the first and the second standard lines, on
which are explained in the first paragraph in Section III1 (“Front-view Face Judgment”). This parameter is used
to evaluate the rotation around the Y axis (Fig. 6).
Here, to eliminate the influence of the hair style and thethroat region on the segmented face image as much as
possible, the top and bottom of both the first and second
standard lines are set inside the face region by eachconstant pixel, which is also decided experimentally(Fig. 6). The feature parameter rotate tends to increase
when the rotation from the front-view face around the
Y axis increases.
D. Parameter lean
The feature parameter lean, which is the angle
between the second standard straight line and thehorizontal line, is used to estimate the rotation around
the Z axis. The value of lean is never greater than 90
degrees. The rotation is estimated to be small when thedeviation of the value of lean from 90 degrees is small.

from the upper to the lower edgepoints of the centerline.
The two standard lines are used to estimate thedeviation from the front-view face. To evaluate the face
direction, we use four feature parameters (nod, up,
rotate, lean), explained below.

Fig. 3. The three-dimensional coordinate system

Fig. 4. Downward rotation; (a) visible image,(b) thermal image, (c) binary image,(d) schematic diagram for the parameter nod

Fig. 5. Upward rotation; (a) visible image,(b) thermal image, (c) binary image,(d) schematic diagram for the parameter up©ISAROB 2011221

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011use the position where the absolute value of theamplitude of the wave form is the maximum while
speaking the vowel. For the timing position of the lastvowel, we apply the same procedure used for the firstvowel.

Fig. 6. Images expressing the face rotation aroundthe Y axis; (a) visible image, (b) thermal image,(c) binary image, (d) schematic diagram
for the parameter rotate

Fig. 8. Speech waveform of “Taro” and timing
positions for image frame extraction [8]3. Learning and Recognition
For the static images obtained from the extracted
image frames, the process of erasing the area of theglasses, extracting the face area, and standardizing theposition, size, and rotation of the face are performed
according to the method described in our previously
reported study [7]. In the next step, we generate
difference images between the averaged neutral face
image and the target face image in the extracted faceareas in order to perform a 2D discrete cosine transform(2D-DCT). The feature vector is generated from the 2DDCT coefficients according to a heuristic rule [6], [7].
As stated above, we use the speech recognition
system named Julius. Julius sometimes makes a mistakein recognizing the first and/or last vowel(s). For
example, /a/ for the first vowel might be misrecognized
as /i/. For the training data, we correct themisrecognition. However, a correction cannot be madeon the test data. The facial expression is recognized by
the nearest-neighbor criterion in the feature vector spaceby using the training data just before speaking and justspeaking the phonemes of the first and last vowels.

Fig. 7. Images expressing face rotation around theZ axis; (a) visible image,(b) thermal image, (c) binary image,(d) schematic diagram for the parameter lean
2. Speech Recognition and Dynamic Image Analysis
We use a speech recognition system named Julius[14] to obtain the timing positions of the start of speech,
and the first and last vowels in a WAV file [8], [9]. Fig.
8 shows an example of the waveform of the Japanesename “Taro”; the timing position of the start of speech
and the timing ranges of the first vowel (/a/) and the lastvowel (/o/) were decided by Julius. By using the timing
position of the start of speech and the timing ranges ofthe first and last vowels obtained from the WAV file,
three image frames are extracted from an AVI file at thethree timing positions. As the timing position just before
speaking, we use the timing position of 84 ms before thestart of speech, as determined in our previously reported
study [7]. As the timing position of the first vowel, we©ISAROB 2011

IV. EXPERIMENTS
1. Condition
The thermal image produced by the thermal video
system (Nippon Avionics TVS-700) and the soundcaptured from an Electret condenser microphone (SonyECM-23F5), amplified by a mixer (Audio-Technica ATPMX5P), were transformed into a digital signal by an
A/D converter (Thomson Canopus ADVC-300) and

222

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

three images at the timing positions of just before
speaking and just speaking the phonemes of the first andlast vowels. We had the training data of only /a/ for the
first vowel and only /o/ for the last vowel. If Julius
misrecognized the vowel of the test sample, thecorresponding image was not used for facial expression
recognition. We had four cases on misrecognition forvowel(s); (1) no misrecognition for the first and lastvowels, (2) misrecognition only for the first vowel, (3)misrecognition only for the last vowel, (4)misrecognition for both of the first and last vowels. We
prepared feature vectors of the training data in each of
the four cases.

input into a computer (DELL Optiplex 760, CPU: Intel
Core 2 Duo E7400 2.80 GHz, main memory: 2.0 GB,
and OS: Windows XP (Microsoft) with an IEEE1394
interface board (I·O Data Device 1394-PCI3/DV6). We
used Visual C++ 6.0 (Microsoft) as the programming
language. To generate a thermal image, we applied thecondition that the thermal image had 256 gray levels forthe range 5 to 12.9 K. Accordingly, one gray level
corresponded to 1.95  10-2 to 5.04  10-2 K. The
temperature range for generating a thermal image was
decided for each subject in order to easily extract theface area on the image. We saved the visual and audio
information in the computer as a Type 2 DV-AVI file, inwhich the frame had a spatial resolution of 720  480
pixels and 8-bit gray levels and the sound was saved ina PCM format of a stereo type, 48 kHz, and 16-bit
levels.
Six subjects exhibited in alphabetic order each of theintentional facial expressions of “angry,” “happy,”“neutral,” “sad,” and “surprised,” while speaking thesemantically neutral utterance “Taro.” Fig. 9 shows
examples of the thermal image of each subject. Subjects
A and B were males with glasses. Subjects C and D
were females without glasses. Subject E was a malewithout glasses. Subject F was a male with a cap. Fig.
10 shows examples of the images of subject A.
In the experiment, all subjects kept intentionally
front-view faces in the AVI files saved as the training
data, and they freely changed their face direction in the
AVI files saved as the test data. We obtained the featureparameter ranges for judging the front-view face withthe use of the training data and the proposed method
mentioned in Section III-1 (“Front-view Face
Judgment”).
The range wascalculatedasai   i  xi  ai   i for the nod, up, and rotate

Fig. 9. Examples of thermal images having a neutral
facial expression just before speaking

parameters of subject A, and ai  2 i  xi  ai  2 ifor the lean parameter of subject A and all featureparameters of other subjects, where x1 , x2 , x3 , x4 werenod, up, rotate, and lean, and a1 , a2 , a3 , a4 were theirmean values and  1 ,  2 ,  3 ,  4 were their standard
deviation values, respectively. In the case of subject A,
the difference of nod, up, and rotate between the frontview and the non-front-view faces was relatively small,and so the feature parameter ranges for judging thefront-view face were set to be narrower than those of
the other subjects. We assembled twenty samples as
training data and ten or less samples as test data, inwhich all facial expressions of all subjects were judged
as front-view faces by the proposed method. The
number of test data was decided as a result of the frontview face judgment. From one sample, we obtained©ISAROB 2011

Fig. 10. Examples of thermal images of subject A
having each facial expression in speaking

223

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

rays image processing. Proc. of the Annual Convention
IPS Japan, 2:339-340[2] Yoshitomi Y, Kimura S, Hira E, and Tomita S(1997), Facial expression recognition using thermal
image processing. IPSJ SIG Notes, CVIM103-3:17-24[3] Yoshitomi Y, Miyawaki N, Tomita S, and Kimura S(1997), Facial expression recognition using thermal
image processing and neural network. Proc. of 6th IEEE
Int. Workshop on Robot and Human Communication
380-385[4] Sugimoto Y, Yoshitomi Y, and Tomita S (2000), A
method for detecting transitions of emotional statesusing a thermal face image based on a synthesis of
facial expressions. J. Robotics and Autonomous
Systems 31:147-160[5] Yoshitomi Y, Kim SIll, Kawano T, and Kitazoe T(2000), Effect of sensor fusion for recognition ofemotional states using voice, face image and thermal
image of face. Proc. of 6th IEEE Int. Workshop on
Robot and Human Interactive Communication 178-183[6] Ikezoe F, Ko R, Tanijiri T, and Yoshitomi Y (2004),
Facial expression recognition for speaker using thermal
image processing (in Japanese). Trans. Human Interface
Society 6(1):19-27[7] Nakano M, Ikezoe F, Tabuse M, and Yoshitomi Y(2009), A study on the efficient facial expression using
thermal face image in speaking and the influence ofindividual variations on its performance (in Japanese). J.
IEEJ 38(2):156-163[8] Koda Y, Yoshitomi Y, Nakano M, and Tabuse M(2009), Facial expression recognition for a speaker of aphoneme of vowel using thermal image processing anda speech recognition system. Proc. of 18th IEEE Int.
Symp. on Robot and Human Interactive Communication
955-960[9] Yoshitomi Y (2010), Facial expression recognition
for speaker using thermal image processing and speech
recognition system. Proc.of 10th WSEAS Int. Conf. on
Applied Computer Science 182-186[10]Kuno H (1994), Infrared rays engineering (inJapanese). Tokyo, IEICE 22[11]Kuno H (1994), Infrared rays engineering (inJapanese). Tokyo, IEICE 45[12]Yoshitomi Y, Tsuchiya A, and Tomita S (1998),
Face recognition using dynamic thermal image
processing. Proc. of 7th IEEE Int. Workshop on Robotand Human Communication 443-448[13]Yamazaki S, Kamakura H, Tanijiri T, andYoshitomi Y (2004), Three-dimensional CG expressionof face rotation using fuzzy algorithm and thermal face
image (in Japanese). Trans. Human Interface Society,6(3):321-331[14]http://julius.sourceforge.jp/[15]Yoshitomi Y, Asada T, Shimada K, and Tabuse M(2011), Facial expression recognition of a speaker using
vowel judgment and thermal image processing. Proc. of
16th Int. Symp. on Artificial Life and Robotics inpress

2. Results and Discussion
The mean value of the front-view face judgment
accuracy for all subjects was 99.3%. The mean value ofthe recognition accuracy of vowels decided by Julius forall subjects was 93.6% for the first vowel (/a/) and
88.0% for the last vowel (/o/). Table 1 shows the facial
expression recognition accuracy for all subjects. The
mean recognition accuracy was 87.7%. The mean
recognition accuracy for each subject was 88% for
Subject A, 90% for Subject B, 88% for Subject C, 82%
for Subject D, 84% for Subject E, and 94% for Subject
F. We have applied the present method to a speaker of
any utterance to prepare the training data for all
combinations of the first and last vowels [15]. We
expect the proposed method to be applicable forrecognizing facial expressions in daily conversation.
Table 1. Recognition accuracy for all subjects
Input facial expression
Angry
Angry
Happy
Output

Happy

90
3.3

Neutral

1.7
93.3

NeutralSad
Surprised

1.7

10
80
5

5
5

Sad

15

3.3
86.7

Surprised

3.3
1.7
1.7
5
88.3

V. CONCLUSION
We developed a method for facial expression
recognition for a speaker by using thermal image
processing and a speech recognition system. In thispaper, we propose a pre-processing module that has ajudgment function for discriminating the front-view
face. Using the module, a frame of the front-view facein a dynamic image is selected by estimating the face
direction. The mean judgment accuracy of the frontview face was 99.3% for six subjects, who changed theirface direction freely. Using the proposed method, fivekinds of facial expressions of six subjects werediscriminable with 87.7% accuracy when the subjectchanged his or her face direction freely.
Acknowledgment
We would like to thank all the subjects who
cooperated with us in the experiments. This work was
supported by KAKENHI(22300077).

REFERENCES[1] Yoshitomi Y, Kimura S, Hira E, and Tomita S(1996), Facial expression recognition using infrared©ISAROB 2011

224

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Facial Expression Recognition of a Speaker Using Vowel Judgment andThermal Image Processing
Y. Yoshitomi1, T. Asada1, K. Shimada2, and M. Tabuse1
1

Graduate School of Life and Environmental Sciences Kyoto Prefectural University,
1-5 Nakaragi-cho, Shimogamo, Sakyo-ku, Kyoto 606-8522, Japan,
E-mail: yoshitomi@kpu.ac.jp, t_asada@mei.kpu.ac.jp, tabuse@kpu.ac.jp
2
Nova System Co., Ltd., 3-3-3 Nakanoshima, Kita-ku, Osaka 530-0005, Japan

Abstract: We previously developed a method for the facial expression recognition of a speaker. For facial expression
recognition, we previously selected three images: (i) just before speaking, and speaking (ii) the first vowel and (iii) thelast vowel in an utterance. By using the speech recognition system named Julius, thermal static images are saved at thetiming positions of just before speaking, and just speaking the phonemes of the first and last vowels. To implement our
method, three subjects, who spoke 25 Japanese first names providing all combinations of the first and last vowels, wereused to prepare first the training data and then the test data. Julius sometimes makes a mistake in recognizing the first
and/or last vowel(s). For example, /a/ for the first vowel is sometimes misrecognized as /i/. In the training data, we
correct this misrecognition. However, the correction cannot be performed in the test data. In the implementation of our
method, the facial expressions of three subjects were discriminable with the mean accuracy of 80.1% when the subjects
exhibited one of the intentional facial expressions of “angry,” “happy,” “neutral,” “sad,” and “surprised,” and the mean
accuracy of the speech recognition of vowels by Julius was 71.0%.
Keywords: Facial expression recognition, Speech recognition, Vowel judgment, Thermal image processing(IR), which show the thermal distribution of the face[7]–[16]. Although a human cannot detect IR, it ispossible for a robot to process the information in thethermal images created by IR. Therefore, as a new modeof robot vision, thermal image processing is a practicalmethod that is viable under natural conditions.
In addition, the timing of recognizing facial
expressions is important for a robot because theprocessing can be time-consuming. We adopted anutterance as the key to expressing human feelings ormental states because humans tend to say something to
express feeling [11]–[16].
In this paper, we briefly introduce our method [14]for the facial expression recognition of a speaker. For
facial expression recognition, we select three images: (i)just before speaking, and speaking (ii) the first voweland (iii) the last vowel in an utterance. A frame of thefront-view face in a dynamic image is selected byestimating the face direction [16]. To apply our method[14], three subjects, who spoke 25 Japanese first names
providing all combinations of the first and last vowels,
were used to prepare first the training data and then thetest data.

I. INTRODUCTION
To better integrate robots into our society, a robot
should be able to interact in a friendly manner withhumans. The goal of this research is to develop a robotthat can perceive human feelings and mental states. For
example, a robot could encourage a person who looks
sad, advise an individual to stop working and rest when
he or she looks tired, or take care of an elderly person.
The present investigation concerns the first stage ofthe development of a robot that has the ability to detect
visually human feeling or inner mental states. Although
the mechanism for recognizing facial expressions hasreceived considerable attention in the field of computer
vision research [1]–[6], at the present stage, it still falls
far short of human capability, especially from theviewpoint of robustness under widely varying lighting
conditions. One of the reasons for this is that nuances ofshade, reflection, and local darkness influence theaccuracy of facial expression recognition through theinevitable change of gray levels.
To develop a robust method for facial expression
recognition applicable under widely varied lighting
conditions, we do not use a visible ray (VR) image.
Instead, we used an image produced by infrared rays

©ISAROB 2011

225

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

II. IMAGE ACQUISITION
The principle behind thermal image generation is the
Stefan–Boltzmann law, expressed as W  T 4 , where is emissivity,  is the Stefan–Boltzmann constant(=5.6705  10-12 W/cm2K4), and T is the temperature(K). For human skin,  is estimated as 0.98 to 0.99[17], [18]. In this study, the approximate value of 1 wasused as  for human skin because the value of  foralmost all substances is lower than that of human skin[17]. Consequently, the human face region is easily
extracted from an image by using the value of 1 for 
when a range of skin temperatures is selected to produce
a thermal image [7]–[16], [19]. Fig. 1 shows examplesof face images obtained by VR and IR of a male.
Thermal images of a face can be obtained without light,that is, even at night. In principle, the temperature
measurement by IR does not depend on skin color [18],darkness, or lighting condition, and so the face region
and its characteristics are easily extracted from athermal image.

Fig. 2. Flow chart of our method [16]the module for learning and recognition, we embedded
the module for front-view face judgment. The procedure,
except the pre-processing module for front-view facejudgment [16], is explained in the following.
1. Speech Recognition and Dynamic Image Analysis
We use a speech recognition system named Julius[20] to save the timing positions of the start of speech,
and the first and last vowels in a WAV file [14]–[16].
Fig. 3 shows an example of the waveform of theJapanese first name “Taro”; the timing position of thestart of speech and the timing ranges of the first vowel(/a/) and the last vowel (/o/) were decided by Julius. By
using these three timing positions obtained from the
WAV file, three thermal image frames are extracted
from an AVI file. As the timing position just before
speaking, we use 84 ms before the start of speech, as
determined in our previously reported study [13]. As thetiming position of the first vowel, we use the position
where the absolute value of the amplitude of the wave
form is the maximum while speaking the vowel. For thetiming position of the last vowel, we apply the sameprocedure used for the first vowel.

Fig. 1. Examples of a face image at night:(a) VR with lighting, (b) IR with lighting, (c) VR
without lighting, (d) IR without lighting [9]III. PROPOSED METHOD
As a pre-processing module, we added a judgment
function [19] of a front-view face to our method [14] forfacial expression recognition [16]. Therefore, we canchoose a front-view face as the target for recognizing
facial expressions in daily conversation.
Fig. 2 illustrates the flow chart of our method. We
have two modules in our system. The first is a module
for speech recognition and dynamic image analysis, andthe second is a module for learning and recognition. In©ISAROB 2011

226

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011expression is recognized by the nearest-neighborcriterion in the feature vector space by using the training
data just before speaking and while speaking thephonemes of the first and last vowels.

IV. EXPERIMENTS
1. Condition
The thermal image produced by the thermal video
system (Nippon Avionics TVS-700) and the soundcaptured from an Electret condenser microphone (SonyECM-23F5), amplified by a mixer (Audio-Technica ATPMX5P), were transformed into a digital signal by an
A/D converter (Thomson Canopus ADVC-300) andinput into a computer (DELL Optiplex GX620, CPU:
Pentium IV 3.4 GHz, main memory: 2.0 GB, and OS:
Windows XP (Microsoft)) with an IEEE1394 interface
board (I·O Data Device 1394-PCI3/DV6). We used
Visual C++ 6.0 (Microsoft) as the programming
language. To generate a thermal image, we applied thecondition that the thermal image had 256 gray levels forthe detected temperature range. This range was decided
independently for each subject in order to best extract
the face area. We saved the visual and audio informationin the computer as a Type 2 DV-AVI file, in which aframe had a spatial resolution of 720  480 pixels and8-bit gray levels, and the sound was saved in a PCM
format of stereo type, 48 kHz, and 16-bit levels.
All subjects exhibited in alphabetic order each of theintentional facial expressions of “angry,” “happy,”“neutral,” “sad,” and “surprised,” while speaking thesemantically neutral utterance of each of the Japanese
first names listed in Table 1. Fig. 5 shows examples of
the thermal image of each subject. There were threesubjects. Subject A was a male without glasses. Subject
B was a male with glasses. Subject C was a female
without glasses. Fig. 6 shows examples of the images ofSubject A.

Fig. 3. Speech waveform of “Taro” and timing
positions for image frame extraction [14]2. Learning and Recognition
For the static thermal images obtained from theextracted image frames, the process of erasing the areaof the glasses, extracting the face area, andstandardizing the position, size, and rotation of the faceare performed according to the method described in ourpreviously reported study [13]. Fig. 4 shows the blocks
for extracting the face areas in a thermal image having
720  480 pixels. In the next step, we generate
difference images between the averaged neutral face
image and the target face image in the extracted faceareas in order to perform a 2D discrete cosine transform(2D-DCT). The feature vector is generated from the 2DDCT coefficients according to a heuristic rule [12], [13].

Fig. 4. Blocks for extracting face areas in the thermal
image [14]As stated above, we use the speech recognition
system named Julius, which sometimes makes a mistakein recognizing the first and/or last vowel(s). For
example, /a/ for the first vowel is sometimes
misrecognized as /i/. For the training data, we correct
the misrecognition. However, a correction cannot bemade on the test data. For example, when Julius
correctly judges the first vowel at the utterance of“Taro” but misjudges the last vowel as /a/, the training
data in speaking “Ayaka” are used for recognition
instead of those for speaking “Taro.” The facial©ISAROB 2011

Table 1. Japanese first names used in the experimentFirst vowel
a
i
u
e
o
shinnyatsubasa keita
tomoyaa ayakayuki
megumikoji
i kazuki hikari
Lastshigerufuyu
megu
noboruvowel u takeru
misae
yusukekeisukekozue
e kaedehirokoyuto
keiko
tomokoo taroIn the experiment, all subjects kept intentionally
front-view faces in the AVI files saved as both thetraining and test data. Accordingly, the pre-processing

227

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Subject A

B

C

Table 2. Accuracy (%) of speech recognition for Subject A
Firstlast
vowel
a-a

Fig. 5. Examples of thermal images having a neutral
facial expression just before speaking “Ayaka”

Angry

Happy

NeutralSad

Surprised

Mean

56

70

80

0

59

a-i

90

100

100

90

90

94

a-u

80

40

100

100

80

80

a-e

20

20

30

70

100

48

a-o

100

80

100

100

100

96

i-a

100

100

100

100

70

94

i-i

90

90

100

100

80

92

90

i-u

90

90

90

100

90

92

i-e

40

90

100

88

0

64

i-o

100

100

90

60

100

90

u-a

90

100

50

100

40

76

u-i

90

100

40

10

100

68

u-u

100

100

90

100

100

98

u-e

100

100

100

100

100

100

u-o

100

100

90

100

100

98

e-a

63

50

60

40

0

43

e-i

100

100

100

100

100

100

e-u

40

60

100

100

90

78

e-e

22

90

80

90

50

66

e-o

90

100

100

100

100

98

o-a

100

90

100

100

50

88

o-i

100

100

100

100

100

100

o-u

100

100

100

100

70

94

o-e

90

100

100

100

80

94

o-o

100

100

100

100

100

100

Mean

82

87

88

90

76

84

Table 3. Accuracy (%) of facial expression recognition
Input facial expression
Subject A
Fig. 6. Examples of thermal images of Subject A
having each facial expression in speaking “Ayaka”
module for judging the front-view face [16] was not
used in the experiment. We assembled 20 samples as
training data and 10 samples as test data. From one
sample, we obtained three images at the timing
positions of just before speaking and just speaking thephonemes of the first and last vowels. We obtained
training data for all combinations of vowel type of thefirst and last vowels.
2. Results and Discussion
The mean value of the recognition accuracy for the
first vowels and last vowels of all subjects was 71.0%.
The mean value of the recognition accuracy for the firstvowels and last vowels was 84% for Subject A, 70%
for Subject B, and 59% for Subject C. Table 2 shows
the recognition accuracy for the first and last vowels of©ISAROB 2011

OutputAngry
Happy
NeutralSad
Surprised

Subject B

OutputAngry
Happy
NeutralSad
Surprised

Subject C

Output

228

Angry
Happy
NeutralSad
Surprised

Angry

Happy

NeutralSad

Surprised

90.8
2.8
0.0
3.2
1.6

0.4
94.8
0.0
3.2
1.6

0.0
2.4
94
0.8
2.8

4.8
4.4
0.0
79.2
11.6

3.6
8.0
0.0
15.2
73.2

Input facial expression
Happy
NeutralSad

Surprised

Angry

64.1
10.8
0.0
7.2
17.9
Angry

78.8
2.4
10.8
2.8
5.2

5.2
77.6
3.6
7.6
6

6.7
6.9
0.4
78
8

9.4
12.6
0.0
10.9
67.3

Input facial expression
Happy
NeutralSad

Surprised

4.4
77.1
2.0
11.3
5.2

5.2
0.8
83.6
2.0
8.4

7.2
3.6
71.9
12.9
4.4

5.2
2.4
3.2
78.8
10.4

0.8
0.0
2.0
5.2
92

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 4. Accuracy (%) of facial expression recognition of
Subject A

Table 5. Accuracy (%) of facial expression recognition of
Subject B

Firstlast
vowel
a-a
a-i

Angry

Happy

Neutral

100

100

100

100

100

70

Surprised

Mean

100

0

80.0

Firstlast
vowel
a-a

100

90

92.0

a-i

Sad

Angry

Happy

NeutralSad

Surprised

Mean

10

70

10

90

90

54.0

50

50

100

40

44

56.8

a-u

90

100

100

100

90

96.0

a-u

80

90

100

89

89

89.6

a-e

100

100

100

100

100

100.0

a-e

22

100

100

78

80

76.0

a-o

100

100

100

60

90

90.0

a-o

30

100

100

60

50

68.0

i-a

60

80

100

60

60

72.0

i-a

100

30

90

70

80

74.0

i-i

100

100

70

100

100

94.0

i-i

40

80

100

90

10

64.0

i-u

100

100

100

100

40

88.0

i-u

70

100

100

100

100

94.0

i-e

90

100

100

90

90

94.0

i-e

90

100

90

90

78

89.6

i-o

90

100

90

90

100

94.0

i-o

60

100

100

90

80

86.0

u-a

90

100

60

40

70

72.0

u-a

60

10

70

90

30

52.0

u-i

100

100

100

0

100

80.0

u-i

0

90

90

40

70

58.0

u-u

100

100

90

90

100

96.0

u-u

100

40

40

90

70

68.0

u-e

100

100

100

100

100

100.0

u-e

100

100

100

83

100

96.6

u-o

100

100

90

100

90

96.0

u-o

80

40

100

80

90

78.0

e-a

100

80

100

50

10

68.0

e-a

90

90

90

90

90

90.0

e-i

80

100

100

70

100

90.0

e-i

90

90

100

90

0

74.0

e-u

100

80

100

90

100

94.0

e-u

10

80

100

90

20

60.0

e-e

100

90

80

80

0

70.0

e-e

100

100

10

100

70

76.0

e-o

10

100

100

50

30

58.0

e-o

90

100

100

100

80

94.0

o-a

100

60

100

100

10

74.0

o-a

50

90

100

70

90

80.0

o-i

100

90

100

90

100

96.0

o-i

50

60

100

10

50

54.0

o-u

100

100

100

30

90

84.0

o-u

50

80

100

80

80

78.0

o-e

90

100

100

90

90

94.0

o-e

80

90

0

100

70

68.0

o-o

70

90

100

100

80

88.0

o-o

100

60

100

40

70

74.0

Mean

90.8

94.8

94.0

79.2

73.2

86.4

Mean

64.1

77.6

83.6

78.0

67.2

74.1

image processing and a speech recognition system. To
implement our method, three subjects spoke 25
Japanese first names, which provided all combinationsof the first and last vowels. These subjects were used toprepare first the training data and then the test data for
all combinations of the first and last vowels. The mean
accuracy of the recognition of vowels by Julius was
71.0% for all subjects. Using our method, the facial
expressions of three subjects were discriminable with
80.1% accuracy when he or she exhibited one of theintentional facial expressions of “angry,” “happy,”“neutral,” “sad,” and “surprised.” We expect our method
to be applicable for recognizing facial expressions indaily conversation.

Subject A. Table 3 shows the facial expression
recognition accuracy as mean values over all
combinations of first and last vowels. The mean
recognition accuracy of the facial expressions of all
subjects was 80.1%. The mean recognition accuracy of
the facial expressions was 86.4% for Subject A, 74.1%
for Subject B, and 79.7% for Subject C. Tables 4 to 6
show all the values of the recognition accuracy of the
facial expressions of each subject. We have not yet
found a relationship between the recognition accuracyof vowels and the recognition accuracy of facial
expressions. As a continuation of our work, we will
apply our method for facial expression recognition byusing other subjects.

Acknowledgment
We would like to thank all the subjects who
cooperated with us in the experiments. This work was
supported by KAKENHI (22300077).

V. CONCLUSION
We previously developed a method for facial
expression recognition for a speaker by using thermal©ISAROB 2011229

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 6. Accuracy (%) of facial expression recognition of
Subject C
Firstlast
vowel
a-a

Angry

Happy

Neutral

Sad

100

67

30

40

Surprised

Mean

80

63.4

a-i

80

50

100

30

60

64.0

a-u

70

100

80

70

100

84.0

a-e

60

70

10

60

60

52.0

a-o

100

50

78

60

100

77.6

i-a

80

100

100

100

90

94.0

i-i

80

100

100

60

90

86.0

i-u

40

70

90

60

60

64.0

i-e

70

40

70

30

100

62.0

i-o

60

80

90

80

90

80.0

u-a

80

100

90

90

90

90.0

u-i

90

10

50

90

100

68.0

u-u

60

90

90

100

100

88.0

u-e

80

100

100

80

100

92.0

u-o

90

10

10

90

100

60.0

e-a

100

100

70

80

100

90.0

e-i

90

100

100

100

100

98.0

e-u

100

80

40

60

100

76.0

e-e

90

100

100

100

100

98.0

e-o

90

100

30

100

100

84.0

o-a

40

80

80

100

100

80.0

o-i

90

60

80

90

80

80.0

o-u

70

90

20

100

100

76.0

o-e

90

100

100

100

100

98.0

o-o

70

80

90

100

100

88.0

Mean

78.8

77.1

71.9

78.8

92.0

79.7

REFERENCES[1] Yuille AL, Cohen DS, and Hallinan PW (1989),
Feature extraction from faces using deformabletemplates. Proc. of IEEE Conf. on Computer Vision and
Pattern Recognition 104-109[2] Harashima H, Choi CS, and Takebe T (1989), 3-D
model-based synthesis of facial expressions and shapedeformation (in Japanese). Human Interface 4:157-166[3] Mase K (1990), An application of optical flow –extraction of facial expression. IAPR Workshop on
Machine Vision and Application 195-198[4] Mase K (1991), Recognition of facial expression
from optical flow. Trans. IEICE E74(10):3474-3483[5] Matsuno K, Lee C, and Tsuji S (1994), Recognitionof facial expressions using potential net and KL
expansion (in Japanese). Trans. IEICE J77-DII(8):1591-1600[6] Kobayashi H and Hara F (1994), Analysis of neuralnetwork recognition characteristics of 6 basic facial
expressions. Proc. of 3rd IEEE Int. Workshop on Robotand Human Communication 222-227©ISAROB 2011230

[7] Yoshitomi Y, Kimura S, Hira E, and Tomita S(1996), Facial expression recognition using infrared
rays image processing. Proc. of the Annual Convention
IPS Japan, 2:339-340[8] Yoshitomi Y, Kimura S, Hira E, and Tomita S(1997), Facial expression recognition using thermal
image processing. IPSJ SIG Notes, CVIM103-3:17-24[9] Yoshitomi Y, Miyawaki N, Tomita S, and Kimura S(1997), Facial expression recognition using thermal
image processing and neural network. Proc. of 6th IEEE
Int. Workshop on Robot and Human Communication
380-385[10] Sugimoto Y, Yoshitomi Y, and Tomita S (2000), A
method for detecting transitions of emotional statesusing a thermal face image based on a synthesis of
facial expressions. J. Robotics and Autonomous
Systems 31:147-160[11] Yoshitomi Y, Kim SIll, Kawano T, and Kitazoe T(2000), Effect of sensor fusion for recognition ofemotional states using voice, face image and thermal
image of face. Proc. of 6th IEEE Int. Workshop on
Robot and Human Interactive Communication 178-183[12] Ikezoe F, Ko R, Tanijiri T, and Yoshitomi Y(2004), Facial expression recognition for speaker using
thermal image processing (in Japanese). Trans. Human
Interface Society 6(1):19-27[13] Nakano M, Ikezoe F, Tabuse M, and Yoshitomi Y(2009), A study on the efficient facial expression using
thermal face image in speaking and the influence ofindividual variations on its performance (in Japanese). J.
IEEJ 38(2):156-163[14] Koda Y, Yoshitomi Y, Nakano M, and Tabuse M(2009), Facial expression recognition for a speaker of aphoneme of vowel using thermal image processing anda speech recognition system. Proc. of 18th IEEE Int.
Symp. on Robot and Human Interactive Communication
955-960[15] Yoshitomi Y (2010), Facial expression recognition
for speaker using thermal image processing and speech
recognition system. Proc.of 10th WSEAS Int. Conf. on
Applied Computer Science 182-186[16]Fujimura T, Yoshitomi Y, Asada T, and Tabuse M(2011), Facial expression recognition of a speaker using
front-view face judgment, vowel judgment, and thermal
image processing. Proc. of 16th Int. Symp. on Artificial
Life and Robotics in press[17]Kuno H (1994), Infrared rays engineering (inJapanese). Tokyo, IEICE 22[18]Kuno H (1994), Infrared rays engineering (inJapanese). Tokyo, IEICE 45[19]Yoshitomi Y, Tsuchiya A, and Tomita S (1998),
Face recognition using dynamic thermal image
processing. Proc. of 7th IEEE Int. Workshop on Robotand Human Communication 443-448[20]http://julius.sourceforge.jp/

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A Human-machine Cooperative System for Generating Sign Language
Animation Using Thermal Image Processing, Fuzzy Algorithm, andSimulated Annealing
T. Asada and Y. Yoshitomi
Div. of Environmental Sciences, Graduate School of Life and Environmental Sciences, Kyoto Prefectural University
1-5 Nakaragi-cho, Shimogamo, Sakyo-ku, Kyoto 606-8522, Japan
E-mail: t_asada@mei.kpu.ac.jp, yoshitomi@kpu.ac.jp
Abstract: We propose a method for sign language animation by skin region detection applied to a thermal infrared
image. In a system incorporating the proposed method, a 3D CG model corresponding to a person’s characteristic
posture while using sign language is generated automatically by pattern recognition of the thermal image, and then aperson’s hand in the CG model is set. The hand part is made manually beforehand. If necessary, the model can be
replaced manually by a more appropriate model corresponding to training key frames and/or the same generated model
can be refined manually. In our experiments, three hearing-impaired people, experienced in using sign language,
recognized the Japanese sign language gestures of 70 words expressed as animation with 94.3% accuracy. We further
improved the system by correcting the position and the direction of the hand of the automatically generated modelthrough the use of a fuzzy algorithm and simulated annealing.
Keywords: Japanese Sign Language, Thermal Image Processing, Computer Graphics, Model Fitting, Fuzzy Algorithm,
Simulated Annealing.
of the automatically generated model through the use ofa fuzzy algorithm and simulated annealing.

I. INTRODUCTION
Sign languages enable hearing-impaired people tocommunicate with each other. However, it isinconvenient for communication with non-hearingimpaired people because there are not enough sign
language interpreters. Therefore, there is a strong needfor an automatic sign language translation system. In
addition to a sign language recognition function, ananimation function is necessary for such a system.
Several systems for sign language animation have beenstudied [1-4]. For animations expressing personality
and/or emotion appropriately, conventional systems
need many extra manual operations. To lower theworkload of sign language systems, one of the mostpromising approaches is to improve the animation
automatically acquired from dynamic images of real
motion, with only the refinement performed manually.
We previously proposed a method for expressing a sign
language gestures as computer graphics (CG) animationthat uses a human thermal infrared image taken without
placing special restrictions on the person providing theimage [5].
In this study, we evaluated a system implementing
our method [5] for generating sign language animationfrom a dynamic image of a hearing-impaired person bymeasuring the recognition accuracy of other hearingimpaired people. Then, we further improved the system
by correcting the position and the direction of the hand

©ISAROB 2011

II. PROPOSED METHOD
The flowchart for generating sign language
animation from an input dynamic image is shown in Fig.
1. In the next sections, important elements in the total
procedure are described.
1. Thermal image generation
The thermal image is produced by a thermal video
system (NEC Avio Infrared Technologies Co., Ltd.,
Neo Thermo TVS-700), then transformed into a digital
signal by an A/D converter (Thomson Canopus ADVC300), and input into a computer with an IEEE1394
interface board (IO Data Device 1394-PCI3/DV6). Then,the image for each signed word is recorded as an AVI
file on a PC. The images that are input into thecomputer have a spatial resolution of 720 × 480 pixels
and 256 possible gray levels.
2. Selection of key frames from training images
After erasing the noise on each image frame in the
AVI file, the sum of the differences of the gray levels of
all pixels between the present frame and the previous
frame is calculated. On the assumption that thecharacteristic postures in sign language gestures
correspond to frames showing slower movement, aframe having a small gray level difference from theprevious frame is considered to be a candidate of231

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig. 2. Selection of key frames from candidates [5]Fig. 3. Human model (a), standard posture (b) [5]Fig. 1. Flowchart for generating sign language
animationa suitable frame (hereafter called a key frame) formaking a 3D CG model. That is, the sum of thedifferences of the gray levels for the previous frame isused for picking out several key frames. As the selection
criterion, we first select the top β % frames of theinverse value of the sum. However, similar successiveframes can be selected. To select the key frames, weremove all frames except the first and last frames (whenthree or more successive frames are selected) with thesame value of β for the inverse value of the sum of thegray level differences (Fig. 2). We use 25, 50, and 75 as
the values of β. All frames selected with each of thesethree values for a sign are designated as key frames. The
feature vector used for pattern recognition is made fromthe mosaic image after smoothing.

Fig. 4. Structure of the human model [5]Fig. 5. Manual model fitting [5]3. CG model generation for thermal images
The CG model has a hierarchical structure of 34
joints (Figs. 3 and 4) and is described with rotation
angles for the corresponding joints. The 3D CG model
corresponding to each key frame for a sign is mademanually (Fig. 5). We store the feature vectors of the
key frames and the corresponding 3D CG models as
training data.

recognition using the “nearest neighbor” criterion for
the feature vectors in the training data. When the user
judges that the CG model corresponding to the key
frame acquired from the input dynamic thermal image is
not appropriate, the model can be replaced manually bya more appropriate model corresponding to the one
made for the training key frames (Fig. 6) and/or thesame model can be refined manually. We assume that
we know the meaning of each sign for making theanimation. The hands in all CG models are replaced bythose made manually as training data. Then, theanimation is generated with the CG models

4. Animation
To animate a sign, key frames are selected according
to the process mentioned in Section II-2, followed by©ISAROB 2011232

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

recognition were used to make the training images andtest each sign. Three subjects (B, C, and D), who were
hearing-impaired and knew Japanese sign language,
evaluated the sign language animation produced by oursystem. The three subjects wrote down the meaning ofeach sign and rated it according to one of the following
three categories—Level 1: instantly understandable,
Level 2: understandable through checking the sign
language gesture expressed by the visible ray image fortraining, Level 3: not understandable (no answer). The
animations were produced under two conditions—
Condition 1: with neither the modification nor the
manual refinement of the models, Condition 2: withboth the modification (Fig. 6) and the manualrefinement of the models. Table 1 shows the correct
answer rates at each level. As shown in Table 2, somemisunderstandings occurred because subjects B and C
did not often check the sign language gestures
expressed by the visible ray image for training. The
modification of the models, an example of which isshown in Fig. 6, and the manual refinement of themodels improved the animation, especially at Level 2,
as shown in Table 1. The average correct answer ratio of
subjects B, C, and D, except for the misunderstandings
caused by their lack of referring to the sign language
gesture expressed by the visible ray image for training,
was 51.3% and 94.3% under Conditions 1 and 2,
respectively. The time required was approximately 20
minutes to make an animation for the sign language
gesture using the training image, whereas it tookapproximately 25 seconds and 15 minutes to make ananimation for the sign language gesture using the test
image under Conditions 1 and 2, respectively.

Fig. 6. Modification of model for key frame [5]corresponding to the key frames by smoothing eachrotation angle around the axis for each joint according
to the appropriate time sequence.

III. EXPERIMENTS AND DISCUSSION
1. Conditions
In the experiments, a personal computer, Dell
Dimension 8300 (CPU: Pentium IV 3.2 GHz, main
memory: 2.0 GB, OS: Windows XP), was used, andMicrosoft Visual C++ 6.0 was used for programming.
The temperature range for detection was 302.3 to 306.7 K.
First, 70 signs were selected for the training data andfor investigating the performance of our system
according to the following necessary sign conditions: anoun that includes a large motion observable by thelistener or the camera and does not include a meaning
movement of the head, a movement in the direction of
the listener or the camera, the crossing of hands, or theneed for an initial pose. With the thermal video systemfor obtaining the thermal images and a CCD camera forobtaining the visible images, 70 signs by subject A, who
was hearing-impaired and knew Japanese sign language,
were recorded twice as an AVI file for each word. The
first sign for the word was used to make the training
data, and the second sign for the word was used forrecognition and animation generation. In our
experiments, the sign language was Japanese.

Table 1. Number of correct answers at each level
Subject B

Subject C

Subject D

Level 2

Total

27/37 (73.0%)

3/7 (42.9%)30/70 (42.9%)

Condition 2

52/57 (91.2%)

6/6 (100%)

58/70 (82.9%)

Condition 1

24/33 (72.7%)

10/26 (38.5%)

34/70 (48.6%)

Condition 2

51/59 (86.4%)

8/10 (80.0%)59/70 (84.3%)

Condition 1

22/26 (84.6%)

15/18 (83.3%)

37/70 (52.9%)

Condition 2

54/55 (98.2%)

10/13 (76.9%)

64/70 (91.4%)

Table 2. Number of misunderstandings

2. Results
For the evaluation of our system, the training image
expressing the same meaning as that used forrecognition was not used. Therefore, 69 signs which did
not involve the sign having the same meaning as that for©ISAROB 2011

Level 1
Condition 1

Subject B
Subject C
Subject D

233

Reference shortage

Poor CG

Numberofincorrect
answers

Level
1

Level
2

Level
3

Level
1

Level
2

Level
3

12
11
6

2
6
0

0
1
2

6
0
1

3
2
1

0
1
1

1
1
1

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Max error (cm)

3. Estimation of the position errors in CG space
The accuracy rate under Condition 2 was muchhigher than that under Condition 1, whereas theworkload to generate an animation under Condition 2
was larger than that under Condition 1. To find somestrategies for improving the accuracy rate under
Condition 1, the position errors in several sign
languages gestures using only the movement of one
hand (the right hand) were evaluated in CG space. We
investigated the causes of the poor animation under
Condition 1 when Condition 2 provided a correctly
understandable animation but Condition 1 did not. To
find the cause of the position errors, we attachedmarkers to the hand. We compared each position of themarkers attached on a subject’s hand in physical spacewith the corresponding position on the model in CG
space. For simplicity, we selected animations in which
only the right hand moved while performing the sign
language gesture.
A. Inspection Method
We used the motion capture system named Radish(Library) for measuring the position of the markers
attached on the subject. We attached 15 markers each on
subject A and the model in CG space (Fig. 7). Then themodel was manually refined to fit the human posture.
The markers on the model were set as accurately as
possible on the positions in CG space so that they had
the same positions as those of subject A in physicalspace.

55
50
45
40
35
30
25
20
15
10
5
0

○ correct answer▲ incorrect answer (with left hand movement)● incorrect answer (without left hand movement)

Subject B

Max error (cm)

3

5

6

7

8

9

10

Avarage error (cm)Subject C
3

Max error (cm)

4

55
50
45
40
35
30
25
20
15
10
5
0
4

5

6

7

8

9

10

Avarage error (cm)

55
50
45
40
35
30
25
20
15
10
5
0

Subject D
3

4

5

6

7

8

9

10

Avarage error (cm)Fig. 8. Position errors of markers on subjects B, C, and D
causing the poor animation was found, as discussed in
Section III-4.
4. Discussion
The sign language animation for “Meiji Era”
generated under Condition 1 was compared with thatgenerated under Condition 2. Though the difference
between the animations generated under Conditions 1
and 2 was relatively small with regard to the hand orbit,
the rotation of the elbow under Condition 1 wasremarkably different from that under Condition 2 attime instances important for recognizing the animation(Fig. 9). In the case of the sign language animation for“younger brother”, the same phenomenon was observed(Fig. 10). Therefore, we thought that a technique tocorrect the direction of the arm including the elbow wasnecessary. We used the angular information and thevector expressing the hand direction for the key frameFig. 7. (a) Subject with attached markers, and (b) modelwith attached markers
B. Results
Fig. 8 shows the position errors of the markers
attached to the model in CG space. When an animation
under Condition 1 was correctly understood, theposition errors were, on the whole, smaller than those in
the case of misunderstanding the animation. However,even when the position error was small, the animation
was sometimes misunderstood. This phenomenon was
primarily caused by the unwanted movement of theother hand (left hand). However, another phenomenon©ISAROB 2011234

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011selected in the training images to produce a fuzzy
system to improve the hand orientation. When fitting amodel, as described in Section II-3, the hand direction
and the center of gravity coordinates of the threemarkers (A, B, and C) in the 3D CG model
corresponding to the key frame for the training image
were recorded. A hand direction is expressed as thevector from marker A to marker B (hereinafter called
vector B) and the vector from marker A to marker C(hereinafter called vector C) (Fig. 11).
Using cluster analysis, 487 key frames for the
training images of sign language gestures in which only
one hand moved were investigated. For this analysis,
Ward’s method was applied to 18 elements composed of
the angles of the shoulder, elbow and wrist, vector B,
vector C, the center of gravity coordinates of markers A,
B, and C. The number of groups divided by the cluster
analysis was used as the number of rules of the fuzzyalgorithm. This value was experimentally decided as 24.
The fuzzy rules are described in (1). The fuzzyalgorithm is described in (2) to (5).

Fig. 9. One scene of the sign language animation for the
Meiji Era under (a) Condition 1, (b) Condition 2

Fig.10. One scene of the sign language animation for“younger brother” under (a) Condition 1, (b) Condition 2

Fig. 11. Hand model with attached markers
and y *i denotes the output for yi . In this study, thecoordinates of the center of gravity for markers A, B,
and C attached on the model were assigned for( x1 , x2 , x3 ) ; the elements of vectors B and C wereassignedfor( x 4 , x5 , x6 )and( x 7 , x8 , x9 ) ,respectively; and the rotation angles on the shoulder, theelbow, and the wrist in the model were assigned for
y1 , y 2 ,, y9 , respectively. Each norm of vectors B
and C had the value of 1. Fig. 12 shows a schematic
diagram of the membership functions used for both theantecedent and the consequent, where Ci denotes a
fuzzy label of the antecedent or consequent, for example,
A11 , whereas Z corresponds to a variable of theantecedent or consequent, for example, x1 . Eachmembership function had a pentagonal shape and avalue of 1 at the corresponding measured mean value
and a value of 0.5 at the corresponding measured
minimum and maximum values. We used a fuzzy
where x1 , x2 ,, x9 denote the parameters of theantecedent; y1 , y 2 ,, y9 denote the parameters of theconsequent; Ai1 , Ai 2 ,, Ai 9 , Bi1 , Bi 2 , Bi 9 denotethe corresponding fuzzy labels; wi denotes the fitness
value of the i th rule to the input ofx *1 , x *2 ,, x *9 ;©ISAROB 2011

Fig. 12. Membership functions

235

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

algorithm having 24 rules, 9 antecedent valuables, and 9
consequent variables. The rotation angles of theshoulder, elbow, and wrist are inferred using this fuzzyalgorithm.
Next, we propose a method to improve the rotation
angle set to the joint of the elbow by decreasing thedifference of the hand direction for the test image to thatfor the training image, and the difference of the centerof gravity coordinates of markers A, B, and C of themodels before and after using the fuzzy algorithm. The
rotation angle of the elbow joint was acquired bysimulated annealing (SA). The amount of themodification in the rotation angle of each rotation axisof the elbow joint is nq ( q  1,2,3) . The value of a

Fig. 13. One scene of the animation of “Meiji Era”;(a) Condition 1, (b) Condition 2, and (c) Condition 3

uniform random number generated with the value of T to T is nq (T ) . We used the experimentally
decided (6), shown below, as the cooling schedule.
T0
T ( s) 
s  ,
(6)

log 2.0 

20
.
0


where T0 is the initial temperature and s is theFig. 14. One scene of the animation of “younger
brother”; (a) Condition 1, (b) Condition 2, and (c)Condition 3

number of steps of the iteration operation. As initialvalues for (6), the angles of the model obtained from therecognition result were used. The total error to be
reduced is defined as the sum of the errors in the hand
direction to that for the training image, and the error in
the center of gravity of the three markers attached on
the hand with respect to that obtained from therecognition result. When the total error obtained at an
iteration step is smaller than the minimum value
obtained before the step, the rotation angles of eachrotation axis of the elbow join are updated to those atthat step. The iteration operation is terminated at apreviously decided step. Condition 3 denotes thecondition for the sign language animation generated
under Condition 1 followed by improvement using the
fuzzy algorithm and SA. The sign language animations
generated under Condition 3 are more natural than thoseunder Condition 1 (Figs. 13 and 14).

simulated annealing, we improved the system to correct
the position and the direction of the hand of theautomatically generated model.
Acknowledgment
We would like to thank all the subjects who
cooperated with us in the experiments.

REFERENCES[1] Sagawa H, Sakou H, Oohira E, et al (1994), SignLanguage Recognition Method Using Compressed
Continuous DP Matching (in Japanese). IEICE Trans.
Inf. & Syst 77(4):753-763[2] Watanabe K, Iwai Y, Yagi Y, et al (1998), Manual
Alphabet Recognition by Using Colored Gloves (inJapanese). IEICE Trans. Inf. & Syst 80(10):2713-2722[3] Igi S, Watanatabe R, and Lu S (2001), Synthesis andEditing Tool for Japanese Sign Language Animation (inJpanese). IEICE Trans84 (6):987-995[4] Kurokawa T (2004), Representation of sign
animation for Japanese-into-Japanese sign languagetranslation (in Japanese). Proc. of 32nd Symp. ofvisualization 24(1):273-276[5] Asada T, Yoshitomi Y and Hayashi R (2008), A
Human-machine Cooperative System for Generating
Sign Language Animation Using Thermal Image.
Journal of Artificial Life and Robotics 13:36-40

IV. CONCLUSION
We developed an enhanced system for generating
sign language animation using skin region detection ona thermal infrared image. In this system, a 3D CG
model corresponding to a characteristic posture used insign language is generated automatically by patternrecognition of the thermal image, whereas the human
hand in the CG model is created manually. In
experiments, three people experienced in using sign
language recognized with good accuracy the Japanesesign language gestures of 70 words expressed asanimations. Then, using the fuzzy algorithm and©ISAROB 2011236

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Music Recommendation System Using the Time-series Discrete Wavelet
Transform and the FastICAK. Horiike1, Y. Yoshitomi2, T. Tokuyama3 and M. Tabuse2
1: Taka Dance Fashion Co., Ltd., 1-13 Ikutamamae-machi, Tennouji-ku, Osaka 543-0072, Japan
2: Graduate School of Life and Environmental Sciences, Kyoto Prefectural University,
1-5 Nakaragi-cho, Shimogamo, Sakyo-ku, Kyoto 606-8522, Japan,
E-mail: yoshitomi@kpu.ac.jp
3: Shofu Inc., 11 Kamitakamatsu-cho, Fukuine, Higashiyama-ku, Kyoto 605-0983, Japan
Abstract: We previously proposed a content-based music recommendation system that uses several strategies for music
recommendation. Each strategy, based on the user’s music evaluation history up to the present time, is composed of afeature vector and a decision rule for music recommendation. Vocal signals are an important factor in music
recommendation. Therefore, we use FastICA to separate vocal components from music. In an experiment to evaluate
the proposed system, 10 users rated 100 music files in a music database. Of these 100 music files, 49 contained vocal
components. For all 100 music files, the mean recommendation accuracy by the proposed system was 67.2%, and themean recommendation accuracy of random recommendation was 32.1%. For songs containing vocal signals, the mean
recommendation accuracy by the proposed system was 45.3% with FastICA and 35.8% without FastICA.
Keywords: Content-based music recommendation, Vocal characteristics, Independent component analysis, FastICA.

sk( 0) data are decomposed into the multi-resolution

I. INTRODUCTION

representation (MRR) and the coarsest approximation
by repeatedly applying the discrete wavelet transform(DWT). The wavelet decomposition coefficient

Currently, many computer users enjoy a large
number of music files stored on their computer or using
the Internet. The increasing amounts of available audio
data require the development of a music
recommendation system. There are two kinds of music
recommendation
systems:
content-based
recommendation and collaborative filtering. We
previously
proposedacontent-based
music
recommendation system [1].
In this study, in our music recommendation system,
we add the time series of Mel-frequency cepstralcoefficients (MFCCs) [2], [3] as the feature parameters.
Moreover, because vocal signals are an important factor
in a listener’s musical tastes, we add a module using
FastICA [4] for separating vocal signals from music.

( j)sequence sk

at level j is decomposed into two

wavelet decomposition coefficient sequences at levelj 1, as shown below in (1) and (2):s k( j  1 )   p n  2 k s n( j ) ,(1)

w k( j  1 )   q n  2 k s n( j ) ,

(2)

n

n

where pk and qk denote the scaling and wavelet
sequences, respectively, and w k( j  1 ) denotes thedevelopment coefficient at level j 1. The development
coefficients at level J are obtained by using (1) and(2) iteratively from j  0 to j  J  1 .
In the present study, we use Daubechies wavelet for
the DWT. As a result, we obtain the following relation
between p k and q k :II. FEATURE PARAMETERSq k  (－1) k p1－ k .

(3)

In the content-based music recommendation system,
we use three kinds of feature parameters forcharacterizing music: time series of wavelet transform
coefficients, time series based on MFCCs, and rhythmic
content [5]. We explain these feature parameters in thefollowing subsections.

It is known that the histogram of the wavelet
coefficients of each domain of MRR sequences has a
distribution centered at approximately 0 when the DWT
is performed on audio data [6]. We use the time series ofwavelet coefficients of each domain of the MRR
sequences as elements of the feature vector.

1. Wavelet transform coefficients

2. MFCCs
MFCCs are obtained for each frame of a soundsignal by using reported techniques [3]. The following
are used as elements of the feature vector: the time-

Original audio datask( 0 ) , where k denotes theelement number in the data, are used as the level-0
wavelet decomposition coefficient sequence. The©ISAROB 2011237

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

user, the principal components up to the l th component
are selected under the condition that the accumulated
contribution ratio first exceeds 80% at the l thcomponent. The score s for music m * file having
the maximum value of similarity to music m fileamong the number N Ms of evaluated music files isseries of the mean values and the standard deviations of
12-dimensional MFCCs, the logarithmic power of 12dimensional MFCCs, the 12-dimensional MFCC
difference between frames, and the 12-dimensional
MFCC logarithmic power difference between frames.
3. Rhythm content
Rhythmic content feature parameters are obtainedby using reported techniques [5]. A set of featureparameters based on a beat histogram are calculated.
These are as follows:
・A0, A1: the relative amplitude (divided by the sum of
the amplitudes) of the first and second histogram peaks,
respectively [5];・RA: the ratio of the amplitude of the second peak
divided by the amplitude of the first peak [5];・P1, P2: the period of the first and second peaks,
respectively, in beats per minute (bpm) [5];・SUM1, SUM2, SUM3: sum of beat strength in thehistogram in the range of 40–90, 90–140, 140–250
bpm, respectively.
Each of the three kinds of feature parameters, which are
a: [SUM1], b: [A0, A1, SUM1, SUM2, SUM3], c: [A0,A1, P1, P2, RA, SUM1, SUM2, SUM3], is used as afeature parameter in the proposed system described in
Section IV.

assigned to the score of music m file. The similarity
is calculated as the inverse value of the Euclidean
distance in the l -dimensional feature vector spaceobtained by the above PCA. When the assigned score is
4 or 5 (“slightly favorite” or “favorite”), the unevaluatedmusic m file is recommended for the user.
B. Decision rules using two or three kinds of feature vectors
Table 1 shows the decision rules for music
recommendation using two or three kinds of featureparameters.
Table 1. Decision rules for music recommendation
using two or three kinds of feature parameters

2. Recommendation system
The system finds a suitable strategy for music
recommendation for each user among 74 combinationsof decision rule and feature parameter(s), as shown in
Table 2, based on the user’s music evaluation history up
to the present time. Assuming that we have evaluatedK  1 music files, one unevaluated music m file isjudged recommendable or not recommendable
according to the following two criteria.
Criterion 1:
One music m * * file among K  1 evaluated music
files is considered an unevaluated music file, and eachstrategy of the 74 combinations of decision rule andfeature parameter(s) is used to judge whether musicm * * file is recommendable by using K  2
evaluated music files as training samples. Then, each
recommendation is checked using the score of musicm * * file to determine whether each judgment iscorrect. The recommendation accuracy for each strategy
is obtained by selecting one music m * * file in orderamongevaluatedmusic
files.
TheK 1
recommendation strategy having the best accuracy among

III. FastICA
Independent component analysis (ICA) [7], which isa statistical and computational technique for separating
hidden factors in a signal, is a promising technique forseparating vocal signals from songs [8]. In this study,
we use FastICA [4], for performing ICA.

IV. PROPOSED SYSTEM
To explain the decision rules for music
recommendation, we describe a set of music indices as
M  {m | 1,  , N M } , an evaluation by a user as a scores (1  s  5) , where 1, 2, 3, 4, and 5 mean “dislike,”“slightly dislike,” “neutral,” “slightly favorite,” and“favorite,” respectively, and a set of evaluated music
indices as Ms  {ms | 1,, NMs} .
1. Elements of decision rules for the recommendation
A. Decision rules using one kind of feature parameter
After principal component analysis (PCA) on thefeature vectors obtained from unevaluated music mfile and the number N Ms of evaluated music files of a

©ISAROB 2011238

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 2. Strategies for music recommendation in our system.
Strategy Decision
rule
no.

Main

1. Conditions
Fig. 1 shows the flowchart for the evaluation of thesystem. K was set as 100 in using 100 music files inthe database, while it was set as 49 in using the 49
music files with vocal.
We used 100 music files in the real world
computing (RWC) music database, which is available
for research [9]. To evaluate the performance of the
proposed system, all music files in the database wereassigned scores s (1  s  5) by 10 users. Of the 100

Other(s)Wavelet

1
2
3
4
5
6
7
8
9
10

V. PERFORMANCE EVALUATION

Feature vector

Using only
one kinds offeatureparameter(s)MFCC
Rhythm a
Rhythm b
Rhythm c
MFCC
WaveletWavelet11

MFCC

Rhythm cRhythm a15

Rhythm b

16

Rhythm c17

Rhythm a18

Rhythm b

19

Rhythm c

Wavelet

MFCC

B（Same as those described at No.6～19）

C（Same as those described at No.6～19）
MFCC, Rhythm a48
49

Wavelet

MFCC, Rhythm b
MFCC, Rhythm c

50
51

Wavelet, Rhythm a52

Wavelet, Rhythm b

D

MFCC

Wavelet, Rhythm c

53
54

Rhythm a55

Rhythm b

56

Rhythm c

57～65
66～74

music files, 49 with vocal signals were used to evaluate
FastICA as a pre-processing module. The featureparameters from wavelet transform coefficients andthose from rhythmic content were calculated under theconditions described in our previous reported paper [1].
To calculate the feature parameters from MFCCs, asound signal of 10 to 65 seconds from the beginning ofa music signal was used. The sound signal was obtained
under the following conditions: window length 30 ms,
shift pitch 10 ms, Hanning window used as the windowfunction, and 24 filter banks. The sound signal was
equally divided into five signals as the time-series. In
this experiment, a sound signal of 55 seconds was
divided into five sound signals of 11 seconds, and 52

Rhythm a
Rhythm b

A

14

20～33
34～47

Rhythm b
Rhythm c12
13

Rhythm a

Wavelet, MFCC

E

（Same as those described at No.48～56）

F（Same as those described at No.48～56）

all strategies is used to judge whether each unevaluatedmusic m file is recommendable by using K  1music files as training samples.
Criterion 2:
When an unevaluated music m file is judged to berecommendable under criterion 1 and the main feature
vector having the second highest similarity to thefeature vector obtained from the unevaluated music m
file is obtained from the music file having a score of 4or 5, the unevaluated music m file is judged to berecommendable.
Fig. 1. Flowchart for evaluation of the system©ISAROB 2011239

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

feature parameters were obtained for a sound signal of
11 seconds according to the method described in
Section II-2. As a result, we obtained a 260-dimensionalfeature vector from the sound signal of 55 seconds.

composed of a feature vector and a decision rule formusic recommendation. The strategy for each user
is based on the user’s music evaluation history up to
the present time. Since vocal signals are an importantfactor of a listener’s musical tastes, we use FastICA as apre-processing module to separate vocal signals fromsongs. In the experiment, 100 music files in the RWC
database were used for estimating the performance ofthe proposed system. The mean recommendation
accuracy by the proposed system was 67.2%, and themean accuracy of random recommendation was 32.1%.
For 49 songs containing vocal signals, FastICA
improved the mean accuracy of music recommendation.

2. Results and discussions
Table 3 shows the recommendation accuracy for 100
music files. The recommendation accuracy of criterion 2
was almost always better than that of criterion 1. The
mean recommendation accuracy by the proposed system(criterion 2) was 67.2%, while that of random
recommendation was 32.1%. For songs containing
vocal signals, the mean recommendation accuracy by
the proposed system with FastICA was 45.3%, but thatwithout FastICA was 35.8% under only criterion 1 (Fig.
2). These results confirm that FastICA improves therecommendation accuracy of songs with vocal signals.

Acknowledgment
We would like to thank all the participants who
cooperated with us in the experiments.

Table 3. Recommendation accuracy for 100 music files
Criterion 1

User
No.

Recommendation
accuracy

1
2
3
4
5
6
7
8
9
10
Mean

REFERENCES

Criterion 2[1] Tanaka M and Yoshitomi Y (2008), A music
recommendation system based on user preferencesusing the discrete wavelet transform (in Japanese). Proc.
of the 7th annual convention of forum on information
technology. 2:245-248[2] Davis S and Mermelstein P (1980), Experiments insyllable-based recognition of continuous speech. IEEE
Trans. Acoust., Speech Signal Processing. 28:357-366[3] Shikano K, Itou K, Kawahara T, Takeda K, andYamamoto M (2001), Speech recognition system.
Ohmsha (in Japanese)[4] Hyvärinen A (1999), Fast and robust fixed-point
algorithms for independent component analysis. IEEE
Trans. on Neural Networks. 10(3)626-634[5] Tzanetakis G and Cook P (2002), Musical genre
classification of audio signals. IEEE trans. speech audio
process. 10(5):293-302[6] Murata S, Yoshitomi Y and Ishii H (2007),
Optimization of embedding position in an audio
watermarking method using wavelet transform (inJapanese). Abstracts of autumn research presentation
forums of ORSJ 210-211[7] Hyvärinen A and Oja E (2000), Independent
component analysis: algorithms and applications.
Neural Networks. 13(4-5)411-430[8] Feng Y, Zhuang Y and Pan Y (2002), Popular song
retrieval based on singing matching. PCM '02 Proc. of
the Third IEEE Pacific Rim Conference on Multimedia:
Advances in Multimedia Information Processing. 639646[9] Goto M, Hashiguchi H, Nishimura T, and Oka R(2004), RWC music database: database of copyrightcleared musical pieces and instrument sounds forresearch purposes (in Japanese). Trans. of IPSJ.
45(3)728-738

Difference toDifference to
Recommendation
randomrandomaccuracy
recommendation
recommendation

55.6
40.0
66.7
59.1
44.4
50.0
84.6
68.8
35.5

15.6
11.0
32.7
17.1
15.4
23.0
50.6
33.8
-4.5

83.3
28.6
100.0
60.0
30.8
60.0
100.0
71.4
37.5

43.3
-0.4
66.0
18.0
0.8
33.0
66.0
36.4
-2.5

57.1

47.1

100.0

90.0

56.2

24.2

67.2

35.1
(%)

Fig. 2. Effect of FastICA on recommendation accuracy

VI. CONCLUSION
We propose a content-based music recommendation
system with several possible strategies. Each strategy is©ISAROB 2011240

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Music Recommendation System Aimed at Improving Recognition Ability
H. Konishi1 and Y. Yoshitomi2
1: Department of Environmental Information, Kyoto Prefectural University,
1-5 Nakaragi-cho, Shimogamo, Sakyo-ku, Kyoto 606-8522, Japan,
2: Graduate School of Life and Environmental Sciences, Kyoto Prefectural University,
1-5 Nakaragi-cho, Shimogamo, Sakyo-ku, Kyoto 606-8522, Japan,
E-mail: yoshitomi@kpu.ac.jp
Abstract: Recently, music therapy has been used for improving the recognition ability of people. Music therapy may be
more effective when the favorite music of each person is adopted. We propose a music recommendation method thatfuses content-based music recommendation and collaborative filtering. For characterizing the music in our contentbased music recommendation system, we use three kinds of feature parameters for characterizing music: time series ofwavelet transform coefficients, time series based on Mel-frequency cepstral coefficients, and parameters characterizing
the rhythmic content. Several strategies for music recommendation are prepared. Each strategy is composed of a feature
vector and a decision rule. The system can determine a good strategy for music recommendation for each user by not
only exploiting the user’s music evaluation history up to the present time but also the music evaluations by other users.
In the experiments, 12 users rated 52 songs coming from a textbook database of songs for elementary schools. The
number of recommended songs by the proposed method was 6.75 per user, and the number by collaborative filtering
was 5.17 per user. The recommendation accuracy of the proposed method was 81.8%, and that of collaborative filtering
was 74.1%.
Keywords: Recognition ability, Music therapy, and Music recommendation.

I. INTRODUCTION

1. Wavelet transform coefficients
Original audio data

In Japan, the average age of the population has been
increasing, and this trend is expected to continue.
Recently, music therapy has been used for improving
the recognition ability of people, particularly older
people. Music therapy may be more effective when thefavorite music of each person is adopted. To the best of
our knowledge, no research reports exist on thetechnology of music recommendation aimed at
improving recognition ability. In the present study, we
propose a music recommendation method that combines
content-based music recommendation [1] andcollaborative filtering to improve recognition ability. We
evaluate the proposed method with children’s songs,
which tend to be familiar to older people.

element number in the data, are used as the level-0(0)wavelet decomposition coefficient sequence. The sk

data are decomposed into the multi-resolution
representation (MRR) and the coarsest approximation
by repeatedly applying the discrete wavelet transform(DWT). The wavelet decomposition coefficient( j)sequence sk

at level j is decomposed into two
wavelet decomposition coefficient sequences at levelj 1, as shown below in (1) and (2):s k( j  1 )   p n  2 k s n( j ) ,(1)

w k( j  1 )   q n  2 k s n( j ) ,

(2)

n

n

where p k and q k denote the scaling and wavelet
sequences, respectively, and w k( j  1 ) denotes thedevelopment coefficient at level j 1. The development

II. FEATURE PARAMETERS

coefficients at level J are obtained by using (1) and (2)iteratively from j  0 to j  J  1 .
In the present study, we use Daubechies wavelet for
the DWT. As a result, we obtain the following relation
between p k and q k :In the content-based music recommendation system,
we use three kinds of feature parameters forcharacterizing music: time series of wavelet transform
coefficients, time series based on Mel-frequency
cepstral coefficients (MFCCs) [2], [3], and parametersdescribing the rhythmic content [4]. These parameters
are described in the following subsections.

©ISAROB 2011

sk( 0 ) , where k denotes theq k  (－1) k p1－ k .

(3)

It is known that the histogram of the wavelet
coefficients of each domain of MRR sequences has a
distribution centered at approximately 0 when the DWT
is performed on music [5]. We found out that the241The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

file and the number N Ms of evaluated music files of astandard deviation of the wavelet coefficients of eachdomain of the MRR sequences of music changed in its
time series. In addition, the change depended on themusic. Therefore, we use the time series of the wavelet
coefficients of each domain of the MRR sequences as
elements of the feature vector.

user, the principal components up to the l th component
are selected under the condition that the accumulated
contribution ratio first exceeds 80% at the l thcomponent. The score s for music m file having
the maximum value of similarity to music m fileamong the number N Ms of evaluated music files is

2. MFCC features
MFCCs are obtained for each frame of a soundsignal by the conventional method [3]. The following
are used as elements of the feature vector: the timeseries of the mean values and the standard deviations of
12-dimensional MFCCs, the logarithmic power of 12dimensional MFCCs, the 12-dimensional MFCC
difference between frames, and the 12-dimensional
MFCC logarithmic power difference between frames.

assigned to the score of music m file. The similarity
is calculated as the inverse value of the Euclid distancein the l dimensional feature vector space obtained by
the above PCA. When the assigned score is 4 or 5(“slightly favorite” or “favorite”), the unevaluated musicm file is recommended for the user.
2. Methods using two or three kinds of feature vectors
Table 1 shows the conditions for music
recommendation by methods using two or three kinds offeature parameters.

3. Rhythm content features
Rhythmic content feature parameters are obtainedby using the techniques described in Ref. [4]. A set offeature parameters based on a beat histogram arecalculated. These are as follows:
・A0, A1: the relative amplitude (divided by the sum of
the amplitudes) of the first and second histogram
peaks, respectively [4];・RA: the ratio of the amplitude of the second peak
divided by the amplitude of the first peak [4];・P1, P2: the period of the first and second peaks,
respectively, in beats per minute [4];・SUM1, SUM2, SUM3: the sum of beat strength in thehistogram in the range of 40–90, 90–140, and 140–
250, respectively, in beats per minute.
Each of the three kinds of feature parameters, which are
a: [SUM1], b: [A0, A1, SUM1, SUM2, SUM3], c: [A0,A1, P1, P2, RA, SUM1, SUM2, SUM3], is used as afeature parameter in the method described in Section III.

Table 1. Conditions for music recommendation bymethods using two or three kinds of featureparameters

IV. PROPOSED METHOD
Fig. 1 shows two music recommendation methods.
Method 1 is collaborative filtering only and it is used
for comparison with Method 2, which is our proposed
method that combines the content-based method andcollaborative filtering. Our system recommends music
stored in a database to user u , as described in Fig. 1.
In the flowchart of Method 2 described in Fig. 1,
the estimation of user u for music mR is set as 1
when the score of user u for music mR is 4 or 5(“slightly favorite” or “favorite”), and it is set as 0 when
the score is 1 to 3 (“dislike,” “slightly dislike,” or“neutral”). Moreover, mRCB is decided for user u by
using the most suitable recommendation method andfeature parameter(s) selected among 74 combinations[1], [6] of method and feature parameter(s) by ourpreviously reported method [1], in which we used thecontent-based recommendation method described inIII. CONTENT-BASED METHOD
To explain some of the methods for music
recommendation, we describe a set of music indicesas M  {m | 1,  , N M } , an evaluation by a user as ascore s (1  s  5) , where 1, 2, 3, 4, and 5 mean
“dislike,” “slightly dislike,” “neutral,” “slightly favorite,” and“favorite,” respectively, and a set of evaluated music indicesas Ms {ms |1,, NMs} .

1. Methods using one kind of feature parameter
After principal component analysis (PCA) on thefeature vectors obtained from unevaluated music m

©ISAROB 2011242

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Section III and the feature parameter(s) described in
Section II. In selecting the most suitable
recommendation method and feature parameter(s) in
Method 2, we use the scores of user u for music thathas already been recommended for and evaluated byuser u and the scores of other users in a reference user
list (UL) for music not yet recommended for user u .

V. PERFORMANCE EVALUATION
1. Conditions
Because older people tend to prefer children’s songs[7], we selected a CD described as an anthology of goodolder songs enjoyed by older people with dementia [8],and then we selected 52 songs on the CD that were also
included in a music textbook database for elementary
schools [9]. For evaluating the music recommendation
methods, all 52 of selected songs in the database wereassigned scores s (1  s  5) by 12 users (teens: 1,
twenties: 6, fifties: 5). Using the same conditions as
used in our previous research [1], the feature parameters
were obtained by the method described in Section II.
We used 10 as the value of K in Fig. 1. For
evaluating the two music recommendation methods
described in Section IV, we chose each of the 12 users
as user u and put the remaining users in the referenceuser list UL described in Fig. 1. Then, we obtained theresult of the music recommendation for each user foreach method described in Section IV.
2. Results and discussions
Table 2 shows the process of the music
recommendation process for user 8. As shown in Table
2, Method 2 tended to recommend more music and havea higher accuracy of music recommendation than did
Method 1. Fig. 2 shows the performance of the two
methods. The number of recommended songs by the
proposed method (Method 2) was 6.75 per user,
whereas that of collaborative filtering (Method 1) was
5.17 per user. The recommendation accuracy of the
proposed method was 81.8%, whereas that ofcollaborative filtering was 74.1%. For both therecommendation accuracy and the number ofrecommended songs, the proposed method was better
than collaborative filtering. In both methods, therecommendation process was terminated at the rate of
5/6, when the number of users staying in the UL became
zero. Accordingly, an increase in the number of users in
UL might contribute to an increased number ofrecommended songs.
Fig. 1. Flow chart of music recommendation methods©ISAROB 2011

243

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

VI. CONCLUSION

Table 2. Music recommendation process for user 8[Method 1]
Order

Recommended

AcceptanceUser No. in UL

We propose a music recommendation method
combining our previously reported method based onmusic features and collaborative filtering. We showedthat the proposed method is more effective for music
recommendation than using only collaborative filtering
when used on a music database composed of children’ssongs. In future work, we will increase the number ofusers who evaluate the music in the database and apply
the proposed method to people who are much older
and/or have a cognitive impairment.

music No.
152○

1,2,3,4,5,6,7,9,10,11,12

2

41

○

1, 3,4,5,6,7,9,10,11,12

3

50

○

1, 3,4,5, 7,9,10,
12

4

21

○

1, 3, 5, 7,9,

5

37

×

9

6

49

×

none

AcceptanceUser No. in UL

12[Method 2]
Order

Recommended

Acknowledgments
We would like to thank Associate Professor M.
Tabuse of Kyoto Prefectural University for his valuableadvice and support on this research. We would also liketo thank all the participants who cooperated with us in
the experiments.

music No.
152○

1,2,3,4,5,6,7,9,10,11,12

2

16

○

1,2, 3,4,5, 7,9,

3

3

○

1, 3,4,5, 7,9

4

26

1, 3,4,5, 7,9

5

21

○
○

6

5

○

1,

7

17

○

1,

8

4

○

1

9

13

×

(a)

12

REFERENCES

1, 3, 5, 7,9
5,[1] Tokuyama T, Yoshitomi Y, Tanaka M and
Tabuse M (2009), A study on music recommendation
system using the time-series discrete wavelet transform(in Japanese). Proc. of the 8th annual convention offorum on information technology. 2:325-328[2] Davis S and Mermelstein P (1980), Experiments insyllable-based recognition of continuous speech. IEEE
Trans. Acoust., Speech Signal Processing. 28:357-366[3] Shikano K, Itou K, Kawahara T, Takeda K, andYamamoto M (2001), Speech recognition system.
Ohmsha (in Japanese)[4] Tzanetakis G and Cook P (2002), Musical genre
classification of audio signals. IEEE trans. speech audio
process. 10(5):293-302[5] Murata S, Yoshitomi Y and Ishii H (2007),
Optimization of embedding position in an audio
watermarking method using wavelet transform (inJapanese). Abstracts of autumn research presentation
forums of ORSJ 210-211[6] Horiike K, Yoshitomi Y, Tokuyama T, and Tabuse
M (2011), Music recommendation system using thetime-series discrete wavelet transform and the FastICA.
Proc. of 16th Int. Symposium on Artificial Life andRobotics. in press[7] Takahashi T (1997), Research report on songs
familiar to people advanced in years (in Japanese)
Journal of Japanese music therapy associate 15(1):68-75[8] Akahoshi T (2009), Good old anthology enjoyablefor people advanced in years and troubled withdementia. Kirara shobo (in Japanese)[9] Music textbook database for elementary school by
Kanagawa prefectural education center (in Japanese);
http://kjd.edu-ctr.pref.kanagawa.jp/daizai_music/

7
7

none

Method 1

Method 2

Accuracy (%)

100
90
80
70
60
50
40
30
20
10
0

Number of recommended songs

1
10
9
8
7
6
5
4
3
2
1
0

2

3

4

5

6

7

8

user

9

10 11 12 Total

8

9

(b)

1

2

3

4

5

6

7

10

11

12

user

Fig. 2. Performance of music recommendation methods;(a) recommendation accuracy,(b) number of recommended songs©ISAROB 2011244

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

The study of path error for an Omnidirectional Home Care Mobile Robot
Jie-Tong Zou1and Feng-Chun Chiang2

1.

Department of Aeronautical Engineering,National Formosa University, Taiwan(scott@nfu.edu.tw)
2.
Institute of Opto-Mechatronics and Materials,WuFeng Institute of Technology, Taiwan(q053751519@yahoo.com.tw)
Abstract:
The first objective of this research is to develop an Omnidirectional Home Care Mobile Robot. The PC-based
controller can control the mobile robot platform. This service mobile robot is equipped with “Indoor positioning system”
and obstacle avoidance system. The indoor positioning system is used for rapid and precise positioning and guidance ofthe mobile robot. The obstacle avoidance system can detect static and dynamic obstacles.
In order to understand the stability of three wheeled omni-directional mobile robot, we make some experiments tomeasure the rectangular and circular path error of the proposed mobile robot in this research. From the experimentresults, the path error is smaller with the guidance of the localization system. The mobile robot can return tothe starting point. The localization system can successfully maintain the robot heading angle along a circular path.
Keywords: Home care; intelligent mobile robot; omni-directional; localization system; heading angle.
6. With the remote control system, remote family
member can control the robot and talk to the elderly.

I. INTRODUCTION
Nowadays, intelligent robots were successfully
fielded in hospitals [1], museums [2], and officebuildings/department stores [3], where they perform
cleaning services, deliver, educate, or entertain [4].
Robots have also been developed for guiding blind
people, as well as robotic aids for the elderly.
Rapid progress of standard of living and healthcare resulted in the increase of aging population. Moreand more elderly people do not receive good care fromtheir family or caregivers. Maybe the intelligent service
robots can assist people in their daily living activities.
Robotics aids for the elderly have been developed, butmany of these robotics aids are mechanical aids. [5] [6][7]. The intelligent service robot can assist elderly
people with many tasks, such as remembering to takemedicine or measure blood pressure on time.
A service mobile robot for taking care of elderly
people was developed in out laboratory [8]. This service
mobile robot is equipped with “Indoor positioning
system”. Five reflective infrared sensors are placedaround the robot for obstacle avoidance.
On the aid of an internet remote control system,
remote family member can control the robot and talk to
the elderly. This intelligent robot also can deliver the
medicine or remind to measure the blood pressure or
blood sugar on time. We hope this intelligent robot canbe a housekeeper or family guard to protect our elderly
people or our family. The functions of the proposed
robot are illustrated as follows:
1. Deliver medicine or food on time
2. Remind to measure and record the blood pressure or
blood sugar of the elderly on time
3. Remind the elderly to do something important
4. Assist the elderly to stand or walk
5. Send a short message automatically underemergency condition©ISAROB 2011

Indoor positioning
systemWireless network
moduleWireless IP
cameraHandlebar

Touch screen

Medicine, bloodpressure gauge
Power monitoring
systemEmergency
STOP

Omni-directional
wheel
Reflective infrared
sensors

Fig.1.The proposed Omnidirectional Home Care Mobile
Robot.
The proposed service mobile robot for taking careof elderly people is shown in Fig. 1. Hardware structureof the proposed mobile robot is shown in Fig. 2. A PC
based controller was used to control three DC servo
motors. The indoor positioning system was used for
rapid and precise positioning and guidance of themobile robot. Five reflective infrared sensors areconnected to an I/O card for sensor data acquisition.
The GSM modem can send a short message
automatically under emergency condition. The live
image of the wireless IP camera on the robot can betransferred to the remote client user.

245

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Wireless
network

IndoorH u ma npositioning machine
systeminterfacer=radius of wheel
R=distance from wheel to center of the platform

III. Indoor Localization System
GSM
modem
Wireless I
P cam

PC basedcontroller

I/O card

Infrared
sensors x5

Switching p
ower

24V battery

Driver 1

Driver 2

Driver 3

Motor 1

Motor 2Motor 3Fig. 4. Indoor localization system (Hagisonic co.)Fig.2. Hardware structure of the Omnidirectional Home
Care Mobile Robot.

As shown in Fig. 4, Indoor localization system[12], which used IR passive landmark technology, wasused in the proposed service mobile robot. The
localization sensor module (see Fig. 5) can analyze
infrared ray image reflected from a passive landmark
with characteristic ID. The output of position andheading angle of a mobile robot is given with very
precise resolution and high speed. The position
repetition accuracy is less than 2cm; the heading angle
accuracy is 1 degree.

II. The Omni-directional Robot Platform
The proposed omnidirectional home care mobile
robot is shown in Fig. 2. Many wheeled mobile robots
are equipped with two differential driving wheels. Since
these robots possess 2 degrees-of-freedom (DOFs), they
can rotate about any point, but cannot perform
holonomic motion including sideways motion[9]. To
increase the mobility of this service robot, three omnidirectional wheels driven by three DC servo motors areassembled on the robot platform (see Fig. 1). The omnidirectional mobile robots can move in an arbitrary
direction without changing the direction of the wheels.
The three-wheeled omni-directional mobile robots
are capable of achieving 3 DOF motions by driving 3
independent actuators [10] [11], but they may havestability problem due to the triangular contact area withthe ground, especially when traveling on a ramp withthe high center of gravity owing to the payload they
carry.

Fig. 5. localization sensor module (Hagisonic co.)IV. Experimental Results
In order to understand the stability of threewheeled omni-directional mobile robot, an experiment
for the straight line path error had been discussed [8].
From these experimental results, when the robot moves
faster or farther, the straight line error will increase. We
make some experiments to measure several different
paths error of the proposed mobile robot in this research.
1. Rectangular path error test for the omnidirectional robot platform.
In this experiment, the proposed mobile robot will
move along a rectangular path with or without theguidance of the indoor localization system. As shown in
Fig. 6, the mobile robot moves along a rectangular path(a→b→c→d→a) without the guidance of thelocalization system. The localization system is onlyused to record the real path in this experiment.
In Fig. 6, solid line represents the ideal rectangularpath, dot lines (■:Test1，▲:Test2) are the real paths ofthe mobile robot without the guidance of thelocalization system. The vertical paths (path b→c andd→a) have larger path error. Finally, the mobile robot
cannot return to the starting point “a”.

Fig.3. (a) Structure of Omni-directional wheel; (b)Motor layout of Robot platform
Fig. 3(a) is structures of the omni-directional
wheel, Fig. 3(b) is the motor layout of the robot
platform. The relationship of motor speed and robot
moving speed is shown as:(1)
Where:
Vi=Velocity of wheel iωi=rotation speed of motor i
ωP= rotation speed of robot©ISAROB 2011

246

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Vertical path (cm)

10
0
‐10
‐20
‐30
‐40
‐50
‐60
‐70
‐80
‐90
‐100
‐110
‐120

a

b

Fig. 8, the mobile robot moves along a circular path
without the guidance of the localization system. The
robot heading angle is 90°(upwards) during this test. The
localization system is only used to record the real pathin this experiment.
The circular path without the guidance of thelocalization system is shown in Fig. 9. The shape of thereal path is similar to a circle, but the starting point andthe end point cannot overlap. The heading angle error
with different circular angle (θ) of the robot is shownin Fig. 10. The maximum heading angle error is about 8°.

dc

Robot heading angle:90°-10 0 10 20 30 40 50 60 70 80 90 100110120

Horizontal path (cm)Fig. 6 Rectangular path error without the guidanceof the localization system.

Vertical path (cm)

10
a
0
‐10
‐20
‐30
‐40
‐50
‐60
‐70
‐80
‐90
‐100
‐110 d‐1200°bFig. 8 Circular angle (θ) of the robotc-10 0 10 20 30 40 50 60 70 80 90 100110120

Horizontal path (cm)Fig. 9As shown in Fig. 7, the mobile robot moves alonga rectangular path (a→b→c→d→a) with the guidanceof the localization system. In Fig. 7, solid line
represents the ideal rectangular path, dot lines(■:Test1，▲:Test2) are the real paths of the mobile
robot with the guidance of the localization system. With
the guidance of the localization system, the mobile
robot can pass through the corner points a, b, c, d. The
rectangular path error in Fig.7 is smaller than that in Fig.
6. The maximum path error is under 10 cm in Fig.7.
Finally, the mobile robot can return to the starting point“a”. The rectangular path is closed at point “a”.

Circular angle (θ) of the robot
Fig. 10 Heading angle error with different c
ircular angle (θ) of the robot
The circular path with the guidance of thelocalization system is shown in Fig. 11. The shape of
this path is more similar to a circle; the starting point
and the end point are overlapped. The heading angle
error with different circular angle (θ) of the robot isshown in Fig. 12. The maximum heading angle error isabout ±1°. From this experiment result, thelocalization system can successfully maintain the robot
heading angle along a circular path.

2. Circular path error test for the omni-directional
robot platform.
The omni-directional mobile robot can move in anarbitrary direction without changing the direction of thewheels. In this experiment, the proposed mobile robot
will move along a circular path with or without theguidance of the indoor localization system. As shown in©ISAROB 2011

Circular path without the guidance of thelocalization system.

Heading angle error

Fig. 7 Rectangular path error with the guidance ofthe localization system.

247

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ʼ11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

this experiment result, the localization system cansuccessfully maintain the robot heading angle along a
circular path.

REFERENCES
King, S., and Weiman, C, “Helpmate autonomous
mobile robot navigation system”, Proceedings of
the SPIE Conference on Mobile Robots, Vol. 2352,
pp.190–198, 1990.
[2]
Burgard, W.; Cremers, A.; Fox, D.; H¨ahnel, D.;
Lakemeyer, G.;Schulz, D.; Steiner,W.; and Thrun,
S. “Experiences with an interactive museum tourguide robot”, Artificial Intelligence,1999.
[3]
Endres, H.; Feiten, W.; and Lawitzky, G., “Field
test of a navigation system: Autonomous cleaningin supermarkets.”, Proceedings of the 1998 IEEE
International Conference on Robotics & Automation (ICRA 98),1998.
[4] Schraft, R., and Schmierer, G., “Serviceroboter”,
Springer verlag. In German,1998.
[5] Song, W.-K.; Lee, H.-Y.; Kim, J.-S.; Yoon, Y.-S.;
and Bien, Z. Kares, “Intelligent rehabilitation robotic system for the disabled and the elderly. Proceedings of the 20th Inter. Conf. of the IEEE Engineering in Medicine and Biology Society, Vol. 5,
pp. 2682–2685,1998.
[6] Dario, P.; Laschi, C.; and Guglielmelli, E., ”Design and experiments on a personal robotic assistant.”, Advanced Robotics 13(2) ,pp.153–69,1999.
[7] Takahashi, Y.; Kikuchi, Y.; T.Ibaraki; and Ogawa,S., “Robotic food feeder.”, Proceedings of the 38th
International Conference of SICE, pp. 979–
982,1999.
[8] Jie-Tong Zou , Kuo L. Su andFeng-Chun
Chiang, “The development of the Omnidirectional
Home Care Mobile Robot” , The Fifteenth International Symposium on Artificial Life and Robotics(AROB 15th ’10),B-Con Plaza, Beppu, Oita, Japan,
Feb. 4- 6, 2010.
[9] Jae-Bok Song and Kyung-Seok Byun, “Design
and Control of an OmnidirectionalMobile Robot
with SteerableOmnidirectional Wheels”, Mobile
Robots, Moving Intelligence, pp. 576, 2006.
[10] Carlisle, B. “An Omnidirectional Mobile Robot”,
Development in Robotics, Kempston, pp.79-87,
1983.
[11] Pin, F. & Killough, S., “A New Family of Omnidirectional and Holonomic Wheeled Platforms for
Mobile Robot”, IEEE Transactions on Robotics
and Automation, Vol. 15,No. 6, pp. 978-989. 1999.
[12] http://www.hagisonic.com/[1]

Fig. 11
Heading angle error

Circular path with the guidance of thelocalization system.

Circular angle (θ) of the robot
Fig. 12 Heading angle error with different circular angle(θ) of the robot

V. CONCLUSION
The objective of this research is to develop a
service mobile robot for taking care of elderly people.
This service mobile robot is equipped with “Indoor
positioning system”. The indoor positioning system isused for rapid and precise positioning and guidance ofthe mobile robot. Five reflective infrared sensors areplaced around the robot for obstacle avoidance.
In order to understand the stability of threewheeled omni-directional mobile robot, we make someexperiments to measure the rectangular and circularpath error of the proposed mobile robot in this research.
Firstly, the mobile robot moves along a rectangularpath without the guidance of the localization system.
The experimental paths have larger path error. Finally,
the mobile robot cannot return to the starting point.
With the guidance of the localization system, themobile robot can pass through the corner points. The
rectangular path error is smaller than that without theguidance of the localization system. The mobile robot
can return to the starting point. The rectangular path isclosed at point “a”.
Secondly, the proposed mobile robot can movealong a circular path with or without the guidance of theindoor localization system. The circular path without theguidance of the localization system cannot be closed.
The maximum heading angle error is about 8°.
The circular path with the guidance of thelocalization system is more similar to a circle; thestarting point and the end point are overlapped. The
maximum heading angle error is about ±1°. From

©ISAROB 2011248

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A* Searching Algorithm Applying in Chinese Chess Game
Cheng-Yun Chung1, Te-Yi Hsu2, Jyh-Hwa Tzou3, Kuo-Lan Su4,
1. graduate school Engineering Science and technology, National Yunlin University of Science & Technology
2. Industrial Technology Research Institute, Taiwan
3. Department of Aeronautical Engineering, National Formosa University, Taiwan
4. Department of Electrical Engineering, National Yunlin University of Science & Technology, Taiwan
E-mail: g9610808@yuntech.edu.tw, teyihsu@itri.org.tw, scott@nfu.edu.tw, sukl@yuntech.edu.tw
Abstract: The article presented A* searching algorithm based to be applied in path planning of Chinese chess game, andused multiple mobile robots to present the scenario. The mobile robot has the shape of cylinder and its diameter, height
and weight is 8cm, 15cm and 1.5kg. The controller of the mobile robot is MCS-51 chip. We play the Chinese chessgame using multiple mobile robots according to the evaluation algorithm of Chinese chess game, and calculate thedisplacement by the encoder. The A* searching algorithm can solve shortest path problem of mobile robots from thestart point to the target point on the chess board. The simulated results can found the shortest motion path for mobile
robots (chesses) moving to target points from start points in a collision-free environment. Finally, we implement thesimulated results on the Chinese chess broad using mobile robots. Users can play the Chinese chess game using thesupervised computer via wireless RF interface. The scenario of the Chinese chess game feedback to the user interface
using image system.
Keywords: A* searching algorithm, Chinese chess game, mobile robot, wireless RF interfaceI.

searching ability and to estimate the forgotten value [9]. We use A*
searching algorithm to program motion paths for multiple mobile
robots in the article.
The paper is organized as follows: Section II describes thesystem structure of the multiple robot based Chinese chess gamesystem, and describes the structure of the mobile robot. Section III
explains the evaluation method of the Chinese chess game using
multiple mobile robots, and described A* searching algorithm forpath planning of mobile robots from the start point moving to thetarget point. The experimental results are implemented in section
IV. Section V presents brief concluding comments.

INTRODUCTION

Chinese chess [1] game is one of the most popular games. A
two-player game with a complexity level is similar to Westernchess. In the recent years, the Chinese chess game has graduallyattracted many researches’ attention. The most researchers of the
fields is belong to computer science, expert knowledge andartificial intelligent. Chinese chess game is not only the mostold-line chess game in the world, but also more complex thanother chess game. There are many evolutionary algorithms to be
proposed. Darwen and Yao proposed the co-evolutionary
algorithm to solve problems where an object measure to guide thesearch process is extremely difficult to device [2]. Yong proposed
multiagent systems to share the rewards and penalties of successes
and failures [3].
The application of co-evolutionary models to learn Chinesechess strategies, and uses alpha-beta search algorithm, quiescencesearch and move ordering [4]. Wang use adaptive genetic
algorithm (AGA) to solve the problem of computer Chinese chess[5]. Lee and Liu take such an approach to develop a software
framework for rapidly online chess games [6]. Zhou and Zhang
present the iterative sort search techniques based on percentage
evaluation and integrate percentage evaluation and iterative sort
into problem of Chinese chess computer game [7]. Su and Shiau
develop smart mobile robot using microchip, and program the
trajectories for multiple mobile robot system [8]. The article usedthe evolutionary method to build up the rules of the Chinese chessgame. In some condition, the mobiles robot (chess) must avoided
the other chess moving to the next position. We used A* searchingalgorithm to program the short motion path on the Chinese chessgame. A* heuristic function are introduced to improve local©ISAROB 2011

II. SYSTEM ARCHITECTURE
The system architecture of the multiple robot based Chinesechess game system is shown in Fig 1. The system contains asupervised computer, an image system, a wireless RF interface, aremote supervised computer, a color CCD and thirty-two mobile
robots. There are two algorithms (evaluation algorithm and A*
searching algorithm) to be implemented in the supervised
computer. The Chinese chesses (Mobile robots) are classified redside and black side. There are sixteen chesses in each side. The
supervised computer can control mobile robots to present the
motion trajectory of the mobile robots, and receives the status of
the mobile robot via wireless RF interface. The signals contain the
ID code, orientation and displacement of mobile robots. The
supervised computer can transmit the ID code and command tothe mobile robot. The mobile robot moves the next pointaccording to the command, and transmits the ending code to thesupervised computer via wireless RF interface. The Chinese chesssystem can transmit the real-time image to the supervised

249

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

chess board, and the width of the corridor is 12cm. The mobile
robot uses IR sensors to detect obstacles, and decides the crosspoints of the chess board.

computer via image system. Users can play the Chinese chessgame with others on the supervised computer using mouse, or play
the game on the remote supervised computer via wireless Internet.
The mobile robot has the shape of cylinder, and it’s equippedwith a microchip (MCS-51) as the main controller, two DC
servomotors and driver devices, some sensor circuits, a voice
module, three Li batteries, a wireless RF interface and some reflect
IR sensors. Meanwhile, the mobile robot has four wheels toprovide the capability of autonomous mobility. The structure of
the mobile robot is shown in Fig. 2.

Fig. 3 The chess board of the Chinese chess gameIII. ALAGORITHM ANALYZE
The evaluation algorithm of the Chinese chess game isdefined by the attribution of the chesses. The definition of theboard is shown in Fig. 4. Then we define the initial position all
chess pieces. Such as the position of “red king” is (5,1), and “blackking” is (5,10)…etc. We plot the possible motion trajectory using
black line for the chesses on the board. Then we define theevaluation algorithm, and move the chess piece to the target point.
Such as the chess piece “black horse” can move to the position(1,8), (3,8) or (4,9). But the chess piece can’t move to the position(4,9) according the rules of the Chinese chess game. The chess“black horse” has an obstacle (black elephant) on the right side.
We define the start positions of the chesses to be ( x, y ) , anddefine the movement rules of the chesses as following. n is
movement displacement on the x axis, and m is movement
displacement on the y axis. n and m must be plus integrate.
The rules of the Chinese chesses are listed in the reference [10].

Fig. 1 The architecture of the Chinese chess system

Fig 2. The structure of the mobile robot
The controller of the mobile robot can acquires the detection
signal from sensors through I/O pins, and receives the command
via wireless RF interface, and transmits the detection results to thesupervised computer via wireless RF interface. The switch input
can turn on the power of the mobile robot, and selects power input
to be Li batteries or adapter. The encoder of the servomotor cancalculates the moving distance. We can set the pulse number for
per revolution to be P , and the mobile robot move pulse number
from the encoder to be B . We can calculate the displacement
D of the mobile robot using the equation.

D= 4 . 25× π

BP

Fig. 4 The definition of the Chinese chesses
A* searching algorithm is proposed by Hart in 1968, and solve
the shortest path problem of multiple nodes travel system. The
formula of A* searching algorithm is following

f (n ) = g (n ) + h(n )(1 )The core part of an intelligent searching algorithm is thedefinition of a proper heuristic function f (n ) . g (n ) is theexact cost at sample time n from start point to the target point.
h(n ) is an estimate of the minimum cost from the start point to
the target point. In this study, n is reschedules as n′ to

The diameter of the wheel is 4.25 cm. The chess board is grid
platform to be shown in Fig. 3. The arrangement of the chessboard is 11 grids on the horizontal direction (X axis), and is 12
grids on the vertical direction (Y axis). The distance is 30cm
between the center of corridor on the X axis and Y axis of the©ISAROB 2011

(2)

250

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

can’t find the path Cst moving to the target position using the
proposed method. The total distance Cst as

generate an approximate minimum cost schedule for the next point.
The equation (2) can be rewritten as follows:f (n ) = g (n ) + h(n′)

Cst = ∞

(3)

We make an example to explain algorithm. Such as a mobile
robot move to the target point “T” from the start point “S”. The
position of the start point is (2,6), and the target position is (2,9).
We set some obstacle on the platform. The white rectangle isunknown obstacle. The black rectangle (obstacle) is detected by
the mobile robot using A* searching algorithm. We construct twolabels (Open list and Close list) in the right side of Fig. 5. The
neighbour points of the start point fill in the “Open list”. The“Close list” fills the start point and evaluation points. We constructlabel on the first searching result to be shown in Fig. 5. We
calculate the values of f (n ), g (n ) and h(n ) function, anduse the proposed method to compare the values of the function.
We select the minimum value of the function f (n ) to be stored
in “Close list”. We can find the target point on the final searching
result to be shown in Fig. 5, and we can decide a shortest path tocontrol the mobile robot moving to the target point.

(5)

IV. EXPERIMENTAL RESULTS
We implement some experimental results on the multiple
mobile robots based Chinese chess game system. The firstexperimental scenario is “red elephant”. The user moves the chess“red elephant” using the mouse to be shown in Fig. 7 (a). The start
position of the “red elephant” chess is (3,1). The supervised
computer orders the command to the mobile robot “red elephant”.
The chess piece moves forward two grids, and turn left angle to beπ / 4 . Then the mobile robot moves two grids to the next position(5,3), and turn right angle π / 4 to face the black side and stop.
The user interface of the Chinese chess game system uses multiple
mobile robots to be shown in Fig. 7(a). The mobile robot cancalculate the displacement via encoder, and speak the movement
status of the mobile robot using voice module, too. The
experimental scenarios use the mobile robot to execute the motion
path of the chess “red elephant” to be shown in Fig. 7 (b)-(e).
There is not obstacle on the motion path of the chess. We don’t the
shortest path using A* searching algorithm.

Fig.5 The final searching result
The total distance of the shortest path Cst can be calculatedasm

Cst = ∑ Gn (m = t − 1) = 2043

(4)

(a)

n =1

(a)

(b)

(c)
(d)
Fig. 7 The experimental result for “red elephant”
Fig. 6 The final searching result

The second experimental scenario is “red cannon”. The user
moves the chess piece “red cannon” using the mouse to be shownin Fig. 8 (a). The start position of the “red cannon” chess is (8,3),

In the other condition, we rebuild the positions of the
obstacles on the platform to be shown in Fig. 6. The mobile robot©ISAROB 2011251

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

computer for the Chinese chess game system. The supervised
computer can control mobile robots using evaluation algorithm
according to the rule of Chinese chess game, and receive the status
of mobile robots via wireless RF interface. The chess (mobile
robot) has obstacle on the motion path, and uses A* searchingalgorithm to program the shortest path moving to the target point.
In future, we want to develop the artificial intelligent rule to decide
which side (red or black) to be a winner.

and the target point is (8,7). The motion path of the mobile robot(red cannon) has obstacle (chess). The supervised computer
reconstructs all possible paths using A* searching algorithm for redcannon step by step. Finally, we can find out the shortest path to
avoid the obstacle (chess). The path can displays on the interface
using red line. The supervised computer controls the mobile robot
moving to the target point from the start point using the shortest
path to avoid the obstacle. The experiment result is shown in Fig. 8(a). The supervised computer calculates the cost f (n ) to belisted on the left side of the Fig. 8 (a).
The supervised computer orders the command to the mobile
robot “red cannon”. The chess piece moves forward two grids, andturn left angle to be π / 4 . Then the mobile robot moves one grid
to, and turn right angle π / 4 . Then the mobile robot moves two
grids, and turn right angle π / 4 . Finally the mobile robot moves
one grid to the next position (8,7), and turn left angle π / 4 toface the black side and stop. Finally, the experiment scenarios areshown in Fig. 8(b)-(e).

ACKNOWLEDGMENT
This work was supported by the National Science Council of
Taiwan, R. O. C. under Grand NSC 99- 2221- E-224-060.
.

REFERENCE
1. S. J. Yen, J. C. Chen, T. N. Yang and S. C. Hsu (2004)
Computer Chinese Chess, ICGA Journal, Vol.27, No. 1,
pp.3-18
2. P. Darwen and X. Yao (2002) Coevolution in Iterated
Prisoner’s Dilemma with Intermediate Levels of Cooperation:
Application to Missile Defense, International Journal ofComputational Intelligence Applications, Vol. 2, No. 1,
pp.83-107
3. C. H. Yong and R. Miikkulainen (2001) Cooperative
Coevolution of Multi-agent Systems, University of Texas,
Austin, USA, Tech. Rep. AI01-287
4. C. S. Ong, H. Y. Quek, K. C. Tan and A. Tay (2007)
Discovering
ChineseChessStrategies
Through
Coevolutionary Approaches, IEEE Symposium on
Computational Intelligent and Games, pp.360-367
5. J. Wang, Y. H. Luo, D. N. Qiu and X. H. Xu (2005) Adaptive
Genetic Algorithm’s Implement on Evaluation Function in
Computer Chinese Chess, Proceeding of ISCIT, pp.1206-1209
6. W. P. Lee, L. J. Liu and J. A. Chiou (2006) A
Component-Based Framework to Rapidly Prototype Online
Chess Game for Home Entertainment, IEEE International
Conference on System, Man and Cybernetics, pp.4011-4016
7. W. Zhou, J. L. Zhang and Y. Z. Wang (2008) The Iterative
Sort Search Techniques Based on Percentage Evaluation,
Chinese Control and Decision Conference, pp.5263-5266
8. K. L. Su, S. V. Shiau, J. H. Guo and C. W. Shiau (2009)
Mobile Robot Based Onlin Chinese Chess Game, The FourthInternational
Conference on Innovative Computing,Information and Control, pp.63
9. Y. Saber and T. Senjyu (2007) Memory-bounded ant colony
optimization with dynamic programming and A* local searchfor generator planning, IEEE Trans. on Power System, Vol.22,
No. 4, pp.1965-1973
10. S. H. Chia, K. L. Su , C. C. Wang and S. V. Shiau (2010)
Multiple Robot System Applying in Chinese Chess Game,
The Innovative Computing, Information and Control –Express Letters (ICIC-EL),Vol. 4, No. 5(B), pp.2027-2032

(a)

(b)

(c)

(d)
(e)
Fig. 8 The experimental result with obstacle

V. CONCLUSION
We have presented a Chinese chess game system using
multiple mobile robots. The system contains a supervised
computer, a wireless RF interface, a remote supervised computer,an image system and thirty-two mobile robots (chesses). The
mobile robot has the shape of cylinder and its diameter, height andweight is 8cm, 15cm and 1.5kg, and executes the chess attributeusing two interfaces. One is wireless RF interface, and the other isvoice interface. We develop the user interface on the supervised©ISAROB 2011

252

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Multi-robot Based Intelligent Security System
Yi-Lin Liao1, Kuo-Lan Su2
1. graduate school Engineering Science and technology, National Yunlin University of Science & Technology
2. Department of Electrical Engineering, National Yunlin University of Science & Technology, Taiwan
E-mail: g9910801@yuntech.edu.tw, sukl@yuntech.edu.tw
Abstract: The article develops a multiple security modules based intelligent security system that has multiple
communication interfaces to be applied in home automation. The interfaces of the intelligent security system containwire RS485, wireless RF and Internet. The detection modules of the system have active security modules and passive
security modules. The passive security modules contain wire security modules and wireless security modules. The
control unit of all security modules is HOLTEK microchip. Each security module has two variety interfaces. They use
voice module to alarm users for event condition, and transmit the real-time event signals to the supervised computer via
wire RS485 or wireless RF interface. If the event occurrence, the supervised computer calculates the belief values using
Dempster-Shafter evidence theory according to the passive wire and wireless security modules. The belief value is over
the threshold. The supervised computer controls the mobile robot moving to the event location, and receives the signal
from the mobile robot via wireless RF interface, and recognizes the final decision output using Dempster-Shafter
evidence theory, and displays detection and decision output values on the monitor of the user interface. Finally, we
present some experimental results using wire passive security modules, wireless passive security modules and active
security modules on the fire detection and gas leakage detection using the platform of the intelligent security system.
Keywords: home automation, wire RS485, wireless RF, Internet, HOLTEK microchip, Dempster-Shafter evidence
theoryI.

INTRODUCTION
In recent years, mobile robots have been widely applied in the
security system as the active security modules. Recently more andmore research takes interest in the field especially intelligent
service robot. There are some successful examples, ASIMO, KHR,
QRIO, WABIAN-2R and AIBO. In our lab, we have been
designed a fighting mobile robot (ISLR-I) [6] and a module based
mobile robot [7]. The research field of mobile robot includes many
directions, such as motion planning, vision system, self-location,
speech recognition, supervised and remote supervisedcommunication and environment detection. Yoichi Shimosasa et
al. developed autonomous guard robot [8, 9] which integrate thesecurity and service system, the robot can guide visitors in daytime
and patrol at the night.
The paper is organized as follows: section II describes thesystem structure of the multiple mobile robots based intelligent
security system for intelligent home. The functions of the mobile
robot are described in Section III. The section IV presents theDempster-Shafter evidence theory to be applied in the intelligent
security system. Section V presents the experimental results of thesystem on the fire detection and gas leakage detection. The brief
concluding comments are described in Section VI.

Intelligent buildings and home can provide safety, convenienceand welfare for human living in the 21st century, and control andmanage resource with minimum life-time costs. An intelligent
building system (IBS) is the integration of various systems. They
contain security system, building heating, ventilating andair-conditioning (HVAC) technologies, computer system,
tele-communication and Internet. The most important role of theintelligent building is security system. In generally, the security
system contains supervised system, active security modules,
passive security modules and appliance control modules, and uses
redundant and complementally information fusion algorithms toenhance system reliability and certainty of intelligent building, andconstruct the safety network using multiple level protection.
In the past literatures, many experts research in the security
system. Wang and So [1] presented the history of development ofbuilding automation system (BAS). The structure of features of amodern BAS was introduced and future trends of BAS are
discussed. Azegami and Fujiyoshi [2] described a systematic
approach to intelligent building design. Kujuro and Yasuda [3]discussed the systems evolution in intelligent building. The qualityof building services can be enhanced by updated information
processing and communications functions of building automation
systems. Finley et al. [4] presented a survey of intelligent building
and reviews issues such as system perspective, subsystem services,and multi-tenant building. Chung and Fu expect to set up thestandard of appliances and communication protocols, and propose
a complete system architecture with integrate control kernel toconstruct an intelligent building system [5].

©ISAROB 2011

II. SYSTEM ARCHITECTURE
The system architecture of the multiple robots based intelligent
security system is shown in Fig 1. The system contains three levels.
There is passive detection level, active detection level and systemsupervised level [10]. We develop the user interface using Visual
Basic language for the security system. The system supervised

253

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011evidence theory.
The user interface of the intelligent security system is shown in
Fig. 3. The user interface contains two parts. The right side is fire
detection function. The other side is gas detection function. The
upper part of each side displays the image from the camera. The
bottom can displays the detection and decision results from three
security modules (wire/wireless security modules and mobile
robot), and controls the direction of the camera using mouse. We
use the green label to present no event status of the security
modules, and use red label to present event status for fire or gas
leakage event.

level receives detection signals via wire RS485 or wireless RFinterface, and control the multiple mobile robots moving to theevent location via wireless RF interface.
In the architecture, there are many security detection modules
and appliance control modules in the system. They are
independent and autonomous, and can work concurrently. Each
module of the security system can transmits the measurement
values, parameter values and decision results to the active security
modules and the supervised computer via wireless RF interface.
The active security and passive security modules can speech
Chinese on real-time event status using voice module. Users canreset the critical values of these modules from the user interface ofthe supervised computer.

Fig 3. The structure of the mobile robot
Fig. 1 The system architecture

III. MOBILE ROBOTThe mobile robot has the shape of cylinder, and it’s equippedwith a HOLTEK microchip as the main controller, two DC
servomotors and driver devices, one compass module, obstacle
detection devices, security modules (fire detection device or gas
detection module), a voice module, three Li batteries and awireless RF interface. Meanwhile, the mobile robot has four
wheels to provide the capability of autonomous mobility. The
mobile robot can carries variety sensors (fire or gas) moving on
the platform of the security system. The structure and the hardware
devices of the mobile robot are shown in Fig. 4.

Fig. 2 The platform of the security system
The controller of the security modules is HOLTEK microchip.
These modules of the passive detection module are classified three
types. There are wire security modules, wireless security modules
and wire/wireless appliance control modules. The active detectionlevel may be some remove platforms. The main device of the
security system is mobile robots. We arrange an ID code in each
module, and identify the module function by the ID code. The
experimental platform of the multiple mobile robots based security
system is shown in Fig. 2. The platform has three floors, andcontains three rooms for each floor. The supervised computer
controls the elevator to carry the mobile robots moving to theevent location via wireless RF interface. The mobile robot
communicates with the elevator via wireless RF interface, andcontrols the status of the elevator. Each room contains more than
one wire security module, one wireless security module and one
appliance control module. We arrange the same detection function
security modules with wire or wireless interface on the same
location enhance the detection precision using Dempster-shafter’s©ISAROB 2011

Fig 4. The structure of the mobile robot
The controller of the mobile robot can acquires the detection
signal from sensors using I/O pins, and receives the commandfrom the supervised computer via wireless RF interface, andtransmits the detection signals to the supervised computer via

254

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

compatible probability distributions. In addition, Dempster-Shafer’s theory allows the fusion of several sources using theDempster’s combination operator. It is defined like the orthogonalsum (commutative and associative) following the equation:

wireless RF interface. The detection signals are fire event or gas
leakage signal. The switch input can turn on the power of themobile robot, and selects power input to be Li batteries or adapter.
The encoder of the servomotor can calculates the moving distance.
We can set the pulse numbers of per revolution to be P on thewheel of the mobile robot, and the mobile robot move pulse
numbers to be B . We can calculate the displacement D of themobile robot using the equation

D = 4.25 × π

BPm( Hn) = m1 ( Hn) ⊕ ⋅ ⋅ ⋅ ⊕ mM ( Hn)

For two sources S i and S j , the aggregation of evidence for ahypothesis Hn ⊆ θ can be written:(1)m( Hn) =The diameter of the wheel is 4.25 cm. The compass module
can transmits the measurement values ( x and y ) of two axes (Xaxis and Y axis) to the controller of the mobile robot. It cancalculate the orientation angle θ using Eq. (2). The supervised
computer can controls the direction of the mobile robot using thecompass module.

θ = Tan −1 (− y / x )(4)

The normalization coefficient Κ evaluates the conflict between
two sources. An additional aspect of the Dempster-Shafer’s theory
concerns the attenuation of the basic belief assignment m j by acoefficient α j for a source S j . For all Hn ⊆ θ , theattenuated belief function can be written as:∑ m( A)(6)(10)

m(α , j ) (θ ) = 1 − α j + α j ⋅ m j (θ )(11)

In the intelligent security system, we use wire and wireless gas
security modules and gas based mobile robot to detect the gas
leakage on the platform. We use one lighter to provide gas on thegas leakage detection modules. The modules can transmit thedetection results to the supervised computer via wire RS485 andwireless RF interface. The label of the gas security modules
displays red. The experimental results are shown in Fig. 5 (a). The
belief value is 0.8522 over the threshold value using
Dempster-Shafter evidence theory. The experimental results areshown in Fig. 5 (b).
The supervised computer controls the mobile robot with gas
detection sensor moving to the event location. The mobile robot
detects the gas leakage event, and transmits the event signal to thesupervised computer. The experimental result is shown in Fig. 5(c).
The supervised computer receives the event signal to calculate thebelief value using Dempster-shafter evidence theory to be shownin Fig. 5 (d). The value is over the threshold value, and alarm theevent to users.

Subsets Hn of θ such that m(Hn ) > 0 are called focal
elements of m . From this basic belief assignment m, thecredulity Bel (Hn ) and plausibility Pl (Hn ) can be computed
using the equations:(5)m(α , j ) ( Hn) = α j ⋅ m j ( Hn)

V. EXPERIMENTAL RESULTS

Hn ≤θ

Bel ( Hn) = ∑ m( A)(9)A∩ B =φThe development of evidence theory began in the 1960s when
Dempster [11,12] developed the mathematical foundations of a
two-value uncertainty mapping, upper and lower uncertainty
measures, between two space. The system is satisfied theassumption with two value output on event detection. A result of
this work is dempster’s rule of combination, which operates on
belief or mass functions as Baye’s rule dose on probability
functions. Shafer [13], a student of Dempster, has extended thedevelopment of belief functions and is the major proponent ofevidence theory.
A brief overview of the Dempster-shafter’s evidence theory isprovided as follows. Let θ represents the set of hypotheses Hn ,called the frame of discernment. The knowledge about the
problem induces a basic belief assignment which allows to defineθa belief function m from 2 to [0,1] such as [14]:∑ m( Hn) = 1(8)

Κ = 1 − ∑ mi ( A) ⋅ m j ( B )IV. ALAGORITHM ANALYZE(3)

1∑ mi ( A) ⋅ m j ( B)
Κ A∩ B = Hn

where Κ is defined by:(2)m(Φ ) = 0A⊆ Hn

Pl ( Hn) =Hn ∩ A≠φ

The value Bel ( A) quantifies the strength of the belief thatevent A occurs. These functions ( m , Bel and Pl ) are
derived from the concept of lower and upper bounds for a set of©ISAROB 2011

(7)

(a)
255

(b)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

the future, we want to increase intelligent security detection
modules, and use multisensory fusion algorithm to enhance theprecision of the security system, and develop remote supervised
system to connect with the security system via Internet. The
security decreases the false alarm condition for the event detection.

ACKNOWLEDGMENT(c)(d)
Fig. 5 The experimental result for gas leakage detection

This work was supported by the project “Development of amodule based education robot” under Ministry of Education of
Taiwan. (99-A010-5).
.

Then we use wire and wireless fire security modules and firebased mobile robot to detect the fire source on the platform, too.
The modules detect fire event, and transmit the detection results tothe supervised computer via wire RS485 and wireless RF interface.
The label of the fire security modules displays red. The
experimental results are shown in Fig. 6 (a). The belief value is0.8522 over the threshold value, and the experimental result isshown in Fig. 6 (b). The supervised computer controls the firebased mobile robot moving to the event location, and transmits theevent signal to the supervised computer. The experimental result isshown in Fig. 6 (c). The supervised computer receives the event
signal to calculate the belief value using Dempster-shafter
evidence theory to be shown in Fig. 6 (d).

(a)

REFERENCES[1] C. W. Wang and A. T. P. So (1997), Building Automation In
The Century, The 4-th International Conference on Advance on
Advances in Power System Control, Operation Management,
APCOM-97, Hong Kong, , pp.819-824.
[2] M. Azegami and H. Fujixoshi (1993), A systematic
Approach to Intelligent Building Design, IEEE Communications
Magazine, pp.46-48.
[3] A. Kujuro and H. Yasuda (1993), Systems Evolution inIntelligent Building, IEEE Communication Magazine, pp.22-26.
[4] M. R. Finley, Jr. A. Karajura et al. (1991), Survey ofIntelligent Building Concepts, IEEE Communication Magazine,
pp.18-20.
[5] W. Y. Chung, L. C. Fu et al. (2001), A Flexible, Hierarchicaland Distributed Control Kernel Architecture for Rapid Resource
Integration of Intelligent System, IEEE International Conferenceon Robotics Automation, Seoul, Korea, pp.1981-1987.
[6] K. L. Su (2006), Automatic Fire Detection System Using
Adaptive Fusion Algorithm for Fire Fighting Robot, IEEE
International Conference on System, Man and Cybernetics (SMC2006), Grand Hotel, Taipei, Taiwan, pp966-971.
[7] S. H. Chia, K. L. Su et al (2010), Multiple Robot System
Applying in Chinese Chess Game, The Innovative Computing,Information and Control – Express Letters (ICIC-EL),Vol. 4, No.
5(B), pp.2027-2032.
[8] Y. Shimosasa, J. Kanemoto et al (2010), Some Results of
The Test Operation of a Security Service System With
Autonomous Guard Robot, IEEE International Conference on
Industrial Electronic, Control, and Instrumentation, pp.405-409.
[9] Y. Shimosasa, J. Kanemoto et al (1999), Security Service
System Using Autonomous Mobile Robot, IEEE International
Conference on System, Man, and Cybernetics, pp.825-829.
[10] S. H. Chia, K. L. Su (2010), Multi-level Multi-sensor Based
Security System for Intelligent Home, International Symposium
on Artificial Life and Robotics (AROB 15th), Beppu, Japan, Feb.
4-6, pp.379-382.
[11] A. P. Dempster (1967), Upper and Lower Probabilities
Induced by a Multivalued Mapping, Ann Math Statistics, Vol38.
[12] A. P. Dempster (1968), Ageneralization of BayesianInference, J. Royal Statistical Soc, Series B, Vol30.
[13] G. A. Shafer (1967), Mathemational Theory of Evidence,Princeton university Press, Princeton, NJ.
[14] E. Lefevre and O Colot (1999), A Classification Method
Based On the Dempster-Shafer's theory and information criteria,
ISIF , pp.1179-7786.

(b)

(c)
(d)
Fig. 6. The experimental result for fire source detection

VI. CONCLUSION
We have presented a multiple mobile robots based intelligent
security system that has multiple interfaces to be applied inintelligent home. The controller of the security modules and active
modules (mobile robots) is HOLTEK microchip. We use
Dempster-Shafter evidence theory to enhance the detection results
using passive and active security modules. The security detection
modules and the mobile robots can transmit real-time event signals
to the supervised computer via wire RS485 or wireless RFinterface. In the paper, we use fire and gas leakage events toimplement the function of the multiple mobile robots based
security system for intelligent home. The experimental results arevery nice to double check the event occurrence on the system. In©ISAROB 2011

256

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Implementation of an Auction Algorithm Based Multiple Tasks Allocation
Using Mobile Robots
Kuo-Lan Su1, Jr-Hung Guo2 , Chun-Chieh Wang3 and Cheng-Yun Chung2
1. Department of Electrical Engineering, National Yunlin University of Science & Technology, Taiwan
2. graduate school Engineering Science and technology, National Yunlin University of Science & Technology
3. Department of Electronic Engineering Chienkuo Technology University, Taiwan
E-mail: sukl@yuntech.edu.tw, g9710801@yuntech.edu.tw, jasonccw@ctu.edu.tw
Abstract: The article uses the ant colony system (ACS) and auction algorithms to solve the path planning and task
allocation problems of multiple mobile robots such that the robots can move from different start points to reach todifferent task points in a collision-free space. Ant colony optimization (ACO) is a new evolvement algorithm that is
proposed by Dorigo M., and solves some task allocation and target searching problems. The utilization of the auction
algorithm improve the efficiently of the tasks allocation. The article uses three performance functions to compare thecost on the motion displacement and waiting time for mobile robots. In this manner, a near optimal assignment ofmultiple task points according to a team objective can be obtained using the proposed algorithms. The simulated resultspresent that Ant colony optimization and auction algorithm find the optimization motion path for multiple mobile robots
moving to task points from start points in a collision-free environment.
Keywords: ant colony system. auction algorithm. path planning. Task allocation. multiple mobile robots.

I.

of natural ant and the algorithm is a kind of random optimization
approach. It solves some difficult problems in the optimization
path planning of the mobile robot system using the ability of
optimization in the process of ant colony searching food. The
algorithm has the following advantages; such as good robustness,
distributed computing and easy combined with other methods. Ant
colony algorithm can combine auction algorithm easily toreinforce its performance [14]. Jones and Dias developed acoordination mechanism which was applied by pickup teams in
the treasure hunt field [15]. Kishimoto and Sturevant use auction
algorithm as solution to multiple robots coordination in routing
problem in terms of computational complexity [16]. Michael andKumar et al have implemented to assign dynamically tasks tomultiple agents using distributed solution in formation control
scenario [17]. Nanjanath and Gini present auction based method
for multirobot dynamic coordination to visited different locationsin the map [18].
The article is organized as follows: Section II describes the antcolony algorithm and auction algorithm for the mobile robot
system, and propose three performance functions to compare thecost of finishing tasl allocation. Section III presents theexperimental results on the target research in the known
environment using the multiple mobile robots system. Section IV
presents brief concluding remarks.

Introduction

There have been a growing interest in multi-robot coordination
research in recent years. Multi-robot cooperation is fundamental
and significant in the robotic research fields. With the increasing
number of robots in one task team efficiency important and energy
consumption reduction research become more important in robots
coordination. Compared to single robot, multiple mobile robots
can operate to faster task completion, higher quality solutions, aswell as increased robustness ability to compensate robot failure [1].
The path planning and task allocation problems of the multiple
mobile robots are important research issues. Its task is to findcollision-free paths from different start points moving to thedifferent target points in an known or unknown environment withobstacles according to a reasonable algorithm.
Motion control and path planning of the wheel based mobile
robots is a currently active area of robot research field. The motion
control of multiple mobile robots for path tracking [2], navigation[3,4], wall following [5], task allocation [6] and path planning [7]has been proposed [8]. How to cooperate effectively with multiple
robot is a challenge. To overcome the challenge, many cooperation
algorithms have been proposed which mainly include
behavior-based approach [9], auction algorithm [10], ant colony
optimization (ACO) algorithm [11], threshold-based approach[12], particle swarm optimization algorithm (PSO) [13], etc.
Especially, ant colony optimization and auction algorithm havereceived significant attention to be growing in popularity [1], andhave been implemented in the article.
Ant Colony Optimization (ACO) is a new computational
paradigm to solve the path planning of the task allocation
problems in Swarm Intelligent. Ant colony algorithm is proposed
by Italian scholars Dorigo M., and simulates the routing behavior©ISAROB 2011

II.

Searching algorithm

The ant system algorithm was developed by Marco Dorigo
and his colleagues in the 1990s. Ants move in random orientationfrom the start point. Pheromones are deposited on the ground fromthe tail as they move around. The ants would choose motion paths
based on the amount of pheromones intensity on all possible

257

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

motion paths from the start point moving to the target point.
Subsequent ants are more likely to choose a shorter path withgreater pheromone trail intensity. The ant decides the motion pathaccording to transition probability, pik, j . The transition probability
is influenced by the pheromone of the ant:(τ ) (η ) , if j ∈ N∑ (τ ) (η )αpik, j =j is assigned to the robot i .

TD (Ri , Ti ) specifies the cost of executing subtasks Ti by the
robot Ri . TW (Ri , Ti ) is defined the waiting time cost of therobot Ri to execute subtasks Ti . We have three performance
function to compare the efficiently for the team robots executing
tasks allocation. There have MINSUM, MINMAX and MINAVE
functions. The MINSUM function is the displacement summationof the total paths for team robots executing all subtasks to beminimized. The MINMAX function is the maximum
displacement of the robot Ri that has been finished the subtask
to be minimized. The MINAVE function is the average cost of the
waiting time for all robots to be minimized. These functions canbe represented as follows:

β

i, j

i, j

α

i, j

β

k
i

(1)

i, j

l∈N ik=0

if j ∉ N ik

The left side of the Eq. (1) represents the transition probability
in which ant k will traverse from point i to point j . The
numerator on the right side of the equation consists of a product of
two terms, τ i, j α represents the intensity of the pheromone trail
between points i and j with a corresponding weight valueof α . On the other hand, ηi, j β represents the heuristic
information between points i and j with corresponding
weight value of β . ηi , j = 1 / d i , j , while d i , j is the distance
between points i and j . N ik is point i' s feasibleneighbourhood at the ant k . The denominator on the right side ofthe equation is a summation of the products of the pheromone
intensity and heuristic information for all possible moving paths[19].
The pheromone value evaporates on all paths by a constantfactor, and adds pheromone on the paths. Pheromone evaporation
is implemented by Eq. (2). The parameter ρ (0 < ρ ≤ 1) is used
to avoid unlimited accumulation of the pheromone. Where Δτ i,k j
is the amount of pheromone for the ant k depositing on the
motion paths it has visited? It is defined as follows:

( )

N

m

i

We implement the simulation results on the known map of
Yunlin University of Science & Technology ElectricalEngineering Department in order to verify the effectiveness of theant colony algorithm and auction algorithm using the multiple
mobile robots. The map is shown in Fig.1, and contains 64 nodes(black rectangles). The mobile robot can moves on the path
between node and node. We use four ants (mobile robots) to
search the twelve different task points from different start points.
We compare the cost of the performance functions (MINSUM,
MINMAX and MINAVE) to find target points. In the map, we use
black point to represent the mobile robot (R1, R2, R3 and R4), andthe black rectangle represent the task points. The motion paths offour mobile robots are presented by variety color lines. The mobile
robots are the same speed in the simulation experiment, andwaiting time is proportion to the displacement of mobile robots.

(2)

Fig. 1 The map of task allocation for four mobile robots
In the simulation results, we use four mobile robots assign
twelve task points from different start points, and use three
performance functions to compare the cost. We can list the cost of
the performance functions in Table 1, Table 2 and Table 3. We cansee the total displacement of the MINSUN function to be

)

i =1

©ISAROB 2011

(6)

i =1

III. Experimental results

}

∑ (

1 N
∑ TW (Ri ,Ti )m i =1

T

Auction algorithm solves the tasks allocation problem using afleet of mobile robots, and is classified single-item auctions andcombinatorial auctions. The single-item auction parallel contains
single-item auction and sequential single-item auction. We use
sequential single-item auction in the paper, and assign the mobile
robots moving to the formation position using ant colonyalgorithm. A formal definition of the auction algorithm is given a
number of tasks t1 , t2 ,......, tm , and subtasks are,T = {T1 , T2 ,......, TN }. A subtask Ti is a set that contains sometasks that is bided and completed by the robot Ri . Then how todecide the optimal allocation methods between robots andsubtasks so that the pattern formation task is achieved efficiently?
A fleet of robot set is defined R = R1, R2, ......, RN . A function
TD Ri , t j that specifies the cost of executing task t j by the
robot Ri , and find the assignment that allocation
one task per robotj
to minimize the global cost defined as
TD Ri , t j , where task{

(5)

MINAVE : min⎧1 / C k , if path (i, j ) belongs to T k ;(3)Δτ ik, j = ⎨0, otherwise;⎩
Where C k , the length of the tour T k that is build by the ant
k moving to the target point successfully, and is computed as thesum of the lengths of the paths belonging to T k , or Δτ ik, j = 0 .

)MINMAX : min max TD (Ri , Ti )
T

k =1

(

(4)

T

( )

τ i , j = (1 − ρ )τ i , j + ∑ Δτ ik, j , ∀(i, j ) ∈ L

MINSUM : min ∑ TD (Ri , Ti )258

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

minimum, and has long waiting time to be shown in Table 1. The
time of each mobile robot moves to each node to be listed in Table
4. We can see the cost of MINSUM to be equal to the total time of
Table 4 (R2). The value is 58.2.

In the Table 2, we can see the four mobile robots that have been
finished the tasks allocation to be faster. The time of each mobile
robot moves to each node to be listed in Table 5. We can see thecost of MINSUM to be equal to the total time of Table 2 (R1). The
value is 37. The MINAVE value is long time, too. The time ofeach mobile robot moves to each node to be listed in Table 6. We
can see the cost of MINSUM to be equal to the total time of Table
3 (R3). The value is 20.

Table 1. The cost of MINSUM

Table 6. The time of each node for MINAVE

Table 2. The cost of MINMAX

Table 3. The cost of MINAVE

Table 4. The time of each node for MINSUM(a) The motion paths of MINSUM function(b) The motion paths of MINMAX function
Table 5. The time of each node for MINMAX(c) The motion paths of MINAVE function
Fig. 2 The experiment results of task allocation©ISAROB 2011259

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 20118.

Finally, we make the minimum of the waiting time of four
mobile robots to allocate twelve task points, and the average valueof MINAVE function is minimum to be shown in Table 3. In thethree cases, the motion paths of four mobile robots according to
the variety performance functions to be shown in Fig. 2. We cansee the displacement of the mobile robot R2 is bigger than theothers to be shown in Fig 2 (a). We reduce the displacement of therobot R2, and allocate some task to the others using the MINMAX
function to be shown in Fig. 2 (b). Fig. 2 (c) is the motion paths offour mobile robots on the MINAVE function.

9.

10.

IV. Conclusion

11.

The article presents the path planning and task allocation
problems of mobile robots using ant colony algorithm and auction
algorithm. It solves the shorter paths of the mobile robots toallocate all task points. We have been implemented variety
performance functions to compare the cost for finishing task
allocation, and implement the simulation results using the
proposed algorithms on the known map. In future, we want toimplement the different task allocation using a fleet of robots, anduse mobile robots to present the scenario on the platform.

12.

13.

14.

ACKNOWLEDGEMENT
This work was supported by National Science Council of
Taiwan, (NSC 99- 2221- E-224-060).
.

15.

REFERENCE
1.

2.

3.

4.

5.

6.

7.

16.

T. Song, X. Yan, A. Liang and K. Chen (2009) A
distributed bidirectional auction algorithm for multirobot
coordination, The International Conference on Research
Challenges in Computer Science, pp.145-148
P. Rusu, E. M. Petriu, T. E. Whalen, A. Cornell and H. J. W.
Spoelder (2003) Behavior-based neuron-fuzzy controller
for mobile robot navigation, IEEE Trans. Instrum. Meas.,Vol. 52, No. 4, pp. 1335-1340
A. Chatterjee, K. Pulasinghe, K. Watanabe and K. Izumi(2005) A pratical swarm-optimized fuzzy-neural networkfor voice-controlled robot systems, IEEE Trans. Ind.
Electron., Vol.52, No. 6, pp. 1478-1489
M. J. Er and C. Deng (2004) Online tuning of fuzzy
inference system using dynamic fuzzy Q-learning, IEEE
Trans. Syst. Man. Cybern. B, Cybern., Vol.34, No. 3,
pp.1478-1489
A. K. Kulatunga, D. k. Liu and G. Dissanayake (2006) Ant
colony optimization based simultaneous task allocation andpath planning of autonomous vehicles, IEEE Conference on
Cybernetics and Intelligent System, pp. 1-6
S Liu, L. Mao and J. Yu (2006) Path planning based on ant
colony algorithm and distributed local navigation for
multi-robot systems, IEEE International Conference on
Mechatronics and Automation, pp.1733-1738
C. F. Juang and C. H. Hsu: Reinforcement ant optimized
fuzzy controller for mobile-robot wall-following control,
IEEE Transactions on Industrial Electronics, Vol. 56, No.
10, pp.3931-3940, 2009.

©ISAROB 2011

17.

18.

19.

260

K. Sugihara and J. Smith (1997) Genetic algorithm foradaptive motion planning of an autonomous mobile robot,
IEEE International Symposium on Computational
Intelligence in Robotics and Automation, Monterey, CA; pp.
138-146
F. Tang and L. E. Parker (2005) ASyMTRe: automatedsynthesis of multirobot task solution through software
reconfiguration, IEEE International Conference on Robotics
and Automation, Barcelona, Spain, pp.1501-1508
M. J. Mataric, G. S. Sukhatme and H. Ostergaarde (2003)
Multi-robot task
allocationin
uncertain
environments, Autonomous Robots, Vol.14, No.2-3,
pp.255-263
P. Tarasewich and P. R. McMullen (2002) Swarm
intelligence: power in numbers, Communication of ACM,
Vol.45, No.8, pp.62-67
N. Kalra and A. Martinoli (2006) A comparative study ofmarket-based and threshold-based multirobot task
allocation, EPFL, Lausanne, Switzerland, Tech. Rep.
SWIS-IPI, February
J. Kennedy and R. C. Eberhart (1995) Particle swarm
optimization, IEEE International Conference on Neural
Networks, Perth. Australia, pp. IV: 1942-1948
Y. Cen, C. Song, N. Xie and L. Wang (2008) Path planning
method for mobile robot based on ant colony optimization
algorithm, IEEE Conference on Industrial Electronics andApplications, pp. 298-301
E. G. Jones, B. Browning and M. B. Dias (2006)
Dynamically formed heterogeneous robot teams
performing tightly-coordination, IEEE Conference on
Robotics and Automation, Orlando, Florida, pp.570-575
A. Kishimoto and N. Sturtevant (2008) Optimized
algorithms for multi-agent routing, The 7th International
Conference on Autonomous Agents and Multiagent System,
L. Padgham, Parkes, D. C., J. Muller, S. Parsons(eds),
pp.1585-1588
N. Michael, M. M. Zavlanos, V. Kumar and G. J. Pappas(2008) Distributed multi-robot task assignment andformation control, The IEEE Conference on Robotics andAutomation, Pasadena, C. A., pp.128-133
M. Nanjanath and M. Gini (2006) Auction for task
allocation to robot: The 9th International Conference onIntelligent Autonomous System, Tokyo, pp.550-557
Y. Z. Cong and S. G. Ponnambalam (2009) Mobile robot
path planning using ant colony optimization, IEEE
International Conference on Advanced Intelligent
Mechatronics, pp. 851-856

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fuzzy Programming for Mixed-Integer Optimization Problems
Yung-Chin Lina, Yung-Chien Linb, Kuo-Lan Sua, Wei-Cheng Linc, Tsing-Hua Chenba

Department of Electrical Engineering, National Yunlin University of Science & Technology, Yunlin, TaiwanbDepartment of Electrical Engineering, WuFeng University, Chiayi, TaiwancDepartment of Electrical Engineering, I-Shou University, Kaohsiung, Taiwan
e-mail: {yclin@wfu.edu.tw; chien-lin@ wfu.edu.tw; sukl@yuntech.edu.tw; byron@isu.edu.tw; thchen@wfu.edu.tw}

Abstract: Mixed-integer optimization problems belong to NP-hard combinatorial problems. Therefore, they are difficult
to search for the global optimal solutions. The mixed-integer optimization problems are always described by precise
mathematical programming models. However, many practical mixed-integer optimization problems inherit more or lessimprecise nature. Under this circumstance, if we take into account the flexibility of constraints and the fuzziness ofobjectives, the original mixed-integer optimization problems can be formulated as fuzzy mixed-integer optimization
problems. Mixed-integer differential evolution (MIHDE) is an evolutionary search algorithm, and has been successfully
applied to many complex mixed-integer optimization problems. In this paper, a fuzzy mixed-integer mathematicalprogramming model is developed to formulate the fuzzy mixed-integer optimization problem. And then the MIHDE is
introduced to solve this fuzzy mixed-integer programming problem. Finally, the illustrative example shows thatsatisfactory results can be obtained by the proposed method. This demonstrates that the MIHDE can effectively handle
the fuzzy mixed-integer optimization problems.
Keywords: fuzzy programming, mixed-integer optimization, evolutionary algorithm.
nonlinear,non-differentiableandmulti-modal
optimization problems [4]. Mixed-integer differential
evolution (MIHDE) [5] is an evolutionary algorithm. A
mixed coding is introduced in MIHDE to implement theevolutionary process of continuous and integer
variables. The MIHDE has been successfully applied tomany complex mixed-integer optimization problems [58].
In this paper, a fuzzy mixed-integer programming
model is developed to formulate the fuzzy mixedinteger optimization problem. The MIHDE is
introduced to solve this fuzzy mixed-integer
programming problem. Finally, the illustrative example
show that satisfactory results can be obtained by the
proposed method. This demonstrates that the MIHDE
can effectively handle the fuzzy mixed-integer
optimization problems.

I. Introduction
Many real-world optimization problems involve
integer or discrete design variables in addition tocontinuous design variables. This kind of problems iscalled mixed-integer optimization problems. Mixedinteger optimization problems belong to NP-hardcombinatorial problems, therefore they are difficult tosearch for the global optimal solutions. On the other
hand, the mixed-integer optimization problems arealwaysdescribed
byprecisemathematicalprogramming models. However, many practical mixedinteger optimization problems inherit more or lessimprecise nature. Under this circumstance, if we take
into account the flexibility of constraints and thefuzziness of objectives, the original mixed-integer
optimization problems can be formulated as fuzzy
mixed-integer optimization problems.

II. Fuzzy Mixed-Integer Mathematical
Programming

In the fuzzy mixed-integer optimization problems,
the constraints and objectives are defined by fuzzy sets
and denoted as “
fuzzy constraints”and “
fuzzy goals”[1]. Combined with fuzzy constraints, fuzzy goals andfuzzy decision, a fuzzy mixed-integer optimization
problem can be transformed into a mixed-integer
optimization problem. Therefore, one can use a mixedinteger optimization approach to solve such a mixedinteger optimization problem.

If we soften the rigid requirements of a mixedinteger optimization problem, the mixed-integer
optimization problem can be stated by a fuzzy mixedinteger programming model as follows:(1)
min f (x, y )x,y~subject to(2)
g j ( x, y ) 0, j 1,..., mi
where x represents an nC-dimensional vector ofcontinuous variables, y is a nI-dimensional vector of
integer variables, and the symbol “~”respectively
denote the softened or fuzzy versions of constraints
g j (x, y ) . It means that the minimized objective

Evolutionary algorithms (EAs) [2, 3] are powerfulsearch algorithms based on the mechanism of natural
selection. Unlike conventional search approaches, they
simultaneously consider many points in the search
space so as to increase the chance of global
convergence. Recently, EAs have exhibited promising
results for solving complex problems such as highly©ISAROB 2011

function can be further improved with properly
softened constraints.

261

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

The fuzzy goal and fuzzy constraints can bequantified by the membership functions. Here a linear
membership function such as triangular function is
employed. The membership functions are represented
by f ( x , y ) and g ( x, y ) as defined by equations
jwhere x represents an nC-dimensional vector of realvalued variables, y is an nI-dimensional vector of
integer-valued variables, and ( x L , y L ) and (x U , y U )are the lower and upper bounds of the corresponding
decision vectors.

(3) and (4).

The procedure of MIHDE includes the following 5
steps.

0
 1
f f ( x , y )f ( x , y )  10
 f f
1


if f ( x , y )  f 0
if f 0  f ( x , y )  f 1

(3)

1)

Representation and Initialization

MIHDE

if f ( x , y )  f 1

0
0
g j g j ( x , y )g j ( x , y )  01
 g j g j
1
if g j ( x , y ) g

0j

if g 1j g j ( x , y ) g 0j

C

Ivariables (genes), x ji and y ji , are directly coded as
real-valued and integer-valued numbers. The
initialization process generates N p decision vectors(x, y ) i randomly, and should try to cover the entirejsearch space uniformly as in the form:(x 0 , y 0 ) i (x L , y L ) 
i {(xU , y U ) (x L , y L )}
, i 1,, N p(11)where i Diag( i ,1 , i ,2 ,  , i ,n n ) is a diagonal

represent the values of f and g j such that thegrades of the membership function f ( x , y ) andg ( x , y ) are 1.
j

C

The fuzzy decision D ( x , y ) is expressed asImatrix, the diagonal elements ( i ,1 , i ,2 ,  , i ,n n )are random numbers in the range 0,1 , the other
C

mi

(5)
D ( x , y ) {f ( x , y )} g j ( x , y ) 1
If we follow the fuzzy decision of Bellman andZadeh [9], the optimal solution (x* , y* ) of fuzzy
decision D ( x , y ) can be selected by maximizing thesmallest membership function such thatD (x* , y * ) max min{f (x, y ), g (x, y ),, g (x, y )} (6)Ielements are zero, and the rounding operatora i ( x U x L ), b i ( y U y L )}in (11) is defined
as (a, INT[b ]) in which the operator INT[b ] isexpressed as the nearest integer-valued vector to thereal-valued vector b.
2)mi

By introducing the auxiliary variable , the maxmin problem can be transformed into the following
mixed-integer nonlinear programming (MINLP)
problem:maximize (7)(8)
subject tof (x, y )Mutation

The i-th mutant individual 
is obtained byuG , vG ithe difference for two random individuals as expressedin the form





G
G
G
G
(uG , v G )i ( xG , y G ) p ρm ( x , y ) k ( x , y ) l G( x , y ) p ( ρm ( x x ), INT[ ρm ( y y l )])Gg j ( x, y ), j 1,..., mi (9)

G

G
k

G
l

G
k

(12)
where random indices k , l 1,  , N p are mutuallydifferent. The operator INT[ b m ( y Gk y Gl )] in (12)In order to solve this MINLP problem effectively,
the MIHDE algorithm is applied to solve this problem
and find the maximizing decision. The details of
MIHDE are described in the following section.

is to find the nearest integer vector to the real vector b.
The mutation factor m is a real random number
between zero and one. This factor is used to control thesearch step among the direction of the differential
variation ( x G , y G ) k ( x G , y G ) l .

III. Mixed-Integer Hybrid Differential
Evolution
Let us consider a general MINLP problem asfollows:(10)
min f (x, y )3)

CrossoverIn crossover operation, each gene of the i-th
individual is reproduced from the mutant vector(u G , v G ) i = (u1Gi , u 2Gi ,, u nGC i , v1Gi , v2Gi ,, vnGI i ) and thex,yx L x xUy L y yU©ISAROB 2011

vectors(x, y ) i , is represented as( x1i ,..., x ji ,..., x n i , y1i ,..., y ji ,..., y n i ) . The decision

if g j ( x , y ) g 1j1

decision

NpG

vector (chromosome),functions f ( x , y ) and g ( x, y ) are 0, and f 1 andg

Gof N p individuals in the G-th generation. The decision(4)

where f 0 and g 0j are respectively denote the valuesof f and g j such that the grades of the membership
1
j

uses

{z } {(x , y )i } , i 1,..., N p to denote a population
G
i

262

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

current

individual(x G , y G ) i=( x1Gi , x2Gi ,, xnGC i ,x Gji 1 x Gjb10, if
2 ; j 1,..., nC , i 1,..., N p , i bdx ji x Gjb1
1, otherwisey1Gi , y 2Gi ,, y nGI i ) as follows:Gxli , if a random number c(13)u liG 1  G,
otherwise;
1
,...,
,
1
,...,
u
l
n
i
N



C
p
li
G

y
,
if
a
randomnumber


 li
c
(14)
vliG 1 Gv,otherwise;
l

1
,...,
n
,
i

1,..., N pIliwhere the crossover factor c 0,1 is a constant and(19)
0, if y Gji 1 y Gjb1; j 1,..., nI , i 1,..., N p , i b (20)dy ji 
1, otherwisewhere dx ji and dy ji are the gene diversity indicesand 2 [0,1] is a tolerance for real-valued geneprovided by the user. According to (19) and (20), we
assign the j-th gene diversity index for the i-th
individual to zero if this gene clusters to the best gene.
We now define the population diversity degree  as aratio of total diversified genes in the population. From(19) and (20) we have the population diversity degreeasthe value can be specified by the user.
4) Evaluation and Selection
The operation includes two evaluation phases. The
first phase is performed to produce the new populationin the next generation as (15). The second phase is used
to obtain the best individual as (16).
(x G 1 , y G 1 ) i argmin{ f ((x G , y G ) i ), f ((u G 1 , v G 1 ) i )}

N p nC

nI

 

(21)

dx

d
(nC n I )( N p 1)
/


ji
ji 
1 
j
1
j
1

ii 

b

From equation (19), (20) and (21), the value of 
is in the range [0,1]. Consequently, we can set atolerance for population diversity, 1  0,1 , to assess

i 1, , N p(15)(x , y ) b argmin{ f ((x , y ) i ), i 1, , N p }(16)
where (x G 1 , y G 1 ) b is the best individual with the
G
1

G 1

G 1

G 1

smallest objective function value.

whether the migration should be actuated. If  issmaller than 1 , then MIHDE performs the migration
to generate a new population to escape a local solution.
Contrary, if  is not less than 1 , then MIHDE
suspends the migration operation to keep a constantsearch direction to a target solution.

5) Migration
In order to increase the exploration of the search
space, a migration operation is introduced to generate adiversified population. Based on the best individual
1,(x G 1 , y G 1 ) b ( x1Gb1 , x2Gb1 ,, xnG b1 , y1Gb1 , y 2Gb1 ,, ynG
b )
C

I

the j-th gene of the i-th individual can be diversified by
the following equations:IV. Computational Example

G 1
xGjb1 x Lj
LG 1
x jb 1 ( x j x jb ), if a random number  U
G 1
x j x Lj
x ji 
G 1
U
G 1
x jb 1 ( x j x jb ), otherwise; j 1,..., nC , i 1,..., N p

Consider a design problem of pressure vessel, as
presented by Sandgren [10], is shown in Figure 1.

(17)

 G1
y Gjb1 y Lj
L
G
1
y

INT[

(
y

y
)],
if
a
randomnumber


jb
2
j
jb
yUj y Lj
y Gji1 
 G1
U
G
1
y jb INT[2 ( y j y jb )], otherwise; j 1,...,nI , i 1,..., N p(18)
where 1 and 2 are the random numbers in therange [0,1].

Figure 1. Pressure Vessel design.

The migration operation in MIHDE is performed only
if a measure for the population diversity is not satisfied,that is when most of individuals have clustered together,
the migration has to be actuated to make some
improvements. In this study, we propose a measure
called the population diversity degree  to check
whether the migration operation should be performed.
In order to define the measure, we first introduce thefollowing gene diversity index for each real-valued
1
gene x G
and for each integer-valued gene y kiG 1 ,ji

©ISAROB 2011

The design variables are the dimensions required for the
specifications of the vessel, i.e.
(x, y ) ( x1 , x 2 , y1 , y 2 ) .
The objective function is the combined costs ofmaterial, forming and welding of the pressure vessel.
The constraints are set in accordance with therespective
ASME
codes. The
mixed-integer
optimization problem is expressed as:

263

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

min f (x, y ) 0.6224(0.0625 y1 ) x1 x 2 1.7781(0.0625 y 2 ) x12

References

x,y

[1]

H. J. Zimmermann,“
Application of fuzzy setstheorytomathematicalprogramming,”
Information Science, vol. 36, pp. 29-58, 1985.
[2] Z.Mi
c
h
a
l
e
wi
c
z
,“
Ge
n
e
t
i
c Al
g
or
i
t
hm + Data
Structure = Evolution Pr
og
r
a
ms
,
”Spr
i
n
g
e
r
-Verlag,
1994.
[3] T. Back, D. Fogel, and Z. Michalewicz,
Handbook of Evolutionary Computation. New
York: Oxford Univ. Press, 1997.
[4] Z. Michalewicz, Z. and M. Schoenauer,“
Evolutionary Algorithms for Constrained
Parameter Optimization Problems,”Evolutionary
Computation, vol. 4, no. 1, pp. 1-32, 1996.
[5] Y. C. Lin, K. S. Hwang, and F. S. Wang, “A
mixed-coding scheme of evolutionary algorithms
to solve mixed-integer nonlinear programming
problems,” Computers & Mathematics withapplications, vol. 47, pp. 1295-1307, 2004.
[6] Y. C. Lin, K. S. Hwang, and F. S. Wang, “Coevolutionary hybrid differential evolution formixed-integer
optimization
problems,”
Engineering Optimization, vol. 33, no. 6, pp. 663682, 2001.
[7] Y. C. Lin, K. S. Hwang, and F. S. Wang, “Anevolutionary Lagrange method for mixed-integer
constrained optimization problems,”Engineering
Optimization, vol. 35, no. 3, pp. 267-284, 2003.
[8] Y. C. Lin, Y. C. Lin, and K. L. Su, “Production
planning based on evolutionary mixed-integer
nonlinear programming,”ICIC Express Letters,
vol. 4, no. 5(B), pp. 1881-1886, 2010.
[9] R. E. Bellman and L. A. Zadeh, “Decisionmaking in a fuzzy environment,”Management
Science, vol. 17, pp. 141-164, 1970.
[10] E. Sandgren, “Nonlinear integer and discrete
programming in mechanical design,”ASME J.
Mechanical Design, vol. 112, pp. 223-229, 1990.

3.1611(0.0625 y1 ) 2 x 2 19.84(0.0625 y1 ) 2 x1
(22)
subject to g 1 ( x, y ) 0.0193 x1 0.0625 y1 0
g 2 ( x, y ) 0.00954 x1 0.0625 y 2 0
g 3 ( x, y ) 750 1728 x12 x 2 4 x13 03
g 4 ( x, y ) x 2 240 0
10.0 x1 100.0
10.0 x2 240.0
10 y1 32
10 y2 32
Using the MIHDE algorithm to solve this mixed-integer
optimization problem, the obtained optimal solution is***(x* , y * ) ( x1 , x 2 , y1 , y * 2 ) (38.8754, 221.4069, 12, 10) ,f (x * , y * ) 6521.3198
Instead of giving the crisp values for this mixedinteger optimization problem, the fuzzy goal and thefuzzy constraints are described in Table 1.

Table 1. Fuzzy goal and fuzzy constraints.
f 0
f 1
or
or
g 0
g 1
j

j

f (x, y)f 0 6200.0

f 1 6500.0

g 1 (x, y )g10 5.0

g11 5.0

02
03

g 2 ( x, y )g 5.0g 12 5.0

g 3 ( x, y )g 5.0g 31 5.0

g 4 ( x, y )g 40 5.0

g 14 5.0

For this fuzzy mixed-integer optimization problem, theobtained optimal solution by MIHDE is***(x * , y * ) ( x1 , x 2 , y1 , y * 2 ) (40.6027, 196.1014, 12, 10) ,**f (x , y ) 6350.6731 ,4.9664
From computational result, the cost of pressure
vessel can be decreased through the fuzzy programming
for the original mixed-integer optimization problem.
This implied that the cost function can be further
improved if the constraints are softened to a morefavorable degree.

V. Conclusions
In this paper, a fuzzy mixed-integer mathematicalprogramming model is developed to formulate thefuzzy mixed-integer optimization problem. And thenthe MIHDE is introduced to solve this fuzzy mixedinteger programming problem. Finally, the illustrative
example shows that satisfactory results can be obtainedby the proposed method. This demonstrates that theMIHDE can effectively handle the fuzzy mixed-integer
optimization problems.

©ISAROB 2011264

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Develop a Vision Based Auto-recharging System for Mobile Robots
Ting-Li Chien
Department of Electronic Engineering, Wu-Feng University, Taiwan
cdl@mail.wfc.edu.tw
Abstract: The article develops a vision based auto-recharging system that guides the mobile robot moving to thedocking station. The system contains a docking station and a mobile robot. The docking station contains a docking
structure, a control device, a charger and a safety detection device and a wireless RF interface. The mobile robot
contains a power detection module (voltage and current), an auto-switch, a wireless RF interface, a controller and acamera. The controller of the power detection module is Holtek chip. The docking structure is designed with one active
degree of freedom and two passive degrees of freedom. In image processing, the mobile robot uses a webcam tocapture the real-time image. The image signal transmits to the controller of the mobile robot via USB interface. We use
Otsu algorithm to recognize the distance and orientation of the docking station from the mobile robot. In theexperiment results, the proposed method can guided the mobile robot moving to the docking station.
Keywords: auto-recharging system. mobile robot. docking station. degrees of freedom. USB interface. Otsu
algorithm
system for the mobile robot, and explains the functionsof the power detection module and the power
measurement and prediction algorithms. Section III
presents hardware structure and docking processing of
the station. The section IV explains the image
processing method to find out the docking station.
Section V presents the experiment results of the autorecharging processing for the mobile robot moving to
the station using the proposed method. Section VI
presents brief concluding remarks.

I. INTRODUCTION
Mobile robots have been widely applied in many
fields with passing year. Such as factory automation,
dangerous environment detection, office automation,
hospital, entertainment, space exploration, farm
automation, military, education, learning and security
system and so on. The mobile robot has been working inlong time, and the power of the mobile robot isweakness, and moves to the docking station
autonomously before under control.
In the past literature, many researches have been
proposed power detection methods. Levi was one of thefirst to comment upon the characteristics of CMOS
technology which make it special amenable to IDD
Testing [1]. Malaiya used IDD testing and estimating theeffects of increased integration on measurement
resolution [2]. Frenzel proposed the likelihood ration
test method applying on power-supply current diagnosis
of VLSI circuits [3]. Horning et al reported on
numerous experiments where current measurements
have forecast reliability problems in devices which had
previously passed conventional test procedures [4].
Maly et al. proposed a build-in current sensor which
provides a pass/fail signal when the current exceeds a
set threshold [5]. In image processing, many techniques
have been proposed. Weszka et al. use the valley
sharpening method to restrict the histogram on the
pixels with large absolute value [6]. Watanabe used thedifferent histogram method to select the threshold atgray level with the maximal amount of difference [7].
The article is organized as follows: Section II
describes the system structure of the auto-recharging©ISAROB 2011

II. SYSTEM ARCHITECHTURE
The hardware structure of the auto-recharging
system is classified two parts: one is designed indocking station. The other is designed in the mobile
robot. The relation is shown in Fig. 1. In the docking
station, there are a limit switch, a landmark, a safety
detection device, a wireless RF interface and a charger.
The landmark can guide the mobile robot moving to thedocking station using vision system. The safety
detection device contains safety switch and power
detection module. The limit switch can detect themobile robot touching the station or not. The wirelessRF interface can communicate with the mobile robot via
RS232 interface, and transmits the status of therecharging processing between the docking station andthe mobile robot.
The power of the mobile robot is weakness on thefree space. The camera can find out black bars of thedocking station using the proposed method. The mobile
robot decides the direction from the docking stationaccording to the relation position of the two black bars.

265

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

the curve from the power detection value of the mobile
robot. Next the user can set the critical value of thepower. The main controller of the mobile robot cancalculate the extrapolation value from the critical value,and can calculate the residual working time for themobile robot.
We fit a second-order polynomial regression

The wireless RF interface can communicate with thedocking station, and transmits and receives the real-time
signal. The power detection module can measurescurrent and voltage values of the auto-recharging
processing, and transmits to the controller of the mobile
robot via RS232 interface. The prototype of the power
detection module is shown in Fig. 2. The IR sensor can
decides the mobile robot touching the docking station,and transmits the signal to the controller of the mobile
robot. The mobile robot opens the auto-switch, andtouches with the connective pins of the docking station.

y = a0 + a1 x + a2 x 2 + e(1)The sum of the squares of the error isn

S r = ∑ ( yi − a0 − a1 xi − a2 xi2 ) 2
i =1

(2)

To generate the least squares fit, we take thederivative of Equation (2) with respect to each of theunknown coefficient of the polynomial, and we can get

na0 + (∑ xi )a1 + (∑ xi2 )a2 = ∑ yi(∑ xi )a0 + (∑ xi2 )a1 + (∑ xi3 )a2 = ∑ xi yi(∑ xi2 )a0 + (∑ xi3 )a1 + (∑ xi4 )a2 = ∑ xi2 yi

Fig. 1. The auto-recharging system(3)

The mobile robot has the shape of cylinder and its
diameter, height, and weight are 40 cm, 70 cm, and 25kg, respectively. It has four wheels to provide thecapability of autonomous mobility [8]. The mobile robot
contains six systems, including structure, motion anddriver system, software development and supervised
system, detection system, auto-recharging system andcamera.

Finally we can calculate a0 , a1 , and a2 from Equation(3). Then we set the power critical value to be PS anda2 x 2 + a1 x + a0 = PS(4)

We can calculate the x value (the unit is second) from
Equation (4). The sample time of the power detection
module is 1 second.

III. Docking station
The docking station is shown in Fig. 3, and provides
two connective points to provide charging current to themobile robot. The docking station is designed with two
passive DOF (Degree Of Freedom) and one active DOF.
It can rotate in the Z-axis and use compression spring tomove for various docking condition. The weight of thedocking station is almost 6 kg, and its length, and height,
and width are 70cm, 50cm and 80cm, respectively. The
station extends 15 cm to touch the mobile robot
providing a 300 entry window. The connective docking
mechanism is mounted on the back of the mobile robot.
The mobile robot docking mechanism is shown inleft side of the Fig. 3 (c). The recharging adapters are at
the inside of the holes. Each hole is shaped as a cone inorder to help the docking smoothly. Since both theconnection pins and adapters are rigid, the docking

Fig. 2. The power detection module
In the power detection module, we use four current
sensors to measure the current variety of the mobile
robot and charging current, and use two multisensor
fusion methods (redundant management method and astatistical prediction method) to detect current andvoltage signals status. We can get an exact measured
value for power detection [9]. Then we want to predictthe residual power of the mobile robot. First we must fit©ISAROB 2011

266

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

[1,2,LL, k ] , and C1 denotes pixels with levels[k + 1, k + 2,LL, L] . Then the probabilities of classstation is designed for providing compliance forreasonable robot docking angle and offset. When themobile robot is approaching to the docking station withoffset and docking angle, the guiding stick mounted on
the docking station functions to guide the recharging
pins for inserting into the adapters. We plot two blackbars on the front of the docking station to be shown in
Fig. 4. The mobile robot recognizes the position of thedocking station according to the two black bars using
vision system.

occurrence and the class mean levels, respectively, aregiven byk

ω0 = Pr (C0 ) = ∑ pi = ω (k )(6)

ω1 = Pr (C1 ) = ∑ pi = 1 − ω (k )

(7)

i =1
L

i = k +1kk

ω(k ) = ∑ pi ,μ(k ) = ∑ipi ,

i =1

i =1

L

μG = μ(L) = ∑ipi

Then the optimal threshold kmaximize η [10,11].

η (k ) = σ B2 (k ) / σ G2 ,σ B2 (k * ) = max σ B2 (k )σ B2 (k ) =

(8)

i =1

*

can be funded to[μGω (k ) − μ (k )]2 (9)ω (k )[1 − ω (k )](10)1≤ k ≤ L

V. Experimental results
We make an experiment to implement the autorecharging processing using the Otsu algorithm for the
robot moving to the docking station. The power of themobile robot is under the critical power. The mobile
robot searches the docking station using camera to beshown in Fig. 5. The mobile robot finds out thelandmark of the station using camera. Then it moves
closed to the docking station using the proposed
algorithm to decide the direction of the mobile robot.
The experiment result is shown in Fig. 5(a). The
original gray-level picture of the station is shown in Fig
5(b). We focus on the landmark, and set two points (X1and X2) on column projection, and set four points (Y1,
Y2, Y3 and Y4) on row projection. We use Otsu
algorithm to process the picture from the camera, andcalculate the results of the threshold, and plot the row
projection image and the column projection image to beshown in Fig. 5(c) and (d). The mobile robot cancalculates the direction and distance from the docking
station.

Fig. 3. The structure of the docking station

Fig. 4. The docking station

IV. Algorithms analysis
We use Otsu algorithm to recognize the landmark of
the station docking. In image reorganization, we define
the pixels of a given picture be represented in L in graylevels [1,2,......, L] . The number of pixels at level i isdenoted by ni , and the total number of pixels is
N = n1 + n2 + L + nL . In order to simplify the
discussion, we can rewrite the gray level histogram tobe normalized and regarded as a probability distribution:pi = ni / N ,pi ≥ 0,L∑ p =1
i =1

i

(5)

Now suppose that we dichotomize the pixels into
two classes C0 and C1 (back ground and objects) by athreshold at level k ; C0 denotes pixels with levels©ISAROB 2011

(a)

267

(b)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

station. The proposed algorithm can guides the mobile
robot moving to the docking station. In future, we want
to implement the auto-recharging processing and path
planning on the multiple docking stations for mobile
robots.

(c)

(d)

(e)

(f)

REFERENCES[1] M. W. Levi: (1981), CMOS is most testable,
Proceedings of International Test Conference,
pp.217-220.
[2] Y. K. Malaiya and S. Y. H. Su, (1982), A new
fault model and testing technique for CMOS
devices, Proceedings of International Test
Conference, pp.25-34[3] J. F. Frenzel, (1994), Power-Supply Current
Diagnosis of VLSI Circuits, IEEE Transaction on
reliability Vol. 43, No.1, pp.30-38.
[4] L. K. Horning et al., (1987), Measurements ofquiescent power supply current for CMOS ICs in
production testing, Proceedings of International
Test Conference, pp.300-309.
[5] W. Maly and P. Nigh, (1988),: Build-in current
testing- Feasibility study, Proceedings ofInternational Conference on Computer-Aided
Design, pp.340-343.
[6] J. S. Weszka, R. N. Naget and A. Rosenfeld,(1974), A threshold selection technique, IEEE
Transaction on Computer, Vol. C-23, pp. 13221326.
[7] S. Watanabe and CYBEST Group: An automatedapparatus for cancer prescreening: CYBEST,
Comp. Graph., Image Process., Vol.3, 1974,
pp.350-358.
[8] K. L. Su, T. L. Chien and J. H. Guo (2005),
Decelop a Seft-diagnosis Function Autorecharging Device for Mobile Robot, IEEE
International Workshop on Safety, Security, andRescue Robot, pp.1-6.
[9] K. L. Su, Jyh H. Tzou and Chien C. Liu (2007),Development of a Multisensor-Based Residual
Power Prediction System for Mobile Robots, IEEE
Workshop on Advanced Robotics and its Social
Impacts 2007 (ARSO '07), Hsinchu, Taiwan,
December 9-11, pp.114-119[10] K. Fukunage, (1972), Introduction to statistical
pattern recognition, New York, Academic, pp.260267.
[11] N. Otsu, (1979), A threshold selection methodfrom gray-level histograms, IEEE International
Conference on System, Man, and Cybernetics,
Vol.9, No. 1, pp.62-66.

(g)(h)
Fig. 5. The experiment result of auto-docking for themobile robot
The mobile robot moves to the front of the docking
station to be shown in Fig. 5(e), and turn left to face thestation. Then it moves the station, and modifies the
direction according to the position of landmark. The
experiment scenario is shown in Fig 5(f). The mobile
robot moves to the docking station, and touch the limit
switch. The docking station transmits RF signal to themobile robot. The mobile robot must stop, and prepares
to execute the charging processing. The experimental
results are shown in Fig. 5 (g) and (h). We make sometests on variety direction of the robot moving thedocking station to determine the performance using the
proposed method, 100 experimental results were
performed, and show a 99% successful rate for thedocking processing.

VI. CONCLUSION
We have developed a vision based auto-recharging
system that had been integrated in the mobile robo. The
system contains a docking station and a mobile robot.
We can get an exact measured value for power detection,and calculate the residual working time using secondorder polynomial regression. We use Otsu algorithm torecognize the two black bars of the docking station, if
the mobile robot locates on the front of the docking©ISAROB 2011.

268

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Super-Twisting Second Order Sliding Mode Control for a Synchronous
Reluctance Motor
Huann-Keng Chiang1, Wen-Bin Lin2, Chang-Yi Chang1, and Chien-An Chen3
Department of Electrical Engineering, National Yunlin University of Science & Technology, Taiwan, R.O.C.
2
Graduate School of Engineering Science & Technology, National Yunlin University of Science & Technology, Taiwan,
R.O.C.
3
Electronic Vehicle and System Verification Group R & D Division, Automotive Research and Testing Center, Taiwan,
R.O.C.
Fax :+886-5-5312065
E-mail: chianghk@yuntech.edu.tw
1

Abstract: This paper presents design and implementation of a super-twisting second order sliding mode controller(SOSMC) for a synchronous reluctance motor. The second order sliding mode control is an effective tool for thecontrol of uncertain nonlinear systems since it overcomes the main drawbacks of the classical sliding mode control, i.e.,
the large control effort and the chattering phenomenon. Its real implementation implies simple control laws and assures
an improvement of the sliding accuracy with respect to conventional sliding mode control. This paper proposes a novel
scheme that based on the technique of super-twisting second order sliding mode control. First, the SOSMC is derived
by mathematics. Finally, the performance of the proposed method is verified by simulation and experiment. The
proposed SOSMC shows the robustness for the motor parameters variation and the improvement of chattering
phenomenon.

Keywords: Super-Twisting Algorithm, Second Order Sliding Modes, Synchronous Reluctance Motor, Chattering
Phenomenon.
motor, FEM fuel cells and automatic docking [9-12].
There is no paper adopting super-twisting SOSMC
in SynRM speed control so far. Therefore, this paper
proposes a novel scheme that based on the technique ofsuper-twisting SOSMC. Finally, the performance of the
proposed method is verified by simulations.

I. INTRODUCTION
Fast accurate dynamic response is of primary
importance in control systems. The motor control
system with the high robustness is an important issue in
research. Synchronous reluctance motor (SynRM) havea mechanically simple and robust structure. They can beused in high speed and high temperature environments.
The rotor circuit of the SynRM is open circuit such thatthe flux linkage of SynRM is directly proportional to thestator currents. The torque of SynRM can be controlled
by adjusting the stator currents. Therefore, there has
been renewed interest in SynRM [1-4].
Sliding mode control (SMC) has attracted increasing
attention in recent years because it is an effective androbust technology for parameter variation and external
disturbance rejection. It has been applied to robot andmotor control [3,5-7]. Sliding mode control (SMC) is arobust control for nonlinear systems. However, sliding
mode is a mode of motions on the discontinuity set of adiscontinuous dynamic system. Hence, reducing thechattering is very important for SMC. The second-ordersliding mode technique has the same properties ofrobustness to uncertainties of model and external
disturbances. Second-order SMC (SOSMC) [8]
improves the chattering phenomenon. Due to few
literatures about SOSMC in SynRM control application,
therefore, it has valuable on research in SynRM control
application for SOSMC.
Different from the conventional first order SMC, the
SOSMC is belonging to the region of higher-ordersliding mode (HOSM). Levant [8] had discussed thetheory of HOSM. HOSM control have been applied to©ISAROB 2011

II. MODELING OF THE SYNRM
The d-q equivalent voltage equations of ideal
SynRM model with a synchronously rotating rotor
reference frame are shown in Fig. 1:
id s− ω r L q iq s

Rs

+

+

V ds

−
Ld

iq s

ω r L d i ds

Rs

+

+

V qs

−
Lq

−

−

Fig.1. The d-q axis equivalent-circuit of SynRM
diVds = Rsids + Ld ds − ωr Lqiqs
(1)
dt
diqs
Vqs = Rsiqs + Lq
+ ωr Ld ids(2)
dt
The corresponding electromagnetic torque Te is:()

3
Pole Ld − Lq idsiqs(3)
4
The corresponding motor dynamic equation is:dω r
Te − TL = J m+ Bm ω r(4)dt
where Vds and Vqs are direct and quadrature axis
Te =terminal

269

voltages,

respectively; i ds

and

i qs are,

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

respectively, direct axis and quadrature axis terminal
currents or the torque producing current; Ld and Lqof (8) can be defined as equation (10):u (t ) = ueq (t ) + u n (t )(10)where ueq (t ) is used to control the overall behavior of
the system and u n (t ) is used to suppress parameter
uncertainties and to reject disturbances. By making
mathematical calculation, we get the overall control
u (t ) as equation (10) [3]:1 ∗
ηu (t ) =ω& r − a 0ω r + ce (t ) + ( K + ) sgn( S ) (11)b0bo

are the direct and quadrature axis magnetizing
inductances, respectively; R s is the stator resistance;and ω r is the speed of the rotor. Pole , T L , J m , and

Bm are the poles, the torque load, the inertia momentof the rotor, and the viscous friction coefficient,
respectively. In this paper, the maximum torque control(MTC) strategy [3,4] is adopted. The torque current
commands are shown in equation (5) and (6) [3]:
Te

∗
ids
=

(

3
Pole Ld − Lq
8

)

π 
cos 
4Teπ 
sin  3
4
Pole Ld − Lq8∗
= sgn (Te )
iqs

(

)

[

where f ≤ K .

(5)

IV.

We can rewrite the equation (4) as= aωr + b(Te −TL )SUPER-TWISTING SECOND-ORDERSLIDING MODE CONTROLLER

In conventional sliding mode control design, thecontrol target is let the system state move into sliding
surfaces S = 0 . But a second-order sliding mode
controller aims for S = S& = 0 . The system states
converge to zero intersection of S and S& in state
space.
Super-twisting method mainly develops relative oneorder system for reducing chattering phenomenon [7].
The state trajectory of S and S& phase plane is shown in
Fig.2. It twists together and approaches in state space.
Finally, it converges to the origin of phase plane.

(6)III. INTEGRAL VARIABLE STRUCTURESLIDING MODE CONTROLLERdωr  Bm 1= − ωr + (Te −TL )dt  Jm 
J

]

(7)

= (a0 + ∆a)ωr + (b0 + ∆b)(Te −TL )= a0ωr + b0 (u(t) + f )

S&

where
a≡−
b≡

B
= a 0 + ∆aJ1
= b0 + ∆bJSu ≡ Tef ≡1(∆aωr + ∆bu (t ) − bTL )b0

J m ≡ J 0 + ∆J
Bm ≡ B0 + ∆B
The subscript index “ o ” indicates the nominal system
value; “ ∆ ” represents uncertainty, and f represents
the lumped uncertainties.
Define the velocity error as e(t ) = ω r∗ − ω r , where

Fig.2. The phase plane trajectory of Super-twisting
Consider sliding variable dynamics given by asystem with a relative degree two:y&1 (t ) = ϕ ST ( y1 , t ) + γ ST ( y1 , t )u ST (t )(12)where y1 (t ) means the sliding variable S , in which ϕ ST
and γ ST are uncertain functions with the upper andlower bounds of equation (13), and u ST (t ) is the scalar
control input.

ωr∗ is the velocity command. The velocity error
differential equation of SynRM can be expressed as
equation (8):de(t )= ω& r∗ − a0ω − b0 [u (t ) + f ]dt(8) ϕ ST ≤ Φ ST0 < Γm ST ≤ γ ST ≤ ΓM ST

Let

S = e(t ) + ct∫ e(t )dτ , c > 0−∞(9)The control u ST (t ) can be given as a sum of two
components as shown in equation (14) [6,7]：

The input control u (t ) (the electromagnetic torque Te )©ISAROB 2011

(13)

270

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011u ST (t ) = u1 (t ) + u 2 (t )where− u ST ,if
u&1 (t ) = 
− W sgn( y1 ), if

u ST > U
u ST ≤ U(14)

Table 1. Parameters of SynRM (0.37kW)
Rs = 4.2 Ω
Pole = 2(15)

Lds = 0.328( f = 60 Hz)

Lqs = 0.181 H ( f = 60 Hz)

Bm = 0.00012
Ntm/rad/sec
In Fig.4, the simulation velocity response of the
SMC due to ωr∗ = 600 rev/min without machine load− λ S ρ sgn( y ), if y > S
0
1
1
0
(16)
u 2 (t ) = ρ − λ y1 sgn( y1 ) , if y1 ≤ S 0
where U is control value boundary, and S0 is aboundary layer around the sliding surface S 。
We define state variable as shown in equation (17):
t

x
(
t
)
=
x2 (τ )dτ
1

−∞
(17)

 x2 (t ) = e(t ) = ω r∗ − ω rWe define sliding function y1 as
y1 = x2 + cx1(18)Then, the system equation can be expressed as

J m = 0.00076 Kg-m2

in the nominal motor inertia and friction coefficientcondition is depicted. In Fig.5, the simulation velocity
response of the SOSMC due to ωr∗ = 600 rev/min∫without machine load in the nominal motor inertia andfriction coefficient condition is depicted. The velocity
response of SOSMC is smoother than the convention
SMC. In Fig.6, the simulation velocity response of the
SOSMC due to ωr∗ = 600 rev/min under an 0.3 Nt-m
machine load at the beginning and an 0.9 Nt-m machineload at 3seconds is added for the 2 times nominal caseof the motor inertia and friction coefficient condition ispresented. Hence, the SOSMC is a robust controller andimprove the chattering phenomenon when the system
has external disturbances and parameter variations.

 B

B
1
y&1 = ω& r∗ +  − m + c  x2 +TL + m ω r∗ + u ST (t )
(19)
Jm
Jm
 Jm

where


B
1
∗  Bm
TL + m ω r∗
+ c  x 2 +ϕ (⋅) = ω& r +  −
Jm
Jm

 Jm

γ (⋅) = 1
1
u ST (t ) = − J Te
m


H

(20)

According to equation (20), the practicalcontrollable signal Te of SynRM is a continuous
controllable signal. Different from the discontinuous
controllable signal of conventional SMC, it can improvethe chattering problem in SOSMC of SynRM apparently.

V. SIMULATION RESULTS
A block diagram of the experimental SynRM drive
and the super-twisting second-order sliding mode
controller speed control block diagram of the SynRM
servo drive are shown in Fig. 3. The proposed controller
was applied to a 0.37 kw three-phase SynRM whose
nominal parameters and proposed controller parameters
are shown in Table 1. The detail specifics of SynRM are
shown in Appendix 2.
Outer loop speed control

inner loop torque control
∗
e

∗
ds

u =T

ωr∗ +
−

SOSMC

limiteri
MTC

Strategy +
∗
qs

i

v

+
−

−

∗
ds

PI Current
Controller

v

v

v

∗
β

V
S
V
gating
P
W

M

ids
iqs
ωr

d
dt

l motor inertia and friction coefficient condition

hardware driver circuit∗
α

2φ / 2φ∗qs

Fig.4. Simulation velocity response of the SMC due toωr∗ = 600 rev/min without machine load in the nomina

ias ics
3φ / 2φ−∑θr

Fig.5. Simulation velocity response of the super-twisting
SOSMC due to ωr∗ = 600 rev/min without machine

Fig.3. Super-Twisting SOSMC speed control block
diagram of SynRM servo drive©ISAROB 2011

load in the nominal motor inertia and friction coefficientcondition

271

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

REFERENCES[1] Bianchi N., Bolognani S., Bon D., and Pre M. D.
(2009), Rotor flux-barrier design for torque ripple
reduction in synchronous reluctance and PM-assisted
synchronous reluctance motors. IEEE Transactions on
Industry Applications 45(3):921-928[2] Kim W. H., Kim K. S., Kim S. J., Kang D. W.,
Go S. C., Chun Y. D., and Lee J. (2009), Optimal
PM design of PMA-SynRM for wide constantpower operation and torque ripple reduction.
IEEE Transactions on Magnetics 45(10):46604663[3] Chen C. A., Lin W. B., and Chiang H. K. (2009),
Design and implementation sliding mode controller
based on radial basis function neural network forsynchronous reluctance motor. ICIEA 2009, 4th IEEE
Conference on Industrial Electronics and Applications:281-286.
[4] Betz R. E., Lagerquist R., Miller T. J. E., andMiddleton R. H. (1993), Control of synchronous
relutance machines. IEEE Transactions on Industry
Applications 29(6):1110-1122[5] Shyu K. K., and Lai C. K. (2002), Incremental
motion control of synchronous reluctance motor via
multisegment sliding mode control method. IEEE
Transactions on Control Systems Technology 10(2):169-176[6] Huang Y. S., and Sung C. C. (2010),
Implementation of sliding mode controller for linear
synchronous motors based on direct thrust control
theory. IET Proceedings on Control Theory &Applications 4(3):326-338[7] Bartolini G., Ferrara A., Levant F., and Usai E.
(1999), Variable Structure Systems, Sliding Mode andNonlinear Control. Springer-Verlag[8] Levant A. (1993), Sliding order and sliding accuracy
in sliding mode control. International Journal of Control
Automation and Systems 58(6):1247-1263[9] Rashed M., Goh K. B., Dunnigan M. W.,
MacConnell P. F. A., Stronach A. F., and Williams B.
W. (2005), Sensorless second-order sliding-mode speed
control of a voltage-fed induction-motor drive using
nonlinear state feedback., IEE Proceedings on ElectricPower Applications 152(5):1127-1136[10] Kunusch C., Puleston P. F., Mayosky M. A., andRiera J. (2009), Sliding mode strategy for PEM fuelcells stacks breathing control using a super-twistingalgorithm. IEEE Transactions on Control Systems
Technology 17(1):167-174[11] Tournes C., and Shtessel Y. (2007), Automatic
docking using second order sliding mode control. 2007
American Control Conference:3455-3460[12] Mohamadian M., Pedram M. M., and Ashrafzadeh
F. (2004), Digital second order sliding mode control fora synchronous reluctance motor. 2004 IEEE Industry
Applications Conference:1899-1902

Fig.6. Simulation velocity response of the super-twisting
SOSMC due to ωr∗ = 600 rev/min under a 0.3 Nt-m
machine load at the beginning and a 0.9Nt-m machineload at 3 seconds in the 2 times nominal case of the
motor inertia and friction coefficient condition

VI. CONCLUSION
In this paper, a super-twisting second-order sliding
mode (SOSMC) speed control design for robust
stabilization and disturbance rejection of SynRM drive.
The simulation results show good performance of
SOSMC under uncertain load subject to variaitons ininertia and system frcition. Also with SOSMC, there isno need for acceleration feedback. An experimental
setup has been prepared to assess the perfromance of the
proposed controller. Employing the experiments, thiscontroller provides a fast and good response for theparameter variations and external disturbances. Apartfrom the robustness feature, the proposed second-order
SMC laws have the advantage of being continuous, thus
eliminating the chattering effect apparently and being
more acceptable in application.

ACKNOWLEDGMENT
This work is supported by the National Science
Council in Taiwan, Republic of China, through grant
NSC99-2221-E-224-072.

APPENDIX
Table 2. Rated parameters of SynRM
Specifics
Value
Rated voltage
230/400 V
Rated current
4.7/2.7 V
Rated speed
3600 rpm
Rated power
0.37 kWPower factor
0.37
Insulation classF
Waterproof and dust-proof
IP 20
class

©ISAROB 2011272

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Shape Recognition Applied in a Semi-Autonomous Weapon Robot
Chun-Chieh Wang1, Chyun-Luen Lin2, Kuo-Lan Su3
Department of Electronic Engineering, Chienkuo Technology University
No. 1, Chieh Shou N. Rd., Changhua City, Taiwan(Tel : 886-4-7111111ext3350; Fax : 886-4-7111139)(Email address: jasonccw@ctu.edu.tw)
2
Department of Electronic Engineering, Chienkuo Technology University
No. 1, Chieh Shou N. Rd., Changhua City, Taiwan(Tel : 886-4-7111111ext3350; Fax : 886-4-7111139)(Email address: a829327@yahoo.com.tw)
3
Department of Electrical Engineering, National Yunlin University of Science & Technology,
No. 123, Section 3, University Road, Douliu, Yunlin, Taiwan(Tel : 886-5-5342601ext4249; Fax : 886-5-5312065)(Email address: sukl@yuntech.edu.tw )1

Abstract: A weapon robot with semi-autonomous shooting is implemented in this paper. The principal aim is theapplication of a shape recognition method so that the MP5K electric BB gun can shoot semi-autonomously. In addition,
we designed a human–machine interface surveillance system via the LabVIEW graphical programming environment,such that the supervisor can control the weapon robot by a keyboard or a specially adapted mouse. In order toaccomplish all these achievements, there have been major additions and overhauls in both system software codes andsystem circuit board developments. The experimental results have shown the practicality of the shape recognition, the89C51 microcontroller, the LabVIEW graphical programming environment, and ZigBee wireless technology applied toweapon robots.
Keywords: LabVIEW, ZigBee, Semi-autonomous shooting, Shape recognition, POB-Basic system.
the practicality of the shape recognition, the 89C51
microcontroller, the LabVIEW graphical programming
environment, and the ZigBee wireless technology
applied to weapon robots.

I. INTRODUCTION
There are times when a rescue team is unable to enter
the scene of an accident owing to various problems thatmight endanger the lives of the rescuers. In order toovercome these possible obstacles, researchers havebuilt several robots that can enter such dangerous sites
instead of the rescuers. However, as regards HMIguided control, few researchers have used the
LabVIEW platform to experiment with tracked robots.
In 2001, Priya Olden et al.1 presented an open-loop
motor speed control with LabVIEW [1]. In 2006,
Prasanna Ballal et al. [2] proposed a LabVIEW-based
test-bed with off-the-shelf components for research into
mobile sensor networks.
Therefore, we extended previous research [3–7] toimplement a weapon robot. The principal aim is theapplication of a shape recognition method so that the
MP5K electric BB gun can shoot semi-autonomously.
In addition, we designed a human–machine interface
surveillance system via the LabVIEW graphicalprogramming environment, such that the supervisor cancontrol the weapon robot by a keyboard or a specially
adapted mouse.
In order to accomplish all these achievements, there
have been major additions and overhauls in both system
software codes and system circuit board developments.
To illustrate the effectiveness of the design, we planned
an urban fight as the scenario in which the robot could
use all its functions. The experimental results validate©ISAROB 2011

II. SELF-MADE WEAPON ROBOT
2.1 Mechanism of the Robot
Figure 1 shows an overview of the platform of theweapon robot. Figure 2 shows the right-hand side view.
The structure has two active wheels and one small
assistant wheel. The specifications of the robotic
platform are (1) length 41 cm, (2) width 31 cm, and (3)height 20 cm. The home-made weapon robot is shownin Fig.3. We put an MP5K electric BB gun on theplatform of the robot, and set one camera behind thegun sights. The specifications of the weapon robot are(1) length 52 cm, (2) width 31 cm, and (3) height 43 cm.
41cm

Fig.1. Vertical view of the platform

273

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 201143cm

control. A complete system can be constructed very fastwith hardware modules for data accomplishment, image
processing, motion domination, or communication
available from National Instruments (NI). Eachapplication created in LabVIEW is referred to as a
virtual instrument (VI). A VI consists of a user interface
front panel and a block diagram. A VI can also be calledfrom another VI, called a sub-VI. The standard
LabVIEW package comes with various VIs in the formof libraries and drivers to permit rapid program
development.
Here, we use LabVIEW graphical programming todesign a human machine interface surveillance system.
From the transmission of RS-232 and ZigBee modules,
a command will be delivered to the controller andplaced on the weapon robot so that the vehicle will be
arrive at assigned place. The LabVIEW front panel isshown in Fig.6. There are eight function-blocks in thisfigure. Block 1 is the safety push-button. If we put the
button, all of the functions can’t be fulfilled. Block 2
displays a keyboard/genius mouse switch device. We
can choose the keyboard or the mouse to control therobot. Moreover, we can set the RS232 I/O port andbaud rate by Block 3 and Block 4, respectively. Block 5
displays the character string when the keyboard ispressed. The ASCII code is shown in Block 6. Block 7
indicates the frame of controlling the order through themouse. Block 8 displays the scene, taken by the wirelessnetwork camera.

20cm

Fig.2. Right side view of the platform

Fig.3. Weapon robot
2.2 Wireless Network Camera and Router
In this paper, an AXIS 207MW wireless network
camera, as shown in Fig.4, is applied to take a scene.
Figure 5 shows the D-LINK DIR-615 router.

Fig.4. AXIS 207MW wireless network camera

Fig.5. D-LINK DIR-615 router

III. LabVIEW [4]Fig.6. LabVIEW front panel

The Laboratory Virtual Instrument Engineering
Workbench (LabVIEW) is an easy-to-use graphicaldevelopment environment which allows users to rapidly
develop applications for experiment, measurement and©ISAROB 2011

IV. ZIGBEE TRANSPARENT-P2P MODE
ZigBee is an intelligent digital protocol, operating at
three frequencies, with the commonest one being at 2.4

274

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

because it must be done 8129 times. That is why using
the technique “1 bit = 1 pixel” will need less memory
space. Another way of drawing a pixel on the screen is
to consider that 1 byte draws 1 pixel. In that case they
will be no delay or mask when switching on or off apixel. Nevertheless, such a technique will take 8 times
more memory than the previous one. However, forshape recognition, we have to transfer 2-D array into 1D array, as shown in Fig.10, such that the externals(length, width, and height) of the figure will be stored in
the POB-EYE memory.

GHz. At this operating frequency, data rates up to 250
kbit/s are claimed. This is a relatively low bandwidthcompared to other protocols such as Bluetooth. In
addition, the features of ZigBee contain the robustness
and simplicity of IEEE 802.15.4 standard, the versatilityof the ZigBee compliance platform, low consumption
and cost, and the standard-based short-range wirelessnetworking.
Here, ZigBee is developed for point-to-pointtransmission and area positioning. IP-Link 2220H, asshown in Fig.7, provides a host of AT commands toallow easy configuration of the key attributes of an IPLink 2220 module. Users can use any terminal
emulation utility or UART communication library on aparticular host platform to issue these AT commands to
IP-Link 2220. Figure 8 shows the transparent point-topoint mode. The signal can be transmitted byconnecting RS-232 series port and ZigBee transmitterso that it can be received. In addition, a tag (placed on
the robot) transmits a continuous signal so that the node(placed on the scenario setting space), which is closest
to the robot, can receive the signal and display it on themonitor. Then, the supervisor will know the present
position of the robot.

Fig.9. 2-D Array
Fig.10. 1-D Array
Figure 11 and Figure 12 display the hardware of
POB-EYE and the main electronic board of POB-EYE,
respectively.

Fig.7. IP-Link2220H

Fig.8. Transparent P2P Mode
Fig.11. Hardware of POB-EYE

V. POB-BASIC SYSTEM AND SHAPE
RECOGNITION
In general, a black/white LCD screen uses one bit tocolor a pixel and the screen displays 8 pixels with a byte,
as shown in Fig.9. In addition, the buffer uses 8192 bits(the POB-LCD screen is 128 pixels by 64 pixels) ofmemory on POB-EYE module. The challenge of thisbuffer is to have a ration of “1 bit = 1 pixel”. The
operations on bits are processor-intensive. The
advantage of this “1 bit = 1 pixel” ration is that it does
not use much memory. In fact, we need to apply a mask
in order to change a bit value in memory. Moreover, thismask will be different whether the bit is set to 0 or to 1.
Thus this computation can be relatively time consuming©ISAROB 2011

Fig.12. Main electronic board of POB-EYE
The graphics resources can be displayed using thelibrary supplied with the POB-compiler tools. The
graphic functions allow us to manage the transparencyof the images and to carry out the superposition ofimages on the LCD screen. Images are displayed using

275

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

the “DrawBitmap” function, as shown in Fig.13. Thisfunction uses a number to display the desired image.
For more clarity in our code, POB-Bitmap generates in“.h” a series of “#define” from the image’s name. The
call to “DrawBitmap” can be becomes (if the name of
the file is cross.bmp): DrawBitmap(X, Y, 1, Tableau)→ DrawBitmap(X, Y, IDB_CROSS, Tableau).

VII. CONCLUSION
A weapon robot with semi-autonomous shooting has
been implemented. A shape recognition method wasapplied to design a controller so that the MP5K electric
BB gun can shoot semi-autonomously. We also
designed a human–machine interface surveillancesystem via the LabVIEW graphical programming
environment so that the supervisor could control theweapon robot by either a keyboard or a specially
adapted mouse. The experimental results validate thepracticality of the shape recognition, the 89C51
microcontroller, the LabVIEW graphical programming
environment, and ZigBee wireless technology applied toweapon robots.

Fig.13. POB-Pattern

REFERENCES[1] Olden P., Robinson K., Tanner K., Wilson R., andBasher A.M.H. (2001), Open-loop motor speed
control with LabVIEW, Proceedings of 2001 IEEE
Southeast Conference, 259-262.
[2] Ballal P., Giordano V., Dang P., Gorthi S., Mireles
J., and Lewis F. (2006), A LabVIEW based test-bed
with off-the-shelf components for research inmobile sensor networks, Proceedings of the 2006
IEEE International Symposium on Intelligent
Control, 112-118.
[3] C.C. Wang, K.H. Hsia, K.L. Su, Y.C. Hsieh, andC.L. Lin (2010), Application of a Remote Image
Surveillance System in a Robotic Weapon,
International Journal of Artificial Life and Robotics,
vol. 15, pp.284-290.
[4] C.C. Wang, C.L. Lin, K.H. Hsia, and Y.C. Hsieh(2010), Obstacle Avoidance and Wireless Network
Surveillance of a Weapon Robot, 2010 International
Symposium on Computer, Communication, Control
and Automation, May 5-7, Tainan, Taiwan, pp.261264.
[5] C.C. Wang, K.L. Su and, C.T. Shen (2009),
Implementation of a Tour Guide Robot via Shape
Recognition and Path Planning, Proceedings of The
Fourth International Conference on Innovative
Computing, Information and Control (ICICIC 2009),
Dec. 7-9, Kaohsiung City, Taiwan.
[6] C.C. Wang, C.L. Lin, and, K.L. Su (2009), Shape
Recognition for an Entertainment Robot with afive-axle Arm, Proceedings of 2009 CACS
International Automatic Control Conference, Nov.
27-29, NTUT, Taipei, Taiwan, pp.377-380.
[7] C.C. Wang, K.L. Su, K.H. Hsia, Y.C. Hsieh, C.L.
Lin (2009), Implementation of a Robotic Weapon
via an AVR Microcontroller, The 17th National
Conference on Fuzzy Theory and Its Applications,
NUK, Kaohsiung, Taiwan, Dec. 18-19, pp.681-686.

VI. EXPERIMENTAL RESULTS
Figure 14 shows the target. The MP5K electric BB
gun can shoot the target semi-autonomously via shaperecognition techniques.
For the scenario setting and the experimental test, weplanned the indoor orientation diagram, as shown in
Fig.15. The robotic weapon will look for the objects(balloons), aim at them, and shoot them. Moreover, it
can pass along a difficult route and look for that route.
Then the tracking vehicle can move toward the objects
and shoot them again.

Fig.14. Mark

Fig.15. Indoor orientation diagram©ISAROB 2011276

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Camera Position Estimation and Feature Extraction from Incomplete Image ofLandmark
Kuo-Hsien Hsia1, Shao-Fan Lien2 and Juhng-Perng Su3
1

Department of Management Information System, Far East University
No. 49, Chung Hua Rd., Hsin-Shih. Tainan County 74448, Taiwan, R.O.C.
(Tel : 886-6-597-9566#5862; Fax : 886-6-597-7010)(E-mail: khhsia@cc.feu.edu.tw)
2
Graduate School of Engineering Science & Technology (Doctoral Program)
National Yunlin University of Science & Technology
No. 123, University Road, Section 3, Douliou, Yunlin 64002, Taiwan, R.O.C.
(E-mail: g9510801@yuntech.edu.tw)
3
Department of Electrical Engineering, National Yunlin University of Science & Technology
No. 123, University Road, Section 3, Douliou, Yunlin 64002, Taiwan, R.O.C.
3
Department of Information Technology, Overseas Chinese University
No. 100, Ciao Kwang Rd., Taichung 40721, Taiwan, R.O.C.
(E-mail: sujp@yuntech.edu.tw)

Abstract: On the autonomous unmanned helicopter landing problem, the position of the unmanned helicopter relative tothe landmark is very important. A camera carried on the unmanned helicopter could capture the image of the landmark.
In the earlier researches, the camera position could be estimated by the extracted features of the landmark image.
However, it is necessary that the landmark image should be complete or with slight deficiency in order for theestimation process. In this paper, the innovated method is designed for camera position estimation from single image
with incomplete landmark. An ANFIS is utilized for constructing the mapping relation between the features of complete
and incomplete landmark image. It will be verified that it is possible to estimate the camera position from a landmark
image with defects more than half via the proposed method.
Keywords: Camera position estimation, Incomplete landmark image, Feature extraction, ANFIS.
vehicle is landing. Fig. 1 shows the situation concernedin [5-6] that an unmanned helicopter is approaching thelandmark. The position of an unmanned helicopter canbe described by θ and R. The θ and R can be estimatedfrom the captured landmark image with the method
proposed in [6].

I. INTRODUCTION
Unmanned helicopters are very useful vehicles foraerial photography or investigation in hazardous locations.
The research of autonomous unmanned helicopter control
has been ongoing for several years. GPS (Global
Positioning System) is the most commonly used sensor
for position control of unmanned helicopters.
Unfortunately, GPS error can range up to 6 meters [1].
Greater accuracy can be achieved by incorporating
image-guided methods in conjunction with a GPS system.
In earlier researches, image servo control systems
for a small scale autonomous unmanned helicopter are
discussed in [2] and [3]. In these two references, twomarks are used for assisting unmanned helicopter
positioning. However, in practice when an unmanned
helicopter is landing, the landmark constitutes a single
reference on the ground. Consequently, image-guided
methods are proposed for locating the landmark [4] andestimating the position information [5-6].
Image-guided methods are usually employed to
guide an unmanned helicopter on close approach to atarget. The distance between the vehicle and target isusually smaller than the range of GPS error while the©ISAROB 2011



Fig.1. Situation of an unmanned helicopter
approaching a landmark
In [5], it is necessary that the captured landmark
image must be complete. Since the helicopter may yaw
due to the effect of wind or the image size of thelandmark may be larger than the image available to the277

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

camera this restriction has to be overcome. The casein which more than half the landmark image is available,
called “Case 1”, has been discussed in [6].
In this paper, we will focus on the case in which morethan half of the landmark image is defective, called “Case2”. This scenario presents serious problems for extracting
useful features. A feature extraction method is developedfor extracting the features from the incomplete image of
landmark in which more than half of the landmark image
is defective. We use ANFIS to capture and estimate theuseful features from an incomplete landmark image.
Simulation results verify that under some conditions
it is possible to estimate the camera position from alandmark image in which less the than half the total
image is available.

ĥk̂

Fig.2. Incomplete landmark image and features
The projective relation between the landmark andcamera position is shown in Fig. 3. Definitions of  , and  e are provided as1  f  b  e  r 
1  f  b  e   tan   tan 
h
h






1  f  b  e 
1  f  b  e  r   tan  (7)   tan 
h
h






1  f  b  e 
1  f  b   tan   e  tan 
h


 h 


II. MAPPING RELATIONS AND CAMERAPOSITION ESTIMATING METHOD
1. Projective geometry of landmark image
The five quantities, as defined in (1-5), obtained via
geometric projection [7] are the projection of thefeatures of the landmark image. We defined (6) for
estimating the camera position [5].
k  tan( γ  σ) - tan γ  * R(1)h  tan(γ  σ  ω) - tan( γ  σ)  * R(2)

Q  tan( γ  σ) * R(3)tan λ * R   tan δ * R r  S   R r
cos θ  R  T  - sin θ * R 
d

2

2

2

2



(4)
Fig.3. Projective geometry of landmark features(5)

Fig. 4 illustrates the projective relations of features
I2, I3 and I4. From Fig.3 and Fig.4, the derivations ofĥ , k̂ , I1, I2, I3, I4 and I5 can be obtained as shown in (813) and the estimated ĥ and k̂ in (14).
I 5  tan       e R(8)
I1  tan R  I 5(9)

 Ratio project = h k Ratioheight = r d(6) Ratioangle = h r Ratio position = k QWe further consider the cases of incomplete
landmark images, specifically, the case of the landmark
image in which more than half of the image is available[6]. The four quantities: k, r , x, and y, related to thecompleted landmark image, are selected for estimating
the approximate h. The estimated h is called ĥ . Here k
and r have been derived from (1) and (2), respectively.
Consider the case in which more than half thelandmark image is defective, as shown in Fig.2. The
five quantities: I1, I2, I3, I4 and I5 which are related to thecomplete landmark image are selected for estimating theapproximate h and k, called ĥ and k̂ , respectively.

©ISAROB 2011

 Cut lim



2

2



k̂

ĥ

  e r  e  I  I   R
2

I2 



2

1

5

2

2

(10)

H 2  b  f 2

k

g

(a)
(b)
Fig.4. (a) Top view of the landmark features(b) Projections of the landmark features

278

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

I3 

I4 

 f  tan( k ) I1  I 5 2  R 2
H 2  b  f 2

2. Camera position estimation method
The proposed method is illustrated in Fig.5. There
are two major parts in this method. The first part is“Landmark image detection algorithm”. The algorithm
is designed for computing the landmark image anddetecting:
1. Is the landmark image complete?
2. Is more than half of the image defective?
3. Is the landmark image beyond sensor range?
The basic rectangle is employed not only forexecuting efficient detection but also for fastcomputation of the features.
The second part is “Image feature based camera
position estimation method”. The incomplete landmark
image features database could be constructed by (1-15).
The ANFIS is employed for recognizing the mapping
relations between the image features and the camera
positions. Furthermore, the camera positions can be
estimated quickly via the established mapping model.

(11)



tan    e      cut lim R I tan 1  3 
 I1  I 5  g b g  tan 1 
      e  
 h 





(12)



(13)



kˆ  tan     R  I 5(14)hˆ  tan    e   R  I1  I 5
Obviously, from Fig. 2, I4 is an important quantity
for estimating ĥ and k̂ . Therefore the restriction ofthis method is that the tangent I4 must be available, i.e.,
I1 + I5 has to be less than Cutlim defined by
Cutlim = r 2  s 2(15)

where s  sin  k   f  e 2  r 2 .

Fig.5. Flow chart of camera position estimation method

III. SIMULATION RESULTS

3. ANFIS training
The model for “Case 1” has been established andconfirmed in [6]. We only discuss “Case 2” in this paper.
Two ANFIS are provided for estimating the ĥ , and k̂ .
In addition, the database is established by (7-15), andthe network is illustrated in Fig. 6.

In this section, we will demonstrate the estimation
results of the ĥ and k̂ from the incomplete landmark
features I1, I2, I3, I4 and I5 via ANFIS. The simulation isbased on the following qualities:
1. The image size of the camera is 320×240 pixels.
2. The diameter of the circular landmark is 3 cm.
3. The range of R is 20 ~ 300 cm.
4. The range of θ is 0 ~ 62 degrees.
The estimation results of features ĥ and k̂ are
illustrated in Table 1. There are 10 sets of data utilized
for testing the ANFIS training results. In Table 1 theFig.6. Illustration of ĥ and k̂ computation with dual ANFIS©ISAROB 2011

279

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011[2] Mori R, Hirata K and Kinoshita T (2007), VisionBased Guidance Control of a Small-Scale Unmanned
Helicopter, International Conference on Intelligent
Robots and Systems, 2648-2653.
[3] Mori R, Kubo T and Kinoshita T (2006), VisionBased Hovering Control of a Small-Scale Unmanned
Helicopter, SICE-ICASE International Joint Conference,
1274-1278.
[4] Wang CC, Lien SF and Hsia KH et al (2009),
Image-Guided Searching for Landmark, Artificial Lifeand Robotics, 14(1):95-100.
[5] Lien SF, Hsia KH and Su JP (2010). Image-Guided
Height Estimation for Unmanned Helicopter Landing,
ICIC Express Letters, 4(6B:2299-2304.
[6] Hsia KH, Lien SF and Su JP (2010), Camera
Position Estimation by ANFIS from Incomplete
Landmark Image, Conference on Fuzzy Theory and Its
Applications, Accepted.
[7] Hartley R and Zisserman A (2003), Multiple View
Geometry in Computer Vision, Cambridge University
Press.

RMSE of ĥ and k̂ are 0.000615 and 0.000471,
respectively. The ĥ and k̂ are utilized to approximate
the complete landmark image features. Table 2
illustrates the estimation results of camera position withthe features ĥ and k̂ . The RMSE of the θ, R and Q
are 0.45868, 0.506617 and 0.240798, respectively.

IV. CONCLUSION
In this paper, a scenario in which more that half of alandmark image is defective has been analyzed. The
method for estimating the approximate features ofincomplete landmark images is provided. Fromsimulation results, it is clear that the approximatefeatures can be very closely estimated via the known
quantities. Moreover, we have shown that theapproximate features can be used for camera position
estimation. The inherent camera position estimation
error range is acceptable for practical applications. The
method proposed in this paper is a functional tool forcamera position estimation.

Acknowledgement
This research was supported by the National Science
Council, Taiwan, R.O.C., under Grant No. NSC992221-E-269-020.

REFERENCES[1] GPS 18 Technical Specifications (2005), Revision D,
Garmin International, USA.

Table 1. The estimation results of features ĥ and k̂
No.

θ

R (cm)kk̂

Error of feature khĥ

Error of feature h

1
2
3
4
5
6
7
8
9
10

32°
21°
42°
53°
19°
60°
10°
35°
13°
29°

31
44
78
113
157
163
173
201
240
289

1.588912.87088
2.067100.160881.780140.771892.78694
0.92258
3.222092.92995

1.588232.870232.067900.16187
1.78001
0.77171
2.78656
0.92308
3.22278
2.929510.000680.00065-0.00080-0.00099
0.000130.000180.00038-0.00050-0.00069
0.000441.704742.73393
1.96832
0.15886
1.762720.740552.77071
0.91298
3.222092.91630

1.704632.73451
1.96786
0.15934
1.763470.739762.770910.91274
3.22176
2.916110.00011-0.00058
0.00046-0.00048-0.00075
0.00079-0.00020
0.000240.00033
0.00019

Table 2. The estimation results of camera position with the features ĥ and k̂
No.

θ

R (cm) Q (cm)

1
2
3
4
5
6
7
8
9
10

32°
21°
42°
53°
19°
60°
10°
35°
13°
29°

31
44
78
113
157
163
173
201
240
289

©ISAROB 2011

10
23
189
78
63
284
174
124
254
365

Error of estimated θError of estimated R

Error of estimated Q-0.619-0.3540.475-0.4870.221-0.334-0.5310.437-0.634-0.3120.421-0.572-0.435-0.3760.636
0.551-0.4170.623-0.581
0.352-0.1740.243
0.314-0.1970.231
0.299-0.1430.354-0.2080.146

280

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A Novel Variable Structure Theory Applied in Design for Wheeled Mobile
RobotsChun-Chieh Wang
Department of Electronic Engineering, Chienkuo Technology University
No. 1, Chieh Shou N. Rd., Changhua City, Taiwan(Tel : 886-4-7111111ext3350; Fax : 886-4-7111139)(Email address: jasonccw@ctu.edu.tw)

Abstract: This paper is concerned with the control of wheeled mobile robots (WMRs) using a modified variable
structure theory. Firstly, we introduce the dynamic characteristic of a WMR. Secondly, the conventional variable structure control is reviewed. To remarkably improve the transient response during the reaching phase, a modified variable
structure control is proposed. The validity of the proposed variable structure theory is verified by means of a simulation
testing on a homemade wheeled mobile vehicle. The simulation results validate the superiority and practicality of themodified variable structure for WMRs.
Keywords: Variable Structure theory, Sliding mode control, Wheeled mobile robot.
chattering, a continuous approximation of the discontinuous sliding control is usually employed in the literature. Though, the chattering can be made negligible if
the width of the boundary layer is chosen large enough,the guaranteed tracking precision will deteriorate if theavailable control bandwidth is limited. To reach a better
compromise between small chattering and good tracking precision in the presence of parameter uncertainties,
various compensation strategies have been proposed.
For example, integral sliding control [4], [5], [9], sliding
control with time-varying boundary layers [5] etc., were
presented. Alternatively, applying so-called reaching
law approach, Gao et al. [12] proposed sliding controllers such that the trajectories are forced to approach thesliding surface faster when they are far away from thesliding surface. This approach seems to be an efficientmethod capable of increasing the approaching speed tothe sliding surface; however, the behavior of the system
dynamics, governed by the transformed first-order equation, can only be predicted through the measurement of
the generalized error; hence the transient response during the reaching phase may not be remarkably improved.
In this paper, it is concerned with the control of a
WMR using a modified variable structure theory in theboundary layer. Then we will review the conventionalvariable structure control scheme and present the modified variable structure control theory. The effectivenessof the newly developed control scheme will be demonstrated through the control of the WMR. We’ll showthat the transient response during the reaching phase has
been remarkably improved by the proposed control.

I. INTRODUCTION
Mobile robots have a wide background of application,and motion control of WMRs has found considerable
attentions over the last decades. The path-tracking control problem of WMRs has received sustained attention[1][2][14]. However, the above researches are mainly
based on kinematic models of nonholonomic mobile
robots. Moreover, the velocity commands must be converted into the actual control input for vehicles. Hence,
dynamic models of systems should be considered. Recently, several authors also consider the dynamic modelof the WMR [7][13].
Variable structure control was initiated in Russia bymany researchers, like Barbashin [3], Utkin [10], Emelyanov [8]. The control scheme has successfully been
applied to many engineering problems including automatic flight control, chemical processes, helicopter stability augmentation systems, electric motors, robots, etc.
Variable structure control law is deliberately changed
according to some defined rules which depend on thestate of the system. The scheme has been mainly considered for continuous-time systems in the form of sliding mode control.
Sliding mode control is known to be very robust
against parameter variations and external disturbancesand has been widely accepted as an efficient method for
tracking control of uncertain nonlinear systems. It has
been shown to be able to achieve ‘perfect’ performancein principle in the presence of parameter uncertainties,
bounded external disturbance, etc [6]. However, in orderto account for the presence of parameter uncertainties
and bounded disturbances, a discontinuous switching
function is inevitably incorporated into the control law
to achieve so-called sliding condition [11]. Due to imperfect switching in practice it will raise the issue ofchattering, which is usually undesirable. To suppress©ISAROB 2011

II. HOME-MADE WMR
The home-made WMR is shown in Fig.1. It consistsof a vehicle with two driving wheels mounted on thesame axis and a free front wheel. The motion and orientation are achieved by independent actuators. Hence in
the WMR model, we assume the coordinates of the281

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

mass center of the WMR is located in the middle of thehind driving wheel.

Fig.1. Home-made WMR◎ Dynamic Equations of a WMR
The 2-D figure of a WMR is shown in Fig. 2. Fig. 3
shows 2-D of the car motion. The dynamic equations
are described as follows, x  cos  0 
 y    sin  0   vl 
(1)
  
 v 
   0
1   r 

Fig.3. 2-D of the car motion

III. CONVENTIONAL VARIABLE STRUCTURE CONTROL

where vl is the linear velocity and vr is the angle velocity.
Let( xr , yr ,  r )be the reference coordinate, ( xe , ye ,  e )  ( xr  x, yr  y,  r   ) be the coordinate errors and y cos   x sin   0 be the nonholonomic constraint. By mathematical processing, wehave the error equationse1  vl  e2 vr  vl R cos e3e2  e1vr  vl R sin e3e3  vr  vr

Consider the double integrator given byy (t )  u (t )(4)
Let the feedback control law beu (t )  ky (t )(5)where k is a strictly positive scalar. We have  kyyyy
(6)Integrating this expression gives the following relationship between velocity and positiony 2  ky 2  c(7)where c is a strictly positive and constant value.
Since y and y remain bounded for all time, theclosed-loop systems are stable. For asymptotic stability,
the control law of the form given in (5) is not appropriate. Consider instead the control law
k y (t ), if yy  0
u (t )   1(8)
k2 y (t ), otherwise
where 0  k1  1  k2 . Then the phase portrait must spiral in towards the origin and an asymptotically stablemotion result. Next, consider a second-order systemx1  x2(9)x2  2 x2  x1  u(2)Rwhere vl R is the reference linearly velocity and vr R is
the reference angle velocity. If e1 and e3 converge tozero, the e2 will also converge to zero. So we concern
the stability of the error equations.
e1  vl R cos e3  e2 vr  vl
(3)e3  vr R  vr
The object is to design vl and vr such that e1  e3  0 .

with

4 x1 , if s ( x1 , x2 )  0u
4 x1 , if s ( x1 , x2 )  0
where the switching function is defined bys ( x1 , x2 )  x1

  0.5 x1  x2

(11)

The system structure varies along the switchinglines: x1  0 and   0 . Figure 4 shows the phase portrait of the subsystem under u  4 x1 . Similarly, Figure

Fig.2. 2-D of a wheeled mobile©ISAROB 2011

(10)

282

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

5 shows the phase portrait of the subsystem underu  4 x1 . Evidently, both subsystems are unstable. However, the origin can be made asymptotically stable by
the switching law, as shown in Fig. 6. Note that thephase portrait under variable structure control consistsof a reaching mode during which trajectories starting offs  0 move toward it and reach it in finite time, followed by a sliding mode (   0.5 x1  x1  0 ) during
which the motion will be confined to s  0 . During thesliding mode, trajectory dynamics are of a lower orderthan the original mode. The sliding mode is a trajectorythat is not inherent in either one of the two subsystems.

IV. MODIFIED VARIABLE STRUCTURE
CONTROL
Consider a simple n th -order SISO nonlinear system( n 1)(n)( n1)y  f (t , y,..., y )  g (t , y,..., y )u
(12)
Assume that f  fˆ  f , where fˆ is the known
part and f is the uncertain part, which includes theinternal noise bounded in in f  F . The object is tofind a sliding control u such that the output y of (12)will approximately track a desired signal, yd which isassumed to be n th -order continuously differentiable
and all of its derivatives are uniformly bounded. Given
the tracking error
e(t )  y (t )  yd (t )(13)
For any q  0 , define the following transformation

VSC Unstable Focus
15

10

n 1

X2

5

d

s (t )    q  e(t )
dt



(14)

s  fˆ  wˆ  u

(15)

Then

0

-5

where
-10

wˆ 
-15
-10

-8

-6

-4

-2

0

2

4

6

8

VSCUnstable Focus
1.5

1

X2

0.5

0

-0.5

-1

-0.3

-0.2

-0.1

0
X1

0.1

0.2

0.3

0.4

n k

 yd ( n )(16)

An n th -order tracking problem can be transformed
into an equivalent 1st-order stabilization problem. It iseasy to show that the control law
1u  uˆ  u f 
(17)
g
with
uˆ   fˆ  wˆ

(18)

 s
u f  sat  will result in a closed-loop system satisfying the reaching condition:ss   s ,  s   ,(19)

Fig.4. Phase portrait of an unstable focus-0.4

 n  1 ( k )  k  1e 
k 1

x1

-1.5
-0.5

n 1

Provided   F   , for some   0 .
This controller ensures that starting from any initialstate the error trajectory will reach the boundary
layer, s   in finite time.

0.5

Fig.5. Phase portrait of an unstable saddle
VSC Stable System
25

V. SIMULATION RESULTS

20

To illustrate the performance of the above, the WMR
is considered, given as the section II. According to thenotations given above, we assume initial values are chosen as ( x0 , y0 , 0 )  (3,1,  ) and expected values
are ( xr , y r , r )  (0,0,  / 4) . By setting the parametersas  y  1,  y  7,  a  15, and a  25.

15
10

X2

5
0
-5
-10

Figure 7 and figure 8 indicate the tracking errors vi
a CVSC and MVSC, respectively.

-15
-20

-15

-10

-5

0
X1

5

10

15

20

VI. CONCLUSION

Fig.6. Phase portrait of the VSC stable system©ISAROB 2011283

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011of PM synchronous motor using boundary layer integral sliding control with sliding load torque observer, IEEE PESC'96 Record, pp.1242-1247.
[5] J.H. Lee, J.S. Ko, S.K. Chun, J.J. Lee, and M.J.
Youn (1992), Design of continuous sliding mode
controller for BLDDM with prescribed tracking
performance, Conference Record, IEEE PESC'92,
pp.770-775.
[6] J.J. Slotine, E. and W. Li (1991), Applied Nonlinear
Control, Prentice-Hall: Englewood Cliffs, NJ.
[7] K.D. Do, Z.P. Jiang, and J. Pan (2004), Simultaneous

We have presented a modified variable structure control scheme in this paper for a WMR. Not only the conventional variable structure control has exhibited good
responses but also the proposed control law has shown
to be capable of improving the transient response as
well as the steady state response. Simulation results
showed good responses to any initial conditions.
1

e-y
e-x
e-theta

0.5

The tracking errors

0

tracking and stabilization of mobile robots: an adaptive
approach, IEEE Trans. On Automatic Control, Vol. 49,
No. 7, pp.1147-1152.

-0.5
-1

[8] S.V. Emelyanov (1985), Binary control systems,
International Research Institute for Management
Sciences, Vol. 1.
[9] T.L. Chern and Y.C. Wu (1993), Design of brushless
DC position servo systems using integral variable
structure approach, IEE Proceedings - Electr. Power
Appl. Vol. 140, pp. 27-34.
[10]U.I. Utkin (1992), Sliding Modes in Control andOptimization, Springer-Verlag.
[11]V.I. Utkin (1997), Survey paper-variable structure
systems with sliding modes, IEEE Trans. Automat.
Contr. Vol. 22, No. 2, pp. 212-222.
[12]W. Gao and J.C. Hung (1993), Variable structure
control of nonlinear systems: a new approach, IEEE
Trans. on Indust. Elect. Vol. 40, pp. 45-55.
[13]Z. Hu, Z. Li, R. Bicker, and C. Marshall (2004),
Robust output tracking control of nonholonomic
mobile robots via higher order sliding mode,
Nonlinear Studies, Vol. 11, No. 1, pp.23-35.
[14]Z.P. Jiang and H. Nijmeijev (1999), A recursive
technique for tracking control of nonholonomic system in chained form, IEEE Trans. On AutomaticControl, Vol. 44, No. 2, pp. 265-279.

-1.5
-2
-2.5
-3
-3.5

0

5

10

15
Time (sec)

20

25

30

Fig.7. Tracking errors via CVSC
1

e-y
e-x
e-theta

0.5

The tracking errors

0
-0.5
-1
-1.5
-2
-2.5
-3
-3.5

0

5

10

15
Time (sec)

20

25

30

Fig.8. Tracking errors via MVSC

REFERENCES[1] A.M. Bloach and S. Drakunov (1996), Stabilizationand tracking in the nonholonomic integrator via
sliding modes, System & Control Letters, Vol. 29,
No. 2, pp. 91-99.
[2] D.H. Kim and T.H. Oh (1999), Tracking control ofa two-wheeled mobile robot using input-output linearization, Control Engineer Practice, Vol. 7, pp.
369-373.
[3] E.A. Barbashin and E.I. Geraschenko (1965), On
speeding up sliding modes in automatic control systems (in Russian), Differentzialniye Uravneniya,Vol. 1, pp. 25-32.
[4] I.C. Baik, K.H. Kim, H.S. Kim, G.W. Moon, and
M.J. Youn (1996), Robust nonlinear speed control©ISAROB 2011284

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Parameter Identification of Lorenz System Using RBF Neural Networks
with Time-Varying Learning Algorithm
Chia-Nan Ko1, Yu-Yi Fu2, Cheng-Ming Lee3, and Guan-Yu Liu 4
1,2

Department of Automation Engineering,
Department of Computer and Communication Engineering,
4
Department of Electronic Engineering,
1,2,3,4
Nan-Kai University of Technology, Tasotun, Nantou 542, Taiwan(E-mail: 1t105@nkut.edu.tw; 2 t098@nkut.edu.tw; 3 t104@nkut.edu.tw; 4 guanyu@nkut.edu.tw )3

Abstract: A hybrid evolutionary algorithm is proposed to identify parameters for Lorenz chaotic system. In the proposed
algorithm, time-varying learning algorithm based on annealing robust concept (ARTVLA) is adopted to optimize aradial basis function neural network (RBFNN) for parameter identification of Lorenz system. In the ARTVLA, thedetermination of the learning rate would be an important work for the trade-off between stability and speed ofconvergence. A computationally efficient optimization method, particle swarm optimization (PSO) method, is adopted
to simultaneously find a set of promising learning rates and optimal parameters of RBFNNs. The proposed RBFNN(ARTVLA-RBFNN) has good performance for identifying parameters of Lorenz system. Simulation results areillustrated the effectiveness and feasibility of the proposed ARTVLA-RBFNN.
Keywords: Parameter identification, time-varying learning algorithm, particle swarm optimization, Lorenz chaotic
system.
RBFNNs for parameter identification of the chaotic
system.

I. INTRODUCTION
A chaotic system is a nonlinear deterministic systemthat has some special features of sensitive dependence on
initial conditions and unstable bounded trajectories in thephase space. Due to their characteristics sensitivity toinitial conditions, chaotic systems are not easy to identify.
Recently, some researchers have endeavored to improve
the identification of chaotic systems[1-3].
Recently, RBFNNs have received considerable applications in various fields, such as function approximation,
prediction, recognition, etc[4,5]. Since RBFNNs have only
one hidden layer and have fast convergence speed, they
are widely used for nonlinear system identifi- cation
recently. Besides, the RBFNNs are often referred to asmodel-free estimators since they can be used toapproximate the desired outputs without requiring amathematical description of how the outputs functionallydepend on the inputs[6,7] .
When utilizing RBFNNs, a learning rate serves as animportant role in the procedure of training RBFNs.
Generally, the learning rate is selected as a time-invariant
constant by trial and error. However, there still exist
several problems of unstable or slow convergence. Some
researchers have engaged in exploring the learning rate toimprove the stability and the speed of convergence[8,9]. In
this article, time-varying learning algorithm (TVLA) isthen applied to train the RBFNN (TVLA-RBFNN), in
which PSO method[10] is adopted to find optimal learning
rates during learning procedure. A typical system,
Lorenz chaotic system, will be given to illustrate thefeasibility and efficiency of the proposed TVLA-©ISAROB 2011

II. PROBLEM FORMULATION
Considering the following n-dimensional chaotic
system:X& = F ( X , X 0 , Q0 )(1)where X ∈ R denotes the state vector, X0 denotes theinitial state, and Q0 is a set of original parameters.
When estimating the parameters, suppose thestructure of the system is known in advance, and thus theestimated system can be described as follows:&Xˆ = F ( X , X , Q )
(2)
n

0

&
n
where Xˆ ∈ R denotes the state vector, and Q is a setof estimated parameters. Therefore, the problem of parameter estimation can be formulated as the following
optimization problem:
1 M
2
X K − Xˆ K
(3)
Min J =∑
M K =1
where M denotes the length of data used for parameter
identification, X K and Xˆ K , denote state vectors of

the original and the identified systems at time K,
respectively.
Obviously, the parameter identification for chaotic
systems is a multi-dimensional continuous optimization
problem, where the decision vector is Q and the optimization goal is to minimize J.

285

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

III. ARTVLA-BASED RBFNNs USING PSO
x1

1. Architecture of RBFNNs
In general, the input-output relation of a nonlinear
system can be expressed asy (t + 1) = f ( y (t ), y (t − 1), L , y (t − n y ),(4)x(t ), x(t − 1), L , x (t − nu ))pyˆ j (t + 1) =∑ G w = ∑ w exp(−ii =1

where xˆ ( t ) ∈ R

m

ij

ij

i =1

xˆ − m i

Σ

ŷ2

M
M

M
M

wL 2

M
M

M
M

wLp

Σ

yˆ p

GL

ρ [e j ; β ] =
(k )

β

⎡

(e j ) ⎤

⎣

β

(k )

ln ⎢1 +

2

2

⎥ for j = 1, 2, L , p⎦(8)

Based on the gradient-descent kind of learningalgorithms, the synaptic weights wij, the centers mi, andthe widths σi of Gaussian functions are updated as
Δwij = −η w
Δm i = −η c

2

)
2
2σ i
for j = 1, 2, L , p ,(5)Δσ i = −ησis the input vector, yˆ j (t + 1) is the

∂J j
j

∂wij
∂J j

∂m i

∂J j
∂σ i

=−

=−

=−

ηw

ηc
N

ησ
N

ϕ j (e j ; β ) =(k )∑ ϕ (e ; β ) ∂w
(k )

j

N

(k )

j

k =1

p

N

(k)

j =1

k =1

p

N

∑ ∑ ϕ j (e j ; β )
(k )

j =1

(9)

ij

∑ ∑ ϕ j (e j ; β )∂ρ (e j ; β )
∂e j

∂e j

N

j

k =1

(k )

(k )

jth output, wij is the synaptic weight between the ith
hidden neuron and the jth output neuron, Gi is theGaussian function at the ith neuron in the hidden layer, miand σi are the center and width of Gi, respectively, and lis the number of the Gaussian functions, which is also
equal to the number of hidden layer nodes.

∂e j

(k)

∂m i

∂e j

(10)

(k)

∂σ i

(11)

(k )

=

ej

1 + (e j )
(k )

2

β (h)

(12)

where η w , η c , and ησ are the learning rates for thejsynaptic weight wij, j = 1, 2, L , p , the center mi, andthe width σi, respectively, ϕ (⋅) is usually called theinfluence function.
In (9) to (11), when the learning rates are constant, thework for selecting an appropriate learning rate is tedious;
moreover, there exists a tendency to get stuck in anear-optimal solution or to converge slowly. To overcome
the stagnation in searching a globally optimal solution,
TVLA is proposed to approach the optimal solution
closely in this paper. In the TVLA, a nonlinear
time-varying evolution concept is adopted over iterations,in which the learning rates ηw, ηc, and ησ with a high valueηmax and nonlinearly decreases to ηmin at the maximal
number of epochs, respectively. This means that themathematical expressions are given as shown as

2. PSO-Based ARTVLAIn the training procedure of the proposed RBFNNs,
the annealing concept in the cost function of robust
back-propagation learning algorithm was adopted toovercome the existing problems in robust backpropagation learning algorithm, such as slow convergence rate and getting into local minimum[11]. A costfunction for the ARTVLA is defined here as
1 N
(k )
(6)
J j ( h) = ∑ ρ ⎡⎣ e j ( h ); β ( h) ⎤⎦ for j = 1, 2, L , p
N k =1
where
(k )
(k )
(k )
e j ( h) = y j − yˆ j ( x )
(7)

pw

h
⎛
⎞
η w = η min + ⎜ 1 −⎟ (η max − η min )⎝ epochmax ⎠(k )h is the epoch number, e j ( h) is the error between thejth desired output and the jth output of the RBFNN at
epoch h for the kth input-output training data in anonlinear system, β(h) is a deterministic annealingschedule acting like the cut-off point, and ρ (⋅) is a logistic loss function defined as©ISAROB 2011

ŷ1

G22

p

l

Σ

Fig. 1. The structure of an RBFNN.

and output, respectively, and f (t ) ∈ R denotes thenonlinear relation to be estimated. One can use a neuralnetwork to estimate the input-output relation of anonlinear system.
A radial basis function neural network (RBFNN)
consists of three layers, the input layer, the hidden layer,
and the output layer. The transformation from the input
layer to the hidden layer is nonlinear. The output layer islinear and gives a summation at the output neurons. The
structure of an RBFNN is shown in Fig. 1. When theGaussian function is chosen as the radial basis function,an RBFN can be expressed in the form
l

w12

x2

xm

where x(t ) ∈ R is the input vector, y (t ) ∈ R is theoutput vector, nu and ny are the maximal lags in the input
m

w11

G1

(13)

pc

h
⎛
⎞
η c = η min + ⎜ 1 −⎟ (η max − η min )⎝ epochmax ⎠

(14)

ps

h
⎛
⎞
ησ = η min + ⎜ 1 −⎟ (η max − η min )⎝ epochmax ⎠

286

(15)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

where epochmax is the maximal number of epochs and h is
the current number of epochs. In the updated procedure,appropriate functions for the learning rate ηw, ηc, and ησ
can promote the performance of RBFNNs. However,simul-taneously determining the optimal combination ofpw, pc, and ps is a time-consuming work.
In this paper, the PSO method is adopted to find the
optimal combination (pw, pc, ps) of learning rates in (13)to (15) and optimal parameters of RBFNNs for parameter
identification of Lorenz system. In the system identification, the goal is to minimize the error between thedesired outputs and the trained outputs. Therefore, theroot mean squares error (RMSE) should be used to define
a fitness function. This means that the fitness function
will be defined as
RMSE =

where y

(k )

1

∑ (y
N
N

(k )

k =1

− yˆ

(k )

)

2

y(k +1)yˆ (k +1)

Fig.2. The proposed PSO-based ARTVLA-RBFNN
scheme for parameter identification of Lorenz system.
where

(16)

are

the

original

Problem 1:
In the ARLA-RBFNNs, various learning rates,
0.5 ≤ η ≤ 3.0 , are used to train the RBFNNs. After 200
training epochs, the RMSE values for various learning
rates are obtained, respectively. The details of thesimulation results are shown in Table 1.
Problem 2:With the nonlinear learning rates, the ARTVLA is
adopted to train the RBFNNs, in which the optimallearning rates are determined by linear time-varying
evolution PSO[13] method. The optimal sets in (13) to (15)areobtainedasfollows:( pw, pc, ps ) = (1.3379, 0.0547, 1.0273),( pw, pc, ps ) = (4.4722, 0.0309, 3.2897),
and( pw, pc, ps ) = (1.4503, 0.3081, 4.1117) for x (t), y(t),and z(t), respectively. Meanwhile, the final values of
RMSE with ARTVLA-RBFNNs are found to be 0.02313,
0.02733, and 0.05276 shown in Table 1. Figure 3 shows
the values of RMSE for x(t), y(t), and z(t) using the
proposed algorithm with optimal learning rates. To show
the feasibility of the ARTVLA-RBFNNs, the errors of
training data after 200 epochs are illustrated in Fig. 4.

output for N sampling data.

IV. SIMULATION RESULTS
The identification scheme of a chaotic system is
depicted in Fig. 2, training input-output data are obtainedby feeding a signal x(k) to the system and measure its
corresponding output y(k+1) Then subject to the same
input signal, the objective of identification is to construct
an ARTVLA-RBFNN using PSO method, which
produces an output yˆ ( k + 1) to approximate y(k+1) as
closely as possible.
In this section, Lorenz system is used to verify thefeasibility of the proposed ARTVLA-RBFNNs. When
applying the proposed algorithm, the population size, themaximal iteration number, and the maximal epoch
number are chosen to be 30, 200, and 200, respectively.
The variables pw, pc, and ps in learning rate functions(13) to (15) are all chosen as real numbers in the range[0.1, 5] . Meanwhile, the values of ηmax and ηmin are set as
3.0 and 0.5, respectively.
Two problems are investigated for Lorenz system.
First, the impact on efficiency of annealing robust
learning algorithms (ARLA) for various learning rates isstudied, in which the best learning rate will be determinedby trial and error. Secondly, the comparison between the
proposed ARTVLA-RBFNN with nonlinear time-varying
learning rates and the ARLA-based RBFNN(ARLA-RBFNN) with a fixed learning rate is illustrated.
The RMSE (16) of the training data is adopted to evaluate
the performance of the RBFNNs.
Example: A typical chaotic system, Lorenz system, isconsidered as an example described as follows[2,12]:⎧ x& = q1 ( y − x )©ISAROB 2011

q1 = 1, q2 = 5.46, q3 = 20

parameters. The initial values of the system are
x (0) = −0.3, y(0) = 0.3, z(0) = 0.2.
With the 2000 training data, two annealing robustalgorithms are then applied to train the RBFN,
respectively.

(k )is the desired output, yˆ
is the trained⎪⎨ y& = q2 x − y − xz⎪⎩ z& = xy − q z3e(k +1)x(k )V. CONCLUSIONSThis paper presented PSO-based ARTVLA to train
RBFNNs for parameter identification of Lorenz chaotic
system. In the proposed ARTVLA-RBFNNs, timevarying learning rates and optimal parameters of
RBFNNs are simultaneously determined by PSO method.
Then the optimal RBFNNs are adopted to identify the
chaotic system. From the simulation results, theeffectiveness and the feasibility of the proposed
ARTVLA-RBFNNs identifying parameter of Lorenz
system has been verified. Meanwhile, the superiority ofthe proposed ARTVLA-RBFNNs with nonlinear
learning rates over the ARLA-RBFNNs with fixedlearning rates for parameter identification has been
illustrated.

(17)

287

1

error (x )RMSE (x )The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

0.5
0

0

50

100

150

200

0.5
0
-0.5

0

500

1
0.5
0

0

50

100

150

200

100

1500

2000

1500

2000

0
-0.5

error (z)RMSE (z)

1

50

2000

0

500

1000
k

0.5
0

1500

0.5

epoch

0

1000
k

error (y)RMSE (y)

epoch

150

200

1
0
-1

0

500

1000

epoch

k

Fig. 3. The values of RMSE after 200 training epochs
using ARTVLA-RBFNN with the optimal learning rates.

Fig. 4. The plots of error = y ( k ) − yˆ ( k ) for 2000
training data after 200 training epoch.

Table 1. The values of RMSE (16) for Lorenz chaotic system after 200 training epochs, in which ARLA with various
learning rates and ARTVLA with time-varying learning rates are applied to train the RBFNNs.
ARLA (learning rate η )ARTVLA
4.0
3.5
3.0
2.5
2.0
1.5
1.0
0.5
0.2
x(t)
0.06570.03920.0386
0.04190.03890.04920.0727
0.09250.02310.0356y(t)0.09290.09270.05840.07500.04310.08240.10770.15600.02730.0393z(t)0.10240.10080.09450.09440.08800.06750.07030.0976
0.05280.0623function networks with hybrid learning algorithm.
Applied Mathematics and Computation 213:
184-196[8] Yu W (2004), Nonlinear system identification using
discrete-time recurrent neural networks with stablelearning algorithms. Information Science 158:131147[9] Yoo SJ, Park JB, Choi YH (2007), Indirect adaptive
control of nonlinear dynamic systems using selfrecurrent wavelet neural networks via adaptive
learning rates. Information Science 177:3074-3098[10] Kennedy J, Eberhart R (1995), Particle swarm
optimization. In Proceedings of the IEEE Int Conf
on Neural Networks, pp. 1942-1948[11] Chuang CC, Su SF, Hsiao CC (2000), Theannealing robust backpropagation (BP) learningalgorithm. IEEE Transactions on Neural Networks
11(5):1067-1077[12] Fu YY, Wu CJ, Jeng JT, Ko CN (2010), ARFNNs
with SVR for prediction of chaotic time series withoutliers. Expert Systems with Applications 37:
4441-4451[13] Ratnaweera A, Halgamuge SK, Watson C (2004),
Self-organizing hierarchical particle swarm
optimizer
withtime-varying
acceleration
coefficients. IEEE Transactions on Evolutionary
Computation 8(3):240-255

ACKNOWLEDGMENT
This work was supported in part by the National Science
Council, Taiwan, R.O.C., under grant NSC
98-2221-E-252-004.

REFERENCES[1] Gu H, Wang HW (2007), Fuzzy prediction of
chaotic time series based on singular value
decomposition.
AppliedMathematics
andComputation 185:1171-1185[2] Pan ST, Lai CC (2008), Identification of chaotic
systems by neural network with hybrid learningalgorithm. Chaos, Solitons and Fractals 37(1):233-244[3] Modares H, Alfi A, Fateh MM (2010), Parameter
identification of chaotic dynamic systems through an
improved particle swarm optimization. Expert
Systems with Applications 37:3714-3720[4] Chuang CC, Jeng JT, Lin PT (2004), Annealingrobust radial basis function networks for function
approximation with outliers. Neurocomputing 56:
123-139[5] Sing JK, Basu DK, Nasipuri M, Kundu M (2007),Face recognition using point symmetry distancebased RBF network. Applied Soft Computing 7:
58-70[6] Kosko B (1992), Neural networks and fuzzy
systems: a dynamical systems approach to machine
intelligence. Prentice-Hall, NJ[7] Fu YY, Wu CJ, Jeng JT, Ko CN (2009),
Identification of MIMO systems using radial basis©ISAROB 2011

288

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Motion Planning of a Landmine Detection Robot
Kuo-Lan Su1, Hsu-Shan Su2, Sheng-Wen Shiao2, Jr-Hung Guo2
1. Department of Electrical Engineering, National Yunlin University of Science & Technology, Taiwan
2. graduate school Engineering Science and technology, National Yunlin University of Science & Technology
E-mail: sukl@yuntech.edu.tw, g9610808@yuntech.edu.tw, g9710801@yuntech.edu.tw
Abstract: The article develops a landmine detection system that contains a landmine-detection mobile robot and afollowing mobile robot. In this system, the landmine-detection mobile robot goes ahead, and uses landmine detector andGPS module to find out landmines, and record the coordinate location, and transmits the landmine’s coordinate to thefollowing mobile robot via wireless RF interface. The following mobile robot can record the location and orientation of
the landmine detection robot and the landmines on the region. The following robot moves closed to the landmine, andprograms the motion path to avoid obstacles automatically. The driver system of the landmine detection mobile robot
uses a microprocessor dsPIC 30F4011 as the core, and controls two DC servomotors to program the motion path. The
user interface of the landmine detection mobile robot and the following mobile robot uses Borland C++ Builder
language to receive the location data. In the experimental results, the landmine mobile robot records the location of thelandmines using GPS module, and transmits the location to the following robot via wireless RF interface. The following
mobile robot avoids landmines quickly, and enhances the safety to carry peoples or materials cross over the landmine
area.
Keywords: mobile robot system, landmine-detection mobile robot, following mobile robot, GPS module, wireless RFinterface, dsPIC 30F4011, Borland C++ Builder

I.

The paper is organized as follows: section II describes thesystem structure of the landmine detection system. The system
contains a landmine detection robot and a following robot, andexplains functions of the two mobile robots. The section III
presents how to program the motion trajectories of the two mobile
robots avoiding landmines in the landmine region. Section V
presents the experimental results of the landmine detection andavoidance using the proposed method. The brief concludingcomments are described in Section VI.

INTRODUCTION

Landmines are one of the great scourges on the human life.
Thousands of people are injured each year by accidental landmine
explosions on the known or unknown environment. There areapproximately 100 million landmines buried throughout the world
in approximately 70 different countries. Current methods canneutralize only about 100000 per year. The procedure forlandmine removal varies greatly depending on such factors as:
location, terrain, landmine distribution, soil density, age ofminefield, covering vegetation and locally available resources [1].
How to develop a safety processing method to detect landmines isa challenge problem.
The currently floating to detect and dispose landmines andexplosive ordnance is to commit an unmanned vehicle that equips
a detection and disposal system to the suspected terrain. In
landmine detection, a huge amount of research has been directed
at sensor technology such as: odor sensors, acoustic sensors,
electromagnetic induction sensors, electrical impedance sensors,
infra-red and microwave, radiometry and radar [2,3].
There are many studies about landmine detection in theliterature. Sweeping, detecting and clearing landmines is possible
thanks to electromagnetic induction spectroscopy [4]. There aresome studies related to a landmine detector assembled on a remotecontrolled mobile platform or vehicle [5]. There are also landmine
detectors with inductive and capacitive [6] RF microcontrollers, aswell as GPS-based [7] and GPR-designed ones [8]. A six-legged
walking robot was equipped with a metal detector andcommunicated with a supervisory computer to mark suspected
landmine locations using a differential GPS system [9].
Furthermore, rolling machines are also faster and more stable thanlegged device on landmine detection. [10,11].

©ISAROB 2011

II. SYSTEM ARCHITECTURE
The architecture of the landmine detection system is shown in
Fig. 1. There are two mobile robots in the system. One is thelandmine detection robot, and the other is the following robot.
The controller of the landmine detection robot is personal
computer, and drives two DC servomotors through dsPIC
30F4011 chip. The mobile robot can find out locations of themines in the dangerous region using landmine detector and GPS
module, and transmits the locations of the landmines to thefollowing mobile robot using wireless RF module. The following
robot programs the trajectory to avoid the landmines, andenhance the safety moving in the landmine region. The driver
system of the following robot is similar to the landmine detection
robot.
The platform of the landmine detection robot is shown in Fig 2.
The mobile robot has the shape of rectangle, and its length, widthand weight is 80cm, 50cm and 10kg. The rubber belt was used asa track on the mobile robot, and rolled tightly on the groundusing fixed wheel. The mobile robot drives the rubber belt using
DC servomotor. The landmine detector is arranged on the frontof the mobile robot, and is driven by stepping motor with reduce289

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

gear. The rotation angle range is about 50 degree, and rotatesback and forth on a fixed joint. The GPS module is on the side ofthe landmine detector to record the location of the landmine.

between 24 and 32 Medium Earth Orbit (MEO) Satellites thattransmit precise microwave signals to determine current location,
time and velocity. These signals travel at the speed of light and thereceiver uses the arrival time to measure the distance and position.
We use the GPS that is produced by GARMIN Company todetermine the coordinate of the robot. GPS takes longitude andlatitude values from satellite [13]. We calculate the angle σ anddistance d using the great-circle distance formula.

σ = arccos(sin φs sin φ f + cosφs cosφ f cos Δλ )(1)

d = rσ(2)

φs is the start value on latitude. φ f is the final value on latitude.
Δλ is the difference on longitude. d is distance from the startlocation to the target location. r is radius of the earth.

Fig. 1 The system architecture of the landmine detection system

The landmine detection mobile robot to detect mines using
the mine detector moving on the free-space, and transmits
locations of the landmines and the landmine detection robot to thefollowing mobile robot. The following robot tracks the landmine
detection robot according the coordinate of the landmine robot.
The experimental area of the system is 20mX10m. The mobile
robot moves forward from the start location to the target location.
The mobile robot detects landmines to stop, and transmits thecoordinate of the landmine to the following robot via wireless RFinterface. The simulation result is shown in Fig. 4. The mobile
robot turns right to avoid the landmine, and crosses over thedangerous area to be shown in Fig. 4. Finally, the landmine
detection robot moves to the target location, and transmits
coordinate of the fixed position from the motion path to thefollowing robot.

Fig. 2 The platform of the landmine detection robot
The platform of the following robot is shown in Fig 3. The
mobile robot has the shape of rectangle, too. Its length, width andweight is 90cm, 71cm and 12kg respectively. The mobile robot isdriven by four DC servomotors. Each rubber belt is driven by two
DC servomotors to increase loading capability. The controller ofthe mobile robot is arranged on the front of the mobile robot. The
GPS module is on the front of the mobile robot to record thecoordinate of the following mobile robot.

Fig 4. The motion path of the landmine detection robot
The following mobile robot moves forward according to the
trajectory that is built using the fixed coordinate from the landmine
detection robot, and receives the landmine location from thelandmine detection robot via wireless RF interface. The mobile
robot moves ahead the landmine to be 3m, and turns right moving
forward 3m to cross over the landmine area. The simulation result

Fig. 3 The platform of the following robot

III. MOTION PLANNING
The GPS is the only fully functional Global Navigation
Satellite System (GNSS) [12]. The GPS uses a constellation of©ISAROB 2011290

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011is shown in Fig. 5. Then the mobile robot turns left moving
forward about 6m. The mobile robot turns left moving 3m to
avoid the landmine. Finally, the mobile robot turns right andmoves to the target location. The simulation result is shown in Fig.
5.
(d)
(e)
Fig. 6 The experimental result of the landmine detection robot
The mobile robot detects the mine on the motion path, anddisplays the coordinate of the landmine and the displacement of
the landmine detection robot on the user interface. The
experimental results are shown in Fig. 6 (b) and (c). Then themobile robot transmits the coordinate of the landmine to thefollowing robot, and moves to the target location step by step.
Finally, the mobile robot stops at the target location. The
experimental result is shown in Fig. 6 (d). The measured
displacement is calculated from the encoder of two DC
servomotors is 20.07m to be shown in Fig. 6 (e). The error
displacement is only 7cm on the test.
Fig 5. The motion path of the following robot

IV. EXPERIMENTAL RESULTS
We implement the experimental scenarios on the four floor ofthe department of electrical engineering in my school. In thelandmine detection, we use a metal bar as the landmine. The
landmine detection mobile robot moves forward from the startlocation. The scenario is shown in Fig. 6 (a). The mobile robot
uses the mine detector to detect the coordinate of landminesaround the region, and transmits the coordinate of the landmine tothe following mobile robot.
(a)

(a)

(b)

(b)

(c)
Fig. 7. The experimental result of the following robot
The following robot receives the coordinate of the motion pathfrom the landmine detection robot, and moves forward to target
location. The scenario is shown in Fig. 7 (a). The user interface ofthe following mobile robot can displays the motion displacement
and the distance far from the target location. The following robot
moves forward closed to the location of the landmine that is(c)©ISAROB 2011291

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011[9] P. G. Santos, M. Armada, J. Estremera and M. Jimenez(1999), Walking machines for humanitarian demining, European
Journal of Mechanical and Environmental Engineering, Vol. 44,
No. 2, pp.91-95.
[10] P. Debenest, E. F. Fukushima and S. Hirose (2002),
Gryphon-I-buggy robot for field applications on uneven terrain, in
SICE Annual Conference, Osaka, Japan, pp.397-402.
[11] D. T. Akeabay, N. Perkins and Z. D. Ma (2004), Predicting
the mobility of tracked robotic vehicles, American Society of
Mechanical Engineers, Dynamic System and Control Division(Publication) DSC, Vol. 73, No. 1, PART A, pp.233-242.
[12] Global Positioning System (GPS) Available at:
http/www.wlius.com/SitePage.aspx?PageID=444.
[13] M. T. Qadri and M. Tayyab (2009), Real time defense
communicator using video processing, Bluetooth and GPS,International Conference on Signal Processing System,
pp.629-632.

recorded on the memory, and avoids the landmine according to the
proposed motion path. The motion processing of the mobile robot
is shown in Fig. 7 (b), and displays the orientation to implement
the programmed trajectory. Finally, the mobile robot moves to thetarget position to be shown in Fig. 7 (c). The error displacement issmaller than 1 meter from the start location to the target location.

V. CONCLUSION
We have presented a landmine detection system that contains
a landmine-detection mobile robot and a following mobile robot.
In this system, the landmine-detection mobile robot goes ahead,and uses landmine detector and GPS module to find out landmines,
and record the coordinate location, and transmits the landmine’s
coordinate to the following mobile robot via wireless RF interface.
The following mobile robot can record the location and orientationof the landmine detection robot and the coordinate of thelandmines on the region. The following robot moves closed to thelandmine, and program the motion path to avoid obstacles
automatically. In the future, we want to develop the curve path to
avoid landmines on the landmine detection robot and following
robot, and program the uniform user interface of the landmine
detection system.

ACKNOWLEDGMENT
This work was supported by the project “Development of amodule based education robot” under Ministry of Education of
Taiwan. (99-A010-5).
.

REFERENCE[1] M. Freese, E. F. Fukushima, S. Hirose and W. Singhose,(2007), Endpoint Vibration Control of a Mobile Mine-Detecting
Robotic Manipulator, The American Control Conference, New
York, USA, pp.7-12.
[2] J. P. Trevelyan (1997), Robots and landmines, Industrial
Robot, Vol. 24, No. 2, pp.114-125.
[3] M. Ghaffari, D. Manthena, A. Ghaffari and E. L. Hall, (2004),
Mines and human casualties, a robotic approach toward mineclearing, SPIE Intelligent Robots and Computer Vision XXI:
Algorithms, Techniques and Active Vision, Vol. 5608.
[4] I. J. Won, D. A. Keiswetter and T. H. Bell (2001),
Electromagnetic induction spectroscopy for clearing landmines,
IEEE Trans. Geoscience and Remote Sensing, Vol. 39, No. 4,
pp.703-709.
[5] Y. Baudoin, E. Colon (1998), Humanitarian demining androbotics, Proceedings of the IEEE International Conference onControl Applications, 1-4, Septmber, Trieste, Italy.
[6] M. E. V. Steenberg, A. Washabaugh and N. Goldfine (2001),
Inductive and capacitive sensor arrays for In Situ composition
sensors, Aerospace Coference, IEEE Proceedings, Vol. 1, pp.
1/299-309.
[7] M. Hussain (2005), RF controlled GPS based hovering mine
detector, 9th International Multitopic Conference, IEEE INMIC.
[8] D. J. Daniels and P. Curtis (2003), Minetect, 2nd International
Workshop on Advanced GPR, Delft, The Netherlands, 14-16,
May.

©ISAROB 2011292

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Optimal posture control of two wheeled inverted pendulum robot on aslanted surface
Youngkuk Kwon, Joonbae Son, Jaeoh Lee, Jongho Han and Jangmyung Lee
Dept. of Electronics Engineering, Pusan National University, Korea(Tel : +82-51-510-1696; Fax : +82-51-510-1693)(kyk1104@pusan.ac.kr, waggish82@pusan.ac.kr, jaeoh2@pusan.ac.kr, gkswhdgh114@pusan.co.kr,
jmlee@pusan.ac.kr)

Abstract: Most of the conventional researches are concentrated on the compensation algorithm of the gyroscope signal
and posture control on the flat ground. But Segway has been considered as next the generation vehicle, and as its
application spread out to the whole of society, its stability and optimal posture on the slanted surface has been discussed
worldwide. So, this paper proposes an optimal posture of two wheeled inverted pendulum robot on a slanted surface.
In order to realize posture balance on the slanted surface, tilted weight should be compensated by the proposed
algorithm. Dynamic state equation was derived from the system’s structure, and an LQR regulator was designed basedon the tilted angle obtained from the ultra-sonic sensor. Optimal posture control experiment was iterated as the slanted
angles varied. Effectiveness of the proposed algorithms has been verified and demonstrated through simulations andreal experiments.
Keywords: Two - Wheel Inverted Pendulum Robot, Optimal Posture Control, ARS (Attitude Reference System).
inverted pendulum. And table 1 shows each parameters
for the mobile inverted pendulum

I. INTRODUCTIION
Conventional researches are highly concentrated on
the posture control under ideal environment,[1] but
posture control for the slanted surface was not studied.
According to the social needs of the next generation
vehicle, this paper proposes an optimal posture controlof the two wheeled inverted pendulum not only on theflat ground but also on the slanted environment. Fromthe experimental results of the proposed algorithm, thedifference between the conventional compensation
algorithm and proposed algorithm has been presented.
Finally, optimal posture for the climbing movementfrom the ARS(Attitude Reference System) will bepresented Optimal posture was defined as a stability ofthe mobile robot during climbing on a slanted surface.
This paper consists of five sections including
introduction. In Section 2, the modeling of the proposed
robot structure and controller design scheme aredescribed, and in Section 3, climbing a slanted surface
algorithm will be stated. In Section 4, experimental
environment and its results are shown. Conclusions andfuture research plans are illustrated in Section 5.

Fig. 1. Two wheeld inverted pendulum.
Table 1. Each parameters for the mobile inverted
pendulum

II. Modeling of The robot structure and Designof Controller
Dynamics modeling from the proposed structure has
been proposed. Fig. 1. illustrates the structure of the©ISAROB 2011293

The angle of rotation of wheel from the center
A tilted angle of body from the center
The angle of rotation of body from rotation of
wheel

MB

The mass of the bodyMWThe mass of the wheel

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011lDistance from center of the body to center ofthe wheelIB

Rotation Inertia of the bodyIW

Rotation Inertia of the wheels( M B rl cos   M Bl 2  I B ) 2 (( M B  M W )r  M B rl cos   IW )   0
 M rl  sin       M gl sin  
S
B
 B


The coefficient of friction of bearing fromwheel
The coefficient of friction of the ground from
wheel

g

2.2 Controller design
From the dynamic analysis for the proposed structure, thelinearized dynamic equation could be derived as follow. Nonlinear dynamic equation (7) and (8) could be linearized as (9).

2.1 Dynamics
Dynamics for the given system has two kinds of parameters,
body angle, and wheel angle. Equation (1) shows the
dynamics according to theO   P   Q   Ru
WS   T    U   Vu  
W   ,d  T   T   U   D 




  Q
dt              (1)d  T   T   U   D 




  Q
dt              (2)

T : Kinetic energy

D : Disturbances

U : Potential energy

Q : generalized force(9)
Equation (10) shows the generalized state spaceequations.
x(t )  Ax(t )  Bu (t )y (t )  Cx(t )(10)Representing the robot structure to the state spaceequations, then (11) could be attained.
0 0
0 1
      OQP      0 
    0    W W W     R 
x    ,    u
      0 0
0 1     0 
       
T U      V       0 S
W W W T,U,D,Q for the Euler-Lagrange dynamics has been
derived as follow,T

1
1
M W ( S2  Z 2 )  M B ( S1  Z1 )
2
2
1
1
1
 IW    I B (    )  I M 2  
2
2
2

U  MW gr  M B gl cos(   )D

1(  s     g  )2

Q  u, Q  0(8)System dynamics could be expressed as equation (7),and (8).

 
 
y  1 0 1 0      

(3)
(4)

(11)

The LQR controller from the equation (11) was
designed by Simulink. Its figure is illustrated in Fig. 2.

(5)

(6)

We can attain following equations by plugging boththe kinetic energy and potential energy to the equation(1) and (2), then( M B l 2  I B  I M 2 ) 2 ( M B rl cos   I M ) 
0          M gl sin   u 
S
B
 S


Fig. 2.
(7)

LQR controller design

III. Climbing a slanted surface algorithm
During the climbing control of the robot, unexpecteddisturbance forces are essentially caused by the irregular©ISAROB 2011294

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

contact force which comes from the irregular contact angle
between the wheel and the terrain.[2] The disturbances haveeffects on the optimal posture of the mobile robot tocompensate the slanted angle. So, In order to realize posturebalance on the slanted surface, tilted weight should becompensated by the proposed algorithm.

IV. Experiment
The proposed system consists of a two-wheeled
Inverted Pendulum Robot shown in Fig. 7. The bodyof a mobile robot is designed with two-wheels andmotor drive controllers. The controller-related hardware
is composed of MCU (Micro Controller Unit, DSP28335), a gyro sensor, an accelerometer and an
ultrasonic sensor

Table 2. Robot parameters for the slanted surface

MWg

The mass of the wheellDistance from the body center of gravitysMoving distance of the mobile robot

fc

Coefficient of friction with the slop

F

External force to the mobile robot

N

External force to the actuator from the robot
body

Slope angle

acceleration of gravity

Table 2 describes the robot parameters while climbing
a slanted surface. The governing equation for the flatground for the equilibrium state was derived in equation(12).

Fig. 3. The configuration of the two-wheeled-inverted robot
systemThe experiment for driving of the two- wheeled
inverted pendulum robot, was conducted on two
different environments; when giving a disturbance tothe robot on the ground, and when driving on slanted
surfaces of 5°,10°,15°,20°,25°. In addition, Fig. 4
shown that the slope was slanted using a clinometer.

F  MW s  MW g sin   fC cos   N(12)
The governing equation for the slanted ground for theequilibrium state was derived in equation (13).

F  (MW  M B )s  M Bl  cos  M Bl 2 sin   MW g sin   fC cos  s (13)Assuming thatis a tiny enough value to ignore,
we can simplify   0, sin    , cos   1(14)Inserting (14) into (13), then we can attain equation(15) as follows.
F  MW g sin   (MW  M B )s  M Bl  fC s (15)Fig. 4. The experiment environment for climbing on theslanted surface.
Using an ultrasonic sensor located under the robot,
which receives the distance information from theground, it is possible to expect information of the slope
degree between the ultrasonic sensor and the ground.
Below, Table 3 shows the changed slope angle
information from the distance obtained by the ultrasonicsensor.[3]For the difference between a pre-linearized equationand linearized equation, MW g sin  should becompensated a under slated environment.
So, if the ground is tilted to the amount of  , theposture control of the mobile robot on the slanted
surface could be realized by compensating amount of
MW g sin  . We can establish a driving command of

MW g sin    where  stands for the moving
direction’s tilted angle.

©ISAROB 2011295

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 3. conversion of the angle from distance between theultrasonic sensor and the groundAngle (Degrees)
Distance (Centimetres)
0˚
10 cm
5˚
8.5 cm10 ˚
7.0 cm15 ˚
5.5 cm20 ˚
4.0 cm25 ˚
2.5 cm30 ˚
1.0 cmFig. 7.

Driving on a 20 and 25 degree slope

VI. Conclusion
The result when given disturbance to the robot on the
ground is shown in Fig. 5, and the environment like Fig.
4 shows driving test results at Fig. 5 and Fig. 6. , 7.
The current measured angle from the slope randomly
gave 5˚ more; it was made the two-wheeled Inverted
Pendulum Robot can be driven on slope. And When the
robot completed climbing at the slope, the two-wheeled
inverted pendulum robot can be stopped by converging
back to the current angle of the slope. Below, the resultsshows the current slope values of the acceleration
sensor, and target values. Sampling Time is thehorizontal axis and the vertical axis represents the Slope
Angle.

In this paper, when controlling the two-wheelinverted Pendulum on the slanted environment, we
acquired the slope information using the ultrasonicsensor. By varying the operating point of the inverted
pendulum and compensating gravity as per the lean ofoperating point, even the slanted environment, can becontrolled for optimal posture. And also, The equationof motion was suggested to satisfy driving speed of thetwo wheeled robot and slope angle conditions according
to the angle of the slope. Therefore, the results of thisstudy will prove useful not only for the mechanism of
the two-wheel-inverted Pendulum but also in
determining the slope of various other types of wheeled
type mobile robot mechanisms.

ACKNOWLEDGEMENT
This research was supported by the MKE(The
Ministry of Knowledge Economy), Korea, under the
Human resource development program for robotics
support program supervised by the NIPA(National IT
Industry Promotion Agency). (NIPA-2010-(C70001001-0009))

Fig. 5. Driving on an even surface anda 6 degree slope

This research was supported by Basic Science
Research Program through the National Research
Foundation of Korea(NRF) funded by the Ministry of
Education, Science and Technology(2010-0024129)

REFERENCES

Fig. 6.

©ISAROB 2011[1] H.J Lee and S. Jung, “Gyro sensor drift compensation by
Kalman filter to control a mobile inverted pendulum robot
system”, IEEE ICIT. PP 1026-1031, 2009[2] G. Sohl and A Jain, “Wheel-Terrain Contact Modeling in
the Roams Planetary Rover Simulation”, Proc. Fifth
ASME International Conference on Multi body Systems,
Nonlinear Dynamics and Control, Long Beach, CA,
September 2005.

Driving on a 10 and 15 degree slope

296

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Efficient CAN-based network
for marine engine state monitoring system
Junseok Lee, Yoseop Hwang, Jaehan Jo and Jangmyung Lee
Dept. of Electronics Engineering, Pusan National University, Korea(Tel : +82-51-510-1696; Fax : +82-51-510-1693)(sukidda@pusan.ac.kr; mmx001@pusan.ac.kr; jjhan@pusan.ac.kr; jmlee@pusan.ac.kr)

Abstract: This paper presents the marine engine state monitoring system implemented with Controller Area Network.
As the marine engine state monitoring system requires various kind of engine information, it consists of a lot of sensor
nodes. So, with the features of Controller Area Network which supports huge numbers of message ID, and its message
arbitration ability, this paper presents scheduling method for the performance of monitoring system. And its
effectiveness and validity have been shown through experiments.
Keywords: CAN(Controller Area Network), Engine State Monitoring System, DPQ(Distributed Precedence Queue)

I. INTRODUCTIION

Module consist of microprocessor and CAN transceiverand transmits data from sensors to the SPU, respectively.
And SPU(Signal Processing Unit) collects entire
network data and transmits engine condition signal to
the ECR(Engine Control Room) via RS-485.

Marine Engine State Monitoring System purposes
precise and rapid transmission of engine condition datafrom various measurement instruments to the engine
control room. A failure of engine in sailing ship affects
critically to the operating the ship, and may causes
enormous financial loss. So, real-time monitoring
system for preventing the unpredictable failure andmaintaining support of marine engine is needed.
A major advantage in using the CAN technology
with respect to the other kinds of field-buses available is
the wide choice of very cheap electronic components.
The Engine State Monitoring System requires lots ofsensor nodes and needs cheaper field-bus solution.
Additionally, CAN can simplify the sensor node wiringand prioritize the message transmissions which satisfy
an urgent message transmission needed at industrial
environment.[1],[2]But if network is overloaded, data transmission isdecreased rapidly due to increasing data transmission
collision. If this state is continuous, a network may be
groggy and non-transmission condition for a long time.
This paper presents a mechanism that can be a fairtransmission and can reduce delay time with assignment
precedence queue to delayed sensor node.

Fig. 1. Engine State Monitoring System Architecture

CylinderLiner

ExhaustGas

Water in Oil

Bearing
Wear

II. Engine State Monitoring System
State Monitoring can be started with measuring ofeach part of engine (Fig. 2). Engine State Monitoring
System can be divided into three parts. Status measuring
sensors which measures temperature, pressure,
perception of flowing and various status indication
factors are equipped with the CAN Module. And CAN©ISAROB 2011

Main
Bearing

Crank
Bearing

Fig. 2. Marine Engine

297

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

III. Controller Area Network Protocolround-robin policy among the stations that want totransmit a message on the bus, and provides two levelsof priority for the frame transmission services. Little ornothing has to be changed at the MAC level; and in thisway it is possible to reuse the same electronics
components developed for the implementation of thestandard CAN protocol.
1. DPQ basic principle
The basic idea of this CAN fairness control
mechanism that is to insert into a global queue all of thenodes that want to transmit over the shared medium. For
Node C, of which transmission is continuously delayed,
a queue is created to transmit Node C and the othernodes that transmit with C. So, several queues can bepartially made in this research, two queue were used.

CAN is based on a CSMA/CS channel accesstechnique, modified to enforce a deterministic
resolution of collisions on a network which uses apriority scheme based on the identifiers of exchanged
objects. The CAN protocol adopts a layered architecture
which is based on the OSI reference model, even though
it is not fully OSI compliant. As with other networks
conceived for the factory automation environment, it
relies on a reduced protocol stack, consisting only of the
Physical layer, Data link layer, and Application layer.
This paper is focused on the data link layer. In fact mostof the features of CAN which concern topics such as thesharing of the bandwidth among the different stations
and the access delays they experience depend on themechanism adopted at this level.[3]Fig. 3. CAN Data Frame format

IV. Distributed Precedence Queue (DPQ)The CAN implicitly assigns to each object
exchanged in the network a priority that corresponds tothe identifier of the object itself. Even though thismechanism enforces a deterministic arbitration that isable to resolve any conflict that occurs when several
nodes start transmitting at the same time, it is clearly
unfair. If many nodes are connected in the network,
nodes that are of low priority rank can continuously losea transmission opportunity. That is, if high priority
objects transmit continuously, finally a low priority
object can miss an important message which isrelatively unimportant compared to that of a highpriority object.
Accordingly, a mechanism that uses a relative
priority according to the consideration of low priority
nodes is necessary although the CAN implicitly assigns
a priority. Fair behavior, which for example enforces around-robin policy among different stations, has to beguaranteed to all the objects exchanged at a given
priority level.
In this paper, it is shown that this kind of behavior
can be obtained by slightly modifying the frame
acceptance filtering function of the LLC sub-layer. In
particular, only the significance of the identifier field in
the transmitted frame has to be modified in some way.
The resulting arbitration mechanism is able to enforce a©ISAROB 2011

Fig. 4. Generation of a precedence queue in DPQThis distributed precedence queue protocol provides
the opportunity to create precedence queues for all
nodes in a network. And, in the case that several
precedence queues exist, each precedence queue
assigned a priority so that they can be implemented
independently.
The DPQ mode ID, which is stored in the 11 bit
standard ID field, indicates the precedence queue orderof each node. Whenever a node carries out atransmission, it moves to the end of the queue, thus
lowering its precedence to the minimum. All of thenodes following the transmitting node advance by one
position in the queue, occupying the space that has justbeen created. Using this round-robin policy, collisions
among messages are avoided.
The queue is not stored in some specific location.
Instead, it is distributed among all the nodes in the
network. Each node is responsible for storing andupdating. That is, if the maximum permission delay
time is reached, it creates a precedence queue, and thenit has to dynamically change priorities to transmit

298

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

The priority bit P specifies whether the frame has tobe transmitted as a high priority frame (P=0) or as a low
priority frame (P=1). When t1 and P are used, thepriority can be assigned a maximum 4 queues.
The next 8 bits represent the precedence level of theframe. Namely, these 8 bits show the transmission
queue order. The DPQ, which was used in this research,uses t0, t1, and then distinguishes the standard CAN
mechanism, and sets each queue using P, and concludes
the precedence in the queue using 8 bits.

preferentially with other nodes. And a precedence queue
has to be dissolved when is completed an urgent task.[4]Fig. 5. Structure of a data field for DPQ
We suppose a network that is composed of Nodes
A to G as shown in Fig 4. If Node C builds up a queue,
the ID that is entered into the data frame queue cantransmit and designate to 7 by lower 7 byte. At thistime, it will be designated precedence priority to higher
byte. Then, each node filters to enter itself into thequeue, and it assigns its queue. After Node C transmits amessage, it will go to the last position in the queue. And
the other nodes will move up one position by order. And
the remaining nodes that to be transmitted aredesignated using the upper 1 byte as shown in Fig 5;
their queues will be dissolved or maintained using theupper 1 byte, as shown the Fig 5 after all transmissions
are completed.
2. DPQ Implementation Issues
The DPQ mechanism can be implemented without
any modifications to the basic format of CAN frames. It
uses an identifier field to designate the priority queue.
Because the length of the conventional identifier fielddefined in the CAN standard is too small, the CAN
extended format can be adopted.

V. Experiment and Result
In this paper, we experimented via Engine State
Monitoring System which is composed of ten CAN
Modules and one SPU to evaluate the effectiveness ofthe proposed method.
CAN Module uses sensors (Resistance Bulb,
Thermocouple, Pressure Sensor...) actually used inmarine engine. And it uses data from measuring
elements composed as shown in Fig.9. And CAN
Module use Microchip's dsPIC30F4012 and SPU uses
TI's TMS320F28335.
Fig. 9 shows the monitoring program for PC via
LabVIEW to monitor the state of network and data inreal-time. Each of vertical fill slides indicate occupied
bandwidth of node and gauge indicates shows
temperature of measuring element. We set the bitrate of
network at 500Kbps considering the length of network
installed in the low-speed marine engine.
To verify the effectiveness of DPQ mechanism, we
presented transmission result comparing normal CAN
transmission with transmission in DPQ mode. For
actual monitoring system for the Low-speed engine inlarge vessel, they use about hundreds of sensor; we set
the transmission period at 500us for more possible
transmission collisions.

Fig. 6. Format of the header of extended CAN Frame
The DPQ uses the first 11 bits of the identifier field
for its control information, whereas the remaining lower
order 18 bits (ID ext.) are used to dynamically store theeffective identifier of the an exchanged object (EID).

Fig. 7. Format of the header DPQ Frame
The first two bits (t0, t1) must be set at the logicalvalue of zero as shown in Fig 7. Then, the protocol isdivided by a standard CAN communication and DPQ
mechanism. So, DPQ always has a higher priority than a
CAN mechanism, and they can exist in this same space.

©ISAROB 2011

Fig. 8. Experiment System Architecture299

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

VI. Conclusion
In this paper, we proposed the Marine Engine Sate
Monitoring System using DPQ mechanism to improve
fairness in network.
Through these methods, messages with low-priority
were ensured to transmit message and delay its time was
improved. Features of the Marine engine monitoring
system are as follows.
1. CAN communication module is connected tosensor used in actual Marine, is distinguished according
to set ID and is designed for adding extra modules easily.
2. SPU (Signal Processing Unit) module forprocessing total data which transmit in CAN module isable to check Whole engine, bearing state and etc.
3. Whole states of Marine engine is able to bechecked through the output values (voltage, current,
resistor) of all sensors using the SPU (Signal ProcessingUnit) module.
4. Conventional communication method as RS232
and RS485 is supported for compatibility with theexisting Marine engine system. Therefore, state ofMarine is able to be checked in ECR (Engine Control
Room).

Fig. 9 Engine State Monitoring Program
5000

RX Number

4000
3000
2000
1000
0

1

2

3

4

5
6
7
Node Number

8

9

10

Fig. 10 Received Messages in normal CAN
transmission
Fig. 10 shows received message numbers for eachsensor nodes when network received 10000 messages.
Because of consecutive collision and short transmission
period, messages with low-priority are not transmitted
while messages with high-priority occupy most of
bandwidth.

ACKNOWLEDGEMENT
This research was financially supported by the
Ministry of Knowledge Economy (MKE) and Korea
Institute for Advancement of Technology (KIAT)
through the Research and Development of Regional
Industry.
This research was supported by the MKE (TheMinistry of Knowledge Economy), Korea, under the
Human resource development program for robotics
support program supervised by the NIPA (National IT
Industry Promotion Agency). (NIPA-2010-(C70001001-0009))

1000

RX Number

800
600

REFERENCES

400[1] Bosch, “CAN Specification Version 2.0.”, Robert
Bosch GmbH. Stuttgard, 1991.
[2] International Standard Organization, “Road vehicles Interchange of digital information - Controller
area network for high-speed communication”, ISO
11898, November 1993.
[3] G. Cena and A. Valenzano, “An Improved CAN
Fieldbus for Industrial Application”, IEEE Trans. On
Industrial Electronics, Vol. 44, No. 4, Aug, 1997.
[4] S. Corriganm, Intriduction to the Controller Area
Network, TI, Application Report. Texas Instruments,
2002.

200
0

1

2

3

4

5
6
7
Node Number

8

9

10

Fig. 11. Received Messages in DPQ mode
Fig. 11 shows result of transmission of DPQ modein identical condition. It shows evenly distributed
received messages.
But data frames derived from the process of making
and dissolving the precedence queue, actual statemonitoring messages are decreased about 10 percent.

©ISAROB 2011300

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Localization Algorithm using Virtual Label for a Mobile Robot in Indoor
and Outdoor Environment
Kiho Yu*, Mincheol Lee**, Junghun Heo*, Youngeun Moon** The School of Mechanical Engineering, Pusan National University,#405 RIMT, Pusan National Univ, Jangjeon, kuemjeong, Pusan, south korea 609-735(Tel : 82-51-510-3081; Fax : 82-51-512-9835)
yukiho1118@pusan.ac.kr** The School of Mechanical Engineering, Pusan National University,#405 RIMT, Pusan National Univ, Jangjeon, kuemjeong, Pusan, south korea 609-735(Tel : 82-51-510-2439; Fax : 82-51-512-9835)
mclee@pusan.ac.kr

Abstract: The scanning laser range sensors provide range data consisting of a set of point-measurements. The laser
sensor (URG-04LX) has a distance range of approximately 0.02 to 4 meter and scanning angle range 240 degrees.
Usually, such a range image is acquired from one view-point by "moving" the laser beam using rotating mirrors/prisms.
The orientation of the laser beam can be easily measured and converted into coordinates of the image. This paper
conducts localization using virtual labels with environment distance data gotten from the 2D distance laser sensors.
This method puts virtual labels on special feature and points which is on the way of mobile robot path. The current
location is calculated by combining the virtual label and the range image of laser range finder.
Keywords: Virtual Label, Laser Range Finder, Encoder, Localizationsuch as the short communication distance and weakness
performance against sunshine [4]. Active beacon systemwith ultrasonic sensors is known as relatively highaccurate performance and low power consumption.
However, this method gets influences with an obstacle
between a transmitter and a receiver, external
temperature, and noise [5].
The localization method using a landmark or a RFID
is robust to external environment such as changes intemperature or solar light. However, it has fatal defect inthat it can be applied only to limited space where RFID
tag or landmark is able to be attached. In addition, it is
impossible to use outdoors [3], [4]. In outdoors, GPS(global positioning system) is most powerful
localization method to recognize the absolute position.
However, it has a limitation area of GPS signal and low
position accuracy [6].
In order to solve these problems, this paper proposes
the localization method with virtual labels to becomerobust against space constraints and indoor/outdoor
environments. The proposed method is based on virtuallabeling techniques. This paper uses a laser range finderand an encoder in indoor environment. And, in outdoor
environment, a GPS only is adopted to calculate initial
position.

I. INTRODUCTION
Mobile robots have been applied in various fields
such as logistics automation, security, industrial
automation and consumer products, and etc. The
absolute positioning of mobile robot becomes essential
part. The research for the localization problems arefocused on robust and minimum error against unknown
environment. As advance of mobile robot technologies,
localization for mobile robots is widely researched anddeveloped such as a dead reckoning system, an active
badge system, an active beacon system and the
localization system using a landmark, a RFID and a
GPS [1-8].
The dead reckoning system is one of well-known
localization method based on measuring the relative
position. The dead reckoning system has an advantage
of high sampling rate and outstanding accuracy in shortinterval. However, this system cannot avoid inevitable
accumulation of the error because of time integral termwith current direction and distance. Especially,
cumulative error of the direction angle may cause
infinitely large positioning error as time passed [1], [2].
In order to detect the location of people insidebuilding, the active badge system with infrared ray has
been researched. However, this system has problems©ISAROB 2011301

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

The paper is divided into the following sections:generation of range finder scans (section 2), andlocalization with virtual label algorithm (section 3).
Section 4 shows experiment results of implementation
within typical indoor/outdoor scenes. And conclusionand future work are discussed in Section 5.

II. LASER RANGE FINDER SCANING
This study constructs a mobile robot equipped with alaser range finder as shown in Fig. 1. 2D laser range
finder (URG-04LX) made by HOKUYO Co. LTD isused. URG-04LX has 100 ms cycle time and 765
resolutions from 60 mm to 4,095 mm distance value
divided by 0.36 degree. Specifications of the laser range
finder are listed in Table 1.

Figure 2. Distance data of a laser range finder

Figure 1. Laser range finder & Indoor environment
TABLE I.

SPECIFICATIONS OF URG-04LX
Spec.

Measuring area

60 to 4095 mm , 240

Figure 3. Distance data conversion of 2D coordinates

Accuracy(Repeatability)

60 to 1000 mm : ± 10 mm
1000 to 4095 mm : 70 mm

Angular resolution

Step Angle : 0.36 (360 /1024 steps)

Scanning time

100 ms/scanFigure 2 shows output data of the laser range finder
in indoor environment. This sensor is available to scan
240 degree and 765 distance output data according to
distance each cycle at 100ms. In this paper, localization
is calculated in 210 degree areas. 600 distance output
data are used to build 2D coordinates including thevirtual label as shown in Fig. 3. This distance output datais transformed to 2D coordinates by using equation (1)and (2).
3.14 , i = 0,1, 2,..., 600
)
0.5π

(1)

3.14 , i = 0,1, 2,..., 600)=
yi Di (sin(i× 0.3515625))(
0.5π

(2)

=
xi Di (cos(i× 0.3515625))(

Figure 4. Segmentation area

It is necessary to classify movable area ( Mk x , y ) andnon-movable area ( Nk x , y ) in transformed coordinates( xi , yi ). In Fig. 4, Nk xy is used to map virtual label andWhere, θi (0.3515625) is the angular resolution from1 point to 600 point and Di is the distance output data.
Figure 3 shows distance conversion of 2D coordinates.

©ISAROB 2011

Mk xy is used to determine mobile robot’s path. After

determining Mk x , y and Nk x , y , the map is generated withconsidering radius of robot ( Rr ), moving distance ofrobot ( M d ), and direction of robot ( Rθ ). As repeating and

302

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

accumulating of above proposed process using equation(3), (4), virtual label is named on the map
simultaneously.
X map j = Nk x

(3)

Ymap j = Nk y

(4)

Where, Dn is the distance from mobile robot tovirtual label ( VLmx , y ).

Figure 6 is the block diagram of virtual label
allocation.

By using the proposed virtual label algorithm whichis explained in section 3, a map is effectively generated
in indoor or outdoor environment. Only for the outdoor
circumstance, GPS is used to get initial position
additionally.

III. VIRTUAL LABEL ALGORITHM
Before moving the mobile robot, the virtual label
should be allocated on the map which can be generated
by scanning the current circumstance with laser range
finder. The virtual label is not needed to be on real points,
lines and space in the circumstance. As the name‘virtual’, the label can be generated on virtual
structures which do not exist in real space. Then, thetemporary virtual wall is allocated in the left and right of
45 degree space.

methodaccomplished as following procedure,The

proposedvirtualFigure 5. Virtual labeling for indoor environment

is

label

[Step 1] In order to initialize current position of mobile
robot, scan the circumstance.
[Step 2] Label on the map using scanned data, featurepoints, lines and spaces.
[Step 3] If feature points, lines and spaces cannot befound, virtual wall and label will be generated
on the map as shown in Fig. 5. The virtual
walls are built on both sides. And the virtuallabels ( VLmx , VLmy ) are attached on 45

Figure 6. Block diagram of a virtual labelingdegree diagonal position on the virtual walls.

IV. EXPERIMENTS
In order to evaluate the localization and map[Step 4] Let a current robot position localize by using
distance ( Dn ) and angle ( Rθ ) toward thelabel.

formation of the proposed virtual label algorithm, themoving experiments of mobile robot were carried out inindoor environment showed as Fig. 1 and outdoor
environment showed as Fig. 7.

[Step 5] Movable domain ( M 1x , y , M 2 x , y ) and not
movable domain ( N1x , y , N 2 x , y ) have to be
added on the map involving the label.

[Step 6] After moving mobile robot, returned to theStep 1.
The position of mobile robot is acquired by using
distance Dn and VLmx , y . The current position is

calculated by using the following equation (5) and (6).

©ISAROB 2011=
Rx (VLmx ± Dn )(5)

Ry = (VLm y )(6)

Figure 7. Outdoor environment

Experiment (Indoor)

In indoor environment, the virtual labelingalgorithm is applied to generate the map for

303

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

localization. The map applied the virtual label
algorithm is shown as Fig. 8. The distance between
start and goal point is 10 m straight path. The test
condition is selected to compare localization effect byusing encoder data with localization effect by using
the virtual label algorithm in indoor environment. The
mobile robot speed is 0.54 m/s. Test result with thevirtual label algorithm is shown as Fig. 9.

If there is not the space to make label in the scan data ofrange finder, label is allocated to virtual wall for
localization.

After experiments over 5 times, repetitive errorof mobile robot is about ±4cm because of rough road in
outdoor. And the repetitive error by only using encoder

was about ±10cm . Thus, localization with the virtuallabel algorithm is superior to localization
method with the GPS.

After 5 times experiments over 5 times, location
error of mobile robot was found as about ±2cm . In
previous study, repetitive error was about ±3cm byusing encoder [3]. However, localization by using
encoder data is not much different. Thus, the experimentshows that virtual label algorithm is precise localization
method.

Figure 10. Map building of an outdoor environment

Figure 8. Map building of an indoor environment

Figure 11. Moving trajectory of mobile robot with the virtuallabeling algorithm in outdoor

Figure 9. Moving trajectory of mobile robot with the virtuallabeling algorithm in indoor

V. CONCLUSION
In order to localize and build a map inindoor/outdoor environments, this study proposed the
localization algorithm with the virtual label algorithm.
And this paper carried out experimental test using
mobile robot to evaluate the proposed virtual label
algorithm. In the experimental test, the proposed virtuallabel algorithm performed precise localization resultthat location error of mobile robot is about ±2cm inindoor environment and about ±4cm in outdoor
environment. In the future work, it will be improved tohave localization maximum error within ±5mm in thevarious indoor/outdoor environments. In addition, it will
be compared with other localization algorithms.

Experiment (outdoor)

In outdoor environment, with mobile robot
equipped with a laser range finder and a GPS, thevirtual labeling algorithm is applied to generate maps
for localization. GPS sensor only is adopted to
calculate an initial position. The map obtained byapplying the virtual label algorithm is shown in as
Fig. 10.
The distance between start and goal point is 17 m

straight path. The mobile robot speed is 0.54 m/s. The
result of 5 times repetitive driving is shown as Fig. 11.

©ISAROB 2011304

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

ACKNOWLEDGE
This research was supported by the MKE(The
Ministry of Knowledge Economy), Korea, under the
Human resource development program for robotics
support program supervised by the NIPA(National IT
Industry Promotion Agency). (NIPA-2010-(C70001001-0009))

REFERENCES[1] F. Dellaert(1999), D. Fox, W. Burgard, and S.
Thrun., Monte carlo localization for mobile robots,
Proceedings of the IEEE International Conference on
Robotics and Automation (ICRA)[2] P.Buschka(2002) and A. Saffiotti, A Virtual Sensor
for Room Detection, Proc. IEEE/RSJ Int. Conf. On
Intelligent Robots and Systems EPFL , pp.637-642.
[3] H. J. Lee(2006) and M. C. Lee, Localization of
Mobile Robot Based on Radio Frequency Identification
Devices, 2006 SICE-ICASE International Joint
Conference, pp.5934-5939.
[4] J. Borenstein(1996) and L Feng, Measurement andcorrections of systematic odometry errors in mobile
robots, IEEE Trans. on Robotics and Automation, vol.
12, no. 6, pp.869-880.
[5] T. Venet(2002), T. Capitaine, M. Hamzaoui and F.
Fazzino One active beacon for an indoor absolutelocalization of a mobile vehicle, IEEE International
Conference on Robotics &Automation .Washington,
May[6] J. Borenstein(1997), H. R. Everett, L. Feng, andD.Wehe, Mobile robot positioning:Sensors andtechniques, J. Robot. Syst., vol. 14, Aug., pp.231–249.
[7] Albert Diosi(2005), Geoffrey Taylor and Lindsay
Kleeman, Interactive SLAM using Laser and Advanced
Sonar, Proc. IEEE Int. Conf. On Robotics andAutomation, pp.1103-1108.
[8] Yusuke Misono(2007), Yoshitaka Goto, Yuki
Tarutoko, Kazuyuki Kobayashi and Kajiro Watanabe,
Development of Laser Rangefinder-based SLAM
Algorithm for Mobile Robot Navigation, SICE Annual
Conf., pp.392-396.

©ISAROB 2011305

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Potential field method applied on the navigation of multiple mobile robots
with limited ultrasonic sensing
Chiyen Kim*, Mincheol Lee**, Junyoung Beak*, Chibeom Noh*
* The School of Mechanical Engineering, Pusan National University,#405 RIMT, Pusan National Univ, Jangjeon, kuemjeong, Pusan, south korea 609-735(Tel : 82-51-510-3081; Fax : 82-51-512-9835)
chiykim@pnu.edu** The School of Mechanical Engineering, Pusan National University,#405 RIMT, Pusan National Univ, Jangjeon, kuemjeong, Pusan, south korea 609-735(Tel : 82-51-510-2439; Fax : 82-51-512-9835)
mclee@pusan.ac.kr

Abstract: Multi-robot manipulation can accomplish complex task with simple robot systems comparing with singularmobile robot. In spite of systemic simplicity, multi-robot cause subsidiary problem, group driving. Each mobile robots
have to consider not only target and obstacles but also other member robots. Previous researches attempt to solve it byusing communication and formation method. As this paper studies to solve the problem in the simplest way, it is
proposed the formatted navigation method using simple microrobots equipped with ultrasonic sensor without network
communications. The navigation is based on potential field algorithm, and virtual hill concept is applied for theformation. Before the experimental test, this paper shows the simulation test to evaluate proposed method.
Keywords: multiple mobile robots, formation, virtual hill, navigation, platoon, potential field

contribute for reducing operating load for driving and insame time increasing program space for other task.
In this paper, whole robots’ driving is proposed
based on platoon formatted navigation. The formation ismade up by virtual hill method which helps mobile
robots make a set-upped distance with other ones. In the
proposed method, all mobile robots follow potential
ingredient decent direction not only avoiding obstacle
but also keeping distance with other member robots.
The previous research done by P. Song, and V.
Kumar showed that potential field method is adoptablein multi mobile robot navigation [4]. The study was
based on vision system. But, this paper wishes tocontribute to make a multi robot navigation method
affordable to very simplified system.
In this paper, potential field method is briefly
explained and virtual hill method is introduced. The
adoptable simple multi mobile robots are modeled for
simulation test. And simulation test evaluates the
proposed method before experimental test.

I. INTRODUCTION
As working as human performance, robot requests
several sensors and actuators. However, increasing
sensors and actuators occur system complexity
problems. Especially as mobile robot basically has topursue safe and right driving as well as anotherrequested task, the robot can meet serious problem atthe several addition of sensors and actuators for the
special purpose. For the problem, many researches
approach the problem with cooperative manipulation bymulti robots [1]. However, in the multi mobile robot
system, corporative driving and manipulation occurs
new problem. Each mobile robot has to consider not
only target and obstacles but also other member robots.
To solve it, many researches adopt network
communication and machine vision system [2][3]. But,
these solutions are still contraposed against system
simplicity.
This study researches the solution laying emphasison the system simplification. The system simplificationis acquired by only using economic ultrasonic sensors
for the environmental recognition without any othersensors and network devices. For a start, this paper
considers only multi mobile robot navigation before
researching the manipulation. This challenge can©ISAROB 2011

II. Potential Field Method
1. Goal Navigation
A variety of effective attraction and repulsion
potential fields are summarized in [5], [6]. Let

306

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

rij =|| ri − rj ||be the Euclidean distance between robot
and the object or the goal, a simple quadratic attractive
potential function can be expressed as
Vioa = 1/ 2kio rio2
(1)
ri and ro are the position vectors of the robot andthe object, respectively.
An expression of the repulsive potential is
Vijr = kij / rij(2)
r =|| ri − rj ||where ij
is the Euclidean distance
between robots i and j. In each control mode, the force
vector generated by potential fields and an appropriately
designed dissipative function provide the driving force
to the robots.

Fig. 3. Global potential with a virtual obstacle

2. Artificial Virtual Hill
The artificial virtual hill was proposed in previous
research to avoid local minimum [7]. In unknown
environments, the robot initially does not have any
information about the environment, and it has a limited
sensing range to detect obstacles. The environment in
Fig. 1(a) does not contain a local minimum, so robots
may successfully reach the goal using the generalpotential field approach. In the environment of Fig. 1(b),however, robots may be trapped in a local minimum as
Fig.2(a). In the study, the virtual hill in global potential
was proposed as Fig. 2(b). And then final global field
become as Fig. 3.

Fig. 4. Virtual hill for one line formation
3. One Line Formation
In this paper, the virtual field is applied to make aformation of multi robots as Fig. 4.Fig 4 is simple one
line formation potential field. In the figure, high hill isshown. This hill can keep robot follow front robot. But
between robots, there occurs a few repulsive potential
energy to keep a few distance between robots. The
repulsive hill to make line formation is paralleled withthe line made between goal and leader robot. If there arethree mobile robots in the field, only one or two of themcan recognize the goal point. Although a few member
know and pursue the goal, if other members unknowing
the goal just drive to make a formation, the all of themcan reach the goal by cooperative navigation in the end.
Except all robots do not recognize the goal, there arethree cases of states. The one is that only one robot
knows the goal. The second case is that two robot
recognize the goal and the other doesn’t. The third oneis that all member find the goal same time.
A. Only one robot recognize goal
The first case is beginning state. If robot find only
goal without other member, the robot will approaches
the goal only. Remaining robots follows the first
moving members.

(a) Open aisle(b) Closed aisle
Fig. 1. Two types of obstacles with aisle shapes(a) The potential field(b) virtual hill
Fig. 2. The potential field and virtual hill

©ISAROB 2011307

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

DG

DG

Two virtual hills are set as Fig 6. One is set on the line
to leader. The other is set on the opposite site to virtual
sink with same angular between the sink and leader. The
distance is same as the one to sink. Thus, after making
line formation, the relative potential field becomes as
Fig. 4.

DL
DL

III. Simulation Test
1. Robot Modeling
The whole study will evaluate the proposed
technique using experimental test. This paper does itusing simulation test before experimental test. Thus, thischapter describes the simulation model based on
experimental system. Fig. 7 shows the concept designsof the robot figure on the stocks. The robot iscentralized 2 wheels robot equipped rotatable ultrasonicsensor scanner, 2 IR photodiodes and IR ramp array.
The purpose of IR usage is recognition of other robot.
But the communication is not implied. The IR sensors
are also attached on the ultrasonic sensor scanner to get
the direction of the member robots. Fig. 8(a) shows the
prototype of ultrasonic sensors scanner. 2 directional
sensors is designed to be rotated 90 degree to get theenvironment distance within 180 degree. Fig. 8(b) is theexperimental test result. Table 1 is the designed
specification of robot and environment.

Fig. 5. The Potential field of the first followers
B. Two robot recognize goal
If there are goal and one other robot in the sensor
view, the robot compares the distance to goal and otherrobot. From the result of distance, the shortest distancerobot becomes the leader which just pursues the goal.
And remaining robot becomes follower.
If the robot becomes follower, then it makes 2
virtual hills as shown in Fig. 5. The first hill is locatedon the line to goal with the distance to the leader. And
the second hill is set on the deposited point to leader
with same angle between goal and leader. The distancetoward second hill is the same one to goal.
C. Remaining follower
If there are two more robots and goal in the view ofrobot, the robots distinguish the leader and the front
follower by comparing relative distance with goals. If
two more robots is shown and goal is not founded in theview of robot, the robot searches the first moving robot
who will become the leader. Then by comparing the
distance to the leader, the robot decides its order.
The second follower robot set the virtual sink whichis temporary goal and two virtual hills as shown in Fig.
6. When the robot decides the lower priority order, thevirtual sink is set at the nearest member to leader robot.

Fig. 7. Concept designs of the experimental robot

9

8

7

6

5

4

3

2

1

0
-8

-6

-4

-2

0

2

4

6

8

10

(a) Ultrasonic Scanner (b)Experimental test result
Fig. 8. Concept designs of the experimental robot
Fig. 6. The Potential field of the second followers©ISAROB 2011308

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 1. Specification of Multi mobile robot system
Sort

Specification

Robot Size

200(D)150(H)Field Size

4,000(W) 4,000(L)The range of Ultrasonic sensor

3,000mm

The degree of Ultrasonic sensor

180 degree

Maximum Velocity of

20mm/sec

Number of Robot in flied

3

during simulation test. From the result, it is known thatthe robot can approach in the tail of front robot or target
goal and find out only front robot and goal in thesensored range result.The whole study will evaluate theproposed technique using experimental test. This paper
does it using simulation test before experimental test.

VI. CONCLUSION
This paper studies the multi mobile robot formationand navigation method in the simplest system by only
using ultrasonic sensors for environmental recognition.
The potential field method is adopted to drive mobile
robot. To achieve corporative navigation, this paper
proposed line formation method using virtual hilled
potential field navigation. The proposed method wasevaluated simulation test. In the future, experimental
test has to be executed and compared with simulation
test

ACKNOWLEDGE
This research was supported by the MKE(The
Ministry of Knowledge Economy), Korea, under theSpecialized Field Navigation/Localization Technology
Research Center support program supervised by the
NIPA(National IT Industry Promotion Agency)" (NIPA2010-(C7000-1001-0004))

Fig. 9. Simulation test result

REFERENCES[1] T. Arai(2002), E. Pagello, L. E. Parker, Editorial:
AdvanceinMulti-Robot
Systems.
IEEE
TRANSACTIONSONROBOTICSANDAUTOMATION. Vol. 18. No. 5: 655-661.
[2] B. MacLennan(1991), Synthetic ethology: An
approach to the study of communication, In Proceedingsof the 2nd interdisciplinary workshop on synthesis andsimulation of living systems. 631–658[3] A. Das(2002), R. Fierro, V. Kumar, et al, A visionbasedformation
controlframework.
IEEE
TRANSACTIONSONROBOTICSANDAUTOMATION. Vol. 18. No. 5: 813-825.
[4] P. Song(2002), and V. Kumar, A Potential Field
Based Approach to Multi-Robot Manipulation,”
ICRA’02, Proceedings:1217-1222[5] O. Khatib(1986), Real-time obstacle avoidance formanipulators and mobile robots, International Journal of
Robotics Research 5:90–98.
[6] R. Volpe(1990) and P. Khosla, Manipulator control
with superquadricartificial potential functions: Theory
and experiments, IEEE Trans. on Syst., Man, andCyber., 20(6):1423–1436.
[7] M. K. Park(2001), M. C. Lee, Object Tracking
Algorithm for a Mobile Robot using Ultrasonic Sensors,
ICCAS’01 , Proceedings:333-336.

Fig. 10. Sensors monitoring in Simulation test
2. Simulation Test
The proposed formation method using potential
virtual hill is evaluated by simulation test. The robots
were located in the 4 by 4 meter simulation fieldarbitrarily. Fig. 9 is the simulation test result showing
the progress of formation and navigation. From therepetitious test and various case of initial setting, theresult reveals that the proposed algorithm make a line
formation and goal arrival in simulation test. Fig. 10shows the ultrasonic sensor recognition of each robot©ISAROB 2011309

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Adaptive tuning of a Kalman filter using Fuzzy logic
for attitude reference system
Taerim Kim, Joocheol Do, Eunkook Jung, Gyeongdong Baek, and Sungshin Kim
School of Electrical Engineering, Pusan National University,
Jangjeon-dong, Geumjeong-gu, Busan, 609-735, Republic of Korea(Tel : 82-51-510-2367; Fax : 82-51-513-0212)(Email : {rimkt, colorhorse, silverkook, gdbaek, sskim}@pusan.ac.kr)

Abstract: This paper proposed parameter control method for covariance of Kalman filter using fuzzy logic. Most
people use accelerometer and gyro for calculating attitude and Kalman filter for fusion algorithm of two sensors.
However, parameter of Kalman filter isn’t correctly predicted and attitude reference system (ARS) using Kalman filter
which has non-controlled parameter has many errors when it is moved a great variation. For improving this problem,
we proposed parameter of Kalman filter tuning method using fuzzy logic and then we compared our ARS module with
Crossbow Nav420CA.
Keywords: ARS, Kalman filter, Fuzzy logic, Accelerometer, Gyro, Euler angle.

I. INTRODUCTION

II. ATTITUDE DETERMINATION FROMEACH SENSOR

Recently, the development in micro-electromechanical system (MEMS) technology has made it
possible to make cheap chip like accelerometer andgyro sensors, which have been adopted into many
applications but traditionally inertial sensors have been
too costly [1-2]. Attitude reference system (ARS)
estimates body’s roll and pitch. It is widely used inrobot, aircraft, ship and so on. Particularly, there isgetting popular robot market. Humanoid and military
robots have been interesting topics for the past decade.
Therefore, inexpensive and high efficient attitude sensor
is necessary for an effective attitude control [2].
Most of the ARS is consist of accelerometer andgyro. Accelerometer measures acceleration of the bodyand gyro measures angular rate of the body. Before time,
attitude calculation is possible by simple integration
using gyro. However, there is difficult to attitude
calculation as using low cost MEMS sensors because ofaccumulated error of output. Therefore, there is recently
study about calibrated method of gyro error using
accelerometer. Fusion method of two different sensors
uses widely Kalman filter [2-3]. But it has trend that
unstable filter by parameter of Kalman filter which isdifficult to estimate. We had known through experimentthat if variation of accelerometer is large then Kalmanfilter error is large. Using this fact, this paper proposed
calibration method that controls parameter of Kalmanfilter using fuzzy logic.

©ISAROB 2011

1 Attitude determination from gyro
The roll(∅), pitch(θ), yaw(ψ) rate of the body aremeasured using gyro and accelerometer. We need toconsider its body axis system so that we can calculate
attitude. Attitude calculation method is divided intoquaternion method, cosine matrix method and Euler
angle method. Among these methods, we used Euler
angle method because it can easily recognize user toattitude of the body. The following equation is direction
cosine matrices of each axis [3-4].

0
0 ù
é1
ê
Cx = ê0 cos f sin f úú
êë0 - sin f cos f úûécos q 0 - sin q ù
C y = êê 01
0 úúêë sin q 0 cos q úûécosy siny 0 ù
Cz = êê - siny cosy 0 úú
êë 0
0
1 úû

(1)

(2)

(3)

The transformation from reference frame to body
frame can be represented in Fig. 1. Therefore, therelation between gyro output and Euler angles can be
described using (1)-(3) as follows :310

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011éw x ù éf& ù
é0 ù
é0 ù
ê ú ê úê &úê ú
êw y ú = ê 0 ú + Cx êq ú + Cx C y ê 0 ú
êw ú ê 0 ú
ê0 ú
ëêy& ûúë ûë zû ë û

III. KALMAN FILTER USING FUZZYLOGIC SYTEM(4)

1. Kalman filter
Kalman filter is used for fusion of gyro andaccelerometer. Kalman filter is consisted of processmodel and measurement model.

where each ωx, ωy, ωz are the gyro outputs in the x, y,z axis. We get next term by applying the Euler angles togyro output.

éf& ù é1 sin f tan q cos f tan q ù éw x ù
ê&ú ê
ê ú
- sin f úú êw y ú
cos fêq ú = ê 0
êy& ú ê 0 sin f sec q cos f sec q ú ê úû ëw z ûë û ë

xk = Axk -1 + Buk + wk -1
zk = Hxk + vk
(5)

where wk and vk represent the process andmeasurement noise respectively. We are assumed to be
independent and with normal probability distributions.

This paper proposed ARS development so that f&
and q& is only used.

wk ~ N (0, Q)
N

Xy

vk ~ N (0, R )

Xz

N, X
ϕ

E

θ

E, Y
Yx
Yz

ψ

Zy Zx D, Z

Fig.1. Definition of Euler angle
2. Attitude determination from accelerometer
Accelerometer output is used to obtain roll and pitch.
The relational equation expressed accelerometer output
equals to add acceleration of body frame andacceleration of gravity[5].

é g sin qùê
f = v& + w ´ v - g » ê - g sin f cos q úúêë - g cos f cos q úû
b

b
nb

b

b

(6)
Fig.2. Structure of attitude reference system
2. Fuzzy logic for controlling parameter
We used fuzzy logic for modifying process andmeasurement noise covariance. Before Kalman filter isused, we have to calculate Fuzzy logic using output of
sensors and reflect changed parameter. Input of fuzzy
logic is acceleration’s variation of x axis and y axis andoutput is variation of R, Q. We used mamdani method
and simple rules. The following two fuzzy rules areused :
Rule1. If |∆a| is small then R is increased and Q isdecreased.
Rule2. If |∆a| is large then R is decreased and Q isincreased.

where f b is specific force and gb is the gravity vector
in body frame. v is linear velocity component of eachaxis and it can’t be measured without external sensors
so that we ignore those terms. Therefore Eq. (6) is
reduces to the following as :fg

q = sin -1 ( x ),f = sin -1 (- fy
g cos q)(7)

We can calculate roll and pitch using simple
equation. Therefore, we can calculate roll and pitch byfusion of gyro and accelerometer.

©ISAROB 2011(9)where Q is the process noise covariance and R is themeasurement noise covariance[6-7].
We used that process model is gyro output andmeasurement model is calculated attitude ofaccelerometer as shown in Fig. 2. However, parameterof Kalman filter can’t estimate accuracy values.
Through experiment, we could know that ARS has large
errors when it is greatly changed so we used fuzzy logic
for improving performance [8].

D

b

(8)

311

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

where |∆a| is variation of each axis’ accelerometer
output and absolute value. If variation of acceleration islarge then we can’t trust output of accelerometer and R
must be changed to be increased because sensor
movement is supposed to be measured. The membershipfunctions of fuzzy sets are shown in Fig. 3.

Roll
50
40
30

degree(˚)

20
10
0
-10
-20

Kalman filter
Fuzzy-Kalman filter
Nav-420CA

-30
-40
-50

μ

1

1

0

300

2048 |∆a |

0
-1

0

1 ∆R or ∆Q

Fig.3. Membership functions
Range of ∆a is determined by output range ofaccelerometer sensor and heuristic method. Output
range of using accelerometer is from -1024 to 1024.

2

4

6

8

10

12

14

16

18

20

50
40
30
20
10
0
-10
-20
-30
-40
-50

Kalman filter
Fuzzy-Kalman filter
Nav-420CA

0

2

4

6

8

10
12
time(sec)

14

16

18

20

Fig.5. Roll and pitch rotation test result
Roll

degree(˚)

IV. EXPERIMENT

degree(˚)

The proposed parameter tuning method of Kalmanfilter using fuzzy logic has been tested and comparedwith Crossbow Nav420CA. Nav420CA is INS(Inertial
Navigation System). It can measure roll, pitch and has
high accuracy which is 0.75°rms. We made ARS
module and it consists of microprocessor, 3-axis
accelerometer and 2-axis gyro. We used LIS3LV02DQ
accelerometer and IDG-300 gyro. Microprocessor isused ATmega128. We measured roll and pitch of ARS
module and Nav420CA at the same time using the
stewart platform in shown as Fig.4.
The results obtained that proposed algorithm isbetter than Kalman filter in shown as Fig. 5. We
confirmed response delay and it is due to response rateof microprocessor and parameter of Kalman filter. To
knowing accuracy, we calculated errors based upon asimple root mean squared error (RMSE). Here, we
assumed truth model is Nav420CA. When we used only
Kalman filter, RMSE is in shown as Table 1. Because of
reducing errors, we can tell that proposed algorithm isbetter than simple Kalman filter.

50
40
30
20
10
0
-10
-20
-30
-40
-50

50
40
30
20
10
0
-10
-20
-30
-40
-50

Kalman filter
Fuzzy-Kalman filter
Nav-420CA

2

3

4

5

6

7

8

time(sec)
Pitch

3

4

5

6
7
time(sec)

Kalman filter
Fuzzy-Kalman filter
Nav-420CA
8
9
10

Fig.6. Zoom in roll and pitch rotation test result
Table 1. RMSE of the test result
Fuzzy-Kalman
Kalman filter
filter
roll
pitch
roll
pitch
Static
0.20
0.10
0.20
0.03
Dynamic
1.43
1.07
0.72
0.71

V. CONCLUSION
This paper proposed parameter of Kalman filter
tuning method using fuzzy logic. In order to attitude
calculation, we used gyro calibration by Euler angle
method. Fusion of sensors is usually used Kalman filter.
We determined process model is output of calibrated
gyro and measurement model is calculated attitude ofaccelerometer. However, result of this algorithm haslarge errors when it moves greatly. To this problem, wechanged parameter of Kalman filter using fuzzy logic. If

Fig.4. ARS module and experimental environment©ISAROB 2011

0time(sec)
Pitch

degree(˚)μ

312The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

REFERENCES

ARS module is moved large, then we trust gyro so that
Q is decreased and R is increased. We tested to confirm
proposed method. We made ARS module using 3-axis
accelerometer and 2 axis gyro. The result is reduced
errors than simple Kalman filter. We confirmed that
proposed algorithm is very simple but it is highperformance. If we have a chance, we want to researchmagnetic sensor. Magnetic sensor is used calculation ofyaw. We will add our ARS module to magnetic sensor
and then we will be able to make AHRS(Attitude
Heading Reference System) module.

[1] Isaac Skog[1], Peter Handel[2](2006), Calibration of
a MEMS Inertial Measurement Unit, XVII IMEKO
World Congress[2] Chul Woo Kang[1], Chan Gook Park[2](2009),
Attitude Estimation with Accelerometers and Gyros
Using Fuzzy Tuned Kalman filter, Proceedings of theEuropean Control Conference 2009:3713-3718[3] Rong Zhu[1], Dong Sun[2], Zhaoying Zhou[3],Dingqu Wang[4](2007), A linear fusion algorithm for
attitude determination using low cost MEMS-based
sensors, Measurement Volume 40, Issue 3:322-328[4] Rong Zhu[1], Zhaoying Zhou[2](2006), Calibrationof three-dimensional integrated sensors for improved
system accuracy, Sensors and Actuators A 127:340-344[5] Sung Kyung Hong(2003), Fuzzy logic based closedloop strapdown attitude system for unmanned aerial
vehicle(UAV), Sensors and Actuators A 107:109-118[6] J.Z. Sasiadek [1], Q.Wang [2], M.B.Zeremba[3](2000), Fuzzy Adaptive Kalman Filter for INS/GPS
Data Fusion, Proceedings of the 15th IEEE International
Sysmposium on Intelligent Control:181-186[7] Nebot E.[1], Durrant-Whyte H.[2](1999), Initial
Calibration and Alignment of Low Cost Inertial
Navigation Units for Land Vehicle Applications,
Journal of Robotics Systems, Vol.16, No.2:81-92.
[8] D. Loebis[1], R. Sutton[2], J.Chudley[3],W.Naeem[4](2004), Adaptive tuning of a Kalman filter
via fuzzy logic for an intelligent AUV navigation
system, Control Engineering Practice 12:1531-1539

Acknowledgment
This research was supported by the MKE(The
Ministry of Knowledge Economy), Korea, under the
Human resource development program for robotics
support program supervised by the NIPA(National IT
Industry Promotion Agency). (NIPA-2010-(C70001001-0009))©ISAROB 2011313

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A High-speed 3D Image Measurement Method
Ke SunYundi Yao and Cunwei Lu

Fukuoka Institute of Technology(Tel : 81-92-606-3578; Fax : 81-92-606-0726)(bd08001@bene.fit.ac.jp) and (lu@fit.ac.jp)

Abstract: The intensity modulation projection technique has been in great anticipation of practicability with the
popularization of digital camera and projector recently with reason that it may get much more information withsingle projection pattern through measurement and calculate the high sensitive 3D information. But thetechnique is premised on the situation that it must get observation pattern image with essential number of stripe
and intensity distribution, that is to say, the ideal observation pattern image so as to secure the sensitivity andaccuracy of measurement. So, as for the target object without specification on the surface reflectance anddistribution of surface color, the measurement would encounter several problems such as deficiency of thevolume of information on the observation pattern, uncertainty of the target measurement sensitivity as well as
the trouble of measurement manipulation. In order to solve the problems presented above. We propose an image
analysis method of intensity correction with synthesis image technique in order to extract the stripes precisely
when measuring the 3D shape of an object by using single projection pattern. By using this analysis method, if
the surface reflection of the target is simple, 3D shape measurement is realized using only one observation image.
Keywords: 3D Shape Measurement, Optimal, Intensity Modulation, Pattern Projection, Image Synthesis.

observation image, so the combination of monochrome
projection and monochrome analysis do not take object
color into consideration. Here we use optimal intensity
modulation pattern, which is made by a special
algorithm. This kind of pattern is easier to do the stripe
extraction just because the adjacent stripes have a good
maximum intensity difference, which makes the stripe
easily to be distinguished[2-3].
In this paper, we propose an analysis method basedon the optimal intensity modulation pattern projection
technique, in which a structured monochrome projection
pattern is projected and a color image of the observation
pattern is captured. The paper focuses on the optimal
analysis of optimal intensity modulation projection
technique by using single projection pattern and thecontradistinction against the conventional methods instripe analysis. The intensity distribution stripes offer an
identifiable pattern so that the whole surface patternimage of the object can be obtained to easily analyze
surface depth information by triangulation. Not only can
we get better detection accuracy and faster computation
speed through optimizing the stripe intensity
distribution, but also can get better stripe order detection
by modulating the intensity differences between
adjacent stripes[4-5].

I. INTRODUCTION
Three-dimensional (3D) shape measurement has
different applications in engineering fields, such as
inspection of product quality, face recognition, bodysurface evaluation, and surgery simulation andnavigation in biomedical engineering[1].
At present, popular 3D shape measurement
techniques include stereo vision, structured light, moiré
method, and the other methods. Depending on the lightsource and the receiving style, many techniques havebeen proposed, active systems and passive systems. The
stereo vision is based on imaging the scene from two or
more points of view and then finding correspondences
between the different images in order to triangulate the3D position. Triangulation is possible if cameras arepreviously calibrated. The structured light techniquegenerates a full pattern to cover the object so that thesurface can be captured as a whole field recording.
Therefore, the encoding of the structured pattern
becomes important because it determines the local
features of the recorded images to be identified inspatial recognition and height evaluation.
When we measure a color object, the color
distribution of the object often influences the code wordof project pattern; here we use a monochrome
projection pattern to avoid the influence from color. In
this method, we only do analysis on the intensity of the©ISAROB 2011

II. ALGORITHM

314

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Figure 1 was a composition of typical 3D shapemeasurement system. The projection pattern been

Step5: Confirming the influence of surface
reflectance. If there were no influence, Step 6 should beomitted and jumped to step 7 directly.
Step6: Correcting the influence of the surface
reflectance of the measurement object from the image
for the measurement with the object of detecting stripes
address in high accuracy and sharpening stripes pattern
for the measurement. In the following Chapter 3, we
described the technique and principle.
Step7：Extracting slit pattern address in the light ofa linear relation of each stripes intensity value from thecorrected image for measurement because the intensity
value of each stripes pattern was decided by majority
according to the intensity value of each pixel of each
stripes pattern.
Step8：Calculating the 3D space world coordinates
based on the detected stripes address. The calculation of
three dimension shape by using the principle of thetriangulation was omitted in this thesis.

ObjectFig.1. 3D shape measurement ofpattern projection system

output by the computer was projected to themeasurement object from the projector, camera wasused to take a picture of the projected pattern reflectedto the measurement object as an observation patternimage, finally, the information stored in the camera was
input to the computer. Next, the directions of the stripes
of projection pattern were detected from the observation
pattern image which had been input, and then 3D shapeof the object was restored based on the principle of thetriangulation. The following contained processing ofeach step was stated simply[4].
Step1: Projecting optimal intensity modulation
pattern to measurement object.
Step2: Taking a picture of the image and using it as
the initial observation pattern image.
Step3 ： Doing color analysis of the initialobservation image and synthesizing the image for themeasurement based on the color channel for themeasurement. The color channel that reflection intensity
value was stronger than the others was extracted every
pixel from input image and used it as the channel for themeasurement. The influence caused by other color
components and the surface reflectance in the same partof the measurement object were reduced bysynthesizing single channel image for calculation with achannel for the measurement of each pixel.
Step4: Extracting measurement object from theimage for measurement and deleting the noise withthreshold extraction method.

©ISAROB 2011

III. PRINCIPLE AND METHOD
1. Problem of reflectivity reduction
About the optimal intensity modulation patternprojection measurement, it was ideal that the relation
between the intensity of the reflection pattern stripes
obtained from the input image and a pattern stripes
degree were a one-to-one correspondence in order toobtain three dimension shapes with accuracy. However,
the measurement object was multiple color distribution,and it was difficult to obtain such an ideal relation when
it had reflectivity taken with a versatile digital camera.
Generally, the gray projection pattern was projected tothe measurement object with a variety of color
distribution, information obtained from the reflection
pattern image was only brightness, coordinates, andcolor information on the measurement object decreased
and the measurement range confused became narrow[6].
2. Past method of intensity correction
For the projected pattern intensity P ( x, y ) , if pixel
intensity I (i, j ) in the image was always (1), thecorrection method of traditional reflectivity should be
ideal.
P( x, y ) : I (i, j )  1 : 1(1)
Actually, Surface reflection element O ( x, y ) of theobject can be actually contained, and intensity of eachpixel in the reflection pattern image that hit the object
be shown as follows.

315

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

I l (i, j)  Pn ( x, y)  O( x, y)(2)
At this time, because the surface reflection element
had its influences in (2), stripes degree n ofPn ( x, y) cannot be detected. Then, we projected
projection pattern with uniform intensity Pf ( x, y ) to
the same measurement scene, the relation of (3) was
obtained.
(3)
I f (i, j )  Pf ( x, y)  O( x, y)
The following relation was obtained from (2) and (3).

division calculation, and then to avoid the effect ofreflectivity of object surface color. At last, pattern image
used for measurement which is in linear relation is
obtained.

IV. OUTCOME OF AN EXPERIMENTThe experiment was done in the spaciousness
environment where the lighting was erased. The
experiment system was composed of the liquid crystal
projector, the CCD color camera, and the computer, thestripe number of the optimal intensity modulation
projection pattern was set to 20.

I l (i, j ) Pn ( x, y )I f (i, j ) Pf ( x, y )(4)That is to say, the image for the calculation with the
projection light stripes intensity distribution and linear
correspondence was obtained by the dividing
calculation correction. With correction method, steadystripes pattern for measurement could be obtained, but
confronting the fact, which meant its difficulty to deal
with the moving object, furthermore, it is necessary to
use two pieces of observation image, which made themeasurement cost time[7].
3. Propose method of intensity correction
Previous method can give us a good result whenmeasuring a stationary object. Typically, this method
takes time to send two frames of projection patternwhen measuring. In fact, we do not need high precisionof the full-light observation image which means notvery necessary to do the full-light image capturing. If
we can analyze the distribution through someprocessing algorithm, the original image after gray scale
may be not necessary.
In order to realize 3D shape measurement using one
image, we propose an optimal image analysis method.
Intensity information of no-pattern area is used torecover intensity information pattern area. Colorchanges of object surface have little effect on thechange of reflectivity. So we set a specific filter to do
recovery of pattern pixel in this specific domain. We try
to use the neighbor pixel intensity of the no-pattern area
to fill in the pixel of pattern area, and then let the filter
move to the next pixel to do the same work of thecorrection processing. Under this way, we could obtain
the random pixel intensity distribution, this kind of
information transference still carry over the no-patternpixel intensity approximately. We can obtain a synthesis
full-light image by using this method, in which there isno pattern area. We use this synthesis image to do©ISAROB 2011

(a)

(b)

(c)

(d)

(e)

(f)

(g)
Fig.2

Experimentresult:(a)

Optimalintensity

modulation projection pattern. (b) Column object.
(c) Observation pattern image. (d) Extracted patternimage. (e) Synthesis image. (f) Corrected pattern image.
(g) Intensity distribution image for comparison.

First of all, 20 stripes (Figure 2(a)) optimal intensity
modulation patterns were projected to the column(Figure 2(b)), the reflected pattern, as an observation

316

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Information and Communication Engineers D-II;
Vol.J88-D-II, No.2, pp.325-333, 2005.
[ 5 ] Cunwei Lu and Seiji Inokuchi : Intensity modulated moiré topography; Applied Optics; Vol.38,
No.19, pp.4019-4029(1999).
[6] C. Lu and G. Cho: 3D Image Measurement by
Combination of Monochrome-Projection ColorAnalysis and OIMP Technique; Transactions of The
Institute of Systems, Control and Information
Engineers: Vol.17, No.6,pp.233-240(2006)[7] Cunwei Lu and Limin Xiang: Optimal intensitymodulation projection technique for three-dimensional
shape measurement; Applied Optics; Vol.42, No.23,
pp.4649-4657, 2003.

pattern image (Figure 2(c)), was taken a photo bycamera. And, the observed pattern was extracted, which
formed extracted pattern image (Figure 2(d)). Next, thesynthesis image (Figure 2(e)) by the technique of
propose was generated, Pattern intensity was correctedby the dividing calculation with Figure 2(d) and Figure2(e), which formed the corrected pattern image (Figure2(f)). Figure 2(g) is the comparison of the intensity
distribution of Figure 2(a) and Figure 2(f), just shown
here, it can be seen that the intensity distribution of thecorrected image is almost similar with the one ofprojection pattern image, our method is successfully
proved to be done in this kind of correction processing.

V. CONCLUSION
The paper presents a technique of 3D shapemeasurement by using single projection pattern to do
optimal analysis based on convolution theory. The
optimal intensity modulation projection pattern ischosen to encode the surface area of the object; anduniform intensity projection pattern formed by programis used to reduce the impact of the colors in stripe
region. This method is applied to revise the color
distribution of projection pattern stripes by using thebackground information in 3D projection patternmeasurement technique.
The future task is to measure object surface which
has complicated texture, and verify the robustness of
this technique.

REFERENCES[1] S. Inokuchi and K. Satou: 3D Imaging Techniquesfor Measurement; Shokodo, Toukyou(1990).
[2] G. Cho and C. Lu: 3D Image Measurement for
Object with Multiplex Color Information Analysis;
Proceedings of 9th Symposium on Sensing via image
Information(SSII 03), pp.79-82(2003).
[3] G. Cho and C. Lu: Practical 3D Measurement
Systembased on Projection Pattern Control
Techniques; IEEJ Trans. EIS; Vol.127, No.4, pp561567(2007).
[4] Cunwei Lu, Limin Xiang and Seiji Inokuchi:
Optimization of the Projection Pattern and a QuasiOptimal Combination Generation Algorithm for 3-D
Shape Measurement; The Institute of Electronics,©ISAROB 2011317

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

3-D Face Recognition Method Based on Optimum 3-D Image
Measurement Technology
Hiroya Kamitomo, Yao Xu and Cunwei Lu
Fukuoka Institute of Technology, 30-1 Wajirohigashi3-chome, Higashi-ku, Fukuoka(Tel : 81-92-606-7395)(bd09001@bene.fit.ac.jp, lu@fit.ac.jp)

Abstract: We propose a 3-D face recognition method using the iterative closest point (ICP) algorithm. So far, thereported 2-D face recognition methods have weak points by source of light, make-up and facial positioning. A using 3D shape measurement system includes a technique for optimizing the intensity-modulation pattern projection.
Therefore, we propose 3-D face recognition system which is robustness in the color change. The face recognition
method is possible for facial registrations. This method estimate rotations and translations for different positions anddirections. Consequently, experimental results obtained the high accuracy that the rate of face recognition is 96%.
Keywords: 3-D shape measurement, 3-D image, face recognition, iterative closest point, registration

practicable 3-D image measurement system based on
optimal intensity-modulation projection technique. Thissystem is possible to obtain 3-D facial images in highspeed and high-accurate. The registration of 3-D facial
image is easy under changes with rotation andtranslation. We propose technique of recognition using
3-D facial image.

I. INTRODUCTION
The biometrics is the security technology used foraccess control. The two main types of biometrics are
behavioral method and physiological method. The
examples of behavioral method are voice, gait andhandwriting. This method’s problem is influenced by
the condition of person. The examples of physiologicalmethod are fingerprint, iris and face recognition. Thismethod is more highly accurate than behavioral method.
However, fingerprint and iris recognition are difficult to
get people’s approval. Many people become unpleasantfeelings because of measurement. Face recognition iseasy to measure biometric information. Therefore, facerecognition has a lot of researches. Past face recognition
uses the two-dimensional image. The biometric system
extracts a feature from the facial image. The first time afeature data is stored. In subsequent uses, the feature
data is compared. However, face recognition does notwork well under conditions of lighting and make-up.
The matching of feature data fails by appearancechanges of the facial image. To deal with this problem,
the three-dimensional face recognition attracts attention.
This technique uses 3-D shape measurement system tocapture information about the shape of a face. One
advantage of 3-D face recognition is that it is notaffected by changes in lighting like other techniques.
Popular 3-D recognition algorithm is the matching of zrange images. The z-range image is 2.5-D information.
The problem of this information is the changes of facial
position with rotation and translation. We proposed the©ISAROB 2011

II. 3-D SHAPE MEASUREMENT
The method of triangulation is the radical principleof 3-D shape measurement based on the pattern light
projection. The pattern light is projected to the target
object from the source of light, and the camera captures
images of the appearance from a different angle.
Formula (1) is used to measure a depth distance oftarget point by the triangulation principle.

zbtan  tan (1)Here, z is a depth distance of the measurement point;b is a base length (the distance between source of lightand the center of lens of camera); α is stripes projection
angle; β is an observation angle degree; β is computableby stripes coordinates on the observation image.
When the strength modulation pattern light is used
as shown in Fig 1, stripes projection angle could be
calculated by a single piece of picture theoretically.
However, this pattern is influenced by the surface colorof the measurement object and reflection. The intensity
value of the projection stripes in the observation image

318

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig.1. Intensity modulated pattern(a) Initial observation(b) Projection patternFig.2. Optimal intensity modulated pattern(c) Observation

Fig. 3. Projection pattern and capture images

Fig. 4. 3DCG of facething. Thus, it can be said that d  p1 , p2 , p N  seek
combination  p1 , p2 , p N  at the maximum as its
optimum combination. Projection pattern shown in Fig
2 is replaced by the stripes order of projection pattern in
Fig 1. We know that the strength difference between
adjacent stripes has been increasing as graph. As a result,
stripes projection angle should be computableaccurately as long as the strength change rule ismaintained even if the error margin is included in thedetection of stripes projection light intensity.

changed and the correspondence to the projection angle
collapsed. There is an optimum strength combination
pattern light projection method to solve such a problem.
It is a technique to optimize projection pattern and toseek for regulation of variations in intensity at a
maximum about intensity difference of projection
between adjacent stripes[1].
In a set of stripes addresses 1,2, , N  , when theintensity order of the address pi  1,2, , N  , theprojection light intensity of the stripes strength order isdefined as the following magnitude correlations.

I min  I p1  I p2    I pN  I max(2)

III. 3-D FACE RECOGNITION

Here, I p i is the projection light intensity of stripe
address, I min and I max are minimum value andmaximum value of the projection light intensity
respectively. The function is defined when the
projection light intensity difference of the stripes
strength distribution combination I1 , I 2 , I N  is at
the maximum, evaluation function d I 1 , I 2 ,  I N  isalso at the maximum.

d I1 , I 2 ,  I N  N

We obtain 3-D facial images by proposed 3-D shapemeasurement system. A using projection pattern andcaptured images show in Fig 3 (a) (b) (c). The obtained
3-D image can be displayed as 3DCG (Fig 4). The 3-D
face recognition needs registration of 3-D facial images.
The registration algorithm is used as an aligning two 3D shape data. A propose recognition method uses the
Iterative Closest Point (ICP) algorithm[2]. We mounted
ICP for 3-D face recognition on the system. ICP iscomprised of four steps.
1. Selection ; select 3-D point cloud
2. Matching ; detect correspondence point-pairs
3. Rejecting ; reject illegal point-pairs
4. Minimizing ; minimize error between shapesM k I  I
j

i

i j

(3)

i M 1 j 1Here, kj is a weight factor. M is width of the filter onwhich evaluation function has been set. As the relationof formula (2), when d I1 , I 2 , I N  is at the
maximum, d  p1 , p 2 ,  p N  amounted to the same

©ISAROB 2011319

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011(a) Model texture(b) Scene texture
Fig. 6. Initial position of two faces

Fig. 5. Camera captured images(a) Measurement-area(b) Boundary-area

Fig. 7. Mask images for rejecting(a) Before minimizing(b) After minimizing

Fig. 8. Registration of two faces
A concrete steps are shown in the following.

3. Rejecting
The purpose of this is to eliminate outliers. The
corresponding point-to-point pairs include the illegal
point-pairs. This method eliminates pairs containing
points on mesh boundaries. Mask images used forrejecting are shown in Fig 6. Fig 6 (a) is 3-D
measurement-area. Fig 6 (b) is boundary-area. The
corresponding pairs are eliminated on boundary-area.

1. Selection
The first time a system obtains the 3-D point cloudfrom 3-D facial images. 3-D point cloud is comprised ofmany three-dimensional coordinates. 3-D point cloud inenrollment time is called Model, 3-D point cloud in
unknown input time is called Scene. However, Model ismeasurement data of frontal faces. Images that 3-D
measurement system captured are shown in Fig 5. Fig 5(a) and Fig 5 (b) are texture color images of Model and
Scene. At first, two facial positions are different. 3DCG
of Model and Scene shows in Fig 6.

4. Minimizing
The purpose of this is to minimize error ofcorresponding point-to-point. This method finds therigid transformation that minimizes a weighted leastsquared distance between the pairs of points. The
transformation is comprised of rotation and translation.

2. Matching
System finds a point-to-point correspondence
between Model and Scene. The corresponding point of
Scene is the closest point in Model. We usually takes
long time of O(N) that search a corresponding point of
one query point. Here, N is number of 3-D points inModel. We use k-dimensional tree (k-d tree) structure to
make the search processing faster[4]. We take time ofO(logN) for searching with k-d tree. This algorithm
finds the corresponding points of all 3-D points in Scene.
The point-to-point pair between Scene and Model is( pi , qi ) i  0,1,2, M 1 . Here, M is number of 3-D
points in Scene, p is 3-D point in Scene, q is 3-D point
in Model.

©ISAROB 2011M 1

 q  Rp  t  min
2

i

i

(4)

i 0

Here, R is rotation matrix, t is translation vector. t is
obtained using center of gravity.

t  q  Rp

(5)Here, p is center of gravity in Scene. q is centerof gravity in Model.

pi  pi  pqi  qi  q(6)p and q is translated using center of gravity.
Formula (4) is changed to formula (7).

320The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011M 1

 q  Rp  min
2

i

i

FAR

(7)

i 0

Formula (7) and formula (8) are the same meanings.



T r R T N  max(8)

Formul (8) can be solved by the eigen problem ofcorrelation coefficient matrix. R can be calculated out
from formula (8). Then t can be found from formula (5)with R. 3-D points in Scene are transformed into 3-Dpoints in Model using rotation and translation. Modeland Scene shows in Fig 7.
5. Verification
Scene and Model are aligned using iteration of four
steps. At the last step, we verify identities of 3-D facial
images of personal person. The distance of Model and
Scene is distinguished with the threshold. The
recognition system accepts Scene when the distance issmaller than the threshold. The recognition systemrejects Scene when the distance is larger than thethreshold.

FRR
Fig. 9. ROC of 3-D face recognition
is depended on the improvement of hardware andrefactoring of software.

V. CONCLUSION
In this paper we proposed a face measurement andface recognition method, our system is possible tomeasure 3-D facial shape with moving. This recognition
method is possible to verify personal persons withmake-up and lighting. The 3-D face recognition systemaligns 3-D facial images when a person moves a face.
This method calculates the distance between two faces.
The verification step compares a face to stored face. The
rate of false is small when a person moves a face.
Serious disadvantage is that our system is less effective
if the facial expressions vary. A weakness of registrationis the changes of 3-D shape. Therefore, our system now
allows only neutral facial expressions. The future work
is to deal with facial expressions and facial accessory
problem.

IV. EXPERIMENT AND DISCUSSION
This section describes experiment steps and results.
The first time, we measure Models of 15 persons. Modelis 3-D measurement data of frontal face. They moveface when measure Scene. A number of 3-D
measurement data is 105 samples. We recognize all
samples by an application using proposed method. We
measures rate of misidentification about false rejection
rate (FRR) and false acceptance rate (FAR). The
threshold is variable parameter in verification step. FAR
and FRR shows in Fig 9. The receiver operating
characteristic (ROC) plot is a visual characterization of
the trade-off between the FAR and the FRR. In thisexperiment, FAR = 0.038, FRR = 0.047. The rate offalse is small when a person moves a face. This method
using registration is effective face recognition withmoving. The lighting and make-up does not influence
this method because without facial color information.
However, this method is used for a one to one
comparison of a 3-D facial image. The stored template
data is individual person’s Model. This method takes
long time when it is used for a one to many comparison.
Therefore, 3-D face recognition system using thismethod is in conjunction with an ID card or username.
And, we are necessary to shorten the processing time. It©ISAROB 2011

REFERENCES[1] Lu C (2003), Optimal Intensity-Modulation Projec
tion Technique for Three-Dimensional Shape Measur
ement. Applied Optics-IP 42(23):4649-4657[2] Besl P J (1992), A method for registration of 3-D
shapes. IEEE Trans Pattern Analysis and MachineInteligence 14(2):239-256[3] Horn B K P (1987), Closed-form solution ofabsolute orientation using unit quaternions. J. Opt. Soc.
Am A-4:629-642[4] Berg M, Kreveld M, Overmars M, Schwarzkopf O(1997),Computational Geometry Algorithms andApplications. Springer Verlag

321

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

User study of a life-supporting humanoid
directed in a multimodal language
T. Oka1, T. Abe2, K. Sugita3, and M. Yokota3
1

Nihon University, 1-2-1 Izumicho, Narashino, Chiba, 275-8575 JAPAN
Nihon Computer Kaihatsu Co. Ltd., 6-24-9, Minami-Ooi, Shinagawai-ku, Tokyo, 140-0013, JAPAN
2
Fukuoka Institute of Technology, 3-30-1, Wajiro-higashi, Higashi-ku, Fukuoka, 811-0295, JAPANTel : 81-47-474-9693; Fax : 81-47-474-2669
oka.tetsushi@nihon-u.ac.jp, t-abe@nck-tky.co.jp, {sugita, yokota}@fit.ac.jp

2

Abstract: This paper describes a user study of a life-supporting humanoid directed in a multimodal language anddiscusses the results. Twenty inexperienced users commanded the humanoid in a computer simulated remote homeenvironment in the language by pressing keypad buttons and speaking to the robot. The results show that they
comprehended the language well and were able to give commands successfully. They often chose a button press
action in place of verbal phrases to specify a direction, speed, length, angle, and/or temperature value, and preferredmultimodal commands to spoken commands. However, they did not think that it was very easy to give commands inthe language. This paper discusses the results and points out both strong and weak points of the language and robots.
Keywords: Life-supporting robot, multimodal language, user study, humanoid, human-robot interaction
verbal messages and nonverbal messages such as hand
gestures, body touch actions, and button press actions
have been developed and evaluated with novice users.
A small humanoid that can be directed in an earlier
version of RUNA was investigated [5]. Novice users
remotely commanded the humanoid in RUNA to
explore a room. All the users completed their task withhelp from a three-page leaflet and most of themconceived a fairly high opinion of the robot and thelanguage, although there were some shortcomings andlimitations in the robot. The overall command successrate during the task was about 70 % and the robot
reacted to noises due to our poor cheap microphone.
Because of the limitation of the onboard computer, therobot had to choose among 80 actions of walking,turning, looking around, and so on; it were able to turnby 30 degrees but not by 40 degrees, so it rejected somegrammatical commands in RUNA. It was difficult for
novice users to inform the robot one out of five turning
angle values by pressing a button for a definite period oftime; they had no idea how long they should press it toturn the robot as much as they want, because they were
given no instructions or opportunities to practice.
After the study, we developed a simulated humanoidwith a more sophisticated command interpreter andeliminated some of the shortcomings found in our firsthumanoid. The new robot has a larger repertoire ofactions, including 540 left turns. We redesigned a set of
button press actions to specify action parameter values
such as speed, angle, length, temperature values.
This paper presents a user study of the new lifesupporting humanoid. All of the twenty users completed
their tasks, to explore a remote room and operate an air
conditioner, successfully commanding the robot in
RUNA. The success rate of each user’s commands wasover 90 % and several of the problems in the oldhumanoid were resolved. Most of the users preferredmultimodal commands to single modal spoken

I. INTRODUCTION
It is predicted that in near future life supporting
robots will help people in homes, streets, hospitals,
offices, etc. In order to design and develop such robots,
one should take into account both cost and user
friendliness, since such robots must be affordable forpeople in need and designed for those untrained.
Although menu-based conventional GUIs are costeffective, they are not suited for untrained people andthrough such interfaces one cannot communicate withrobots in a natural manner even if they look like humans.
It is understandable that we expect robots tocommunicate with us just like ourselves. In fact, there
are thousands of robots which can speak and understandverbal messages [1]. However, verbal communication isjust one aspect of human communication; we use
gestures, postures, eye contacts, paralanguage, etc. toconvey different kinds of information [2]. Therefore, we
can well image that humanoids will communicate withus both verbally and nonverbally. Obviously, suchrobots will be more user friendly than robots operated
through a conventional user interface [3]. Unfortunately,in order to communicate like humans, robots will need
many kinds of sophisticated sensors to perceive verbal
and nonverbal signals, high-performance computers to
disambiguate messages and infer users’ intensions [4],and an articulated body which look and move smoothly
like a human. For this reason, precise imitation of ahuman will prevent us from developing affordable user
friendly life-supporting robots.
The authors have designed RUNA (Robot Users’
Natural Command Language), a multimodal commandlanguage [5-8], taking into account both costeffectiveness and natural human-robot communication.
In RUNA, one can command an action without
ambiguity by specifying its type and parameter values.
Several versions of RUNA, which combine spoken©ISAROB 2011322

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

commands and often selected a button press action to
specify parameter values. However, the users did notthink that it was very easy to command the robot in the
new version of RUNA. The results shed light on someproblems of our multimodal language as well as
advantages. Most importantly, it will be easier for
novice users to communicate with a life-supporting
robot, if they can convey parameter values one by one
in nonverbal messages and can always omit someparameter values which are obvious in the context orvalues which they do not care about. Since there arecases in which it is hard to determine parameter values
beforehand, users should be allowed to modify
commands at any time.

mentioning them. This reduces average number ofwords in a command and speech recognition errors. One
can command a robot saying “turn” and pressing abutton simultaneously instead of saying “turn 60
degrees left slowly!” Furthermore, multimodal
commands are often more natural than spoken
commands: e. g. pointing a glass and saying “pick thisup” or saying “lower the temperature” pressing a button.
If a button event has been arrived within a shortperiod of time, a spoken command will be modified asshown in Table 2. The twelve buttons are assigned to
specific parameter values (Fig. 1). The direction andspeed of a turning action command are determined by
the key pressed most recently by the user. A single key
press action conveys an angle value (10, 20, 40, 50 or
60 degrees) and a step value (1, 2, 4, 5, or 8 step(s))
depending on the duration, while a multiple key press
action conveys values (90 or 180 degrees and 10, 20,
and 30 steps). Likewise, the robot will make the preset
temperature two degrees higher, if a key has beenpressed twice before a spoken command “raise the room
temperature!”
Finally, the repeat button and query button allow
users to command robots without speaking. The empty
button convey default parameter values, such as 3 steps,
30 degrees, right, normal speed, etc.

II. MULTI-MODAL LANGUAGE
In the study presented in this paper, we used aversion of the multi-modal language, RUNA [6], which
comprises a set of grammar rules and a lexicon for
spoken commands, and a set of non-verbal events
detected using keypad buttons. The spoken language
enables users to command a humanoid in Japaneseutterances, completely specifying an action to be
executed. Commands in the spoken language can be
modified by nonverbal events.
An action command in RUNA consists of an action
type such as walk, turn, report, and lowertemp (forlowering the temperature setting) and action parameterssuch as speed, direction, angle, object and temperature.
Table 1 shows examples of action types and commands.
The action types are categorized into 24 classes basedon the way action parameters are specified in Japanese.
There are more than 300 generative rules for the fullversion of RUNA. These rules allow Japanese speakers
to command robots actions by speech alone. In RUNA,
a spoken action command is an imperative utterance
including a verb to determine the action type and otherwords to specify action parameters. There are more than
250 words, categorized into about 100 groups identified
by non-terminal symbols.
Table 1 Examples of action commands
Type
walk
turn
move
look

Commandwalk_s_3steps
turn_f_l_30deg
move_m_r_2steps
look_f_lraisetemp

raisetemp_room_2deg

settempsettemp_aircon_22deg

query

query_aircon_all

Table 2 Button event and action parameters

English Utterance
Take 3 steps slowly!
Turn 30 deg. left
quickly!
Move 2 steps right!
Look left quickly!
Raise the temperatureof the room by 2
degrees!
Set the air-conditioner
temperature around 22
degrees!
Report the status of
the air-conditioner!

durationcount

distance

distance

turn etc.

angle

angle

look etc.
raise/lowertemp-temperature

←
Left
←
Left
←
Left
empty

↑
up

buttonspeed
direction
speed
direction
speed/target-

Moderate

↓
down

→
right
→
right
→
right

query

repeat

Cue

Fast

Slow

Fig. 1 Key assignment for action parameters

III. USER STUDY METHOD
We conducted a user study of a humanoid which canbe commanded in RUNA using a remote interface. Fiveusers who had experienced another version of RUNA
and 15 novice users (male and female, aged 13-30
years) commanded the robot to achieve two different
tasks: exploring a remote room and operating an air
conditioner in the room. Before commanding the robot,
the users watched a short demonstration movie for 70
seconds and read an eight-page document illustrating
how to give commands in RUNA in diagrams andfigures for five minutes. Then, we explained them how
to give spoken and multimodal commands showing the

In RUNA, non-verbal events modify the meaning of
spoken commands. They convey information aboutparameters of action commands. Table 2 shows
examples of non-verbal events; users can use keypadbuttons to specify action parameters values instead of©ISAROB 2011

action typesidestep/walk
etc.

323

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

same document. They were allowed to practicecommanding the robot for up to 20 minutes.
In order to test their comprehension and competence,
we also gave them a comprehension test and asked themto give some extra spoken and multimodal commands
precisely as printed in sheets of paper. In addition, they
were asked to answer questions about the robot and ourmultimodal language at the final stage.

time to look at the diagrams.
Table 6 shows that the users were poor at conveying
one of five values by the duration of a button press
action even after achieving their tasks, while it wasstraightforward for them to give parameter values bypressing a button twice or three times. The users
selected quick/fast actions most frequently and there
were more multimodal commands to turn the robot to
the right than to the left for some reasons (Table 7).
After some practice, most of the users spoke clearlyand fluently in most cases. However, they failed toconvey action types and/or parameter values by speech
for several reasons. Some users hesitated to givecommands including many words a few times. Seven
users failed to change the temperature setting of the air
conditioner using a wordy speech only command,saying “change the temperature setting of the air
conditioner to 23 degrees” and failed instead of giving asimpler multimodal command, pressing a button twiceand saying “lower the room temperature!” There werealso ambiguous and ungrammatical commands such as
a button press action followed by an utterance “lower it!”A few users tried to modify or restate a commandand failed while the robot was executing an action.
Some users used words which are not included in thelexicon of RUNA. Some spoken messages were utterly
clear and fluent but misrecognized by the speech
recognition system. Finally, Table 8 shows some
important comments from users.

IV. RESULTS
Table 3 summarizes the data of each user. Users 1-15
commanded the robot in RUNA for the first time, whileusers A-E had already contributed to our previous
studies. All the users completed their tasks: theyanswered three questions about the remote room
correctly and switched on, changed the temperature
setting, and switch off the air conditioner as instructed.
As shown in Tables 4 and 5, they gave many spoken
and multimodal commands to move forward and turnthe robot to explore the virtual remote home. They often
repeated to move the robot forward or turn it to thesame direction as if to look how the camera view image
on the screen changed. Some users gave repetitioncommands to do so by pressing the “repeat” button in
Fig. 1, but the other users did not. For some types ofactions there were more multimodal commands thanspeech only commands (Table 5). Table 6 shows how
the users specified angles and distance values. About
70% of the commands were given within ten seconds
after the previous command was completed.

Table 4 Modality choice for parameter values
type

Table 3 Representative data of each user
ID
SR
nc nw
time
test
Q1 Q2 Q3 Q4
A 100
42
1.0
7:30
8
6
7
M
Y
B 100
48
2.3
9:55
8
4
3
M
N
C
97
33
3.4
11:16
8
4
4
M
Y
D 100
21
2.4
6:07
7
2
3
S
Y
E 100
41
2.5
10:00
10
4
3
M
Y
1 100
36
1.4
8:20
8
6
4
M
N
2 100
31
2.0
12:00
8
4
6
M
Y
3 100
23
2.1
10:37
8
2
3
M
Y
4 100
33
2.0
19:50
10
5
5
M
Y
5 100
38
1.9
9:45
10
5
5
M
Y
6 100
43
1.5
12:10
9
5
4
S
Y
7 100
27
2.1
8:05
10
4
5
M
Y
8 100
26
2.9
8:03
10
4
4
S
Y
9
99
74
1.9
23:06
8
6
7
M
Y
10
98
43
1.8
10:43
9
5
5
M
Y
11
97
31
3.8
11:35
6
6
6
M
Y
12
96
31
4.2
9:20
7
4
4
S
Y
13
96
25
3.7
11:10
7
2
2
S
Y
14
91
23
2.2
13:30
6
4
5
M
Y
15
90
31
2.5
11:46
8
3
5
S
Y
ave
35
11:20
8.3
4.3 4.5
SR: Command success ratenc: Number of given commands
nw: average number of words
time: task completion time
test: comprehension test result before the tasks (ten questions)Q1. Did you command the robot in a natural way? (7 pt. scale)
Q2. Was it easy to command the robot? (7 pt. scale)
Q3. Which do you prefer, spoken or multimodal commands?
Q4. Is this robot helpful for you? (Do you want it?)

turn
walk
&
move
forwardduration count speech default
85
8
49
17
18
31
29
30
36
16
109
34
1
5
1
-

Table 5 Number of commands given by users
action typewalk/forward
backwardturnsidestep
look
look aroundswitch on/off
query
settemplowertemp
repetition

spoken
56
2
92
18
12
10
45
15
18
1
-

multimodal
174
1
147
24
9
0
4
-

button
11
53

total
230
3
239
42
21
10
45
26
18
5
53

Table 6 Success rates of parameter specification
modality
durationcount

At the beginning, some of the users consulted theeight-page document, but they did it less frequently atthe end. Some users seldom turned the pages or took

©ISAROB 2011value
0-30 deg.
31-60 deg.
61-180 deg.
0-9 steps
10-50 steps
0-99 cm
1-3 m“much”

324

Value
0-150[ms]700-1300[ms]1300-2000[ms]24Success rate (%)
76.2
81.0
47.6
100
100

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 7 Parameter specifications using a button
parameter
speed
direction

value
fast
moderate
slow
left
right

values while in this study users had to specify two or
three values at once without omission. Therefore, itwould be better if one can specify values one by one
and leave out parameters which are not important. The
language will be more natural if robots can infer users
intentions based on the context. In fact, novice users
often left out words whenever it was natural in daily
communication. In addition, beginners may hesitate orrestate commands, so life supporting robots must beable to deal with hesitant or halting spoken commands.
We think that the current version of RUNA is toocomplicated for novice users and has some defects
which make the language a little awkward. The
language should cover as many natural verbal
commands as possible. We also suspect that the eight
page document might have been a little confusing. We
should not force users to learn what words and phrases
they can say.
The results of our studies show that multimodal
commands are advantageous in some ways andpreferred by novice users of life supporting robots. We
have realized a cost effective humanoid novice users
can successfully direct to a position and make operate
an air conditioner. The robot will be more user-friendly
if one can communicate with it in a simpler, morenatural, and flexible language.

specifications
251
81
9
38
75

Table 8 Users’ comments
C1
C2
C3
C4
C5
C6
C7

It was difficult to specify a parameter value by
the duration of a button press action.
It was hard to measure the distance from therobot to a target in the camera image.
It was difficult to specify an angle value
verbally or nonverbally.
It was difficult to command the robot to move
forward and turn.
It was difficult to change the temperature
setting.
I wish if I had more time to practice.
I could not speak to the robot fluently.

V. DISCUSSION
The results show that the users understood thelanguage well and mostly succeeded in giving
commands during the tasks (Tables 3 and 5). They oftenchose a button press action and specified direction,
speed, and temperature values without difficulty (Tables
5 and 7). They preferred to use buttons and thought thatthe robot was helpful (Q3 and Q4 in Table 3). Each user
spoke only one to four words and well avoided speech
recognition errors, slips of the tongue and wordy
commands. We presume that it gets easier for users tospecify action parameter values using a button and they
will choose button press actions more often, sincecognitively speaking it is easier to press a button than togenerate verbal phrases.
It is obvious that they did not think that the language
was a very natural or easy one to communicate with therobot (Q1 and Q2). Surprisingly, this result is worth
than the previous study [5]. Although it is difficult tofind the good reasons in the data, we can point out somedisadvantages of the language which might have caused
difficulties for the beginners. First, there are action
parameter values, such as angles and walking steps,
which are difficult to determine before giving acommand (C2 in Table 8); many users chose short turns
using a single press and 10-30 step walks using a
multiple press possibly because they did not know theright values. They did not know how to turn the robot toa target or how much to move the robot forward toreach a target point. Therefore, robots should allow
users to modify parameter values at any time. It would
be better if one can send a cue to stop the robot at theright place. Second, they were unable to choose among
angle or step values using a single press action (Table 6
and C1 and C3), none the less because they were given
some time to practice. Although users may adapt, there
should be ways around for beginners. Third, we suspectthat it is difficult for beginners to specify two or morevalues in a single button press action, because in theprevious study button press actions specified one or two©ISAROB 2011

AKNOWLEDGMENT
This work was supported by KAKENHI Grant-in-Aid
for Scientific Research (C) (19500171).

REFERENCES[1] Prasad R, Saruwatari H, Shikano K (2004) Robots
that can hear, understand and talk. Advanced Robotics
18-5:533-564[2] Knapp ML, Hall JA (2010) Nonverbal
Communication in Human Interaction. Wadsworth[3] Perzanowski D, et. al. (2001) Building a multimodal
human-robot interface. IEEE Intelligent Systems, 16-1,
pp. 16-21[4] Jurafsky D, Martin JH (2000) Speech and Language
Processing. Prentice Hall
[5] Oka T, Abe T, Shimoji M, Nakamura T, Sugita K,
Yokota M (2008) Directing humanoids in a multi-modal
command language. The 17th International Symposium
on Robot and Human Interactive Communication[6] Oka T, Abe T, Sugita K, Yokota M (2009) RUNA: amulti-modal command language for home robot users.
Journal on Artificial Life and Robotics 13-2: 455-459[7] Oka T, Abe T, Sugita K, Yokota M (2009) Successrates in a multimodal command language for homerobot users. Journal on Artificial Life and Robotics 142:219-223[8] Oka T, Sugita K, Yokota M (2010) Commanding ahumanoid to move objects in a multimodal language.
Journal on Artificial Life and Robotics 15-1:17-20

325

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A multimodal language to communicate with life supporting robots
through a touch screen and a speech interface
T. Oka, H. Matsumoto, and R. Kibayashi
College of Industrial Technology, Nihon University, 1-2-1 Izumicho, Narashino, Chiba, 275-8575 JAPANTel : 81-47-474-9693; Fax : 81-47-474-2669
oka.tetsushi@nihon-u.ac.jp
Abstract: This paper proposes a multimodal language to communicate with life supporting robots through a touch
screen and a speech interface. The language is designed for untrained users who need support in daily lives from costeffective robots. In this language, the users can combine spoken and pointing messages in an interactive manner inorder to convey their intensions to robots. Spoken messages include verb and noun phrases which describe intensions.
Pointing messages are given when users finger-touch a camera image, a picture containing a robot body, or a button
on a touch screen at hand, which convey a location in their environment, a direction, a body part of the robot, a cue, a
reply to a query, or other information to help the robot. This work presents the philosophy and structure of thelanguage.
Keywords: Life-supporting robot, multi-modal language, speech, touch screen, human-robot interaction
RUNA [8-12] is the first multimodal language
designed for communication between untrained people
and cost effective robots. In the language, one cancommand a robot by combining a spoken message inJapanese and nonverbal message such as a hand gesture,body touch, and button press action. In some recent user
studies, novice users were able to successfullycommand a robot and achieve some tasks without a longtraining. Many of them preferred multimodal commands
to single modal spoken commands. The results of thesestudies imply that more experienced users would be
able to command robots combining verbal andnonverbal messages more efficiently and successfully.
In addition, since the language is designed so that one
can give a command on a context free basis without
ambiguity, one can build a responsive commandinterpreter at low cost.
Although most of the commands by the novice users
were successful, there were some human errors as well
as system errors. Many of them failed to give spoken
commands with many words which specify a robot
action in detail. It looks also difficult for beginners toconvey multiple values in a single nonverbal message.
For example, those users who commanded a robot
simultaneously conveying a speed, direction, and angle
in a gesture or button press action thought that it was
harder to communicate in the language than the otherusers. Therefore, the efficiency of RUNA seems to be an
obstacle for beginners.
Another problem of the current version of RUNA isthat it does not have a means to point at an object or alocation. For this reason, users had to direct a robot to alocation or an object in an indirect manner using spoken
words, buttons and gestures. Direct pointing at locations
and objects would make it much easier to convey
intentions to life supporting robots.
Based on the advantages and disadvantages of
RUNA, this paper discusses a multimodal language inwhich one can communicate with life supporting robots

I. INTRODUCTION
It is expected that life supporting robots will help
people in their daily lives in near future. Such robots
must be easy to communicate with those untrained
including elderly and disabled, and particularly thosepeople must be able to learn how to communicate with
their robot in a short period of time without much effort.
On the one hand, although a conventional user interface
using pull-down menus and buttons is a cost effective
choice, it is difficult to build an easy-to-learn naturalinterface with life supporting robots that are given manykinds of tasks. Unless users learn many short-cuts, they
have to go through a long sequence of choosing among
menu items or buttons. On the other hand, it will be along journey to realize robots that can speak andcommunicate like humans. Such robots will needsophisticated sensors to collect information andpowerful computers to recognize different situations,perceive verbal and nonverbal messages from theircounterpart, and disambiguate them based on various
knowledge sources[1-3]. Therefore, we predict that it
will take decades to develop affordable robots one canconvey intentions to in the same way as to humans.
Thus, in the near future we will need an artificial
language for natural communication with robots.
In human face-to-face communications, nonverbal
messages play a great role [4]. They can fill in gaps inverbal messages and convey some kind of information
more efficiently than explicit words. Since robots areoften given physical tasks, nonverbal messages are
particularly important for smooth and efficient
communication between humans and robots. For thisreason, nonverbal and multimodal communications
between humans and robots are subject of current
research [5-7]. However, there are few arguments on
multimodal or nonverbal languages between humans
and robots.

©ISAROB 2011326

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

through a speech interface and a touch screen, both ofwhich are common and not expensive in recent years.
The language allows users to communicate with a robot
more slowly and in a more interactive manner than
RUNA. Using a touch screen at hand, they can point at
a location or an object in a camera image and send cues,
pieces of advice and parts of intentions to their robot bytapping on a button. In the following sections, we
describe the principles and structures of the language.

value by interpreting verbal and nonverbal messages
from the user. In our language, users can leave out
optional parameter values if they do not care about
them; if one does not care about the speed, one does notneed mention it. As seen in Table 1, all the types ofintension belong to one of the classes, such as action,cue, and advice.

II. COMMUNICATION
The multimodal language proposed in this paper will
be developed so that untrained novice users should be
able to communicate with a life supporting robot
without taking much time to learn about the language
and their robot. Based on the results of studies of
RUNA, we predict that an affordable user friendly life
supporting robot will be realized if we develop a good
multimodal language. Above all, a touch screen and aspeech interface should provide a good means for usersof such a robot. For this reason, the language isstrategically-designed to utilize such devices. Figure 1
depicts a touch screen showing windows for interactionsin the language. The camera image window shows real
images from a camera on the robot in real time. Somebuttons appear in the button window, which guide users
to convey intentions. Users may point at a location or abody part of the robot and use gestures on the screen.
The language is designed based on the following laws:
1. Users must be able to convey their intensions in
an interactive manner.
2. Users must be able to send messages at any
time.
3. Users must be allowed to speak in as natural a
way as possible.
4. A touch screen must help users as much as
possible.
5. Users must be able to point at 3D positions or
locations in their environment without difficulty.
6. Users must be able to point body parts of theirlife supporting robot.
7. Whenever necessary, buttons must appear on
the touch screen.
8. Users must be able to use gestures to convey
their intensions.
9. The touch screen must display messages to help
users and realize smooth communications.
10. Robots must send spoken messages to help
users.

Fig. 1 Touch Screen Windows
Table 1 Intension RepresentationClassactionTypemove forward
turn/move
aside
go to
come toturn topick upbring
place
push / pull
look
grasp
releaserotate wrist
move hand
move object
throw away
vacuum- cleancue

advicedevice opreportsing / dance
start stop
pause
resumecancel ok
subgoalobstacle
landmark
bail out

Parameters
speed(optional)
direction(mandatory) speed(o)destination(m) speed(o)destination(o) speed(o)target position(m)position(m) size(o) hand(o)position(m) size(o) hand(o)position(m)position(m) size(o) hand(o)target position(m)hand(m) size(o) speed(o)hand(m) speed(o)hand(m) direction(m) speed(o)direction(m) hand (o)position(m) destination(m) size(o)position(m) destination(m)type(o) size(o)room(m) location(o) area(o)mode(o)device (m) operation(m) value(m)content(m)genre(o)position(m)position(m)position(m) label(m)direction(o)III. MULTIMODAL LANGUAGE
B. Interpretation
Since inexperienced users may not be able to express
their intensions without effort, our language allows
them to send verbal and nonverbal messages that
contain partial information. In other words, one caninform a robot of the type and mandatory and optionalparameters of their intension bit by bit in separate

A. Semantic Representation
Table 1 exemplifies intensions users can convey torobots in the language. Intensions are identified by their
type and parameter values. For instance, an intension tostart a robot moving forward slowly is formallyrepresented by a type move forward and a speed valueslow. The robot must identify the type and parameter©ISAROB 2011327

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

messages (Fig. 2). For instance, a spoken message “turn”
brings a left button and a right button on the touch
screen to help the user to convey a direction. The robot
also sends spoken messages and screen messages toinform the user of what has been received and what
information is needed. Unmentioned parameter values
are inferred by the robot based on the default values andcontext. In addition, one can restate a type or parametervalue at any time before the robot starts its action.

such as kono/kore (this) and koko (here). For example,
one may give commands like “place the bottle here,”“turn to this quickly,” “pick up this can,” or “move thishere.” Although such words certainly do not help robots
to determine parameter values, one can command arobot in a natural manner by combining a pointing
message and such a word.
Table 2 shows some examples of grammar rules of
our new spoken language. The grammar is similar to
RUNA’s grammar, but it includes more deictic
expressions and less numerical expressions.

V. POINTING MESSAGESPointing messages are such messages that are sent
through a touch screen. A touch on a camera image
conveys a parameter value, e. g. the position of an
object or a location in the robot’s view. One can touch abutton on the screen to send a cue, a parameter value, ora piece of advice. For instance, a touch on a picture of
the robot’s body creates a nonverbal message containing
a location or a body part.
A cue can be sent by a touch on a cancel, start, stop,pause, or resume button. Users can send a cancel cue
and restart commanding at any time, terminate orinterrupt an ongoing action, and resume an interrupted
action by touching a button on the screen. After the typeof an intension is identified in a spoken message,
buttons appear on the screen to allow the user to choose
among parameter values. The user can modify
parameter values at any time before the action
terminates.

Fig. 2 Communication of Intentions
After conveying a complete intension, namely, thetype and mandatory parameter values, one can cue therobot to start by sending a spoken or pointing message.
A cancel cue removes received information and stops
the current action if being executed. While the robot is
executing an action, the user can send a cue, anoverriding parameter value, or advice to the robot.
Table 2 Grammar rules
Rule
S → INTENTION
S → INTETION_PARAMSS → CUES → ADVICEINTENTION → IP1 IT11
INTETION_PARAMS → IP1
IP1 → SPEED SHORTSLIENCE
IP3 → TO_LOCATION
TO_LOCATION → HERE TO
TO_LOCATION → KITCHEN
TO_LOCATION → THIS DOOR TO
IT1 → MOVEFORWARD
CUE → START

Description
intention
parameterscue
adviceintention type andparametersparameters(type1)
speed + shortsilencelocationto herethe kitchen
to this door
move forward
cue to start action

VI. 2D GESTURES
Finger gestures on a touch screen can convey
contours, trajectories, directions, speeds, lengths,
heights, angles, locations, and symbols. First, using
gestures on a camera image and deictic expressions in
spoken messages, one can inform a robot a safe route toa location, a hand trajectory to an object, the contour
and size of an object, a direction, etc. Next, 2D gestures
on a robot picture help users to intuitively communicate
directions, speeds, lengths, angles, body motions,heights, etc. Moving a finger across the body of therobot with a spoken message “put the cup up” is quite anatural way to command a robot. Finally, symbolicgestures substitute for or emphasize spoken words. We
expect that using these gestures users will be able tocommunicate more naturally.

IV. SPOKEN MESSAGESA spoken message of our language is a sentence,phrase, or word in Japanese to partially convey anintension that can be represented as in Table 1. A
sentence includes a word or phrase denoting an
intension type, so one can specify the type and one ormore parameter values in a sentence like “turn to theleft slowly.” It is also possible to send a single phrase
message that contains only a parameter value or type.
Thus, a beginner can convey an intension bit by bit
slowly. It is also easy to modify parameter values thathave been already sent.
The lexicon of the language includes deictic words©ISAROB 2011

VII. INTERACTIONSThis section illustrates some example of interactions
between a robot and its user.
Interaction A (moving to a location)
U: Turn…
R: Left or right? (Displays buttons)
U: (Touches on the “right button”)
R: OK! (Starts moving and displays new buttons)
U: (Touches on the “slow” button)
R: (Slows down)

328

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

and results of some earlier studies on RUNA, andallows beginners to convey their intentions on a moreinteractive basis. A touch screen will help them to selectamong options and interact with a robot in an intuitive
manner. At this moment, we are developing a test bed
for user studies of the new multimodal language. Our
future work include user studies of life supporting
robots based on the language, design of a multimodal
language integrating speech, 3D gesture, body andscreen touch, and other modalities.

U: (Touches on the “stop” button)
R: OK!
U: (Points at the camera image.)
R: (Displays a mark on the point.)
U: Go here!
R: Shall I go here? (Displays a “start” button)
U: (Touches on the “start” button.)
R: OK! (Starts moving and displays buttons.)
Interaction B (moving an object to a location)
U: (Points at an object)
R: (Displays a mark on the object)
U; (Points at a location)
R: (Displays a mark at the point)
U: Move this here!
R: Shall I move this here? (Displays a start button)…

AKNOWLEDGMENT
This work was supported by KAKENHI Grant-in-Aid
for Scientific Research (C) (22500179).

REFERENCES

Interaction C (throwing away a bottle)
U: (Points at a bottle on the floor)
U: Uh, this bottle.
R: This bottle? (Displays a mark)
U: Throw it away!
R: Shall I throw this away? (Displays a “start” button)…

[1] Prasad R, Saruwatari H, Shikano K (2004) Robots
that can hear, understand and talk. Advanced Robotics
18-5:533-564[2] Jurafsky D, Martin JH (2000) Speech and Language
Processing. Prentice Hall
[3] Bos J, Oka T (2007) A spoken language interface
with a mobile robot. Journal of Artificial Life andRobotics 11-1:42-47[4] Knapp ML, Hall JA (2010) Nonverbal
Communication in Human Interaction. Wadsworth[5] Perzanowski D, et. al. (2001) Building a multimodal
human-robot interface. IEEE Intelligent Systems, 16-1,
pp. 16-21[6] Iba S, Paredis CJJ, Adams W, Khosla PK (2004)
Interactive multi-modal robot programming, The 9thInternational Symposium on Experimental Robotics(ISER ’04), pp. 503-512[7] Igarashi T (2008) User Interface for Robots, Journalof Robotics Society of Japan, Vol. 28, No. 3, pp.246248 (in Japanese)[8] Oka T, Abe T, Shimoji M, Nakamura T, Sugita K,
Yokota M (2008) Directing humanoids in a multi-modal
command language. The 17th International Symposium
on Robot and Human Interactive Communication[9] Oka T, Abe T, Sugita K, Yokota M (2009) RUNA: amulti-modal command language for home robot users.
Journal on Artificial Life and Robotics 13-2: 455-459[10] Oka T, Abe T, Sugita K, Yokota M (2009) Successrates in a multimodal command language for homerobot users. Journal on Artificial Life and Robotics 142:219-223[11] Oka T, Sugita K, Yokota M (2010) Commanding ahumanoid to move objects in a multimodal language.
Journal on Artificial Life and Robotics 15-1:17-20[12] Oka T, Abe T, Sugita K, Yokota M (2011) User
study of a life supporting humanoid directed in a
multimodal
language.
The 16th International
Symposium on Artificial Life and Robotics (AROB11)[13] Oka T, Sugita K, Yokota M (2009) Spoken
command language to direct a robot cleaner. FAN2009

Interaction D (changing the temperature setting)
U: The, uh, air conditioner…
R: Shall I operate the air conditioner? (Displays
buttons)
U: (Touches on the “temperature” button)
R: (Displays the current temperature setting and buttons
to change the temperature)
U: (Changes the temperature and cues OK)

VIII. DISCUSSION
In general, direct pointing and speech are among theeasiest input methods for most users. Our language
allows beginners, who cannot efficiently combine
pointing gestures and speech, to convey their intentions
slowly and interactively. Using key words and phrases
in Japanese, one can inform a robot of the type of an
intention without using a keyboard shortcut or selecting
among menu items, and then convey parameter values
guided by the robot. Experienced users of our language
will be able to achieve their goals more quickly bysending spoken and nonverbal messages.
Needless to say, it is difficult to specify locations,objects, and body parts without pointing at or touchingthem. Spoken commands to life supporting robots tend
to be wordy and tongue-twisting [12, 13]. In addition,
the use of nonverbal messages instead of verbal
messages will often decrease cognitive loads.

IX. SUMMARY AND FUTURE WORKThis paper proposed a multimodal language and auser interface for untrained users of life supporting
robots. The language is designed based on experiences©ISAROB 2011329

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Some Consideration on User Interface Switching Functions for the
Weaker at IT
Shinichi Inenaga1, Kaoru Sugita2, Tetsushi Oka3, Masao Yokota2
1

Graduate School of Fukuoka Institute of Technology, 3-30-1, Wajiro-Higashi, Higashi-ku, Fukuoka, 811-0295,
Japan
2
Fukuoka Institute of Technology, 3-30-1, Wajiro-Higashi, Higashi-ku, Fukuoka, 811-0295, Japan
Nihon University College of Industrial Technology, 1-2-1 Izumicho, Narashino-shi, Chiba 275-8575 Japan1

Tel : 81-92-606-4965; Fax : 81-92-606-4965
mgm08004@bene.fit.ac.jp，2{sugita, yokota}@fit.ac.jp, 3oka.tetsushi@nihon-u.ac.jp

Abstract: We have already proposed a new concept of ‘universal multimedia access’ intended to narrow thedigital divide by providing appropriate multimedia expressions according to users’ (mental and physical)
abilities, computer facilities and network environments. Previous works, have evaluated some types ofmultimedia user interfaces according to users’ (mental and physical) abilities, computer facilities and network
environments. In this paper, we discuss the user interface switching functions.
Keywords: Multimedia, User Interface, Switching Functions.

switching functions and propose a concept of user
adaptive interface for UMA.

I. INTRODUCTION
Recently, immense multimedia information has
come to be exchanged on the Internet, where 3DCG,
video, image, sound, and text are involved in various
circumstances with terminal devices, networks andusers different in their competences and performances.
This fact may easily lead to ‘digital divide’ so called
unless any special support is given to the weaker.
The universal design concept is proposed to support
handicapped people in their social activities1. In thecomputer science field, the universal web2 has been
proposed to evolve this concept. However, this does notsupport to switch the contents, media and its quality ofservice (QoS) function to work the devices and network
environments in their full performances. On the other
hand, many studies about the QoS function proposed to
optimize the video quality for priorities on users’
requests3. These studies focused on performances ofdevices and network environments but neither users’abilities nor contents. Of course, there were also several
studies on ‘universal multimedia access (UMA)’ butthey could not narrow the digital divide because they
concerned ‘content switching’ only4.
Considering this fact, we have already proposed a
new concept of UMA and its switching functions5
intended to narrow the digital divide by providing
appropriate multimedia expressions according to users’(mental and physical) abilities, computer facilities andnetwork environments. In this paper, we redefine these©ISAROB 2011

II. UNIVERSAL MULTIMEDIA ACCESS
The digital divide is caused by the differences inusers’ personal competences, computer facilities andnetwork environments with such detailed items asfollows.
(1) Personal competence: sight ability, hearing
ability, handling ability, language ability, computer skill
and culture,(2) Computer facility: processing power, resolution,color quality, sound quality and battery life,(3) Network environment: bandwidth availability,
specification and transfer mode.
Therefore, multimedia information is necessarily
accompanied by switching user interface, media and
QoS parameters reflecting these differences. Here, we
present a new approach to UMA for handicappedpeople to work their devices and network environments
in full performances. Our purpose is exclusively todevelop a new mechanism for switching appropriately
user interfaces, media and QoS parameters based onsuch a concept as shown in Fig.1.

III. SWITCHING FUNCTIONS
UMA is to selectively provide three kinds ofswitching function, namely, user interface switching330

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

IV. USER INTERFACE SWITCHING(UIS), media switching (MS) and QoS switching (QS).
Fig.2 shows these switching functions working asfollows:(SF1) UIS: switch to user interfaces (UI) appropriate
for users’ competences and display devices,(SF2) MS: switch to media appropriate for users’competences, performances of terminal devices andnetworks,(SF3) QS: control media qualities appropriate forusers’ competences and terminal devices.
These functions are applied in the ascending order(from SF1 to SF3) at beginning to play multimedia
information or in the descending order at playing.

UIS sets up the following items according tocomputer skill and computer facilities.
(U1) Writing style appropriate for language ability,(U2) UI type and annotation option appropriate forcomputer skill,(U3) Media size, font size, number of media andnumber of characters appropriate for display device size.
Additionally, I/O function is reflected by the users’
disability.
1. Operations and MediaUI provides for operations and media according tocomputer skill and computer facilities. Computer skill isgraded to select operation as follows.
(G0) No Knowledge about Computer: Unable tooperate any computer functions.
(G1) Computer Beginner: Able to startup an
application software such as Web browser and play
media.
(G2) General Web User: Able to operate general
Web pages and select to play a media.
(G3) Internet Expert: Able to use efficiently
interactive online applications such as a search engine.
On the other hands, computer facilities arecomposed by some components and classified to 4
levels (None, Low, Middle, High) to setup media.
2. User Interface
A. Template for User Interface
UI is different at each level depending on computer
skills and facilities. Considering such differences, 12
types of UI are expressed in a matrix as shown in
Table.1.
Computer beginners are supposed to selectBroadcast Operation (BO) so as to play media
according to the program without complicatedoperations. The user can get information just likewatching TV because it is not necessary to operate any
application software fundamentally. Choice Operation(CO) is intended for general Web users so as to selectmedia only. But it takes user much time to select one
from a lot of media. Search Operation (SO) is supposed
to support Internet experts by providing a keyword
search function.
Low power terminals are supported to play AA andtext with only low graphics power and narrow
bandwidth of network. Middle power terminals are to

Fig.1. Universal Multimedia AccessFig.2. Switching Functions©ISAROB 2011

331

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011display contents such as combinations of still image andtext. High power terminals play video requiring not only
high power CPU but also high power Graphic device.

readability, including such functions as to filter out
harmful media contents.
3. Approach to User Interface Switching
In order to introduce UIS, we focused on CO
because willing users are supposed to perform thisoperation driven by necessity. On the other hand, they
will not use any operation without CO. From this pointof view, CO is applied to UIS switching from a current
UI to desired one and controlling types of UI withbuttons just like TV remote control.

B. Layout of MediaA layout is used to put media on UI and to specify
the display region, display size and number of media for
resolution of the terminal device and the media. Typicalresolutions in terminal devices are shown in Fig.3 andthe numbers of available media are affected by theseresolutions. These relations enable a layout to specify
the display position according to the display size and thenumber of media. Typical devices can display the
number of media as follows:

V. Implementation
Our ideas were implemented as Flash applications
running on a web browser as shown in Fig.4. These
applications conduct a person the way from the
Fukkodaimae station to Fukuoka Institute ofTechnology providing abovementioned 12 UIs. TheseUIs can be switched to a desired one using ‘software
remote controller (SRC)’ as shown in Fig.5. SRC is
supposed for general Web users and over so as to selecta UI with simple button operation because other users
would not like complicated operations.

(Ex.B1) Cellular phone (Resolution: 240 x 320[pixel])Character (10.5pt): 374
Image (96 x 120 [pixel]): 6
Video (QCIF: 176 x 144 [pixel]): 1(Ex.B2) PDA (Resolution: 800 x 600 [pixel])
Character (10.5pt): 2394
Image (96 x 120 [pixel]): 40
Video (QCIF: 176 x 144 [pixel]): 1(Ex.B3) Notebook computer (Resolution: 1024 x
768 [pixel])
Character (10.5pt): 3942
Image (320 x 240 [pixel]): 9
Video (DV: 720 x 480 [pixel]): 1(Ex.B4) Desktop computer (Resolution: 1280 x
1024 [pixel])
Character (10.5pt): 6643
Image (320 x 240 [pixel]): 16
Video (720p: 1280 x 720 [pixel]): 1

Table 1. User Interface According to User’s Operati
on and Types of MediaC .Expression of MediaAn expression is to facilitate the options of writing
style, Kana-Kanji conversion, alternative media andlanguage in order for better readability.
(Ex. 1) For children (supposed to be lower inlanguage ability): simple Kana text with notes(Ex. 2) For aged person (supposed to be higher inlanguage ability): replacement of loan words by
Japanese traditional words
In addition, cross-media switching, for example text
to video conversion, has been provided for better
Fig.3. Resolution of Terminal Device©ISAROB 2011332

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig.4. Implemented User Interfaces

REFERENCES[1] Ronald L. Mace, Graeme J. Hardie, and Jaine P.
Place (1991), Accessible Environments: Toward
Universal Design, The Center for Universal Design,
North Carolina State University, USA[2] Kazuhiko Yamazaki (2001), Achievement of
universal Web (1) (in Japanese). Design study research
48:330-331[3] Mohan, R, Smith, J.R and Chung-Sheng Li (1999),
Adapting multimedia Internet content universal access,IEEE Multimedia Transcations 1(1):104-114[4] Fernando Pereira and Ian Burnett (2003), Universal
Multimedia Experience for tomorrow, IEEE Signal
Proccesing Magazine, IEEE 20(2): 63-73[5] Y. Maeda, K. Sugita, T. Oka, and M. Yokota (2008),
Proposal of a New Concept of Universal Multimedia
Access, Proc. 13th International symposium on
ARTIFICIAL LIFE AND ROBOTICS (AROB
13th ’08), OS7-6.
[6] Y. Maeda, E. Tsujimura, K. Sugita, T. Oka, and M.
Yokota (2009), Some consideration on user adaptive
interface for universal multimedia access, Proc. 13th
International symposium on ARTIFICIAL LIFE ANDROBOTICS (AROB 14th ’09), OS13-5.
[7] Y.Maeda, K.Sugita, T.Oka, M.Yokota (2009),
Evaluation of sume user interfaces for universal
multimedia access, Proc. the 19Th Intelligent System
Symposium (FAN2009), F3-3:120-124.

Fig.5. Software Remote Controller

VI. CONCLUSION
In this paper, we discussed the UIS and its UI
appropriate for computer skills and facilities. Especially,
we introduced SRC into UIS for simple button
operation. SRC is simple solution to switch a desired UI
employing UIS function. Currently, we are evaluating
SRC and its UI. In near future, we will construct some
types of contents for evaluating UIS and other switching
functions for UMA.

©ISAROB 2011333

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Signal transmission in multilayer asynchronous neural networks
Wataru Kobayashi1 , Makito Oku2 , Kazuyuki Aihara1,2
1

Graduate School of Engineering, The University of Tokyo, Tokyo 153-8505, Japan
2
Graduate School of Information Science and Technology,
The University of Tokyo, Tokyo 153-8505, Japan(Tel : +81-3-5452-6697; Fax : +81-3-5452-6694)(Email: kobayashi@sat.t.u-tokyo.ac.jp)

Abstract: It is believed that common input to nearby neurons leads to their synchronous spiking. However,recent studies have shown that recurrent neural networks can generate an asynchronous state characterized bylow mean spiking correlations despite substantial amounts of shared input. The asynchronous state is generated
by the interaction of excitatory and inhibitory populations, which is called active decorrelation. Here, weinvestigate the advantage of the active decorrelation on signal transmission in multilayer neural networks. The
results of numerical simulations show that the active decorrelation is suitable for transmission of rate code
because it can suppress the layer-by-layer growth of correlation.
Keywords:Neural Networks, Asynchronous States, Active Decorrelation.

1 Introduction
It is thought that brains process information by the temporal and spatial patterns of neuronal firing. Previous studies showed that synchronous firing plays functional roles
such as binding information. However, a recent experiment[1] has shown that the correlation of neuronal firing in monkey’s V1 is extremely low, and a theoretical study[2] hasproposed the mechanism that can generate an asynchronous
state although neurons share a large amount of input. The
asynchronous state is generated by the effect of balanced
excitatory and inhibitory populations, which is called active
decorrelation. In this study, we investigate the advantage ofthe active decorrelation for information processing by numerical simulations. We investigate how the active decorrelation improves transmission of rate code in multilayer
neural networks.

2
2.1

Methods

Figure 1: Transmission of rate code in a multilayer network.
FR, firing rate.

2.1.1 Neuron Model
Here, we use McCulloch-Pitts model[3], which is the simplest and popular neuron model. The model outputs 1 if thesum of input from other neurons exceeds the threshold . If
not, it outputs 0. The model’s equation can be described asfollows:

Model Description∑xi (t + 1) = 1[
wij xj (t) − ],(1)
As mentioned earlier, we consider multilayer neural netj
works that transmit rate code (see Fig. 1). This network
where wij is the conection strength from jth neuron to ith
structure corresponds to the hierarchical organization in theone,  is the threshold, and 1[u] is the step function:
real brains. For example, visual information is transmitted
through retina, LGN, V1, V2, and so on. Although it is
known that brains send top-down signals to process infor{
1 (u ≥ 0)
mation effectively, here, we do not consider such feedback
1[u] =.
(2)0 (u < 0)connections for simplicity.

©ISAROB 2011334

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Network

2.1.2

The network architecture of our model is shown in Fig. 2.
X is external input consisting of excitatory neurons, E denotes an excitatory population, and I denotes an inhibitory
population. The number of neurons in each populations is
N=1000, and neurons are connected with probability p=0.2.
The layers correspond to the regions in Fig. 1. The interlayer connections of the model in Fig. 2 are only from excitatory populations because it is known anatomically that axons of excitatory neurons are longer than those of inhibitory
neurons. Axons mean a part of neurons which play a roleof connection. Detail of parameter settings are the same to
the Renart’s study[2].

Ei (t). We consider the average of rij over all the pairs in a
population as the correlation of the population.
2.2.3 Time Response
Vreeswijk et al.[4] mentioned that the response time of the
population rates is shorter than the time constant of single
neurons in the balanced state. Here, we check the phenomenon by changing the firing rate of external input mX
according to a sine curve and measuring the lag of the peaksof population rates between layers.

3 Simulation Results
3.1 Transmission of Rate CodeThe response against sine input is shown in Fig. 3. Eachline means each layer’s firing rate. The active decorrelation
is ON in Fig. 3 (A) and OFF in Fig. 3 (B). We change
the parameter  , which is the relative time constant of inhibitory neurons against excitatory ones, to switch between
active decorrelaion ON and OFF.  is set to 1 in Fig. 3 (A)and 2 in Fig. 3 (B). The system transmits rate code clearly
when the active decorrelation is ON.

Figure 2: Schematic of the network architecture.

2.2

Evaluation

2.2.1

Rate CodeRate code is one of the important information representations in brains. In this study, we consider mean firing ratethat is the number of firing neurons divided by the population size. So the range is [0,1]. In other words, a neuronal
population represents one scalar value. Here, we investigate
the efficiency of transmission of rate code signals when theactive decorrelation works effectively (active decorrelationON) and when it does not (active decorrelation OFF).
2.2.2

Correlation

We evaluate whether the system is in synchronous states
or in asynchronous states by calculating correlation of neuronal activities in each layer. The correlation is calculated
as follows:T∑

rij = √(Ei (t) − Ei )(Ej (t) − Ej )
√
,
T
T
∑
∑
(Ei (t) − Ei )2(Ej (t) − Ej )2
t=1

t=1

(3)

t=1

where rij is the correlation between ith neuron and jth neuron, E is the time series of neuronal activities represented Figure 3: Time series of the firing rate of each excitatory
by 1 (firing) and 0 (rest). Ei means the time average of population in response to a sinusoidal input.
©ISAROB 2011335

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

The correlation and raster plots are shown in Fig. 4
where external input mX is fixed to 0.1 to check whether
the active decorrelation creates an asynchronous state. The
blue dots in Fig. 4 represent neuronal firing. Vertical stripes
mean synchronous firing of neurons. You can see that layer
3 is in a highly synchronous state when the active deccorelation is OFF. The active decorrelation suppresses the layerby-layer growth of correlations, and it is suitable for transmission of rate code.

Figure 5: Firing rates in excitatory populations against external input. The active decorrelation is ON.
in Fig. 5. If f ′ (a) > 1, a is an unstable fixed point, andthe firing rate converges to 0 or 1 depending on the initialcondition. If f ′ (a) < 1, a is a stable fixed point, and thefiring rate converges to a. For the above reason, the only
perfect linear system can transmit rate code in infinite layers, and such a perfect one can not exist. So what we can dofor good transmission of rate code is to make the response
curve closer to linear one. This problem of response curves
between layers has been mentioned by Litvak[5].
In this paper, we have shown two problems to transmit
rate code in multilayer networks. One is the layer-by-layer
increase of correlation. The other is the nonlinearity in the
response curves. We have shown that the former problem
can be resolved by the active decorrelation.
3.2 Time Response
We investigate another advantage of the active
decorrelation—time response. As the frequency of externalTime Response in Multilayer
1

Figure 4: Raster plots and the correlations of neuronal firingin multilayer networks.

0.9
0.8

Firing Rate

By carefully checking Fig. 3 and Fig. 4, you can find
0.7
some features. In Fig. 3(A), the rough profile of the sine
0.6
curve is transmitted well but the correct values are not. For
0.5
example, the peaks of the firing rates are about 0.8 in exter0.4
nal input, 0.7 in layer 1, 0.65 in layer 2 and 0.60 in layer 3.
It is thought that the values of the firing rates decay because
0.3
of the deviation from linearity in the input-output relation0.2
ship, as shown in Fig. 5.
0.1
The intersection is about 0.5 in Fig. 5. It is the equi0
librium point because signals converge to the intersection
0
200
400
600
800
1000
Time(ms)
when they are transmitted through layers repeatedly. It iseasy to check if the intersection is stable or not. We denotethe value of the intersection as a and the response curve of Figure 6: Responses to a sinusoidal input with increasing
layer 1 as f . Here, a is about 0.5, and f is the red line frequency. The active decorrelation is ON.

©ISAROB 2011336

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Eigen Frequency of Networks

input mX increases in Fig. 6, the time lag becomes apparent. You can see the lag is several milliseconds between
successive layers in Fig. 7 which is an enlarged view ofFig. 6. In this model, the time constant of one excitatoryneuron is 10 ms, so the populations track the signal changes
quickly as compared with the time constant. A previous
study[4] has shown theoretically that such a quick tracking
occurs in balanced states. The√ study showed that the lag decreases in proportion to 1/ K, where K is the number ofconnections per neuron. We have shown that such a quick
tracking also occurs in multilayer networks.

1
0.9
0.8

Firing Rate

0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
100

Time Response in Multilayer
1

120

140

160

180

200

Time(ms)

0.9

Figure 8: Sympathetic vibration in a synchronous network.
The active decorrelation is OFF.

0.8

Firing Rate

0.7
0.6

brain. We have also observed the existance of characteristicfrequencies in synchronous networks. The networks seemto have a range of resonant frequencies, and it is our future
problem to check the generality of the phenomenon.

0.5
0.4
0.3
0.2
0.1
0
800

Acknowledgements
820

840

860

880

900

This research is partially supported by the FIRST program, Grant-in-Aid for Scientific Research (A) (20246026)
Figure 7: An enlarged view of Fig. 6 in the range from 800 and Grant-in-Aid for JSPS Fellows (21·937) from JSPS.
to 900 ms.
Time(ms)

References
3.3

Characteristic Frequency[1] Alexander S. Ecker, Philipp Berens, Georgios A. KeAnother point that we can find from Fig. 3 and Fig. 4 isliris et al., ”Decorrelated neuronal firing in cortical mithat a synchronous network has a characteristic frequency.
crocircuits”, Science 327, 584 (2010).
In Fig. 4(B), synchronous firing which is represented asvertical stripes is observed in layer 3, and it seems to be pe- [2] Alfonso Renart, Jaime de la Rocha, Peter Bartho et al.,”The asynchronous state in cortical circuits”, Science
riodic. We set external input to various frequencies in the327, 587 (2010).
situation of active decorrelation ”OFF” ( = 2) to check its
periodicity. An interesting phenomenon like sympathetic [3] Warren S. McCulloch and Walter Pitts, ”A logical calvibration is observed in Fig. 8. It is not observed against
culus of the ideas immanent in nervous activity”, Bulhigher frequencies. So we can conclude that the network
letin of Mathematical Biology 5(4), 115 (1943).
with the active decorrelation OFF has a characteristic frequency. It can play a role as oscillator in vivo. Further [4] C. van Vreeswijk and H. Sompolinsky, ”Chaos in neuparameter analysis would be required to check whether the
ronal networks with balanced excitatory and inhibitory
phenomenon occurs broadly.
activity”, Science 274, 1724 (1996).
[5] Vladimir Litvak, Haim Sompolinsky, Idan Segev etal., ”On the transmission of rate code in long feed4 Conclusion
foward networks with excitatory-inhibitory balance”,
The Jounal of Neuroscience 23(7), 3006 (2003).
We have investigated the advantage of the active decorrelation for transmission of rate code in multilayer neuralnetworks. However, the active decorrelation seems incompatible with other codes such as temporal code. So it canbe thought that the brains use both synchronous and asynchronous states depending on the situation and areas of the©ISAROB 2011337

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A two-variable silicon neuron circuit based on the Izhikevich model
Nobuyuki Mizoguchi, Yuji Nagamatsu, Kazuyuki Aihara, and Takashi Kohno

Institute of Industrial Science, The University of Tokyo, 4-6-1 Komaba, Meguro-ku, Tokyo 153-8505, Japan(mizoguti@sat.t.u-tokyo.ac.jp)
Abstract: The silicon neuron is an analog electronic circuit that reproduces the dynamics of a neuron. It is a
useful element for artificial neural networks that work in real time. Silicon neuron circuits have to be simple
and at the same time be able to reproduce rich dynamics in order to reproduce various activities of neuralnetworks with a compact, low-power consumption, and easy-to-configure circuit. We have been developing asilicon neuron circuit based on the Izhikevich model, which has rich dynamics in spite of its simplicity. In ourprevious works, we proposed a simple and low-power consumption silicon neuron circuit by reconstructing themathematical structure in the original model using an analog electronic circuit. In this paper, we propose an
improved circuit in which all of the MOSFETs are operated in the subthreshold region.
Keywords:Silicon neuron, Izhikevich model, mathematical structure-based method, subthreshold

I Introduction
The dynamics of neurons and that of neural networks
have been widely researched. Many mathematical models of neurons have been developed[1, 2, 3] to reconstructtheir dynamics and simulations conducted[4, 5, 6] to reproduce various functions of neural networks such as rhythmic
movements, associative memory, and pattern recognition.
The silicon neuron reproduces the dynamics of a neuron in
an electronic circuit. The network of silicon neurons canoperate in real time regardless of its size, which is difficult
by the simulation using digital computers. In addition, it
can be compact if it is implemented into an analog Very
Large Scale Integrated circuit (aVLSI). For those reasons,the silicon neuron is expected to be applied for real time systems such as hybrid systems, medical devices, and robots.
Basically, silicon neurons have been developed by two
different approaches. The first one is to reproduce only significant neuronal behaviors, which is called the phenomenological method. Silicon neuron circuits designed by thismethod[7] tend to be simple but can only reproduce a few
firing patterns because of its oversimplified dynamics. The
second one is to reproduce the neuronal dynamics minutely
by solving the ionic conductance models, which is called
the conductance based method. Silicon neurons designedby this method[8] can reproduce various firing patterns but
their circuitry is complex and large.
Recently, Kohno[9] have proposed another method, themathematical-structure-based approach, which allows us todesign simple circuits with rich dynamics by reproducing
the mathematical structure of the original models. By using this method, Nagamatsu[10] proposed a silicon neuron circuit (we call it the previous circuit in this paper)
which reproduces the mathematical structure of the Izhikevich model[11]. This silicon neuron circuit reconstructs the
dynamics of the original model with a simple circuit whose
average power consumption is about 15 nW. However, in©ISAROB 2011

the circuit that implements jump of the state in the originalmodel (we call this circuit reset circuit) the voltage variationis relatively large in comparison to other parts of the circuit
operated in subthreshold region. Besides this, the input current that triggers spiking of the silicon neuron needs to beconsiderably small. It is likely to be difficult to produce.
In this paper, we propose an improved circuit with a
new reset circuit in which the voltage swing is about 0.4
V, which is lower than that of the previous circuit by about
2.25 V. In addition, we scaled up the input current by altering a part of the circuit. These modifications are expected tolower the difficulty in aVLSI implementation of this circuit.

II

Izhikevich model

The Izhikevich model consists of two-variable differential equations with jump of the state:dv= 0.04v 2 + 5v + 140 − u + Idtdu= a(bu − v)
dt
if v = 30 mV then v ← c, u ← u + d

(1)
(2)
(3)

where v and u are the membrane potential and an internalvariable respectively. This model can reproduce 20 firing
patterns that neurons in the cortical area of the brain exhibit, by tuning the four parameters: a, b, c, and d[12]. The
parameter a controls the time constant of u, and b represents
the slope of the u-nullcline, the line where the derivative ofu is zero, which is shown in Fig. 3(a). The parameters a andb control the characteristics of the equilibrium points wherethe v-nullcline intersects with the u-nullcline. The parameters c and d respectively determine the membrane potential
and the increment of u when a jump of the state occurs.
These two parameters mainly control the characteristics of338

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

v-nullcline

Vdd

Bump-antibump

Vdd

Reset circuit

circuit
Vvb

Iin

Ibias

M1

u-nullcline

Current mode

Differential pairIntegrator

M0
Vvd

Vin

u+d circuit

v

Vub

M0

M2

M3

M2

M3

M4

Vin

M4

Vss

IT

M1

Vud

Vss

Iout

(a)

Figure 1: Block diagram of our silicon neuron circuit.

Iout

(b)

Figure 2: Schematics of the circuits that produce the nullclines. (a) The bump-antibump circuit that produces the vnullcline. (b) The degenerated differential pair circuit that
produces the u-nullcline.

the burst firing.

III

Designed circuit
20500
u [pA]

u

5
0
-5
-10

2

-60
-40
v [mV]-20

0©ISAROB 2011

-300

-250

-200

-150 -100v [mV]

-50

0

(b)

Figure 3: Comparison between the nullclines of the original model and those of our circuit. (a) The v- and the unullclines of the original model. (b)Those of our circuit thathas the same geometry as the original model even though
they don’t have exactly the same shape.

Iout =

Reproducing v- and u-nullclines

To reconstruct the mathematical structure, we reproduced the v- and the u-nullclines by circuits shown in Fig.
2. In the previous circuit, the v-nullcline was reproducedby two differential pair circuits, but in the improved circuit,a bump-antibump circuit[13] is used because it consumes
half as much power as the two differential pairs to get theparabolic output current of the same scale. We have increased the bias voltage to scale up the input current thattriggers spiking of the silicon neuron, yet the improved circuit consumes as low power as the previous one. It is because the new reset circuit consumes less power than theprevious one by about 9 nW. This cancels the increment of
the power consumption of the alternative v-nullcline circuit
which is about 7 nW. The u-nullcline is reproduced by thesame circuit as the previous one: a differential pair circuitwith source degeneration.The characteristic curves of the vnullcline and the u-nullcline circuits are shown in Fig. 3(b)and their theoretical formulae are described in Eqs. (4) and(5) respectively.
)
(
1
(4)
Iout = Ib 1 −vd)
1 + S4 cosh2 κ(V in−V
2

300

100
-80

(a)

This circuit is designed by using mathematicalstructure-based approach proposed by Kohno[9]. In thismethod, we reconstruct the mathematical structure in theoriginal model using output characteristic curves of simple
analog electronic circuits, which allows us to avoid using
complex circuits to directly approximate the equations in
the original model.

400

200

-15

Mathematical-structure-based approachv-nullcline
u-nullcline

600

10

-20
-100

1

v-nullcline
u-nullcline

15

In this section, we show the design of the improved silicon neuron circuit based on the Izhikevich model comparing to the previous one proposed by Nagamatsu[10]. Figure
1 shows the block diagram of our improved circuit.

Ib−κ(V in−V ud)
1 + exp
2UT(5)

As shown in Fig. 3(b), the width of the v-nullcline of ourcircuit is three times as large as that of the original model
but it doesn’t matter if we scale the membrane potential appropriately. We have to add an amplifier that scales v down
to one-third to output the original membrane potential, butthere is no need to do so if silicon neurons are interconnected via silicon synapses that reproduce the dynamics ofsynapses because their circuit can be designed to accept thevoltage scale of our silicon neuron circuit. Also, in our
circuit the v-nullcline is not exactly parabolic and the unullcline not exactly linear but it doesn’t matter because it
has the same geometrical arrangement as the original one
and we can reconstruct the same mathematical structure if
we configure the parameters appropriately.
3 Current-mode integrator
A current-mode integrator is used to integrate the equation for the variable u because it is represented by theamount of the current from the capacitance that represents
the membrane potential v. Figure 4 shows the current-mode
integrator designed to operate in the subthreshold region

339

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Vdd

Vdd

Vdd

Vdd

Vrb

Iout

M0

M1

M3

M6

M2

M4

M7

Vrd

M2

M1

M10

Vss

M3

Vvs

Vsw

Vin

Vdd

Vout

M4
Vrpb

Vin

Vd
Iout1

Vrvd

Figure 4: Schematic of the current-mode integrator[14].
Iout is temporal integration of Iin − Iout , which represents
the variable u. The time constant depends on C and Iτ ,which corresponds to the parameter a

M5

M6

M7

M8

M5

M0

gnd

Iout2

Vsw

M9

Vrvb

M10 M8

M9

Vss

Vss

(a)

Vss

(b)

Figure 5: Schematics of the circuits that implement theand to be suitable for implementation by relatively fine pro- jumps of the state. (a) The reset circuit that switches v fromcess such as .35 µm[14]. This circuit implements the inte- 90 mV to c mV. Both Vin and Iout are connected to the cagration described as follows:
pacitance that represents the membrane potential v. Voutbecomes high while reset operation. (b) The u+d circuit
dIout
Iτ
=
(Iin − Iout )(6) that increases u by d while the reset circuit is switching thedt
CUTvalue of v. The input voltage Vin is connected to Vout of thewhere Iout is the output current of the integrator that rep- reset circuit. The bias voltage Vsw is the switch that shuts
resents u, Iin is the input current that comes from the u- off either Iout1 that represents the positive d or Iout2 that
represents the negative d.
nullcline circuit, C is the capacitance in the integrator, U
T

Reset circuit and u+d circuit

In the previous silicon neuron circuit, the output voltage of the reset circuit swings from -1.65V to 1V when it
switches v from 30 mV to c, which is relatively large voltage variation in comparison to the other parts of the circuit operated in subthreshold region. This considerably increases the difficulty in layout mask design and potential
troubles in its operation. In the improved circuit, we propose a new reset circuit with a circuit that switches u to u+d(we call it u+d circuit) as shown in the Fig. 5. The reset circuit switches the membrane potential v to c when it exceeds
90 mV (30 mV in the original model). In this operation, theoutput voltage of our reset circuit keeps high, which makes
the u+d circuit increase or decrease the variable u by d. The
voltage swing of the output voltage is about 0.4 V, which islower than that of the previous one by about 2.25 V. The parameter c and d can be configured by the bias voltages Vrvb
and Vrvd in Fig. 5(a) and by Vd in Fig. 5(b) respectively.
5

IV Simulation
We validated our silicon neuron circuit by HSpice circuit simulation using TSMC .35µm CMOS mixed signal
process PDK. Vdd and Vss are 1.65V and -1.65V respectively. The simulation results are shown Fig. 7. We havesuccessfully found the parameter voltages for 17 firing patterns out of 20 that the Izhikevich model produces.

20
v-nullcline
u-nullcline (b=0.2)
u-nullcline (b=0.26)

15

500

5
0
-5

-80

-60

-40
v [mV](a)

We use the bias current Ibias to configure the parameterb as in the previous circuit. Current Ibias is injected into or
taken up from the capacitor that represents the membranepotential v. In the u-v phase plane, it means Ibias moves
the v-nullcline up or down and this can configure the char-©ISAROB 2011

stable node

300
200
stable spiralstable spiral-15

Bias current that controls the position of v-nullcline

400

stable node

-10

-20
-100

v-nullcline (Ibias=-520 pA)
v-nullcline (Ibias=-420pA)
u-nullcline

600

10

u [pA]

4

acteristics of the equilibrium points as the parameter b in
the original model (see Fig. 6).

uis the thermal voltage, and Iτ is the constant current that
represents the parameter a.

-20

100
0

-300

-250

-200

-150
v [mV]

-100

-50

0

(b)

Figure 6: (a) In the original model, the parameter b controls
the slope of the u-nullcline. (b) In our circuit, the parameter
Ibias moves the v-nullcline up and down. The characteristics of the equilibrium points are tuned to be same as in (a).

340

0.1

0.1

0.05

0.05

0

0

0

-0.05

-0.05

-0.05

-0.1

-0.1
-0.15

-0.2

-0.2

-0.2

-0.25
0.3

-0.25
0.3

-0.25
0.3

0.35

0.4
t [s]

0.45

0.5

-0.15

0.35

0.1

0.1
0.05

0.6

v [V]

v [V]

0
-0.05

-0.1

-0.1

-0.15

-0.15

-0.15

-0.2

-0.2

-0.1

0.4

0.45
t [s]

0.5

0.55

-0.25
0.3

0.6

0.4

0.45
t [s]

0.5

0.55

0.1
0.05

v [V]

-0.1

-0.25
0.15

-0.2
0.15

0.2

0.25
0.3
t [s]

0.35

0.4

Class1 excitable

-0.05
-0.1
-0.15
-0.2

0.2

0.25
0.3
t [s]

0.35

-0.25
0.3

0.4

0.1

0.1

0.1

0.05

0.05
0
-0.05

v [V]

0
-0.05

v [V]

0
-0.05

-0.15

-0.1

-0.2
-0.25
0.2 0.25 0.3 0.35 0.4 0.45 0.5
t [s]

0.45
t [s]

0.5

0.55

0.6

-0.1

-0.25
0.3

0.55

0.1
0.05

0

0

-0.05

-0.05

-0.05

-0.1
-0.15
-0.2

0.35

0.4

0.45
t [s]

0.5

0.55

Rebound burst

0.1
0.05
0
-0.05

v [V]

0
-0.05
-0.1
-0.15

0.5

0.55

0.6

-0.1

-0.2
0.55

0.6

-0.25
0.1

0.2

0.3

0.4

0.5

0.6

t [s]

Bistability

Threshold variability

0.1

0.45
t [s]

-0.15

-0.25
0.25 0.3 0.35 0.4 0.45 0.5
t [s]

0.6

0.05

-0.1
-0.15

-0.2
-0.25
0.3

v [V]

0.1
0.05

0
v [V]

0.1

-0.2

0.4

Rebound spike

0.05

-0.25
0.3

0.35

Integrator

-0.15

0.5

-0.2

Subthreshold oscillations

-0.1

0.45

-0.15

-0.15

-0.2
-0.25
0.3

0.4

0.4
t [s]

Spike latency

0.05

0.35

0.35

Class2 excitable

-0.1

0.6

0

0
-0.05

-0.15

0.55

Frequency adaptation

Mixed mode

0.1
0.05

-0.2

0.5

-0.25
0.3 0.32 0.34 0.36 0.38 0.4 0.42 0.44
t [s]

0.6

0.1

-0.1

0.45
t [s]

-0.2
0.35

0.05

-0.05

0.4

0.1

0

0.35

0.35

Tonic bursting

-0.05

v [V]

v [V]

0.55

0

-0.15

v [V]

0.5

-0.05

0

v [V]

0.45
t [s]

0.05

Phasic bursting

v [V]

0.4

Phasic spiking

0.05

-0.25
0.3

References

-0.1

-0.15

Tonic spiking

v [V]

v [V]

0.1
0.05

v [V]

v [V]

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

-0.2
0.35

0.4
t [s]

0.45

0.5

-0.25
0.1

0.2

0.3

0.4
t [s]

0.5

0.6

0.7

Accommodation

Depolarizing after-potential

Figure 7: Produced firing patterns.

V

Conclusion

We proposed an improved silicon neuron circuit basedon the Izhikevich model. The voltage swing of the newreset circuit is about 0.4 V which is smaller than that of theprevious circuit by about 2.25 V. In addition, we redesigned
the v-nullcline circuit and changed the bias voltages to scale
up the input current, yet the improved circuit consumes as
low power as the previous one does. These improvements
are suitable for aVLSI implementation in which MOSFETs
are operated in subthreshold region. We have reproduced 17
firing patterns out of 20 that the Izhikevich model produces.

Acknowledgements[1] A. L. Hodgkin and A. F. Huxley, “A quantitative description of membrane current and its application to
conduction and excitation in nerve,” The Journal of
Physiology, vol. 117, no. 4, pp. 500-544, Aug. 1952.
[2] C. Morris and H. Lecar, “Voltage oscillations in the
Barnacle giant muscle fiber,” Biophysical Journal, vol.
35, no. 1, pp. 193-213, Jul. 1981.
[3] J. L. Hindmarsh and R. M. Rose, “A model of neuronal bursting using three coupled first order differential equations,” Proceedings of the Royal Society B, vol.
221, no. 1222, pp. 87-102, Mar. 1984.
[4] A. A. V. Hill, J. Lu, M. A. Masino, O.H. olsen, R.L.
calabrese, “A model of a segmental oscillator in theleech heartbeat neuronal network,” J. Comp. Neurosci.
10, 281-302, 2001.
[5] E. M. Izhikevich, “Weakly pulse-coupled oscillators,
FM interactions, synchronization, and oscillatory associative memory”, IEEE Transactions on Neural Networks, 10, pp. 508-526, 1999.
[6] C. D. Brody, J. J. Hopfield, “Simple networks for spiketiming-based computation, with application to olfactory processing”, Neuron 37:843-52, 2003.
[7] G. Indiveri, “A low-power adaptive integrate-and-fireneuron circuit, Circuits and System,” 2003. ISCAS 3.
Procedings of the 2003 International Symposium, Vol.
4, pp. 820-823.
[8] M. F. Simoni, G. S. Cymbalyuk, and M. E. Sorensen,“A Multiconductance Silicon Neuron With Biologically
Matched Dynamics,” IEEE Transactions on Biomedical
Engineering, Vol.51, No.2, 2004.
[9] T. Kohno and K. Aihara, “A Design Method for Analog and Digital Silicon Neurons -Mathematical-ModelBased Method-,” AIP Conference Proceedings, Vol.
1028, pp. 113-128, 2008.
[10] Y. Nagamatsu, K. Aihara, and T. Kohno, “An Izhikevich type silicon neuron circuit,” Proceedings of International Symposium on Artificial Life and Robotics
2010, OS4-1, 6th, Feb., 2010.
[11] E. M. Izhikevich, “Simple Model of Spiking Neurons,” IEEE Transactions on Neural Networks, Vol. 14,
No. 6, November 2003.
[12] E. M. Izhikevich, “Which Model to Use for Cortical
Spiking Neurons?,” IEEE Transactions on Neural Networks, Vol. 15, No. 5, September 2004.
[13] T. Delbruck, “Bump Circuits,” Caltech, Pasadena,CA, Tech. Rep. CNS Memo 26, May 1993.
[14] T. Kohno and K. Aihara, “A simple aVLSI burst silicon neuron circuit,” Proceedings of the 2008 International Symposium on Nonlinear Theory and its Applications, pp. 556-559, Sep., 2008.

The study was supported by JST PRESTO program andwas partially supported by a Grant-in-Aid for Young Scientists (A) 19680015 from the Ministry of Education, Culture,
Sports, Science, and Technology, the Japanese Government.

©ISAROB 2011341

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A three-variable silicon neuron circuit
Yohei Nakamura1 , Kazuyuki Aihara1 , Takashi Kohno1 ,1Institute of Industrial Science,
The University of Tokyo, Tokyo 113-8656, Japan(Tel : +81-3-5452-6693; Fax : +81-3-5452-6694)(Email: nakamura@sat.t.u-tokyo.ac.jp)
Abstract: The silicon neuron is a type of artificial neuron implemented with electronic circuit. Previously adesign approach based on mathematical structures under neuronal dynamics was proposed. It is based on themathematical techniques such as phase plane and bifurcation analysis. These methods allow us to implement
silicon neuron with smaller circuit area and to strategically adjust the bias parameter voltages without losing
variety of output patterns. In this study we demonstrate a mathematical-structure-based silicon neuron, which
operates a three-dimensional system. This silicon neuron can generate a firing pattern called square-wave
bursting. In this report we show the experimental results of this silicon neuron. We are planning to makepattern generating network using this silicon neuron and silicon synapses.
Keywords:Silicon neuron, MOSFET, Phase plane, Bifurcation analysis

I Introduction

ation under the presence of device mismatch and noises.
Thus it is hard to compose large silicon neural networks of
The neuromorphic hardware is an electronic system that the conductance-based silicon neurons.
mimics functions of nerve systems. Biological studies reIn previous studies [3], Kohno proposed avealed that nervous systems process information in fundamathematical-structure-based
approach to design silimentally different ways from digital computers. They havecon
neurons.
Qualitative
neuronmodels, such as thegood adaptability to their environment and high robustness.
FitzHugh-Nagmo
[4]
and
the
Hindmarsh-RosemodelsTo investigate the mechanism of such splendid information[5]
cangeneratevariousfiringpatterns
withfewer
paprocessing ability and to reproduce it in artificial systems,
rameters.
However,theirequationsarenotsuitableformany biophysical and theoretical studies have been done
implementation
byelectronic
circuit.
Inthemathematicalabout the neuron and the synapse which are the basic component of the nerve system. Silicon neurons have been pro- structure-based approach, the equations of the qualitative
duced through the efforts to reproduce various properties models are modified so that they can be implemented byelectronic circuit effectively while preserving their mathelucidated by these studies using electronic circuits.
ematical structures utilizing phase plane and bifurcation
There have been two major types of approaches, one isanalyses. In this study we present a mathematical-structurephenomenological approach and the other is conductancebased silicon neuron, which has three dimensional system
based one. Phenomenological silicon neurons are based
equations and designed to produce a burst firing pattern
on extremely simplified neuron models such as the leaky
called square-wave bursting when all of the three variables
integrate-and fire model [1]. They could be implemented by
are activated.
relatively simple and compact circuit because phenomenological neuron models focus on the specific properties ofneurons, thus they are suitable for investigating large silicon neural networks. However, they reproduce limited aspects of neuronal dynamics because these models ignore the
II System equations of our silicon neuron
ionic dynamics in neurons. On the other hand, conductancebased silicon neurons, such as [2] are intended to emulate
the dynamics of ionic channels in neurons. They have theability to generate various firing patterns by adjusting theexternally applied parameter voltages to the circuit. HowThe system equations of our silicon neuron containever, they have drawbacks of complexity in their circuitry three variables, v, n and q. Variable v corresponds to theand a number of parameter voltages to be adjusted. These membrane potential of the neuron, n and q are the variables
points raise difficulty in circuit implementation and oper- that represent an ionic channel and a negative feedback cur-

©ISAROB 2011342

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Figure 1: The n-v phase plane of silicon neuron model(simulation). n-nullcline and q-nullcline are respectivelydq
the set of dn
dt = 0 and dt = 0

Figure 2: Block diagram of our silicon neuron

rents. They are written as follows:
Cv

dv
dt
dn
dt
dq
dt

= fm (v) − g(v) − n − q + Ia + Istim (1)==fn (v) − nTnfq (v) − q
Tq

(2)
(3)

where Ia is an ionic current which is independent of themembrane potential. Current Istim is an externally applied
stimulus current. Constant Cv is the membrane capacitance.
Constant Tn and Tq are the time constants for n and q.
Figure 3: Schematics of f (v) (left) and g(v) (right) generaThe functions fx (v) (x = m, n, q) and g(v) are the tor circuits.
sigmoidal characteristic curves of the differential-pair circuitries, which are expressed in the following forms:fx (v)

=

Mx

g(v)

= S

1
1 + exp(− UκT (v − δx ))1 − exp(− 2UκT (v − ))
1 + exp(− 2UκT (v − ))

(4)
(5)

Equations (1) and (2) comprise the basic excitable system, which can reproduce the same mathematical structures
as various non-bursting neuron models, such as HodgikinHuxley and Morris-Lecar models. Equation (3) comprises
the negative feedback system that generates positive current q into the membrane capacitor while the potential v
is high, and negative current while v is low, whose mechanism cause the square-wave bursting when the basic excitable system has a kind of bistability. Figure 1 shows thev-n phase plane of the basic excitable system of our silicon neuron. When no stimulus current is applied, the membrane potential stays at the stable equilibrium (S) (resting
point). If the stimulus current is small, the system statecannot move over a stable manifold of the saddle point (T)©ISAROB 2011and go back to (S). However, if the stimulus is sufficiently
strong, the system state travels around the unstable node (U)and come back to the resting state. In the next section, we
present the circuit experiment results of our basic excitablesystem.

III Circuit of our silicon neuron
In Fig.2 the block diagram of our silicon neuron isshown. It is composed of differential pair (Fig.3), current
mirror, and current-mode integrator (Fig.4) circuits. The
differential pair circuits make the functional curves of thenullclines, and the current-mode integrator integrates thesystem equations. Each of function modules generates output current which depends on membrane potential v, andthe output currents flow into the membrane capacitor whose
voltage represents v through current mirror or current-mode
integrator.

343

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 1: Operating parameter set
Parameters
VDD
VSS
Vδm
VMm
Vθ
VS
VδIa
VSIa

Figure 4: Schematic of the current-mode integrator circuit
where MOSFETs are operated in the subthreshold condition. This circuit realize the integration in Eqs. (2) and (3).

-25

v(mV)

n-nullcline
v-nullcline

-20

n(mV)

-15
-10
-5
0
5
-200

Values[V]

80
60
40
20
0
-20
-40
-60
-80
-100
-120
-140
-160
-180
-200
-220
-10

3.3
0
−0.01
0.31
−0.13
0.3
0
0.35

Parameters

Values [V]

Vδn
VMn
Vτ bn
Vof stVτVτ dVδIstim
VSIstim

0.09
0.395
0.243
0.39
0
0
0
0.35

-30mV
-40mV
-50mV
-60mV
-70mV
-80mV
-90mV

-5

0

5

10
t(ms)

15

20

25

30

Figure 6: Response of the membrane potential to singlet
pulse stimuli. Stimulus is applied to the membrane capacitor through a V -I transmitter circuit in the VLSI chip,
Figure 5: The v- and the n-nullclines drawn by the voltage which converts negative voltage to positive current. The
clamp system implemented in the same chip with silicon duration of the stimulus pulse is 1.0 ms, and the strength isvaried from −30 mV to −90 mV.
neuron.

1

-150

-100

-50

0
v(mV)

50

100

IV

Experimental results

150

200

Drawing phase plane

Our silicon neuron has the voltage-clamp measurement
system inside the same VLSI chip, which can generate theoutput currents of each functional modules while clamping the membrane potential to a specific voltage. Figure
5 shows the v-n phase plane of our silicon neuron, which
was drawn by this system. In the following subsections we
present experimental results of the basic excitable system inour silicon neuron circuit. The values of parameter voltages
are shown in Table 1.
2

Responses to singlet pulse stimuli

negative voltage because stimulus current is applied via a
V -I transmitter circuit that converts negative input voltage
into positive current. These results demonstrate that our silicon neuron has the first four out of the five properties ofsilent neurons described by Zeeman [6]. (1) a stable equilibrium point exists that corresponds to the resting state. (2)and action potential can be generated in response to an external stimulus, and the size of the response is absolutely
larger than that of the stimuli. (3) a threshold of the stimulus magnitude exists for the generation of action potential.
(Fig.6 shows that the threshold voltage of our silicon neuron
exists between −50 mV and −60 mV.) (4) an action potential returns to the resting state more slowly than its risingphase. (5) refractoriness exists after generation of an action
potential.

Figure 6 shows the behaviour of the membrane potential 3 Responses to doublet pulse stimuli
v in response to the singlet pulse stimuli. The duration of
the input pulse is 1.0 ms and the amplitude δIstim is varied
Figure 7 shows the responses to the doublet pulse stimfrom −90 mV to −30 mV. The value of the amplitude is uli. The duration of the input pulse is 1.0 ms and their in©ISAROB 2011344

v(mV)The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

80
60
40
20
0
-20
-40
-60
-80
-100
-120
-140
-160
-180
-200

-60mV
-70mV
-80mV

0

10

20

30
40
t(ms)

50

60

70

Figure 7: Responses to doublet pulse stimuli. The durationof the input pulse is 1.0 ms and their interval is 20 ms. Bothof the pulses have the same amplitude of −60 mV, −70 mV,or −80 mV. This graph shows that the responses to second
pulse is smaller than first one’s. This result indicates theexistence of refractory period after the firing.

Figure 8: The v-q plane of the system equations of our silicon neuron. At the point of q = 33.47pA, minimum of thestable limit cycle reaches to the saddle point.

sented by the phase plane. We determined the externally applied parameter voltages utilizing the structures in the phaseplane. In HSpice circuit simulation, our silicon neuron circuit successfully produced burst firing patterns including
terval is 20 ms. Both of the pulses have the same amplitude square-wave bursting (not shown). We are working on theof −60 mV, −70 mV, or −80 mV. In each amplitude, mem- circuit experiments of the total system in our silicon neuron,brane potential v was less responsive to the second stimulus which will be presented in our future publications.
in comparison to the first one. This indicates the existenceof refractory period which is the fifth property in the ZeeAcknowledgements
man’s characterization listed above.

V

Square-wave burster mode

When the parameters are selected appropriately, asaddle-loop homoclinic orbit bifurcation emerges in the basic excitable system when q is varied. In this situation, there
exists a bistability between a stable limit cycle that represents a tonic firing state and a stable equilibrium that represents a silent state (see Fig.8). When the system state is in
the left side of the q-nullcline, dq
dt is positive, thus the state
point moves to the right direction generating tonic firing.
At the point of the saddle-loop homoclinic orbit bifurcation,
the minimum potential of the stable limit cycle reaches tothe saddle point and the state point is attracted to the stable node along the unstable manifold of saddle node. Thendq
dt becomes negative, thus the state point moves to the left
until jumps to the limit cycle to generate tonic firing again.
These are the mechanism of generating square-wave bursting in our silicon neuron.

VI

The study was supported by JST PRESTO program andwas partially supported by a Grant-in-Aid for Young Scientists (A) 19680015 from the Ministry of Education, Culture,
Sports, Science, and Technology, the Japanese Government.

Conclusion

We introduced a mathematical-structure-based siliconneuron circuit with 3 variables and reported the experimental results of its basic excitable system. The mathematicalstructure which dominates the circuit operation was repre-©ISAROB 2011

345

References[1] Indiveri, G., A low-power adaptive integrate-and-fire neuron
circuit. Proceedings of the 2003 International Symposium on
Circuits and Systems, 2003. ISCAS 03., p.IV-820-IV-823.
[2] Simoni, M.F. et al., 2004. A multiconductance silicon neuron
with biologically matched dynamics. Biomedical Engineering, IEEE Transactions on, 51(2), p.342354.
[3] Kohno, T. Aihara, K., 2005. A MOSFET-based model of aClass 2 nerve membrane. IEEE transactions on neural networks / a publication of the IEEE Neural Networks Council,
16(3), pp.754-73.
[4] Fitzhugh, R., 1961. Impulses and Physiological States in Theoretical Models of Nerve Membrane. Biophysical Journal,
1(6), pp.445-466.
[5] Rose, R.M. Hindmarsh, J.L., 1989. The Assembly of Ionic
Currents in a Thalamic Neuron III. The Seven- Dimensional
Model. Proceedings of the Royal Society B: Biological Sciences, 237(1288), pp.313-334.
[6] Zeeman, EC., 1972. Differential equations for the heartbeat and nerve impulse. In: Toward a Theoretical Biology,
Waddington CH (ed) Edingburgh, UK, Int. Union of Biol.
Sciences and Edingburgh Univ. Press, vol. 4, p 8

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Theory of mind in a microscopic pedestrian simulation model
Ryo Adachi† , Kazuyuki Aihara†,∗†Graduate School of Information Science and Technology,
The University of Tokyo, Tokyo 113-8656, Japan∗
Institute of Industrial Science, The University of Tokyo, Tokyo 153-8505, Japan(Tel: +81-3-5452-6693; Fax : +81-3-5452-6694)(Email: ryo@sat.t.u-tokyo.ac.jp)
Abstract: In this paper, we propose a microscopic pedestrian simulation model which focuses on pedestrians’
anticipatory behavior in collision avoidance. While it is obviously recognized that inferring other pedestrians’
behavior is playing a crucial role when they intend to avoid collision, few models seriously tackled with thismental attribution. Our model assumes that each pedestrian has theory of mind, which refers to the capacity tomake accurate judgments about beliefs, desires and intentions of other people, and he decides his action basedon his current state in cognitive hierarchy. We also present various simulation results to understand how ouranticipatory behavior affects pedestrians’ behavior as a whole.
Keywords: Pedestrian Simulation, Decision Making, Theory of Mind

I Introduction

II The model

A number of researches analyzing microscopic pedestrians’ behavior with computer simulations have been conducted after we experienced vast improvement on computational ability. Such microscopic simulation models adopt
physical forces [1], cellular automaton models ([2], [3]) or
decision making processes [4]. In these papers, authors’cardinal concerns were how accurate their models could
simulate realistic pedestrian behavior in our daily lives,
which are obviously important from a practical perspective,
and they did not pay much attention to collision avoidance
behavior although the game theoretic aspects which occur
when we face other pedestrians on a road are fundamental
to pedestrians’ characteristics, thus should be investigatedfurther. Recent relevant papers which treat collision avoidance game theoretically include [5] and [6].
Although adopting game theory to pedestrians’ collision
avoidance is insightful, classical game theoretic approaches
do not immediately tell pedestrians how they should behave
in a certain environment. What a pedestrian do to make reasonable decisions in ever changing environment is that hesuccessively infers and makes belief about the movementsof other pedestrians and then decides his response based onthat belief. Thus it is natural to incorporate the concept oftheory of mind [7], which refers to the ability to make accurate guessing about the beliefs, desires and intentions ofother people.
In this paper, we model pedestrian behavior with decision making approaches incorporating theory of mind. We
also present various simulation results and discuss anticipatory behavior of pedestrians.

In this section, we first describe the microscopic pedestrian model we have developed and then discuss how weapply the idea of theory of mind to collision avoidance inour model. Our model assumes that pedestrians’ transition
are fully specified by utilities calculated on each available
state. Such assumption and procedure are one of the mostcommon and powerful approaches in past researches of human decision making.

©ISAROB 2011

1 Pedestrians’ admissible directions and speeds
Due to computational constraints, we need to discretize
pedestrians’ applicable directions and speeds at a certain
moment. We adopt the same model as in [6], which is
described in Figure 1, except for some parameter configuration. Here, θid and θi are desired and current direction,

Figure 1: Choice set of pedestrian i
respectively.

346

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

At every moment after a certain time called scan inter- other state si (t + 1) from si (t) asval, pedestrian i decides his new direction within the choice
exp(λ′ U (si (t + 1)))set Θi , that are.
p(si (t + 1)|si (t)) = ∑′′s′i (t+1) exp(λ U (si (t + 1)))2m − nφ, m = 0, 1, · · · , n}Θi (t) = {θi (t)| θi (t − 1) +n
3) Pedestrian i moves according to the probability distribuwhere φ restricts the admissible change in direction. Also tion calculated above.
To determine probability of choosing strategies from thepedestrian i chooses his speed vi from the choice set Vi , thatvalue of utility, three forms, exponential (logit), power andareknormal (probit) have been used in previous researches in
Vi = {vi | vi = vid , k = 0, 1, · · · , l}decision making. Among these, exponential form whichldwe adopt is considered to be the most preferable experiwhere vi is the desired speed of pedestrian i.
mentally. The parameter λ′ controls how likely pedestriani deviates from the rational choice for unexplained rea2 Specifying actual direction and speed
sons. Also it is known that there is positive correlation between the value of λ′ and pedestrian’s rationality. We put′
We assume that pedestrian behavior can be divided into λ = λ /100 in what follows.
two factors, that are goal-directed (GD) behavior and riskoriented (RO) behavior. GD behavior reflects the fact that 4 Theory of mind
pedestrians basically attempt to minimize the time required
to reach their goals and RO behavior reflects the fact that
Theory of mind, namely attribution of mental states,
pedestrians hate physical contacts with other pedestrians orwasoriginally studied in chimpanzees [7] and is now
walls. To represent such behavior, we define the utility ofbroadly
used in researches that investigate our psychologpedestrian i in a simple form as
ical concepts for imputing mental states to others and ourU (si (t + 1)) =αvi (t) cos(θi (t) − θid (t))(1) selves. Also there are some recent mathematical models oftheory of mind, [8] and [9], from which we gained inspira+ βf (|si (t + 1) − sj (t + 1)|)tion for this research.
+ γf (dist(si (t + 1), wall)),
The concept of theory of mind is considered in pedestrian
behavior such that, in a certain environment, a pedeswhere si (t) denotes pedestrian i’s position at step t.
trian
makes belief about other pedestrians’ future moveIn the first term of the equation (1) which represents GD
ments
and decides his action based on that belief. The utilbehavior, a pedestrian chooses his direction and speed so as
ity
we
mentioned in the last section is calculated with thisto maximize the distance traveled into his desired direction
formedbelief.
in anticipatory period. For the function f in the second andWemodel
the first- and the second-order type of theory
the third term, we specify it asin pedestrian behavior denoted as L1 and L2, respectively
f (x) = −e−ax .
(2) as the same manner as in [9]. We assume that pedestrians with type L1 do not represent other pedestrians’ behavApplying this f in our utility function, we can ignore inter- ior, thus do not anticipate others’ actions. In contrast, L2
action between a pedestrian and other pedestrians or walls pedestrians are assumed to model other pedestrians’ planwhen their distance is large enough. On the other hand, as ning processes to predict their future behavior. In practical,
the distance becomes smaller, RO behavior to avoid colli- a pedestrian with type L1 always chooses his desired direcsion becomes dominant and the corresponding value of util- tion and speed. Also the utility of a pedestrian with type L2
ity decreases. Also the parameter a should be tuned so that is calculated assuming that all other pedestrians in his eye
the value of utility be close enough to zero at any distance sight are having type L1. These L1 and L2 types refer towhere pedestrians do not perceive to be uncomfortable due “going” and “giving way” behavior mentioned in [6].
to the existence of other pedestrians or walls.
While we can consider pedestrian types with higherorder contents such as L3, L4, · · · , we only adopt two types
L1 and L2 in this paper. We discuss this topic in detail
3 Transition of pedestrians
later in concluding remarks. Also for simplicity, we assume
that a pedestrian is type L1 with probability p and type L2
After every scan interval, pedestrian i moves to another with probability 1 − p. This assumption is natural in constate following the algorithms described below.
gested situations such as intersections, stations or buildings
1) Given the current state si (t), pedestrian i calculates the where intermittent changes of environment hinders pedesutility for every admissible state s′i (t + 1) after anticipatory trians’ rational decision makings while it would not apply
period.
in cases when interference between only two pedestrians, in2) Pedestrian i calculate the probability of moving to an- other words low density situations, are concerned.

©ISAROB 2011

347

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

III Simulation
In this section, we present various simulation results foralgorithms shown in Section II.
1

Parameter configuration

First of all, let us explain general parameter configuration in our model. Note that we get the parameter values
presented below after conducting various simulations.
α and β represent how each pedestrian balances between one’s weights on GD and RO behavior. Here theproportion of β to α and γ to β are important, thus we put
α = 1, β = 2.7 ± 0.2 and γ = 1.5. Someone who is inhaste has low value of β and more inclined to GD behavior
than to RO behavior, and vice versa. a determines the steepness of the function f in (2) and reflects the distance wherea pedestrian begins to feel uneasy due to the existence ofother pedestrians or walls and RO behavior starts to hinder
GD behavior. We put a = 2.0 and a = 10.0 in the case ofinteraction with other pedestrians and walls, respectively.
λ reflects pedestrians’ rationality as we mentioned earlier
and pedestrians become more rational as the value of it increases. Since our aim is to simulate the role of theory ofmind in settings that contain intermittent interaction wherechanging environment makes it difficult for pedestrians tomake rational choices, we put λ = 1.0. Other parameters
are basically determined according to [6] and summarizedin Table 1.

Figure 2: One directional trajectories with no obstacles

Figure 3: One directional trajectories with an obstacle

5m, starting at [0, 2.5] for fifty seconds. Other parameter
configuration is same as that of Table 1. One directional
flow simulation we adopt here are simple yet provide profound data about fundamental behavior of pedestrians andused in previous researches. All the data we employ forplots below are averages of the results for five times.

Table 1: Parameter settings
Scan intervals
0.2 s
Radius of a pedestrian
0.2 m
Searching area
Fan-shape with 5 m radiusand ±π/3 in range
Desired speed (vid )1.35 ± 0.2 m/s
Set of directions (φ, n)π/3, 12 (every 10◦ )Here we present two typical examples in which interaction with other pedestrians are not concerned (Figure 2) orare concerned only within two pedestrians (Figure 3, with asteady pedestrian at [4.0, 1.5] represented by a red circle),
only to confirm that above parameter settings work. In both
examples, the starting point and the goal are [0, 1,5] and[8.0, 1.5], respectively and we plot trajectories of a hundred pedestrians on the road. Since we defined transitionof pedestrians probabilistically and added fluctuation in thevalue of β and desired speed, we are able to observe trajectories peculiar to each pedestrian.
2

One directional flow

In this section we present simulation results to show
how the difference in the value of p affects pedestrian behavior as a whole.
First let us explain the simulation settings. Fifty pedestrians are assigned to walk along the road whose width is©ISAROB 2011

Figure 4: Velocities

We plot velocity of pedestrians after ten seconds in Figure 4 and frequency of collision which is the number ofcollision divided by the number of nodes among pedestrians in Figure 5 for some values of p. From these figures we
see that the value of p close to 0 or 1 are unfavorable forpedestrians. To examine this effect we obtain the averageof velocity and frequency of collision for all time steps andplot them in Figure 6. It is implied that the best value of plies between p = 0.2 and p = 0.5.

348

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Also, although we assumed types in pedestrians are determined probabilistically with p in this paper, exploring arule that regulates transition in types is valuable since weknow by our experience that we choose our types observing
the behavior of other pedestrians during a few steps before
we make decisions.
These two main interesting topics would be investigatedfurther in our future works.

Acknowledgements
This research is partially supported by the FIRST program from JSPS.

References

Figure 5: Frequency of collision[1] D. Helbing and P. Molnar (1995), “Social force model
for pedestrian dynamics”, Physical Review E, Vol. 51,
No. 4, pp. 4282–4286.
[2] V. J. Blue and J. L. Adler (2000), “Modeling four–
directional pedestrian flows”, Transportation Research
Record, No. 1710, pp. 20–27.
[3] A. Kirchner, H. Klupfel and K. Nishinari et al. (2004),“Discretization effects and the influence of walking
speed in cellular automaton models for pedestrian dynamics”, Journal of Statistical Mechanics: Theory andExperiment, P10011.

Figure 6: Average of velocity and frequency of collision[4] Th. Robin, G. Antonini, M. Bierlaire et al. (2009),“Specification, estimation and validation of a pedestrian walking behavior model”, Transportation Research Part B, Vol. 43, No. 1, pp. 36–56.

[5] M. Asano, T. Iryo and M. Kuwahara (2009), “A pedestrian model considering anticipatory behavior for caIV Concluding remarks
pacity evaluation”, in: Transportation and Traffic Theory 2009, Springer, New York, pp. 559–581.
We proposed a microscopic pedestrian simulation
model incorporating the concept of theory of mind which [6] M. Asano, T. Iryo and M. Kuwahara (2010), “Microwe strongly believe is fundamental to pedestrian behavior.
scopic pedestrian simulation model combined with a
We also presented numerical examples for simulated datatactical model for route choice behavior”, Transportato show that our model exhibits different behavior due totion Research Part C, Vol. 18, pp. 842–855.
variation in pedestrians’ mental attribution represented by[7] D. Premack and G. Woodruff (1978), “Does the chimtypes.
panzee have theory of mind?”, Behavioral Brain SciIn this paper, we only adopted two types of theory inence, Vol. 1, pp. 515–526.
pedestrians, the first-order and the second-order, although
we can consider higher order contents. Such higher-order[8] W. Yoshida, R. Dolan and K. Friston (2008), “Gametypes would lead to infinite repetition of anticipatory betheory of mind”, PLoS Computational Biology, Vol. 4,
havior and become unrealistic, while it could be worth conNo. 12, pp. 1–14.
sidering since we never surely know how pedestrians arecapable of using sophisticated strategies. If we take latest [9] C. Baker, N. Goodman and J. Tenenbaum (2008),
perspectives from researches on cognitive hierarchy into ac“Theory-based social goal inference”, Proceedings ofcount, L3 and probably L4 are worth adopting while conthe thirtieth annual conference of the cognitive sciencetents higher than L5 seem not to be required because suchsociety, pp. 1447–1455.
contents are likely to be mapping from states to actions acquired by experience rather than computation in the brain.

©ISAROB 2011349

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A Neural Network Model for Categorical Effects in Color Me mory
Chihiro Imai1 , Satohiro Tajima 2 , Kazuyuki Aihara 1 , Hideyuki Suzuki11) The University of Tokyo, 4-6-1 Komaba, Meguro-ku, Tokyo 153-8505, Japan(Tel: 81-3-5452-6693; Fax: 81-3-5452-6694; Email: chihi@sat.t.u-tokyo.ac.jp)
2) Japan Broadcasting Corporation, Nagano Station, 210-2 Inaba, Nagano-city, Nagano 380-8502, Japan(Tel: 81-26 -291-5230; Fax: 81-26-291-5298; Email: tajima.s-iu@nhk.or.jp)
Abstract: Human co lor perception is categorical. Prev ious experimental studies have shown that the color category has
profound effects on cortical neural responses, perceptual color discrimination and color memory. However, existing
theoretical studies are not enough to provide an inclusive model accounting for those categorical effects in color
perception and memory. In this study, we propose a computational model for categorical co lor processing, where thecolor memo ry is represented by a population of color selective neurons in cortex. Our model reproduces thecharacteristics of color memo ry reported in the previous experimental studies. Furthermore, it exp lains the properties ofneurons in IT cortex, which change the activity depending on whether the task demands is color categorizat ion or
discrimination. Th is study suggests that perceptual biases found in color processing and task-dependent modulations ofneural responses may be explained as a natural consequence of statistically optimal estimat ion.
Keywords: color memory, categorical co lor, population coding
upon the task. So me IT neurons showed stronger
activity in the categorization task or in thediscrimination task in response to the same color
stimulus.
In spite of these clear ev idences that color
discrimination, color categorization and color memo ry
are related to each other, there is no inclusive model that
account for how our brains fulfill each of thesefunctions depending on the situation. In the present
study, we propose a co mputational model that takes into
account the categorical effects in color perception andmemo ry, assuming that color selective neurons in cortex
express colors in memory. In part icular, our model
reproduces neural activit ies of color-selective neurons in
the IT cortex during categorical judg ment anddiscrimination, and the categorical effects in color
memo ry.

I. INTRODUCTION
We perceive millions of colors based on the
spectrum of incident light flowing into our eyes.
Previous studies show that there are two different ways
in how we see colors in spectral space: to look closely
or to look as groups. Recognizing very slightdifferences between colors is called co lor d iscrimination.
On the other hand, organizing colors with in a certain
color region into one group is called categorical color
perception. Discrimination and categorizat ion are
considered as two different functions in color perception.
Psychophysical studies have shown the categoricaleffects on color discrimination. Harnad (1987) reported
that discrimination was better for colors straddling thecategory boundary than for colors within the same
category. Furthermore , considering time factor, color
memo ry is also involved. Perez-Carp inell et al. (1998)
showed the mean difference between test color andrecalled color increases with the delay t ime. According
to Heider(1972), focal co lors (colors at categoricalcenter) are remembered more accurately than non-focal
colors.
fMRI and electrophysiological studies suggest thatthe visual cortical areas subsequent to area V4 p lay
important roles in higher order functions of color vision
including categorization and discrimination. Koida et al.
(2007) investigated the responses of color-selective
neurons in the inferior temporal (IT) cortex of monkeys
during making a categorical judgment or a finediscrimination of colors. They found the activity ofmany IT color-select ive neurons differed depending©ISAROB 2011

II. COLOR PERCEPTION MODEL

Fig.1. A schematic diagram of the proposed model

350

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Our model exp lains categorical effects in colo r
perception through the interactions between hueselective neurons and category-selective neurons. We
assume a hue space of one dimension as a color space.
The model is summarized in Fig.1.
We assume two groups of color-coding neurons:
the first group („hue-selective neurons‟) in which theneuronal preferred hue is homogeneously distributed,
and the second group („category-selective neurons‟) inwhich the distribution of hue preference is concentrated
around some points, wh ich we call „category center‟ inthis study.
Each hue-selective neuron receives the input signal
fro m earlier processing stages. The strength of the input
signal is expressed as a function of the difference
between presented hue θ and ith neuron‟s preferred hue
θi :i ( )  f (   i )  g  e k cos(  )one, the posterior probability is expressed with theBayes' formu la as follows:
t 1

t 1

P( t 1 | {ri })  P({ri } |  t 1 ) P( t 1 )(5)

We propose that the information of the prior probability
distribution P(θi t+1 ) is given by the top-down signal
fro m k-th category-selective neuron to ith hue-selective
neuron: f(θck - θi ). For Po isson-like spike statistics,
combin ing process of two probability distributions isachieved by a simple linear summation of t wo
population activities (Ma et al., 2006). In the present
study, we propose that the likelihood and the prio r isiteratively reco mb ined through following recurrence
formula :i ( t 1 )  f ( t 1   i )  f ( kc,t   i )

(6)

i

(1)

where α and β control the co mbining ratio. In
discrimination task, the parameter β is set to a s maller
value than in categorization task. In color memo ry task,
the iteration of Eq. (6) is continued for the length ofmemo ry duration. For all of the simulations, g = 1, k = 1,
T = 1, α = 1, β = 1, nu mber of category = 2 andnumber of hue-selective neurons = 2000.

where we used the von-Mises (circu lar Gaussian)function for the hue selectivity; g is a parameter tocontrol the response gain,. When there is no other input
to those neurons, we assume that the neuronal spikes are
generated with the follo wing Poisson process:(i T ) riT iT
P(ri |  ) e(ri T )!

(2)
where ri is the firing rate in one trial; T is theobservation duration. The categorical decoding θck is thehue of kth category center

III. RESULTS
1.

Categorical effect on Col or Memory3

where we used the fact that the summation Σ i f(θ-θi ) is
independent of θ when the neuronal preferred hues arehomogeneously distributed. Equation (3) means that thelog likelihood of hue category can be calculated as aweighted sum of the hue-selective neurons‟ activities(the first term) (Jazayeri et al., 2006). In the present
model, we hypothesize that the category-selective
neurons receive the read-out signals from the hueselective neurons as follows:2ia kc   ri ln f ( kc   i )input hue

1
0
-1
-2

(4)

-3
0

i

20

40

60

80

100

Time

By comparing a ck for various hue category θck, we canestimate wh ich category is the most likely to generate
the color of visual input. This means that the maximu mlikelihood estimat ion of input hue category can be
implemented by a winner-take-all p rocess among thecategory-selective neurons.
Now we consider estimat ing the hue presented attime t+1 based on the knowledge of hue at time t. Given
the assumption that the hue presented in future is likely
to be generated from the same category as the present©ISAROB 2011

category centerln P({ri } |  kc )  T  ri ln f ( kc   i )  const. (3)Fig.2. Temporal evolution of the cortical hue
representation during the color memo ry task. We p lotted
the preferred hue of the neuron that showed the peak
activity in the population. The horizontal axis indicatesthe memory duration. The results for four example
init ial hues are shown.
Figure 2 shows the temporal evolution of hue
values in color memory task. Here, the preferred hue

351

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

the different task demands of color categorization anddiscrimination. We simu lated the neuronal responses to
11 samp le colors (the horizontal axis in Fig.4) wh ich
spread on the axis fro m red to green. Figure 4 shows theactivities of a greenish-preferred neuron. The horizontal
axis indicates 11 sample colors. In Fig.4, the activity ofa hue-selective neuron is shown. Our model reproduces
neural activit ies of color-selective neurons in the IT
cortex during categorical judgment and d iscrimination(Koida et al. 2007).

70

120

60

100

Firing rate[spike/s]

Firing rate[spike/s]of hue-selective neuron that showed the maximu mmagnitudes at each time is plotted. Values at time 0
indicates the input hue presented at the initial point of
memo ry task; Each hue-selective neuron is activatedresponding to the hue input at first. After t ime 0, hue
selective neurons received only the top-down signals
fro m category-selective neurons, and the input signals
were set to zero. When the memory duration evolved,
the represented hues approached to the nearest category
center.

time
50
40
30

80
60
40

20

20

10

0
1

0
-4

-2

0

2

4

3

4

5

6

7

8

9

Sample color(1.red<->11.green)

10

11

Fig.4 Responses of a hue-selective neuron during ca
tegorizat ion (solid line) and discrimination (dashed line).

Preferred hue
Fig.3 Temporal act ivity change of hue selective neurons
in the color memory task. Horizontal axis denotes thepreferred hue of each neuron. Each solid line shows asnapshot of the activit ies of hue-selective neurons, anddashed line indicates the activity induced by an input
signal.

IV. DISCUSSION
Our model reproduces the characteristics found in
the previous experimental studies and provides an
inclusive exp lanation for some of the phenomenon
concerning categorical color perception, color
discrimination and color memo ry. Fro m Bayesian
viewpoint, the neural response properties in IT cortex
can be exp lained as a natural consequence ofstatistically optimal estimat ion. This theoreticalframework is possibly applied to similar phenomena
found in other cortical area involved in d ifferent sensory
or feature modalit ies, such as audition, visual motion orface expression.
We assumed that the main categorical effects are
elicited by top-down signals from category selective
neurons in the response to the input from hue-selective
neurons. In the co mbin ing process of input and topdown signals, we applied a simp le static linear
summation for a purpose of conciseness. Further studies
may cover a saturation and adaptation of neuronal
responses or weight varying with time.

Figure 3 shows the snapshots of the activities ofhue-selective neurons, which evolved during thememo ry task. The peak of the activities distribution of
the hue-selective neurons shifts toward the nearest
category center. The maximu m value of firing rateincreases with time.
As it can be considered that the hue centers in themodel correspond to the focal co lors, these results agreewith the characteristics of the results reported by Heider(1972). When input hue is away fro m each hue center,the model also reproduces the temporal increase in the
difference between memo ry color and init ially presented
color (Perez-Carpinell et al., 1998).
2.

Col or categorization and col or discriminati on
We next investigated how the present model can be
related to the response properties of visual neurons in©ISAROB 2011

2

352

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

[6] W. J. Ma, J. M. Beck et al. (2006) “Bayesianinference with probabilistic population codes .” Nature
Neuroscience, 9:1432-1438

The present model considers only one-dimensionalhue shift with the duration between memory and recall,though shifts of color saturation and lightness are also
reported in psychophysical studies of color memory.
These factors should be considered in an extension of
the current model.
Another topic to be considered is theneurobiological substrate of categorical computation.
Although Fig. 4 only depicts the activities of hueselective neuron, we found that the corresponding
category-selective neuron also showed the similar
pattern. It can be supposed that the category-preferredneurons in Ko ida et al. (2007) correspond
approximately to the hue-selective neurons in thismodel, category-preferred neurons also alike.

V. CONCLUSION
We proposed a computational model for h igher
order mechanisms of colo r perception. Ou r model
reproduces the characteristics of categorical effects
including color memo ry reported in the prev ious
experimental studies. From the perspective of
statistically optimal estimation, this study provides one
possible explanation of the relation among categoricalcolor perception, color discrimination and color
memo ry in the higher visual cortex in the brain.

ACKNOWLEDGEMENT
This research is supported in part by the Japan
Society for the Promotion of Science (JSPS) through its“Funding Program for World-Lead ing Innovative R&D
on Science and Technology (FIRST Program)."

REFERENCES[1] K. Ko ida & H. Ko matsu (2007), “Effects of task
demands on the responses of color-selective neurons in
the inferior temporal cortex.” Nature Neuroscience.
10(1) 108-116[2] E. R. Heider (1972), “Un iversals in color naming
and memory.'' J. Exp. Psychol. 93, 10-20[3] S. Harnad (1987), “Category induction andrepresentation.” In S. Harnad (Ed.), Categoricalperception: The Groundwork of Cognition. 535-565.
Cambridge: Camb ridge University Press.
[4] J. Perez-Carpinell, R. Baldovı et al. (1998) “Color
Memory Matching: Time Effect and Other Factors .”COLOR research and application, 23(4) 234-347[5] M. Jazayeri & J. A. Movshon (2006) “Optimal
representation of sensory information by neural
populations.” Nature Neuroscience, 9(5):690-696.

©ISAROB 2011353

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Study of computational performance of Genetic Algorithm for 3-satisfiability
problem
QingLian Ma, Yu-an Zhang, Makoto Sakamoto, Hiroshi Furutani
Faculty of Engineering, University of Miyazaki
Miyazaki City 889-2192 Japan
Abstract

and problem-specific algorithms. Their result was confirmed experimentally by Fleurent and Ferland (1996),
who reported poor performance of classical GAs whencompared to local search methods. Marchiori designed
a rather successful GA-based algorithm for hard 3SAT problems by combing a native GA with a local
search algorithm [7]. GA for SAT employ heuristic information into the fitness function or into the GA operations (selection, crossover, and mutation) [2, 8]. In
this paper, we study the computational performanceof genetic algorithm on a 3-SAT problem. GA cansolve SAT problems. However it usually needs highperformance computing. To overcome this problem,
we study the mean first hitting time of optimum solution T , success probability S, the mean survival timea and their relations.

In order to improve the computing performance ofGenetic Algorithms (GAs), it is important to study theeffects of crossover and mutation. In this study, we examine the relations of first hitting time T of optimum
solution in population, success probability S, mutation rate pm and crossover rate pc by GA experiments,
which are carried out on the 3-satisfiability (3-SAT)
problem. Here, S is defined as that there is at least
one optimum solution in a population at the stationary distribution. We found that, when mutation rateis small, the effects of crossover on T and S are large.
S with crossover is larger than that without crossover,and T is smaller than that without crossover. We also
observed the relation between T and a/S when mutation rate becomes large, and found that T = a/S.
When pm = 0.02, T ≈ 1/S.

2
1

Introduction

The SAT problem is a task to determine whether
there exists an assignment of truth values to a setof Boolean variables that make a conjunctive normal form (CNF) formula to be true [2]. The SAT
problem can be formulated as follows: given a setof clauses C1 , C2 , . . . , Cm on the Boolean variables
x1 , x2 , . . . , xn , determine if there is an assignment for
the variables such that the formula C1 ∧ C2 ∧ · · · ∧ Cm
evaluates to true, where ∧ is a logical connector and.
A clause is a disjunction of literals, e.g., x1 ∨ x2 ∨ x3 ,where a literal is a Boolean variable x or its negationx̄, and ∨ is a logical connector or. If each Ci contains
exactly z distinct literals, then the problem belongs to
the z-SAT class [7]. In this paper we consider formulas in conjunctive normal form, and each clause has
exactly 3 literals.
We use the following notations, let:
• F be a CNF Boolean formula,• m be the number of clauses in F ,• n be the number of variables in F ,• xk be the kth variable in F (1 ≤ k ≤ n),

The satisfiability (SAT) problem is a core of a large
family of computationally intractable NP-complete
problems [1] with relevant practical applications suchas automated reasoning, computer-aided design, machine vision, database, robotics, computer network design and so on. Methods to solve the SAT problem
play an important role in the fields of efficient computing systems [2].
During the last two decades, several improved algorithms have been developed, and important progresshas been achieved. These algorithms have considerably enlarged our capacity of solving large SAT instances. It can be divided into two main classes:
complete and incomplete algorithms. The first category includes approaches based on the Davis-Putnamalgorithm [3]. Most incomplete algorithms include approaches based on local search [4] and evolutionary
algorithms (EAs) [5, 6]. De Jong and Spears (1989)
proposed a classical GA for SAT problem, and observed that the GA may not outperform highly tuned©ISAROB 2011

3-Satisfiablity problem

354

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 201187

• Ci be the ith clause in F (1 ≤ i ≤ m),• l be the clause length in Ci ,• Qi,j be the jth literal in the ith clause in Ci (1 ≤j ≤ l).
We use bit variable xk = 1 or 0, corresponding totrue or f alse, respectively. We count the number ofclauses that are fulfilled by the variable assignment. A
chromosome is a candidate solution, which is a stringof bits having length equal to n.
A common formulation for CNF formulas exists [9].
The fitness function of F is expressed byf (F ) =

m
X

Ci ,

L=20 pm=0.002 pc=1 m=91
86

AVERAGE FITNESS

85

N=100
N=50

83
N=20

81

(1)

80
0

where Ci is denoted as
Ci = Qi,1 ∨ Qi,2 ∨ Qi,3 , Qi,j = xk or xk (j = 1, 2, 3).
If all clauses are satisfied, then f (F ) = m.
In this study a set of benchmark instance was used.
Problems in this benchmark are downloaded from the
SATLIB-benchmark Problems [10]. The instance provided here is cnf formulae encoded in DIMACS cnf
format. We use a uniform random-3-SAT problem,
uf20-91, with 20 variables and 91 clauses. Each ofclauses is constructed from 3 literals. The input of ourprogram is data file format with 91 rows, 3 columns.
We evolve a population of variable assignments until
we find an assignment that makes the formula true.
In the following we define some variables for evaluating the GA computing performance. The successprobability S(t) is defined as the probability that there
is at least one optimum solution at generation t in stationary distribution [11]. S is the time average of S(t)defined by
3000
X
S =
St .
(2)

100

200

300

400

500

GENERATION

Figure 1: Dependence of average fitness on N . The
horizontal axis represents generation.
uniform crossover. The calculations were performed
repeatedly, and results were averaged over 10000 runs.
Figure 1 shows the dependence of average fitness on
N with pc = 1 and pm = 0.002. By comparing resultswith N = 20, 50, 100, 200, we observe the strong N
dependence of average fitness.
Figure 2 shows the dependence of average fitness on
pm with pc = 1 and N = 200. By comparing resultswith pm = 0.05, 0.005, 0.0005, we find strong pm dependence of average fitness. When mutation becomesstrong, the average fitness is low, and converges into asmaller value.
Figure 3 shows N -dependence of success probability
S with pm = 0.02. As show in this figure, the successprobability shows little difference between calculationsof pc = 0 and pc = 1. The effect of crossover on successprobability is small in this case.
Figure 4 shows pm -dependence of success probability S with population size N = 50. The solid line
is success probability with crossover rate pc = 1, andthe dotted line is success probability with crossover
rate pc = 0. We find that the success probability withcrossover is larger than that without crossover whenmutation rate is small. However, when mutation rate
becomes large, the effect of crossover goes to small,and the success probabilities are almost equal.
Figure 5 shows pm -dependence of the mean first hitting time of optimum solution T with population size
N = 50. The value of T without crossover is larger
than that of with crossover, which means crossover
accelerates the speed of evolution in GA. When muta-

t=1000T is defined as the mean first hitting time of the optimum solution in a population. The mean survival
time a is the average consecutive generations containing optimum solution in the stationary distribution.

Numerical experiments

In this paper, we study the relation between the
success probability S and the mean first hitting time
T by experiments. We performed numerical calculations of GA with roulette wheel selection on the 3-SAT
problem. We carried out the experiments with string
length L = n = 20, number of clause m = 91 andclause length l = 3. Crossover was done with the©ISAROB 2011

84

82

i=1

3

N=200

355

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

0.012

90

N=50

N=200 L=20 pc=1 m=91
0.01

88

SUCCESS PROBABILITY

pm=0.0005

AVERAGE FITNESS

L=20 m=9186

pm=0.005
84

82

0.008

with crossover

0.006

0.004

without crossover

0.002

pm=0.05
80
0

200

400

600

800

0
0

1000

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

GENERATIONMUTATION RATE

Figure 2: Dependence of average fitness on pm . The
horizontal axis represents generation.

Figure 4: pm -dependence of success probability S withcrossover rates pc = 1 and 0. The horizontal axis represents mutation rate pm .

0.002

pm=0.02

L=20

m=91

0.0015

L=20 m=91

5000

4000
0.001

T

SUCCESS PROBABILITY

N=50

6000

3000

0.0005

Without Croccover(pc=0)

2000
With Croccover(pc=1)

0

Success with crossover
Success without crossover
40

80

120

160

1000
200

0.05

0.1

0.15

0.2

0.25

POPULATION SIZE

MUTATION RATE

Figure 3: N -dependence of success probability S withcrossover rates pc = 1 and 0. The horizontal axis represents population size N .

Figure 5: pm -dependence of T with crossover rates
pc = 1 and 0. The horizontal axis represents mutation
rate pm .

©ISAROB 2011356

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

1.4 104

1.2 104

1 10

4tion rate is large, the effect of crossover on T is small.
Figure 6 shows the relation between T and a/S
when pm is 0.02. a is the mean survival time. From
this figure we observe the relation of T = a/S. Thereis strong N dependence of first hitting time of optimum solution T .
Figure 7 shows the dependence of a on N with pm =0.02 and 0.002. When pm = 0.02, a ≈ 1, while pm =0.002, a is large. Therefore, the conclusion of T ≈ 1/S
is established if pm = 0.02.

L=20 pm=0.02 m=91

T

8000

T(N=40)6000

a/S (N=40)

4

4000

T(N=120)

2000

The GA parameters, for example, population size
N , string length L, crossover rate pc and mutation
rate pm , influence the GA performance. In this studyof GA on 3-SAT problem, the effects of crossover andmutation are investigated. The results show that :when mutation rate is small,
1) The effects of crossover on S and T are large.
2) The success probability S with crossover is larger
than that without crossover, which means crossoverplays an important role on S.
3) The value of T with crossover is smaller than thatwithout crossover, which means crossover accelerates the speed of evolution in GA.
We also observed the relation between T and a/S
when mutation rate becomes large, and that T ≈ a/S.
When pm = 0.02, a ≈ 1 and T ≈ 1/S.

a/S (N=120)
0
0

0.2

0.4

0.6

0.8

1

CROSSOVER RATE

Figure 6: The realation of T and a/S with N =40, 120. The horizontal axis represents crossover rate
pc .

8

L=20

Summarypc=1 m=91

7

SURVIVAL TIME

6

References

5[1] J. Gu: Randomized and Deterministic Local
Search for SAT and Scheduling Problems. DIMACS Series in Discrete Mathematics and Theoretical Computer Science, Vol.43, 61–62 (1999).

pm=0.002

432[2] J. Gu: Parallel Algorithms for Satisfiability(SAT) Problem. DIMACS Volume Series in Discrete Mathematics and Theoretical Computer Science, American Mathematical Society, Vol.22,
105–161 (1995).

pm=0.02
1

0
20

40

60

80

100

120

POPULATION SIZE

[3] M. Davis, H. Putnam: A Computing Procedure
for Quantification Theory. Journal of the ACM,
Vol.7, Issue 3, 201-215 (1960).

Figure 7: The dependence of a on N with pm = 0.02
and 0.002. The horizontal axis represents population
size N .

©ISAROB 2011[4] J. Hansen, B. Jaumard: Algorithms for the maximum satisfiability problem. Computing, Vol.44,
Issue 4, 279-303 (1990).

357

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011[5] K. De Jong, W. Spears: Using genetic algorithms
to solve NP-complete problems. Proceedings of
the Third International Conference on Genetic
Algorithms, 124-132 (1989).
[6] C. Fleurent, J. Ferland: Object-oriented implementation of heuristic search methods for graphcoloring, maximum clique, and satisfiability. Discrete Mathematics and Theoretical Computer Science, Vol.26, 619-652(1996).
[7] E. Marchiori, C. Rossi: A Flipping Genetic
Algorithm for Hard 3-SAT Problems. In Genetic and Evolutionary Computation Conference,
Vol.1, 393-400 (1999).
[8] J. Gottlieb, E. Marchiori, C. Rossi: Evolutionary
Algorithms for the Satisfiability Problem. Evolutionary Computation, Vol.10, Issue 1, 35-50(2002).
[9] J. Gu: Effecient local search for very large-scalesatisfiability problem. SiGART Bulletin, Vol.3,
Issue 1, 8-12(1992).
[10] http://www.cs.ubc.ca/ hoos/SATLIB/benchm.html.
[11] Y. Zhang, M. Sakamoto, H. Furutani: Effects ofPopulation Size and Mutation Rate on Results ofGenetic Algorithm. Proceedings of Third International Conference on Natural Computation, Vol.3,
70-75 (2008).

©ISAROB 2011358

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

An Adaptive Resolution Hybrid Binary-Real Coded Genetic Algorithm
Omar Abdul-Rahman1, Masaharu Munetomo2 and Kiyoshi Akama2

1

Graduate School of Information Science & Technology, Hokkaido University,
2
Information Initiative Center, Hokkaido University, Japan(Tel : 81-11-706-7890; Fax : 81-11-706-7890)(omar@ist.hokudai.ac.jp, munetomo@iic.hokudai.ac.jp, akama@iic.hokudai.ac.jp )Abstract: Which is better to be used in Genetic Algorithms (GAs) binary or floating point coding schemes? In
this paper, we try to answer this controversial question by proposing a novel algorithm that shares thecomputational power between two cooperated versions of GAs, a binary coded GA (bGA) and a real coded GA(rGA). The evolutionary search is primarily led by bGA, which is used to identify promising regions in the
search space. While, the rGA is used to increase the quality of the obtained solutions by conduct a detailed
search through these regions. The interactions between two versions are organized by a resolution factor that it
is increasingly adapted during the evolutionary search. Comparison experiments were conducted over a typica
l function to proof the feasibility of the algorithm under critical scenarios of increasing problem dimensi
ons and decreasing precision power.
Keywords: Binary coded GA, Real coded GA and Hybriding.
On other hand, real coded GAs are preferred bymany practitioners. They are on rising usage since thefloating point representation is conceptually closest to
the real design space, and moreover, the string length
reduces to the number of design variables. Real coded
GAs are robust, accurate, and efficient. However, it ispossible to argue that real coded GAs are stillsusceptible to premature convergence especially forcomplex real world problems with a large number ofdesign variables. It is also possible to say that theory ofreal coded GAs is still far from providing plausible
understanding of internal real coded GAs mechanisms,
which is a true hinder before the development ofadvanced techniques in this field.
Therefore, in this paper we propose the adaptive
resolution binary-real coded GA (brGA) that combines
the advantages of both versions. Under the proposed
algorithm, two versions of the same population ofsolutions are kept, binary coded and real coded
populations. They are updated by continuously mappingfrom one version to another during the evolutionary
search. The search is primarily guided by binary coded
GA part, which is used to identify promising regions inthe search space. While, the real coded GA part is used
to increase the quality of the obtained solutions byconduct a detailed search through these regions. Theinteractions between two parts are organized by aresolution factor that it is increasingly adapted during
the evolutionary search. it has a small value at the
begin of the search to allow the exploration of thesearch space by the binary coded GA part, while, itsvalue increases gradually as the search progresses to

I. INTRODUCTION
The intersection between Genetic Algorithms (GAs)and artificial intelligence has a long history. From one
side, GAs play a central role in many artificial life
models. They are currently the most prominent andwidely used models of evolution in artificial life
systems. In artificial intelligence, GAs have been usedboth as a tools for solving practical problems and as ascientific models of evolutionary processes. On theother hand, GAs have been successfully applied to solve
difficult real world problems related to robotics likepath planning, navigation controller optimization andline balancing problem.
In this paper, we try to approach again this long andcontroversial debate over the real coded and binary
coded GAs. For theoretician, binary coded GA is an
attractive solution. The theoretical finding of schemata
theory supports that enhanced schemata processing is
obtained by using alphabet of low cardinality. Binary
coded GAs are efficient and the latest developments inthe field of GAs research adds much to the robustness,
speed and accuracy of such algorithms. However, it ispossible to argue that binary coded GAs suffer fromseveral disadvantages when applied to real-world
problems involving a large number of real design
variables. The direct relationship between the desired
precision and the increased binary string length, and thediscrepancy between the binary representation space
and the actual problem space are good examples of suchdisadvantages.

©ISAROB 2011359

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

allow the exploitation of the search space by real coded
GA part.
The remaining part of this paper is organized asfollows. Related literature is reviewed in Section II. The
proposed algorithm framework and implementation is
explained in Section III. Then, experimental results arereported and discussed in Section VI. Finally, we
conclude the paper in Section V.

2.

II. LITERATURE REVIEW

3.

Dynamic coding is a sophisticated approach to alter
the coarseness of search spaces. An example of suchapproach is the stochastic genetic algorithm aspresented by Krishnakumar et al [1]. In stochastic
GAs, the region represented by each point of a binary
coded GA is adapted during the optimization processusing Evolutionary Strategies (ES). In our work, we
adopted a similar concept. However, in contrast to
stochastic GAs we employed a real coded GA instead of
ES to adapt the region represented by the binary coded
GA.
Another example of dynamic coding is the Adaptive
Range Genetic Algorithm (ARGAs) as presented by
Arakawa and Hagiwara [2]. In ARGAs, mapping rules
from binary to real strings is updated during the
optimization process according to the population
statistics to adapt the population toward promising
design regions. However, in our work we depend on areal coded GA that sample from the regions determinedby a binary coded GA to adapt the population towardpromising region in the search space.
Finally, a different approach is adopted by Barrios etal [3]. in their cooperative binary-real coded GA for
designing and training feed-forward artificial neuralnetworks. They employed two interconnected GAs thatwork parallelly to design and train better neuralnetworks that solves the problem. In our work, we also
employed two cooperative GAs, but they interact witheach others in an interleaving manner.

4.

IV. EXPERIMENTAL RESULTS
The proposed algorithm was implemented using
MATLAB. Griewangk’s function was used to testify theperformance of the algorithm. The function is defined asfollows.

This function has complex structure of numerous
local extremum. It has a value of 0 at global optimum.
In the first set of experiments, the performance of
the algorithm is testified against increasing problem
dimensions. The performance is compared against an
ordinary binary GA[4] and UNDX real coded GA[5].
The value of resolution factor was chosen as 2
increases in step of 2. The settings for real and binary
coded GA were chosen as recommended in [4] and [5].

III. THE ALGORITHM
The proposed algorithm flowchart is shown in Fig.1,
which illustrates the typical optimization cycle. The
main components of the algorithm can be described asfollows.
1. Coding schemes: floating point and binary
coding schemes is used to represent two
equivalent populations of solutions. These
populations are kept updated by continuously©ISAROB 2011

mapping from one version to another during theevolutionary cycle.
Binary coded GA: the evolutionary cycle isusually led by the binary coded GA. This part is
implemented using a binary GA as described by
Haupt and Haupt [4] . The main parts of thisalgorithm are single point crossover operator, one
point crossover operator, rank weighting selection
scheme.
Population handover: the population of binarychromosomes that processed by the binary GA istransmitted to the real coded GA and vice versa at
specific points of the evolutionary cycle. The
process of population handover is controlled by
the resolution factor that has an adaptive valuethat allows the real coded GA to increase thecoarseness of the search space. During theprocess of handover, the best half from thecurrent binary population is directly transmitted
to the real coded population, while the other halfcomes by random sampling within the region
determined by the binary population.
Real coded GA; the role of this version is toadapt the population toward the promisingregions of the search space. In this paper, we have
implemented this part using Unimodal Normal
Distribution (UNDX) as an advanced real
crossover operator [5]. Minimal Generation Gap(MGG) is used as a generation-alteration model.

360

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig.1. The proposed algorithm flowchart
This is due to the fact the implementation of bGA isfaster than brGA. However, its efficiency rates degrades
drastically to 0 in other cases, while, brGA keeps
reasonable efficiency rates. This is due to the fact thatthe bGA loses the ability to progress the population at
higher problem dimensions, which is one of binary
coding disadvantages.
On the other hand, by comparing brGA to rGA it isevident that the performance is comparable at lower
dimensions (10,50). But, the difference in performanceincreases at the higher dimensions (100, 120) for the
benefit of brGA. So, the hybriding enhance theperformance of rGA at complex situation of highproblem dimension. In term of efficiency rates, theresults are comparable.
In the second set of experiments, the performanceof the algorithm is testified against increasing
chromosome length. The same settings as above wereused. However, here the dimension is fixed to 2. In
addition, only bGA and brGA were testified.
The experimental results are shown in Fig. 2. As
expected the performance of bGA is largely depend on
chromosome length. By increasing the number of bits,
coding precision increases. Which, translated into

The results were averaged over 50 experiments. Eachrun consisted of 990 iterations and a population of 6
members was used. The function is minimized over therange of 5 ≤ x ≤ -5. Each parameter is represented by 8
bits.
Experimental results are shown in Table 1.
Efficiency rates shown in the table is used to measurequality- computational effort relationship.
It is defined according to Hillstrom [6] as follows.

Where , and are the best initial, best known and best final fitness, while, T is the elapsed time in seconds.
By comparing brGA to bGA, it is evident from thetable that brGA outperforms in term of performance inall the cases. The difference in performance is clear at
higher problem dimensions. In term of efficiency rate,bGA outperforms only in case of 10 dimensions in spiteof lower solution quality.

©ISAROB 2011361

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 1. Performance against increasing problem dimension
Binary GA

Real GA

brGA

Problem
Dimension

Best

mean

Efficiency

Best

mean

Efficiency

Best

mean

Efficiency

100.1391

0.317

4.61250.004932

0.011739

3.85320.004934

0.012536

3.573950

1.06761.089

00.004305

0.024375

2.67230.004655

0.014299

2.2888100

1.16021.188

00.61367

0.85668

0.178880.284730.450650.37779120

1.18851.226

00.895410.981750.11178

0.57957

0.77130.16254an enhanced performance. It is clear from Fig. 2 that
brGA shows an independent performance against
chromosome length. So, the hybriding is useful indecreasing the dependency of its binary part on
chromosome length to achieve a better performance.
That it is helpful in case of high precision that requires
long chromosome representation.

ACKNOWLEDGMENT
This work is supported by Grant-in-Aid for Scientific
Research (C) by MEXT, Japan.

REFERENCES[1] Krishnakumar, K., Swaminathan, R., Garg, S. and N
arayanaswamy, S., Solving Large Parameter Optimizati
on Problems Using Genetic Algorithms, Proc. of the Gu
idance, Navigation, and Control Conference, (1995), 44
9-460.
[2] Arakawa, M. and Hagiwara, I., Development of
Adaptive Real Range (ARRange) Genetic Algorithms,
JSME Intl. J., Series C, Vol. 41, No. 4 (1998), 969-977.
[3] Barrios, D., Carrascal, A., Manrique, D., Ríos, J.,
Cooperative binary real coded genetic algorithms forgenerating and adapting artificial neural networks,
Neural Computing and Applications 12 (2003), 49-60.
[4] Haupt,R.L. and Haupt,S.E. Practical Genetic
Algorithms. (1998) Wiley, New York.
[5] Ono, I. and Kobayashi, S. ,A Real-coded Genetic
Algorithm for Function Optimization Using Unimodal
Normal Distribution Crossover, Proc. 7th ICGA, 246253 (1997).
[6] Hillstrom, K. E.,A simulation test approach to theevaluation of nonlinearoptimization algorithms. ACM
Transactions on Mathematical Software, 3(4),305– 315(1977)..

V. CONCLUSION
An adaptive resolution binary-real coded GA is
proposed in this paper. It is a framework that integrates
the cooperation between rGA and bGA through an
adaptive resolution factor to combine the advantages ofboth versions. Algorithm flowchart is presented, andexplained. Then, experimental results over a typicalbenchmark problem is reported and discussed. It isevident from the results that the proposed algorithm
greatly enhanced the performance of the binary GA partagainst increasing problem dimensions and decreasing
chromosome length. On the other hand, the algorithm
enhances the performance of rGA in case of higher
problem dimension.

Fig.2. Solution quality versus chromosome length©ISAROB 2011362

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Neural network with exponential output neuron for estimation ofphysiological activities from protein expression levels
Kazuhiro KONDO† Kunihito YAMAMORI‡ Ikuo YOSHIHARA‡†

Graduate School of Eng., Univ. of Miyazaki‡
Faculty of Eng., Univ. of Miyazaki(Tel :+ 81 985 58 7589; Fax :+ 81 985 58 7589)(kondo@taurus.cs.miyazaki-u.ac.jp,{yamamori,yoshiha@cs.miyazaki-u.ac.jp})

Abstract: Many people expect the third function of foods called as physiological activities that affect our healthcondition. However, measurement of a physiological activity is troublesome and measurements for all kinds of foods
are not impractical. Therefore a system which can easily estimate physiological activities is required. We have proposeda method to estimate physiological activities of foods from protein expression levels using artificial neural networks(ANNs). Estimation of physiological activity using conventional ANN has a problem that physiological activities takepositive real number more than 1.0, but dynamic range of ANN is limited from 0.0 to 1.0. So measured physiologicalactivities has to be scaled by dividing the maximum one. But this normalization gives undesirable influence toacceptable error level which is a parameter to terminate training process.
To solve this problem, we employ exponential function as the activation function in output neurons. By using
exponential function in output neurons, ANN can directly handle physiological activities as the training signals thoseare more than 1.0. Experimental results show that our method improves accuracy than conventional ANN with scaled
training samples for anti-oxidant activity, anti-inflammation activity and anti-angiogenic activity.
Keywords: physiological activities, protein expression level, exponential activation function, artificial neural network.

enough estimation accuracy. Tsukuda et al. [3] showedthat artificial neural network (ANN) could achieve good
estimation accuracy of physiological activities.
Estimation of physiological activities using
conventional ANN has a problem. Physiologicalactivities sometimes take positive real number morethan 1.0, but dynamic range of a conventional ANN islimited from 0.0 to 1.0. So physiological activities haveto be normalized for estimation by conventional ANN.
But normalization leads undesirable influence for
acceptable error level to terminate training process. To
avoid this undesirable influence, we employ exponentialfunction as the activation function in output neuron. By
using exponential function in output neuron, ANN candirectly handle physiological activities as the training
signals those are more than 1.0.

I. INTRODUCTION
Foods have three functions; the first is nutrition, thesecond is taste and the third is biological regulation. The
third function of foods attracts attention in recent years
because many people expect that the third function offoods will be useful to keep their health and prevent
diseases. The third function of foods is measured asphysiological activities. However, measurement ofphysiological activity is troublesome because it needsmanysophisticatedmanualoperations,andmeasurements for all kinds of foods are impossible.
Therefore a system which can easily reveal
physiological activities is required. For this objective,
we have proposed a method to estimate physiologicalactivities from protein expression levels because protein
expression levels closely relate to cell condition andsome kinds of protein expression levels can bemeasured at once.
Togo et al. [1] tried to estimate physiologicalactivities by Bayesian classifier. Kamiguchi et al. [2]proposed to estimate physiological activities by multiple
regression analysis. But these methods did not achieve©ISAROB 2011

II. ESTIMATION OF PHYSIOLOGICALACTIVITIES BY CONVENTIONAL ANN
We use artificial neural network (ANN) [4] for
estimation of physiological activity. An ANN has an

363

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

III. ANN WITH EXPONENTIAL OUTPUT

input layer, a hidden layer and an output layer. An
output of neuron j is calculated by Equation (1),(1)
o ij  f ( x i w jk ),NEURONWe employ exponential function f x   e tx as theactivation function in output neuron. Here we introduce
a parameter t into exponential function to adjust its
slope. By using exponential function as the activation
function in output neuron, ANN can directly handle thephysiological activity as the training signals those are
more than 1.0. Since normalization is unnecessary for
proposed ANN, the error does not increase by rescaling.
Exponential function is suitable for handling large
physiological activities that is difficult to handle bysigmoid function.
Fig. 1 shows a sample of three layer ANNs. The
ANN has an input layer, a hidden layer, and an output
layer. All neurons of an input layer connects all neurons
in hidden layer, and a neuron of hidden layer connects
all neurons in output layer. Constitution of our proposed
neural network is three layer model and the activation
function of the output neuron is exponential function,and that in other layers is sigmoid function.

kwhere x i i  0,1,2,, I  is the i-th input vector,
w jk  j  0,1,2, , J , k  0,1,2,  K  is a weightvector between the neuron j and the neuron k .
f x  is an activation function. Weight update
algorithm of artificial neural network is usually backpropagation algorithm. Back-propagation algorithm
minimizes the error between the calculated output oand the desired output y based on the steepest descentmethod. When the sum of the error define by Equation(2) becomes less than acceptable error level E  ,training process is terminated.
IEo  y 
i

i 2

(2)

i

where o i and y i is the output of ANN and thedesired output for the i-th training sample,
respectively.
Estimation of physiological activity by conventional
ANN has a problem. Physiological activities sometimes
take positive real number more than 1.0, but dynamic
range of ANN is limited from 0.0 to 1.0. So
physiological activities have to be normalized for
estimating by conventional ANN. Since dynamic rangeof physiological activities is usually from 0.0 to 3.0,
previous works divide physiological activities by the
maximum one. If the acceptable error level to terminate
training process is decided based on the originalphysiological activity, the scaled acceptable error level
will become too small to terminate training process. If
the acceptable error level is decided based on the scaled
physiological activities, and it is large enough toterminate training process, obtained error will become
too large because it is multiplied by the maximum valueof the physiological activity. Kuno et al. [5] proposed
Amplitude Extended Neural Networks to solve thisproblem. Amplitude Extended Neural Network
introduces two parameters k and l into sigmoid
function as y  k 1  elx . The parameter k adjusts
amplitude of sigmoid function and the parameter ladjusts slope of sigmoid function. Dynamic range of
Amplitude Extended Neural Network is extended from
0.0 to k , but we have to decide two parameters foreach physiological activity.

©ISAROB 2011activation func.

x1

activation func.

x2

y

xi

Fig. 1 Neural network with exponential output neuron
We evaluate proposed ANN by preliminary
exercises. An artificial data are generated by following
equations data1~data4. An input consists of three
dimensional vector a1, a2 , a3  , and an output is a scalary . The elements of input vector is decided at random.
The range of the elements in a input vector is 0.0 to 3.0.
The x in the following equations is sum of theelements of input vector x  a1  a2  a3. All x satisfy
the condition 0.0  x  5.0. The output y isnormalized by dividing the maximum value forconventional ANN. Table 1 shows ANN parameters forpreliminary exercise. We continue the training processuntil it arrives at the maximum epoch.



364

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011



data1: y  x 2

data2: y  x  2 x  1

data3: y  x 3
data4: y  5 1  e  x constituent of the food, and apply simple linear
regression analysis for these combinations. The
combination that has the lowest residual is selected as
the adequate training sample.
We use thirty kinds of constituents for three
concentrations as training samples. A part of theconstituents as shown in Table 3. The constituent isgiven to HepG2 cells, then protein expression levels andphysiological activities are measured, respectively. So adata-set involves 90(=30 × 3) measured values. We
measure the expression levels of following proteins;
Survivin, HSP70, XIAP, FADD, TXNRD1, HSP90,
MxA, tNOX, NQO1, ERK2, p53 and Bcl2.
We compared conventional ANN with proposed
ANN for six kinds of physiological activities. Theseexperiments run ten times with the same parameter.
Table 4 shows training parameters, and Table 5 shows
the number of samples. We evaluate each method by
estimation accuracy defined by Equation (3).
Number of estimated samples
accuracy  100, (3)Total number of samles
where the number of estimated samples means that the
number of samples they satisfied the error less than 0.2.

2

Table 2 shows mean square errors between thetheoretical y and the output of ANN.
Table 1 ANN parameter for preliminary exercises.
Training rate
0.4
Momentum
0.7
Maximum epoch
20,000The number of input neurons
3
The number of hidden neurons
2
The number of output neurons
1
Slope of exponential function t
1/256
The number of training sample
400
Table 2 Mean square error of preliminary exercises.
data1 data2 data3 data4
convetional ANN 0.184 0.204 0.220 0.132
proposed ANN
0.163 0.193 0.134 0.123
Proposed ANN could obtain more accurate
approximation model than conventional ANN as shownin Table 2. Mean square error of conventional ANN
becomes larger than proposed ANN because the error of
conventional ANN becomes large by rescaling. Table 2
also says that proposed ANN is suitable for the model
equation approximation like as quadratic or cubic
function.

Table 3 Samples of constituents andtheir concentrations
concentrations(μM)
LipoicAcid
100
300 1000
EGCG
7
20
50
Genistein
10
20
60
Daizein
25
50
150
Glycitein
1030
100
Qurcetin
5
15
60

IV. PHYSIOLOGICAL ACTIVITIESESTIMATION

Table 4 ANN parameters for
estimation experiments
Training rate
0.4
Momentum
0.7
Maximum epoch
20,000The number of input neurons
13
The number of hidden neurons
6
The number of output neurons
1
Slope of exponential function t
1/256
Acceptable error level
0.2

ANN needs appropriate training samples to make asuitable model. Measured physiological activities andprotein expression levels include relatively large noise
come from cell condition and measurement
environment. We solve this problem by exception ofoutlier using Smirnov-Grubbs test.
Physiological activities and protein expression levels
are measured in multiple times for a constituent of thefood, and these are not corresponded each other. It
means that we do not construct appropriate training
samples consisting of a physiological activity as a
training signal and protein expression levels as input
signals. We use simple linear regression analysis tocorrespond a physiology activity and protein expression
levels. We make all the available combination between
the physiological activities and the protein levels for a©ISAROB 2011

Table 5 The number of sample
physiological
The number of
The number ofactivity
training sample
testing sample
anti-oxidant
627
128
anti-proliferative
642
131
anti-inflammation
637
130
anti-angiogenic
642
131

365

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

proposed ANN

conventional ANN

the x-axis is measured value and the y-axis is estimatedvalue. In Fig.3 and Fig.4, the dotted-line shows boarderthat satisfies ±0.2 of measured values. Proposed ANN
can estimate relatively large physiological activity
shown in circles in Fig. 3 and Fig.4.

100%
90%

Estimation accuracy

80%

70%
60%
50%
40%
30%

V. CONCLUSIONS

20%

In this paper, we proposed a neural network withexponential output neuron and applied it to estimate
physiological activities from protein expression levels.
Experimental results showed that our method achieved
more accurate than conventional ANN withnormalizationforanti-oxidant
activity,
antiinflammation activity and anti-angiogenic activity.
Furthermore it could realize to directly use measured
physiological activities as training samples. However,proposed neural network couldn’t follow small
physiological activity. It is future subject that estimation
by multi-model ANNs to estimate both of large andsmall physiological activities.

10%0%
anti-oxidant

anti-proliferative

anti-inflammation

anti-angiogenic

Fig. 2 Results of estimation accuracy
1.6

1.4

Estimated

1.2

1

0.8

0.6

0.4

0.2

0
0

0.2

0.4

0.6

0.8

1

1.2

1.4

REFERENCES

1.6

Mesured[1] Togo S. (2008), Estimating physiological activities
of functional foods form protein expression levels usingbaysian classifier, The Thirteenth International
Symposium on artificial Life and Robotics 2008,
pp.853-856, OS12-3.
[2] Kamiguchi M. (2008), Estimation of Physiological
Activity Values from Protein Expression Levels with
Multiple Regression Analysis, Memoirs of the Facultyof Engineering, University of miyazaki, No.37, pp.269273.
[3] Tsukuda S. (2007), Estimation of physiology
Activity in Foods with Neural Network from
Insufficient Data, Memoirs of the Faculty ofEngineering, University of miyazaki, No.36, pp.345350.
[4] Sholom WM., Nitin I. (1998), PREDICTIVE DATAMINING., Morgan Kaufmann Publishers, Inc, San
Francisco, California.
[5] Kuno T. (2009), Development of Physiological
Activity Estimation System of Foods Using Amplitude
Extended Neural Networks, The fourteenth International
Symposium on artificial Life and Robotics 2009,
pp.853-856.

Fig. 3 Scatter diagram for anti-angiogenic activity
1.6

1.4

1.2

Estimated

1

0.8

0.6

0.4

0.2

0
0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

MeasuredFig. 4 Scatter diagram for anti-inflammation activity
Fig. 2 shows estimation result of four physiologicalactivities(anti-oxidant,
anti-proliferative,
antiinflammation, anti-angiogenic activity) using proposed
ANN and conventional ANN. Proposed ANN is moreaccurate than conventional ANN with normalization foranti-oxidant activity, anti-inflammation activity andanti-angiogenic activity. Fig. 3 and Fig. 4 show
estimation results for the anti-angiogenic activity andthe anti-inflammation activity by scatter diagram where©ISAROB 2011366

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Asynchronous migration for parallel genetic programmingon computer cluster with multi-core processers
Shingo KUROSE1 Kunihito YAMAMORI2 Masaru AIKAWA3 and Ikuo YOSHIHARA2
1

Graduate School of Eng., Univ. of Miyazaki
2
Faculty of Eng., Univ. of Miyazaki
3
Technical Center, Faculty of Eng., Univ. of Miyazaki(Tel : 81 985 58 7589; Fax : 81 985 58 7589)(kurose@taurus.cs.miyazaki-u.ac.jp,{yamamori,yoshiha,aikawa@cs.miyazaki-u.ac.jp)

Abstract: Island model is a typical implementation model of parallel distributed genetic algorithms, and it is also usedin parallel genetic programming. Island model has migration process that exchanges individuals between subpopulations to leave local optimum. However island model requires synchronous process to exchange individuals at thesame generation, and synchronous process increases computation time.
This paper proposes a new parallel genetic programming model based on the island model with asynchronous
migration. We implement island model using Massage Passing Interface (MPI). Fitness calculation which requires thelongest computation time is processed in parallel by multi-threading. In addition, proposed method employs acommunication thread for migration between computation nodes, and communication thread communicates withanother communication thread to exchange individuals at appropriate intervals. The communication and genetic
operations can be independently processed on each core. Experimental results show that proposed method with fivecomputation nodes and forty threads can reduce computation time about 17% of serial GP.
Keywords: Genetic programming, MPI, Multi-threading, Island model, Asynchronous migration

I. INTRODUCTION

II. PARALLEL GENETIC PROGRAMMING

Genetic Programming (GP) [1,2,3] is one of theevolutionary algorithms for optimization inspired from
biological evolution. GP expresses a candidate ofsolution as a structural individual like a tree. Eachindividual is evolved by genetic operations such as
crossover and mutation, and then only the individuals
that have superior fitness remain for next generation.
Through these evolutionary processes, GP can make amodel automatically. Computation time of GP becomes
longer as increasing of the number of individuals andthe generations to obtain more accurate solution. So GP
is usually implemented in computer cluster. Parallel GP
implementation can be classified into two models;master-slave model and island model [2].
This paper proposes a new parallel genetic
programming model based on the island model with
asynchronous migration. We implement island modelusing Massage Passing Interface (MPI) [3]. Fitness
calculation which requires the longest computation timeis processed in parallel by multi-threading [4]. In
addition, proposed method employs a communication
thread for migration between computation nodes toreduce synchronous overhead. We evaluated proposed
method on computer cluster.

©ISAROB 2011

1. Island modelIsland model [2] is one of the implementation models
for parallel distributed genetic algorithms and it is also
used for parallel genetic programming. Island model
divides a population consisting of individuals into subpopulations, and it assigns a sub-population to acomputation node. Individuals in each sub-population
Serial GP

Computation
node 1

Individuals

Island model
GP

Computation
node 1

Subpopulation1

Migration
Subpopulation
2

Subpopulation
3

Computation
node 2Computation
node 3Fig. 1 An example of island model by threecomputation nodes.

367

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

are independently evolved in parallel. The number ofindividuals in sub-population is reduced, and genetic
operations for individuals in each sub-population areprocessed in parallel, island model can reduce
computation time. In usual, island model has migration
process, that immigrates individuals from subpopulation to another sub-population, to leave local
optimum. Figure 1 shows an example of island modelby three computation nodes.

programming.
Recent processers equip some processing cores on adie, and they share a main memory. This architecture issuitable for multi-threading [4]. Our method creates
threads for fitness calculation, and these threads work inparallel. Shared memory is used to exchange individuals
between main thread and fitness calculation threads.
Proposed method expects high speed processing byusing both inter-nodes parallelization by MPI and intranodes parallelization by multi-threading. Figure 2 shows
intra-nodes parallelization using fitness calculation
threads.

2. Synchronous migration
Migration is a process to immigrate individuals from
sub-population to another sub-population at every some
generations. So the migration on computer cluster
requires network communication to exchange
individuals. Migrated individuals are chosen by various
ways. For example, migrated individual is an elite thathas the highest fitness in sub-population. In other way,some individuals are chosen at random. Before
migration, computation nodes synchronize theirgeneration because independent evolution leads thedifference of computation time for each sub-population.
So some sub-populations have to wait their migration
until the slowest sub-population has finished theiroperations.

2. Asynchronous migration
Proposed method creates a communication thread to
reduce waiting time by synchronization. At first mainthread creates individuals at random, and calculates
their fitness. Then the main thread of each computation
node selects an elite individual and stores it into thetransmission buffer. Here, synchronization is executed
just for once for reliable migration. Next, the mainthread creates a communication thread and it transfer
the individual stored in the transmission buffer to thecommunication thread in the other computation node atappropriate interval. The elite individual is exchanged
through the shared memory between the main thread
and the communication thread. The communication
thread also receives the transferred individual from theother communication thread and stores it into thereceiving buffer. The main thread takes the migrated
individual from the other sub-population into their own
sub-population at appropriate interval when the genetic
operations are finished. Figure 3 shows an example ofindividual exchanging between the main thread and thecommunication thread. Figure 4 illustrates an example

III. ASYNCHRONOUS MIGRATION FOR
PARALLEL GENETIC PROGRAMMING
1. Inter-nodes parallelization andintra-nodes parallelization
Proposed method implements island model using
Massage Passing Interface (MPI) [3] for inter-nodes
parallelization. MPI is a standard library for parallel<Usual state><Fitness calculation>

Computation
nodeComputation
node

Shared memory

Shared memory

Subpopulation

Subpopulation

Write

Read

Write

ProcesserCore 1

Computation node
Shared memory

Subpopulation

MainthreadCore 1

Core 1Read

Read

The transmission bufferCore 2MainthreadFitness
calculation
threadCore 3

Core 4

Fitness
calculation
threadFitness
calculation
threadCore 2

Core 4Read

The receiving buffer

Communication
threadReceive

Another
communication
threadFig. 3. An example of individuals exchanging
between the main thread andthe communication thread.

Fig. 2. Intra-nodes parallelization
using fitness calculation threads.

©ISAROB 2011

Communication
Send

Write

Core 3Main thread

Write

ProcesserCore 2

Processer

368

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table. 3. Parameters of GP.

Computation
node 1

Computation
node 2

Shared memory

Shared memory

Subpopulation

Subpopulation

Write

Read

Write

Processer

Read

ProcesserCore 1

Core 2

Core 1

Core 2MainthreadFitness
calculation
threadMainthreadFitness
calculation
threadCore 3

Core 4

Core 3

Core 4

Communication
threadFitness
calculation
threadFitness
calculation
threadParameters
The number ofgeneration
The number of
Individual
Max depth for
Individual
The number ofTraining samples
Operator

Communication
threadCommunication

Selection

Fig. 4. An example of task allocation for
two computation nodes withfour processing cores in a processor.

Mutation rate
Migrating individual
Migration intervalof task allocation for two computation nodes with four
processing cores in a processor.

We evaluate processing time and accuracy ofobtained solution on computer cluster. Table 1 shows
experimental environments. Computer cluster equips
distributed memory, and a multi-core processer has four
processing cores, and it equips hyper threading
technologies®, so eight threads can work in parallel. We
compare with three implementation models of parallel
GP. Table 2 shows detail of each model. The iGP isparallel GP using basic island model. The tfGP is themodified iGP using fitness calculation threads. In thetfGP, eight fitness calculation threads are created. The
amGP is also the modified iGP using fitness calculation
thread and communication thread for asynchronous
migration. In the amGP, seven fitness calculation
threads and one communication thread are created. We
can evaluate the effect of intra-nodes parallelization by
Table. 1. Experimental environments.
Intel® Xeon E5530
CPU
2.40GHz ×2
Core/Thread
4 cores / 8 threads
Memory8 GB
The number of
12 nodescomputation node
Table. 2. Details of each model.
Island modelThreaded fitness
calculation
Asynchronous migration©ISAROB 2011

tfGP

amGP

○

○

○

×

○

○

×

×

○

300
8
600
+, -, *, /, sin, cos, tan, log
Elitist schemes
Roulette selection
0.1
Elite
Every 50 generationcomparing the iGP with the tfGP. In addition, we canalso evaluate the effect of asynchronous migration bycomparing tfGP with amGP. Table 3 shows parameter of
GP. Individuals are divided into sub-populations equally.
In experimentation, training samples are taken from [5].
Figure 5 shows the computation time with respect tothe number of computation nodes. Serial GP takes
about 1,000 sec. for 1,000 generations. The iGP withfive computation nodes can reduce computation timeabout 46% of serial GP. However, computation time isincreased when more than six computation nodes areused. Figure 6 shows the average number of nodes in an
individual for each number of computation nodes. In theiGP, the average number of nodes in an individual
increases when the number of computation node isincreased. Computation time of fitness calculation isincreased by increasing of the number of nodes in an
individual. As a result, the computation time for fitness
calculation is different among sub-population, and it
leads large overhead for synchronous migration. This is
the reason of increasing of computation time when morethan six computation nodes are used on the iGP and
tfGP.
In Figure 5, the tfGP with five computation nodes andforty threads can reduce computation time about 21% ofserial GP. Computation time is also increased whenmore than six processors are used as well as the iGP. It
is because that synchronous migration takes large
overhead as well as the iGP.
In Figure 5, the amGP with five computation nodes
and forty threads can reduce computation time about
17% of seriasl GP.
Figure 7 shows the average fitness for each numberof computation nodes. Fitness of individual s is

IV. EXPERIMENTAL RESULTS

iGP

1,000

369

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

output of the model for the i -th training sample,
respectively. In Figure 7, the average fitness of amGP
increases as well as the iGP and the tfGP when thenumber of computation node is increased. In Figure 5,
computation time of amGP doesn’t increase even if
more than six processors are used. So, proposed method
is effective method when many computation nodes areused to obtain more accurate solution.

calculated by Equation (1).
1Fitness 
F

(1)

n

F 



1
 yˆ i  y i 
n i 1In Equation (1), n is the number of training
samples, ŷ i and y i are the desired value and theComputation time (sec)

1600

iGP
tfGP
amGP

1400
1200

V. CONCLUSION
This paper proposed a new parallel genetic
programming model based on the island model using
asynchronous communication between computation
nodes. We implement island model using Massage
Passing Interface (MPI). Our method creates threads forfitness calculation, and these threads work in parallel. In
addition, proposed method employs a communication
thread for asynchronous migration between computation
nodes. Experimental results showed that fundamental
parallel island model with five processors can reduce
computation time about 46% of serial GP. Furthermore,
our proposed method with five computation nodes andforty threads can reduce computation time about 17% ofserial GP. In future, we will improve efficiency ofparallelization by investigating the load of eachcomputation node.

1000
800
600

400
200
0
1

2

3

4

5

6

7

8

The number of computation node

The number of node in individual

Fig. 5. Computation time with respect to thenumber of computation nodes.
20

iGP and tfGP
amGP

18
16
14
12

10
8
6

REFERENCES

42[1] Numata N, Sugawara K, Yamada S et al. (1999),
Time Series Prediction Modeling by Genetic
Programming without Inheritance of Model Parameters.
Proc. Fourth Int. Sym. Artificial Life and Robotics:500503[2] Eklund SE (2003), Time Series Forecasting using
Massively Parallel Genetic Programming. Parallel andDistributed
Processing
Symposium:
10.1109/IPDS.2003.1213272[3] Messom CH and Walker MG (2002), Evolving
Cooperative Robotic Behaviour using Distributed
Genetic Programming. Control, Automation, Robotics
and Vision: 215-219[4] Nichols B, Buttlar D, Farrell JP (1996), Pthreads
Programming (A Nutshell handbook). Oreilly &Associates Inc (California).
[5] Kamiguchi M, Yamamori K, Yoshihara I et al.
(2009), An Automatic Model Building for Screening
Functional Foods with GP. ICROS-SICE International
joint Conference 2009:3679-3684

0
1

2

3

4

5

6

7

8

The number of computation nodeFig. 6. Average number of nodes in individuals foreach number of computation nodes.
10.5

iGP and tfGP
10

amGP

Fitness

9.5

9

8.5

8
1

2

3

4

5

6

7

8

The number of computation nodeFig. 7. Average fitness for each number ofcomputation nodes.

©ISAROB 2011370

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Classification of species by information entropy
and visualization by self-organizing map
Kentaro NISHIMUTA1), Ikuo YOSHIHARA1), Kunihito YAMAMORI1), Moritoshi YASUNAGA2)
1)

1-1, Gakuen Kibanadai-nishi, Miyazaki, 889-2192, Japan, University of Miyazaki(Tel : +81-985-58-7384; Fax : +81-985-58-7384)(Email address : { nmuta, yoshiha, yamamori }@taurus.cs.miyazaki-u.ac.jp)
2)1-1-1, Tennoudai, Tsukuba, Ibaraki, 305-8573, Jaban
Graduate School of Systems and Information, Engineering,
University of Tsukuba

Abstract: Most analysis methods of base sequences are based on the idea of pattern matching, but feature patterns
known until today are supposedly a part of all the patterns hidden in base sequences. We develop a novel analyzing
method based on disorder of base sequences, for example variance of data, 1/f fluctuation, information entropy andself-information. We try to classify species based on self-information of base sequences. Relations between species arevisualized by self-organizing map (SOM). It is probable that neighboring regions of the SOM corresponds to near
species. We compare neighboring regions of the SOM with neighboring branches of an evolutionary tree produced by
the Clustal-W system.
Keywords: DNA, self-information, information entropy, self-organizing map

information is calculated from appearance rate ofcodons in sub-sequences cut out from whole the basesequence.
We try to classify species based on self-informationof base sequences without using pattern matching, andvisualize relations of species by self-organizing map(SOM). One region represents one species.
We validate the method of classification of species
and to compare neighboring regions on the SOM withneighboring branches of evolutionary tree produced by
the Clustal-W system[7].

I. INTRODUCTION
Evolution degeneration of species must be remainedin base sequences, because mutations have been
accumulated in them for a long time. Various analysis
methods on base sequences are investigated forrevealing evolution of living creatures.
Most analysis methods of base sequences are basedon pattern matching, but feature patterns known until
today are probably a part of iceberg i.e. enormous kindsof patterns are ought to exist. Rule based analysis
cannot be applicable to genes of unknown species.
We make up our mind to change the viewpoint ofresearch from strict pattern matching to roughcomparison. We propose a novel analyzing method
using disorder of base sequences. Though disorder issomewhat vague, various features of species arereflected in disorder.
There exist many candidate methods of analysis
concerning to disorder, for example, data compression
ratio[1], long range order used in statisticalmechanics[2], 1/f fluctuation[3], variance of data[4][5],information entropy[6] etc .
This article employs information entropy, especiallyself-information as an index of disorder. Self-©ISAROB 2011

II. DISORDER ANALYISIS BY
INFORMATION ENTROPY AND SOM
1.

Composition of base sequences
Base sequences consist of four kinds of bases;
A(Adenine), C (Cytosine), G (Guanine), and T(Thymine).
A sequence of three adjacent bases is called codon.
The number of codons is 64=43, because codon is a
triplet of 4 kinds of bases.

371

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

2.

Information entropy
Information entropy of species is assumed to be ameasure of characteristic of species.
The complete event system (X, P) is determined by
the occurrence probability P(Xi), where Xi is one of 64
codons. { X i } corresponds to AAA, AAG, AAC,

The output layer of SOM is allocated on rectangle
lattice(x,y). The size of output layer is arbitrarily
decided.
Weights connect input layer and output layer. Weightis strength of connection between input layer and output
layer. Two layers unite completely by weight (Fig 1).
The relation between the output(O) ,weight(W) and the
input(I) is shown by the following expressions.

… ,TTT.

AAG
 X   AAA
   ,  ,
 P   P(AAA) P( AAG)

TTT P(TTT)  .

64

O( x, y )   Wi I i .

Self-information is defined by the probability P(Xi),
Self-information has the same number as the codon.

i 1

We use SOM to distinguish species by selfinformation.

I ( X i )   log 2 P( X i ) (i  1,2,,64)

4.

Positional information entropy
Sub-sequences are parts of a base sequence cut out
with a given length of bases and are information entropy
a positional function of the base sequence (Fig 2).

Information entropy H(X) is an expected value ofself-information I(X), i.e. H ( X )  P( X ) log 2 P( X ).
H (X ) derives as follows,64

H ( X 1 , X 2 ,  , X n )   P( X i ) log 2 P( X i ).
i 1

3.

Self-organizing map (SOM)
Self-organizing map (SOM) proposed by Kohonen isan unsupervised-leaning neural network which is used
for clustering and visualization [8].
The codon of self-information as vector of 64th
dimension is used for input of SOM. SOM has twolayer structure. Input layer has nodes the number ofwhich is the same as the number of dimension of the
input.

Fig 2 Base representation of the whole base sequence
Self-information of codons is calculated by theappearance probability of codons in sub-sequence (Fig3).

Output layer
weightInput layer
Fig 3 Codon representation of sub-sequences

Fig 1 Illustration of input and output layers of SOM©ISAROB 2011372

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

III. EXPERIMENTS
1.

Experimental condition
We validate the proposed method by the following
steps.
[Step 1]. Calculation of self-information.
[Step 2]. Visualization by SOM.
[Step 3]. Comparison SOM with evolutionary tree.
Experimental data of base sequences are ribosomal
protein genes of 20 species in Table 1, which are takenfrom the Ribosomal Protein Gene (RPG) Databaseestablished by Frontier Science Research Center,
University of Miyazaki[9].
The ribosomal proteins used are as follows;
RPL3 RPL4 RPL5
RPL6
RPL8 RPL9 RPL10 RPL10A

RPL7
RPL7A
RPL17 RPLP23
Fig 4 Result of pre-experiment (6 species)2.

Pre-experimentFirst of all, we need to determine experimentally anappropriate length of sub-sequence. We conduct small
scale experiments will give us nearly optimal
parameters of SOM. 6 kinds of species are used;{ A(Hs), B(Mm), H(Ag), I(Ce), J(Mg), and O(Um) } in
Table 1.

4.

Experiment
We validate the proposed method employing data of
20 species. Length of sub-sequences is same for all
species i.e.2048 bases. Parameters in experiments are as
follows; map-size is, and iteration of learning
is 100.

We change the length of sub-sequence from 64 bases
to 2048 bases. Sub-sequence cannot cut out more than
2048 bases, because length of base sequence is different
because of species. Parameters of experiments are as
follows; map-size is, and iteration of learning
is 100.

5.

Results and discussions
Fig 5 (A) is a map produced by SOM and Fig 5 (B) isan evolutionary tree produced by the Clustal-W system.
We take a look of parts with same colored mask pattern.
The map of SOM is separated in 21 regions. Thereare 2 regions of C(Rn). We confirmed to correspond
between the neighboring regions of the SOM to theneighboring branches of the evolutionary tree.

3.

Results of pre-experiment
Length of sub-sequence 2048 gives good figure of
SOM(Fig 4). Species are separated in 6 regions. Lengthof sub-sequence is long, species of 6 kinds all separated.

Table 1 Correspondence table for experiment
symbolabbr

speciessymbolabbr

speciessymbolabbr

speciessymbolabbr

speciesA

Hs

H.sapiensF

Dm

D.melanogasterK

Fg

F.graminearumPRo

R.oryzae

B

Mm

M.musculusGAm

A mellifera

L

Yl

Y.lipolytica

Q

Cn

C.neoformans

C

Rn

R.norvegicus

HAg

A.gamiae

M

Sc

S.cerevisiaeR

Dd

D.discoideumDFr

F.rubripes

I

Ce

C.elegans

N

Sp

S.pombeSAt

A.thaliana

E

Ci

C.intestinalis

J

Mg

M.griseaO

Um

U.maydisT

Cr

C.reinhardtii©ISAROB 2011373

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011(A)
Fig 5 Comparison of SOM with evolutionary tree

[3] Y. Koyama, K. Nishimuta, K. Yamamori, I.
Yoshihara, M. Yasunaga (2010), “Quest for genetic
information hidden behind disorder in DNA
sequences”, Proc. of 15th Int. Symp. on Artificial
Life and Robotics (AROB’15),824-827[4] K. Yamamori, Y. Fujita, M. Aikawa, I. Yoshihara(2007), “Identification of Exon-intron Boundaries
by Integration of Base-oriented Genetic
Programming and Statistical Heuristics”, Proc. of
12th Int. Symp. on Artificial Life and Robotics(AROB’12), 657-660[5] T. Abe, S. Kanaya, M. Kinouchi (2003)“Informatics for Unveiling Hidden Genome
Signatures”, Genome research, 13, 693-702[6] S. Miyazaki, H. Sugawara, M. Ohya (1996), “Theefficiency of entropy evolution rate forconstruction of phylogenetic trees”, Genes &Genetic Syst, vol.71,5,323-327[7] Clustal-W system; http://clustalw.ddbj.nig.ac.jp/[8] T. Kohonen (2005), Self-Organizing Maps.
Springer-Verlag Tokyo. (Translated into Japaneseby Tokutaka et.al.)[9] RPG database; http://ribosome.med.miyazakiu.ac.jp/

IV. CONCLUSION
We proposed an analyzing method of base sequences
employing disorder. The method is emposed of
calculation of self information from base sequences of
species and visualization by SOM. A rate of SOM is toseparate all the species into different regions. We
compare neighboring regions of SOM with neighboring
branches of evolutionary tree produced by the ClustalW system.
The neighboring regions of the SOM are
corresponding to the neighboring branches on
evolutionary tree. The experiments lead us to theconclusion that the analysis with disorder and SOM isuseful for classifying species.

REFERENCES[1] I. Yoshihara, H. Takashima, K. Yamamori, K.
Sugawara (2006),”Similarity Comparisons of
Seeds Using Image Compression Technology”,
Memoirs of the Faculty of Engineering, Universityof Miyazaki,35,251-256[2] Y. Sakaguchi, I. Yoshihara, K. Yamamori, N.
Kenmochi(2006),”Criterionof
Evolution
Analyzing DNA Sequences based on Statistical
Mechanics” Memoirs of the Faculty ofEngineering, University of Miyazaki,35,269-274©ISAROB 2011

(B)

374

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Digital-signal-waveform improvement
on VLSI packaging including inductances
Hiroki SHIMADA†, Shohei AKITA†, Masami ISHIGURO†, Noriyuki AIBE†,
Ikuo YOSHIHARA‡, and Moritoshi YASUNAGA††Graduate School of Systems and Information Engineering, University of Tsukuba,
1-1-1 Tennodai, Tsukuba-shi, Ibaraki-ken, 305-8573, Japan(Tel : 81-92-853-5323)(Email:{shimada@islab, akita@islab, masami@islab, yasunaga, susu@islab}.cs.tsukuba.ac.jp)‡Faculty of Engineering, Miyazaki University,
1-1, Gakuen-kibanadainishi, Miyazaki, Miyazaki, 889-2192, Japan(Email: yoshiha@cs.miyazaki-u.ac.jp)
Abstract: As digital-signal frequency in PCBs (Printed Circuit Boards) increases, waveform-distortion, or the Signal
Integrity (SI) problem is getting serious more and more. The reason why the SI becomes serious is that wires, or traces
need to be regard as transmission lines, which are sensitive to electric noises. In order to overcome this problem, we
have already proposed a novel methodology called STL (Segmental Transmission Line), and have also shown its
effectiveness using computer-simulation and fundamental prototypes. In the STL design however, combinatorial
explosion problem occurs as a big problem. To solve this problem, Genetic Algorithms (GA) is used to design STL. In
this paper, we newly apply the STL to the bus system that includes inductances, which come from the VLSI packaging.
We evaluated the STL on the simulation experiments and the actual experiments using prototypes, and obtained the
maximum improvement ratio of 1.53 in the actual experiment.
Keywords: Signal Integrity, Transmission Line, Genetic Algorithms, Inductance, Printed Circuit Board
DigitalSignalI. INTRODUCTION
From hundreds of MHz or more, digital signals
begin to behave as waves, so that the PCB (Printed
Circuit Board) traces must be designed as transmission
lines. The transmission line is characterized by thecharacteristic-impedance Z, which shows the easiness ofthe signal to transmit. And if some devices or electricalcomponents, which are equivalent to capacitors or
inductors, are connected to the transmission line, they
become sources of impedance-mismatching, and cause
waveform-distortion, or decrease SI (signal integrity) as
shown in Fig. 1.
Conventionally, some local-impedance-matching
techniques such as SSTL[1] have been used to solve theimpedance-mismatching problem. These techniques,
however, have not been going well gradually as thefrequency increases and approaches to the GHz-region.
In order to overcome this difficulty in theconventional techniques, we have already proposed anovel technique called “Segmental Transmission Line
(STL)”, and have shown its fundamental effectivenesson some simple transmission models [2][3], where only
capacitors are connected. In the high-speed digitalsignal transfer, however, not only the capacitances but
some inductances, which come from VLSI packages
and connectors connected two PCB boards for example,
also cause impedance-mismatch and affect the
waveform.
In this paper, we apply the STL to the bus system
where inductances are also connected to thetransmission line as well as the capacitances, and show
its effectiveness quantitatively on computer-simulations©ISAROB 2011

Noise superposes
Digital Signal

Reflected or transmitted waves

Dumping
ResistorC

Pulse
SourceFig.1

Capacitors are equivalent
to devices

C

Terminal
Resistor

Decline of Signal Integrity

and actual experiments.

II. SEGMENTAL TRANSMISSION LINE
In order to improve SI in PCB traces, we have alreadyproposed a novel transmission called Segmental Transmission
Line (STL) [2]. In the STL, a transmission line is divided into
multiple segments of individual characteristic-impedance Zi
and segment-length Li. In the strip-line structure, or microstrip line structure in the PCB traces, the characteristic
impedance Z is a function of the segment-width W, i.e.,
Z=f(W). Each segment has thus its own width Wi and length Li
individually as design parameters as shown in Fig.2 if the STL
is applied to the PCB trace.
Low signal integrity
Digital Signal

Z i  f (Wi )High signal integrity
Digital Signal

Reflected or transmitted waves

L1、W1
L：Length
W:Width

Fig.2

375

L2、W2

L3、W3

L4、W4

Impedance mismatching
ObservepointsPoint
Configuration of Segmental Transmission LineThe Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Each segments has individual characteristic impedance Zi
and length Li. （i is a natural number which begin from 1）
Z1 Z2 Z3
Z4 Z5 Z6 Z7 Z8 Z9 Z10 Z11

In the STL, each boundary between adjacentsegments causes an impedance-mismatch-point as amatter of course, and reflection waves and transmitted
waves occurs at each boundary as a result. In the STL,
superposition of the multiple reflection-waves from allboundaries is used to improve the waveform distortion
at target points in the transmission line by adjusting Wi
and Li of each segment.

W1 W2 W3CRD

W4 W5 W6 W7 W8 W9 W10 W11
RTCV
Map into Chromosome Structure
Chromosome

III. GENETIC ALGORITHMS

•
•
•
•

In the STL, however, there is a troublesome barrier
in its design: finding the best combination of allparameters is next to impossible because combinatorial
explosion occurs in its search process. Assuming that
there are 10 segments, each of which consists of 10
width-candidates and 100 length-candidates, the total
number of combinations comes to.
This astronomical number prevents us to evaluating allof them in real time. In order to overcome this problem,
we apply the genetic algorithms (GA) [4], which is oneof optimization algorithms mimicking the biologicalevolution, to the problem. And more positive reason we
use GA is that the STL structure of one dimentional
array will be well mapped onto the chromosome
stroucure in GA as shown in Fig.3.
Figure 4 shows the flowchart of the STL-designusing GA. Genetic operations are executed repetitively
until they reach the finish condition. Difference area
between the target and the observed waveforms shownin the figure is used to evaluate the chromosome, that is,
the reciprocal of the difference area is used as score, or
fitness of the chromosome.

V
RD
C
RT

RD Z1 Z2 Z3 Z4 Z5 Z6 Z7 Z8 Z9 Z10 Z11 RT
L1 L2 L3 L4 L5 L6 L7 L8 L9 L10 L11: Pulse Source: Dumping Resistor: Capacitative Load
: Terminal Resistor

Fig.3

Structure of STL and Chromosome
Start

Output Netlist

Generate Initial
Chromosome

Run SPICE andoutput wave data

Crossover
Mutation

Calculation ofDifference Area

Evaluation
Election
Judge Finish
Condition

Difference Area diffTarget Wave
Observed Wave

FinishFig.4

IV. EXPERIMENTAL RESULTS
We use a bus-system shown in Fig.5 to evaluate the
STL. This bus-system models a transmission line with a
Ball Grid Array (BGA) or a Land Grid Array (LGA)
connected with it as shown in Fig. 6. There are bunchesof boding wires inside the BGA or LGA package, andthey need to be considered as inductances as thefrequency increases.

RD

STL-Design Flowchart using GA

15cm

35cm

Observing
Point

15nH
15pF

V

10cm

RT

RD：Dumping Register
RT：Terminal Register
V：Pulse Source

1.
Simulation Experiments
Figure 7 shows a waveform observed at theobserving point in Fig. 5. As shown in Fig. 7, the logicmargins degrease to critical ranges; high-level-logicmargin is 0.3V and low-level-logic-margin is 0.32V. To
recover the logic margins, STL is thus applied.

Fig.5

Target Bus-system

The cross section of Ball Grid ArrayLSI Packaging
Wire inside thepackage(Bonding Wire）

LSI

Ball

Fig.6

©ISAROB 2011

376

Cross sectional view of Ball Grid Array

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 1 shows the condition of the STL-Designerusing GA, and Fig.8 shows an STL design result for thebus-system including inductances. Figure 9 shows the
waveform at the observing point in Fig. 8 in the STL.
Both of the high-level-logic-margin and low-levellogic-margin increase to 0.5Vs. Table 2 summarizes theexperimental results. High improvement ratios of 1.67
and 1.56 were achieved in high-level-logic-margin andlow-level-logic-margin, respectively compared to aconventional transmission line.
Voltage[V]Voltage[V]
1.0

0.5[V]

0.8

0.4

0.5[V]
0.2

TargetWave

0.0
time[ns]
1.0ns /div

Observed
Wave

Fig.9

0.8

0.3[V]

0.6

0.2

0.32[V]

TargetWave

0.0
time[ns]
1.0ns /div

Waveforms of Conventional Transmission Lineon Simulations

6,7,3

Finish condition

1500 generations

SMA Connecter

Observing
Point
Linearly
implemented
chip capacitor andchip inductor

Linearly
implemented
chip capacitor andchip inductor

STL
320mm

Observing Point

Terminal
Register2.
Prototype Experiments
We fabricated some PCB prototypes to prove thecorrectness of the simulation results. Prototype PCB
boards are shown in Fig. 10: the upper is a conventionaltransmission line and the lower is an STL. On the PCBs,
chip inductors and chip capacitors are used and solderedas the inductors and capacitors in Figs. 5 and 8.

Impedance[Ω]Segment Length[cm]320mm

Terminal
Register

15cm,35cm,10cm(Total:60cm）

Fraction of segments

Normal Transmission Line

SMA Connecter

Table. 1 Conditions of GA
Lengths

Waveforms of the STL on Simurations

Table. 2 Summary of Simulation Results
High-levelLow-levellogic-margin logic-margin
0.3[V]
0.32[V]
Normal
0.5[V]
0.5[V]
STL
1.67
1.56
Improvement
rate

Threshold
0.5V

0.4

Extpanded Picture of STL: Segments have individual width and length

Fig.10 Prototype PCB

RD

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

RT

50

110

85

75

30

70

80

65

95

45

120

60

40

75

60

45

55

51

2.5

2.45

2.45

2.5

2.5

2.6

4.8

5.85

5.1

4.85

4.65

4.9

4.85

3.2

3.25

3.5

15cm

35cm

RD
V

15nH
Observing
Point

Fig.8

©ISAROB 2011

Observed
Wave
Threshold
0.5V

0.6

0.2V /div

1.0

Fig.7

0.2V /div

15pF

Design Results for STL

377

10cm

RT

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Figure 11 shows the measurement environment for the
prototype boards. Clock signals are supplied from thepulse generator into the PCBs, and the waveforms areobserved using an active probe and a digital sampling
oscilloscope.

Table. 3 Summary of Measurement Results Using
Prototype
NormalSTL
Improvement
rate

Oscilloscope
LeCroy Wave Runner 204Xi

Pulse Generator
Agilent 81132A

Band：2GHz
Sampling Rate：10GS/s

SMA Connecter

Low-levellogic-margin
0.36[V]
0.5[V]
1.39

As the Table. 3 shows, the improvement ratio of
high-level-logic-margin is 1.53, and low-level-logicmargin is 1.39. The results indicate that STL is effective
to the system which includes inductances directly before
the devices.

Active Probe
LeCroy HFP2500

SMA Cable

High-levellogic-margin
0.34[V]
0.52[V]
1.53

Prototype PCB

V. CONCLUSIONS
Observing Point

We newly applied the STL to the bus-system
including inductances. The maximum improvement
ratio of 1.67 for the distorted waveform in theconventional transmission line was achieved on thecomputer simulation. We also evaluated the STL using
the prototypes, and high improvement rations of 1.53
and 1.39 were achieved in the high-level-logical-margin,
and the low-level-logical-margin, respectively. Those
results clearly show that the STL is also effective to thebus-system including inductances as well as
capacitances.
Our future works is to make clear the differences
betweensimulation
andactualexperiments.
Furthermore, we will also try to make steeper
rising/falling waves in the STL.

Fig.11 Measurement Environment
Figures 12 and 13 show waveforms in theconventional trace and of the STL, respectively
observed with the oscilloscope. Logical margins of
0.34V in high-level and 0.36V in low-level are well
improved to 0.52V and 0.5, respectively in the STL.
Voltage[V]

0.2V /div

1.0
0.8
0.6
0.4
0.2

0.34[V]

Threshold0.5V

0.36[V]

0.0

REFERENCES[1] M.Taguchi, “ High-speed, small-amplitude I/Ointerface circuits for memory bus application”,
IEICE Trans. Electron., vol.E77-C, no.12,
pp.1944-1950, Dec.1994.
[2] Moritoshi Yasunaga, Ikuo Yoshihara, “TheSegmental Transmission Line and Its Design
Methodology: Line Trace Ensuring High Signal
Integrity in VLSI Mounting Boards”, IEICE
journal, D-I, J-88-D-I (5), pp.915-929, 2005. In
Japanese.
[3] Yuki Shimauchi, Hiroshi Nakayama, Yoshiki
Yamaguchi, Noriyuki Aibe, Ikuo Yoshihara,
Moritoshi Yasunaga, “Evaluation of waveformimprovement performance on the segmental
transmission line”, IEICE technical report109(317), Design Gaea 2009, pp.63-68, Dec. 2009.
In Japanese[4] Hiroshi Sato, Isao Ono, Shigenobu Kobayashi, “A
New Generation Alternation Model of Genetic
Algorithms and Its Assessment”, Journal ofJapanese Society for Artificial Intelligence Vol.12
No.5, pp.734-744, Sep.1997, In Japanese.

Time[ns]

2.0ns /div

Fig.12 Waveforms observed in Conventional
Transmission Line on the Prototype
Voltage[V]

0.2V /div

1.0
0.8

0.52[V]

0.6
0.4
0.2

Threshold
0.5V

0.5[V]

0.0

Time[ns]

2.0ns /div

Fig.13 Waveforms observed in STL on the Prototype©ISAROB 2011378

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Digital-signal improvement-method using Pareto optimization
Shohei Akita†, Hiroki Shimada†, Masami Ishiguro†, Noriyuki Aibe†,
Ikuo Yoshihara‡, Moritoshi Yasunaga††Graduate School of Systems and information Engineering, University of Tsukuba
Tsukuba Ibaraki 305-8573, JAPANE-mail: akita@islab.cs.tsukuba.ac.jp‡Factory of Engineering, University of Miyazaki
Miyazaki, Miyazaki 889-2192, JAPAN
Abstract Recently, propagating digital signals with low distortion in PCB (printed circuit board) traces is getting
difficult more and more as the frequency increases. In order to solve this problem, we have proposed “Segmental
Transmission Line (STL)”. In the STL, a transmission line is divided into multiple segments of individual characteristic
impedance. The multiple segments are designed to improve the waveform distortion on the transmission line by solving
a combinatorial explosion using genetic algorithm. In this paper, we try to improve signal integrity at two points using
the STL. In order to achieve the signal integrity improvement at two points, we use the multi-objective GA andpareto-optimal-solutions. We conducted experiments with simulation and an STL prototype using real PCBs. As a result,at two observation points on simulation, logical margins in the STL were improved 3.3 times and 3.0 times
simultaneously in their maximum, respectively. And at two observation points in the STL prototype, logical margins
were improved 1.7 times and 2.0 times simultaneously, respectively. The result thus indicates that the STL has highability to improve digital-signal waveform at multiple points simultaneously.
Keywords Signal Integrity, STL, Transmission Line, Genetic Algorithms, Pareto-optimal-solutions

I.

conventional methods such as [1],[2], which aim to
decrease noise-emissions.
In the STL, we use counter approach to theconventional ones. Figure 1 shows an overview of STL
and its segment-model. And Fig. 2 shows a model of the
STL. In the STL, we use noises that are generated

Introduction

Signals of GHz frequency, the wavelengths of which
are less than 15cm and are shorter than sizes PCBs, areterribly distorted at the impedance mismatching points in
the PCB traces.
They have used conventional impedance-matching
techniques [1][2] to improve the distorted waveforms
usually. These techniques works well up to hundreds
MHz, but will not work well at more than GHz.
In order to solve this problem and to ensure thesignal integrity we have proposed a novel transmission
line called "Segmental Transmission Line (STL)" [3] .
In the STL, a transmission line is divided into
multiple segments of individual characteristic impedance.
In this structure, noises are generated purposely at thesegments boundaries to cancel the target noises, which
occur due to the impedance mismatching between thetransmission line and the devices connected to thetransmission line.
In our previous paper, we have already shown
effectiveness to GHz-frequency signal at single point
using computer simulation and on STL prototype
fabricated in real PCBs[4][5]. These STL boards were
designed in the MHz-frequency targeting to theGHz-frequency by lengthening the wire (trace) length inproportion to the ration between the MHz and GHz
frequencies.
In this paper, we try to improve signal integrity at
multiple points using STL and to show its effectivenessusing an STL prototype as well as computer simulaition.

purposely at impedance mismatching points tocancel the target noises. To generate these noises
purposely, a transmission line is divided into
multiple segments which have different impedancesand lengths, respectively. Impedance of a segment
is function of a segment-width.
Noises

W1

L1

W2

W3

Z1

Z2

Z3

L2

L3

L1

L2

L3

Impedance mismatching

Segment model of STL

Overview of STL

Fig. 1 Outline of STL
Z1

Z2

Z3

Z4

Z5

Z6

Z7

Z8

Z9

Z10 Z11 Z12 Z13

L1

L2

L3

L4

L5

L6

L7

L8

L9

L10 L11 L12 L13

Rd
Rt
V

C

V： Signal Source
Rd：Dumping Resistance
Rｔ： Terminal Resistance
C： Capacitance
Zi: Characteristic Impedance
Li: Segment length

II. Segmental Transmission Line

Fig. 2 STL Model

The idea of STL is completely different from the©ISAROB 2011

Wi: width of transmission line
Li: length of transmission line
Zi: impedance

379

C

C

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

III. STL design methodology using Genetic
Algorithms

In Family, there are six solutions

Solution

START

CrossoverDifference Area Calculation

Point 0V. Signal-Integrity improvement at multiple
points on the computer simulation
In the STL design, or in the simulation, we used thecircuit simulator ngspice [7] as the SPICE shown in Fig.
3. Figure 6 shows the circuit-diagram of the scale-up
STL prototype for 150MHz clock-signal targeted to1GHz. This model assumes the bus line connecting a
CPU with 3 memory modules. The transmission line of
1-m long is divided into 12 segments. Three capacitorsof 24pF each represent three device-inputs, e.g., memory
modules connected to the transmission line. Point 0 and
Point 1 in Fig. 6 are the target or observation points in
the STL design. And the solution obtained in the STL isshown in Fig. 7.

O(t)Selection

Rd
:誤差面積

End

I(t)：Ideal WaveformO(t)：Observed Waveform

CPU

Rt

DIMM
Point 0Fig. 3 STL-Designer
A GenePoint 1

Fig. 5 Pareto optimization solution

I(t)Fitness Evaluation

Not Pareto Solutions（There are solutions in the dotted area）

Different Area of Point 0

SPICE Starting

Output Waveform Data

Pareto Solutions（There is no solution in the dotted area）

1

1 generation
First Individuals Generation=

Different Area of Point

In the STL, it is necessary to obtain the best, or
semi-best impedance and length combination. In the caseof complete search of combination, for example, 10
widths and 100 lengths for each of 10 segments results inthe combination of (10 * 100)10 = 1.0 * 1030 . It is clearly
impossible to calculate all of them, and this difficulty iscalled combinatorial explosion. In order to overcome thisdifficulty, we use Genetic Algorithms (GA) [6] to the
STL design.
In the GA for the STL design, two kinds of genes areused. One is for the segment-width and the other is forthe segment-length. Figure 3 shows the STL design
system named STL-Designer, which is composed of a
newly developed GA calculation-loop specialized for the
STL and the circuit simulator SPICE. Figure 4 shows thegene and chromosome configuration of GA applied to
the STL. As shown in the figure, a characteristic
impedance and its corresponding length forms a pair ofgenes in a chromosome.

Chromosome

Rd

A Chromosome

30cm

55cm

3.3V
150MHzZ1 Z2 Z3 Z4 Z5 Z6 Z7 Z8 Z9 Z10 Z11 Z12 Z13
L1 L2 L3 L4 L5 L6 L7 L8 L9 L10 L11 L12 L13

Point 1

10cm

24pF

5cm

24pF

24pF

Rt

Fig. 6 Circuit-diagram of STL applied to bus system
Impedance
55
(Ω)

35

60

55

75

60

115 100

85

120

110

35

50

Segment
length(mm) 96.0 108.5 95.5 292.5 54.0 50.0 50.0 50.5 53.0 32.5 33.0 34.5 50.0

Rd
Rt

Fig. 4 Gene and Chromosome Constitution

Z1

Z2

Z3

Z4

Z5

Z6

Z7

Z8

Z9

Z10 Z11 Z12 Z13

L1

L2

L3

L4

L5

L6

L7

L8

L9

L10 L11 L12 L13

Rd

Point 1
24pF

Point 0

24pF

Rt
24pF

IV. Multi-objective GA and Pareto solutions
To achieve the signal integrity improvement at
multiple points, the waveforms at the observation points
must be optimized simultaneously. We thus use themulti-objective GA and pareto-optimal-solution [8] tomeet the design requirement. Figure 4 shows a chart ofthe pareto-optimal-solution. Pareto-optimal-solutions is aset of the solutions which cannot be put above the others.

©ISAROB 2011

Fig. 7 Solution of GA applied to STL
In Fig. 8, distorted waveforms of red waves in theleft column were observed in the conventionaltransmission line of homogeneous characteristic
impedance of 50 ohm at Point 0 and Point1, respectively.
The distorted waveforms were well improved in the STL
as shown in the red waves in the right column. Blue

380

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

The results shown in Fig. 8 are summarized in Tab. 1.
Improvement ratios are also shown in the parenthesis.
High improvement ratios are obtained except thefall-delay time in Point 1.

waveforms in the figure, which are all the same, are thewaveforms observed at Point 0 and Point 1 if nocapacitors are connected in the transmission line. The
blue waveform was thus used as the target or teacher
waveform in the design.

STL

Conventional Transmission Line
2.0

Point 0[V]

1.5

0.66ns0.34nsPoint 0

[V]
2.0

2.0ns

1.5

0.35V

～0ns

1.0

Vth
(0.825V)

0.5

0.3V

0.22ns

1.0
0.5

0

2.0ns

0.86V
Vth
(0.825V)

0.91V

0

2.0

1.5

Point 1[V]

0.58ns

1.5

0.48ns

1.0

0.41V

Vth
(0.825V)

0.32V

0.5

Point 1

[V]
2.0

2.0ns

0

2.0ns
0.22ns0.55ns

1.15V

1.0
0.5

Vth
(0.825V)

0.94V

0Fig. 8 Comparison of waveforms designed at Point 0
times are not measured in Fig. 11 due to some restrictionin the measurement environment.
Declines in amplitudes in the conventionaltransmission line, which come from the waveform
distortions, are well improved in the STL. The
improvement ratios however are not so high as in thedesign, or simulation in Fig. 8. The results shown in Fig.
11 are also summarized in Tab. 2 as Tab.1.

Table. 1 Summary of simulation results() : Logical Margin Improvement Ratio andDelay Time Shortening Percentage
High Logical Low Logical Delay Time Delay Time
MarginMargin（Rise）（Fall）
[V]
[V]
[ns]
[ns]

Point

Conventional
Transmission Line

STL

0

Point

Conventional
Transmission Line

1

STL

0.35

0.3

0.66

0.34

0.86
(2.5)

0.91 < 0.1ns(3.0) (< 0.15)

0.22(0.65)

0.41

0.32

0.58

0.48

1.15
(3.3)

0.94
(3.1)

0.22
(0.38)

0.55(1.1)

Prototype of Conventional Transmission Line

Prototype of STL

VI. Signal Integrity improvement at multiple
points in the STL prototype

Fig. 9 Photographs of prototypes

Figure 9 shows photographs of the prototypes of
conventional transmission line and of STL. Figure 10
shows the measurement-environment schematically.
Signals from the pulse generator are put into the FPGA,
which is used to reshape the waveform, and the outputs
from the FPGA are input to the prototype board. The
waveforms are observed by the digital storage
oscilloscope of 2GHz bandwidth through an active prove.
The waveforms observed by the oscilloscope areshown in Fig. 11, which are corresponding to Fig. 8 (noteacher waveforms in Fig. 8 are shown in Fig. 11). Delay©ISAROB 2011

Fig. 10 Measurement environment

381

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

STL

Conventional Transmission Line
[V]
2.0
1.5

Point 0

2.0ns

0.44V

0.5

0.41V
0

1.5

2.0ns

0.74V

1.0
Vth
(0.825V)

0.5

2.0

Point 0

1.5

1.0

[V]

[V]
2.0

Vth
(0.825V)

0.71V

0Point 1

2.0ns

Point 1

[V]
2.0

2.0ns

1.5

0.41V

0.83V

1.0
Vth
(0.825V)0.5

1.0

0.5

0.8V

Vth
(0.825V)

0.41V
0

0

Fig. 11 Comparison of Waveforms measured at Point0 in prototypes)
integrity at two points needs to be improved andincreased.

Table. 2 Summary of measurement results() : Logical Margin Improvement Ratio andDelay Time Shortening Percentage

High Logical
Margin[V]

Point
0

Point
1

Conventional
Transmission Line

0.44

0.41

STL

0.74
(1.7)

0.71
(1.7)

0.41

0.41

0.83
(2.0)

0.80
(2.0)

Conventional
Transmission Line

STL

References

Low Logical
Margin[V][1] N. Naono，Y. Nakamura: “Details on High-speed
Digital System Design", NikkeiBP INc.，2001.
[2] M. Taguchi，”High-speed, small-amplitude I/Ointerface circuit for memory bus appli-cation", IEICE
Trans. Electron.，vol.E77-C，no.12，pp.1944-1950，Dec.
1994.
[3] M. yasunaga, I. Yoshihara “Segmental Transmission
Line and Its Design Methodology: Line Trace Ensuring
High Signal Integrity in VLSI Mounting Boards”, Thetransactions of Electronics, Information andCommunication Engineers. D- I , vol. J88-D-I, no. 5, pp.
915-929, May. 2005
[4] H. Nakayama, M. Yasunaga, I. Yoshihara et al.,”Experimental Verification of Improved Signal Integrity
Using the Segmental Transmission Line"，The 70th
National Convention of Information Processing Societyof Japan, 1pp.9-10，tsukuba， Mar. 2008.
[5] H. Nakayama，I. Yoshihara， M. Yasunaga et al.,“AHigh-Signal-Integrity transmission-Line for High
Speed VLSI Packaging: Its proposal andPrototypeEvaluation"， DA Symposium 2009，
pp.187-192．
[6] M. Mitchell，H. Iba，”An Introduction To Genetic
Algorithms"，Tokyo Denki University Press，1997.
[7] http://ngspice.sourceforge.net/[8] C. A. Coello. Coello, “A Comprehensive Survey of
Evolutionary-Based Multiobjective Optimization
Techniques”, Knowledge and Information Systems An
International Journal vol.1 no.3 pp.269-308 1999

VII. Conclusions and further works
In the simulation experiments, STL showed the
maximum improvement ratios of 3.0 and 3.3 at Point 0
and Point 1, respectively in terms of the logical margin.
And it also decreased the rise-time delay to less than 15%
at Point 0 and the fall-time delay to 38% at Point 1 at
Point 1, while the fall-time delay at Point 1 increased
10%.
In the prototype experiments, the STL showed the
maximum improvement ratios of 1.7 and 2,0 at Point 0
and Point 1, respectively in terms of the logical margin.
The results clearly indicate that the STL has highability to improve digital-signal waveform improvement
at multiple points simultaneously.
Followings are further works: the fall-delay-time at
Point 1 needs to be improved and accuracy of the signal©ISAROB 2011382

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Basic Position/Force Control of Single-Axis Arm
Designed with an Ultrasonic Motor
K. Ogiwara and F. Nagata
K. Watanabe
Tokyo University of Science, Yamaguchi Okayama University
Sanyo-Onoda, Japan
Okayama, Japan

Abstract
Recently, many studies on assist robots are being
conducted, in which the development of a unique system is required to support aged persons, physicallyhandicapped persons and/or caretakers. One of therepresentative systems is called the assist suit and ispartly practiced. The assist suit is a mechatronics device which can assist physical human actions. However, the current assist suit has a few problems withrespect to cost, size, weight, long-time run and soon. In this article, a fundamental study concerning
a compact assist device is conducted. Where the assist device supports is one spot on the body such as aknee, an elbow and a shoulder. First of all, a simple
single-axis arm is designed by using an ultrasonic motor which can generate high torque from a low velocity
range. Then, a servo system, a torque control system
and a passive torque control system are applied andtheir characteristics are evaluated. Here, the passive
torque control includes a stiﬀness control and a compliance control.

1

DC power of 5V

DC power of 24V

Driver device
Single-axis arm by using
an ultrasonic motor

Figure 1: Single-axis arm designed with an ultrasonicmotor.

arm is equipped to a damaged joint of a human. The
operator can adjust the desired torque, e.g. more stiﬀ
or more compliant, while feeling the behavior and theeﬀectiveness of the assist device.

2

Introduction

Recently, many studies on assist robots are being
conducted, in which the development of a unique system is required to support aged persons, physicallyhandicapped persons and/or caretakers [1]. One of
the representative systems is called the assist suit andis partly practiced. The assist suit is a mechatronics
device which can assist physical human actions. However, the current assist suit has a few problems withrespect to cost, size, weight, long-time run and so on.
In this article, a fundamental study concerning acompact assist device is conducted [2]. Where the assist device supports is one spot on the body such asa knee, an elbow and a shoulder. First of all, a simple single-axis arm is designed by using an ultrasonicmotor which can generate high torque from a low velocity range. Figure 1 shows the experimental setup.
Then, a servo system, a joint torque control system
and a passive torque control system are applied andtheir characteristics are evaluated. The passive torquecontrol includes a stiﬀness control and a compliance
control. Finally, a promising application as an assist
device is considered. It is assumed that the single-axis©ISAROB 2011

A small force sensor

A Single-Axis Arm Designed with an
Ultrasonic Motor

The ultrasonic motor has two features. One isthat it can generate high torque from the low velocity range. The other is that it has a large holding torque when no voltage is given. That is thereason why no brake system is needed and consequently the weight reduction can be realized. Also,
the responsiveness of the ultrasonic motor is generally superior to the one of conventional electromagnetic motors, so that the energy consumption can besuppressed. Considering the above points, the simple
arm is designed based on an ultrasonic motor. Figure 2 shows the hardware block diagram. The ultrasonic motor used in experiments is the model of
USR60-E3T provided by Shinsei Corporation. A DA
board(CONTEC DA12-8) is used to control the motor velocity. A digital IO board(CONTEC PIO-48D)is also used to switch the direction of motor rotation, i.e., clockwise or counterclockwise. Further, acounter board(CONTEC CNT32-8M) is incorporated
to sense the rotation angle. These cards are connectedto the ultrasonic motor via a driver device(Shinsei

383

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

600

Digital I/O
Board
D/A
Board

DC 24 VDC 5 V

SingleUltrasonic
Axis Arm
Motor(Force(Encoder)Sensor)Input(0−3.2 V) Motor

DriverCounter
Board

Joint angle pulse

PC (Controller)

ObjectKp = 0.006

500

Kp = 0.004

400

Kp = 0.002
Kp = 0.001

300
200

Reference
Response

100

A/D
Board

0

Force Sensor
for Input Device

0

1

2

Time s3

4

5

Figure 4: Step responses obtained by using Eq. (1),in which 500 pulses mean π rad.
Figure 2: Hardware block diagram of the experimental
system.

Joint driving voltage V

3.0

140

Rotational speed rpm

120
100
80

2.5
2.0
1.5

Kp = 0.004

1.0

60

0.5

40

0

Kp = 0.002
Kp = 0.001
0200Kp = 0.006

1

2

3

4

5

Time s
0

0.4

0.8

1.2

1.6

2.0

2.4

2.8

3.2

Figure 5: Joint driving voltage calculated from Eq. (1).

Joint driving voltage V

Figure 3: Relation between joint driving torque androtational speed in steady state.

has only to be applied as
τ (k) = Kp {θd − θ(k)}where τ (k) is the joint driving voltage at the discrete
time k, Kp is the p-gain; θ(k) and θd are the joint angle and desired one, respectively. In order to conduct
high accuracy positioning, when θd = θ(k) is detectedin the sampling loop, the excitation power to the motor has only to be oﬀ at the same time. Figure 4 shows
examples of step response with several Kp , in which
500 pulses mean π rad. Note that a linear characteristic suddenly appears from a point, for example, incase of Kp =0.006. To examine the matter a bit moredetail, the relation between the time and the torqueobtained by Eq. (1) is measured as shown in Fig. 5.
It is observed from Figs. 3, 4 and 5 that the joint velocity tends to show a constant value about 8.5 rpm
under the point of 0.6 V. So, in order to cope with thecharacteristic, Eq. (1) is improved as

Corp. D6060 24V). A small force sensor is ﬁxed at the
tip of the single-axis arm to estimate the joint torque.

3

Control System

Basic functions were ﬁrst developed on Microsoft
Visual C++ to give the commands such as the direction of rotation and the rotational velocity and toobtain the rotational angle from the encoder. Then, it
was measured on how the relation between the voltage
and its given time inﬂuenced the dynamic response of
the motor. Figure 3 shows the characteristics, in which
it is conﬁrmed that the velocity tends to be constantunder 0.6 V. The dynamic characteristics were used todesign a position feedforward controller.

3.1

τ (k) = Kp {θd − θ(k)} + 0.6

Basic servo system(2)

Figure 6 shows the step response when Eq. (2) is employed. As can be seen, a desirable response without
an overshoot and delay is observed only by using ap-action. This is the attractive characteristics of theultrasonic motor used in the experiment.

The servo system of an ultrasonic motor is easily
constructed due to the high holding torque and the
responsiveness. Here, a simple proportional control©ISAROB 2011

(1)

384

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

600

3.5

Kp = 0.006

3.0
Joint angle rad

Joint angle pulse

500
400
300

Reference
Response

200

Kd = 2

Kd = 3

Kd = 4

Kd = 5

2.0
1.5
1.0

100
0

Kd = 1

2.5

0.5
0

0

1

2

3

4

5

0

5

Figure 6: Step responses obtained by using Eq. (2).

10Torque Nm

Time s15

Figure 8: Stiﬀness control result in using Eq. (4).

2.4

2.5

Bd = 1

2.0
1.6

Joint angle rad

Joint torque Nm

2.0

1.2
0.8

Reference
Response

0.4
0
0

2

4

6

8

1.5

Bd = 4

Bd = 5

Bd = 31.0

Bd = 20.5

10Time s0

Figure 7: Joint torque control reslut in using Eq. (3).

0

5

10

Time s15

20

25

Figure 9: Compliance control result in using Eq. (5).

3.2

Joint torque control
to the desired position θd , and also θ(k) obtained from
Eq. (4) is given to θd in Eq. (2). Figure 8 shows anexample of the stiﬀness control result.
Next, the compliance control law is written as{}
τs (k) = Bd θ̇(k) − θ̇d + Kd {θ(k) − θd }(5)

In the joint torque control mode, the torque acting
at the joint is actively controlled by a PI controller
written byτ (k)= Kf p {τd − τs (k)}
+Kf i

k
∑

{τd − τs (n)} + 0.6(3)

where Bd is the desired viscosity [Nm·s/rad]. In thecompliance control mode, the transient behavior to an
equilibrium position can be controlled. Of course, theequilibrium position depends on Kd . If it is assumed
that both θ̇d and θd are 0, and τs (k) = τstep (constant),
then θ(k) is obtained by

n=0

where Kf p and Kf i are the p-gain and i-gain, respectively. τd is the desired joint torque, τs (k) is theestimated joint torque which is calculated from theforce value sensed by a small force sensor attached tothe arm tip. Figure 7 shows a torque control result,in which the response desirably follows the reference
2 Nm by setting Kf p , Kf i to 3 and 0.001, respectively.

3.3

θ(k) =In the passive joint torque control mode, an external force given to the arm can be absorbed smoothly.
Here, a stiﬀness control and a compliance control areconsidered. The stiﬀness control law is given by

4(4)

where Kd is the desired stiﬀness [Nm/rad]. Note thatin the stiﬀness control mode, the initial position is set©ISAROB 2011(6)

where ∆t is the sampling width. Compliance control
can be easily realized by giving θ(k) obtained from
Eq. (6) into θd in Eq. (2). Figure 9 shows an example
of compliance control result, in which the transient
behaviors are changed with Bd . In the experiment,
Kd and τstep (k) are set to 1 and 2, respectively.

Passive joint torque control

τs (k) = Kd {θ(k) − θd }K
τstep
− d ∆tk(1 − e Bd)
Kd

Example of Application

In this section, a promising application called an assist device is considered. It is assumed that the single-

385

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011τ d (k )τ S (k )θ (k )

Joint TorqueControllerτ (k ) Ultrasonic SingleMotor(Encoder)

Axis
Arm

Single-axis arm

DamagedJoint of
Human

F(k)
f (k)

Perception of Arm’s Motion

Small force sensors

Sensory
Information
Human body

Figure 11: Experimental scene assumed to be the assist device.

axis arm is equipped to a damaged joint of a human.
A small and light weight force sensor is used so thatthe desired torque can be directly given by the ﬁngersof the operator. The operator can adjust the desired
torque, e.g. more stiﬀ or more compliant, while feelingthe behavior and the eﬀectiveness of the assist device,i.e., the single-axis arm.
The block diagram of the controller is shown in
Fig. 10, in which the desired torque τd (k) is manually given by an operator. The control law is derived
from Eq. (3) by varying the desired joint torque as

2.5

40

Joint torque Nm

2.0

24

1.5
1.0

k
∑

{τd (k) − τs (n)} + 0.6(7)τd (k) = αf (k)(8)τs (k) = LF (k)

(9)

8

0.5
0

0

1

2

4

5
6
Time s

7

8

9

0
10

control and a compliance control. Finally, a promising
application as an assist device has been considered. It
is assumed that the single-axis arm is equipped to adamaged joint of a human. The operator can adjust
the desired torque, e.g. more stiﬀ or more compliant,while feeling the behavior and the eﬀectiveness of theassist device.

where f (k) is the small force generated by an operator’s ﬁngers, α is the gain which transmits the force
to the time-varying desired joint torque. F (k) is theforce acting between the arm tip and the object, L is
the length of the single-axis arm. In the experiment,τs (k) is regarded as the joint torque.
Figure 11 shows the experimental scene assumed
to be the assist device. The control result of τs (k)is shown in Fig. 12, in which the response τs (k) is
desirably ampliﬁed according to f (k) by giving 1 and0.15 to α and L, respectively.

References[1] S. Toyama and J. Yonetake,“Development of
the Ultrasonic Motor-Powered Assisted Suit System,”Procs. of 2007 IEEE/ICME International
Conference on Complex Medical Engineering, pp.
1361–1366, 2007.

Conclusions

In this article, a fundamental study concerning acompact assist device is conducted. Where the assist device supports is one spot on the body such as aknee, an elbow and a shoulder. First of all, a simple
single-axis arm has been designed by using an ultrasonic motor which can generate high torque from a low
velocity range. Then, a servo system, a joint torquecontrol system and a passive torque control systemhave been applied and their characteristics are evaluated. The passive torque control includes a stiﬀness©ISAROB 20113

Figure 12: Joint torque manually controlled by an operator. Force is given by the operator’s ﬁngers.

n=0

5

16

Force f (k )

τ (k) = Kf p {τd (k) − τs (k)}
+Kf i

32

Joint torque τ S (k )Force N

Figure 10: Block diagram of an application called theassist device for assisting a damaged or weakened joint.

[2] K. Ogiwara and F. Nagata,“Basic Position/ForceControl of a Single-Axis Arm Designed with an
Ultrasonic Motor,” in Procs. of the 12th IEEE
Hiroshima Section Hiroshima Student Symposium,(F)–11, 2 pages, 2010 (in Japanese).

386

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Cooperative Swarm Control for Multiple Mobile Robots
Using Only Information from PSD Sensors
T. Yamashiro and F. Nagata
K. Watanabe
Tokyo University of Science, Yamaguchi Okayama University
Sanyo-Onoda, Japan
Okayama, Japan

Abstract
Recently, many studies on swarm robotics are being
conducted, in which the aim seems to be the realization of complex task ability by cooperating with eachother. Future progression and concrete applications
are being expected. The objective of this study is toconstruct an attractive system by using multiple mobile robots. First of all, multiple mobile robots with
six PSD (Position Sensitive Detector) sensors are designed. The PSD sensor is a kind of photo sensors. A
control system is considered to realize such a swarm
behavior as Ligia exotica by using only information of
PSD sensors. Experimental results show interesting
behaviors of multiple mobile robots such as following,
avoidance and schooling. The proposed system was
intriguingly demonstrated to high school students in
OPEN CAMPUS 2010 held in Tokyo University of Science, Yamaguchi.

1

Figure 1: Ligia exotica swarming at seashore.

2

In experiments, a mobile robot is used as shownin Fig. 2, which is developed based on three wheeledomni-directional mobile robot provided by Tosa Denshi. In order to real-timely measure the distances toobjects, the robot is improved with six PSD sensors.
The PSD sensor is mainly composed of an LED, electrical resistance and photodiode, and can calculate thedistance through the triangulation technique. In order
to cope with the problem of narrow directivity of the
PSD sensor, the number of the PSD sensor is increased
to six. If possible, PSD sensors more than six are desirable to further reduce the dead angle. Also, eachrobot has a Bluetooth wireless device to communicate
with a PC server.
As for control scheme, self-control mode and servercontrolled mode are proposed for the mobile robots.
The PC server gives either mode to each robot. Whenthe self-control mode is given, the robots behave basedon their own decisions. This will be applied to a simulation of Ligia exotica swarming at seaside where theyescape around because of the surprise to a sudden surrounding change. In this case, the swarm itself behaves as controlled, however, each individual seems tohave only very simple action pattern. That is the reason why several easy action patterns are given to therobots in the self-control mode. However, the softwaredevelopment environment of the mobile robot using afree C language has two restrictions. The one is thatthe ﬂush ROM of the mobile robot is only 8 kB. The
other is that mathematical standard library such as

Introduction

Recently, many studies on swarm robotics are being conducted, in which the aim seems to be the realization of complex task ability by cooperating witheach other. It also seems that the research of swarm
robotics includes the design of robots, their physicalbody and their behavior as a controller. Future progression and concrete applications are being expected.
In this study, multiple mobile robots with six PSD(Position Sensitive Detector) sensors are designed [1].
The PSD sensor is a kind of photo sensors. A control
system is considered to realize such a swarm behavior
as Ligia exotica as shown in Fig. 1 by using only information of PSD sensors. A lot of organisms who behavewith making a group live in everywhere on the earth.
Ligia exotica is one of such an organism that swarms
at the seashore. Experimental results show interesting
behaviors of multiple mobile robots. They are the following behavior, avoidance behavior and schooling behavior. The collective behaviors such as the following,
avoidance and schooling emerges from the local interactions among the robots and/or between the robots
and the environment.

©ISAROB 2011

Mobile Robots with PSD Sensors

387

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

“exp ( )” cannot be compiled. Thus, for example, it is
impossible to directly apply the potential ﬁeld method
for path planning.
The server-controlled mode is considered to copewith the poor development environment. In theserver-controlled mode, the robots behave according
to commands transmitted from the server. All information measured by PSD sensors of each robot aretransmitted to the server once, the server can send action commands to the robots totally considering the
behavior of the swarm. Of course, the potential ﬁeld
method is available on the server side where the Windows Visual Studio runs. This will be also applied
to a simulation experiment of complex swarm intelligence where software developments with comparatively large-scale are required.
Figure 3 shows the coordinate system of this typeof mobile robot. ωi (i = 1, 2, 3) is the angular velocityof each wheel. Also, vi (i = 1, 2, 3) is the velocity ofeach wheel given by
vi = rωi(i = 1, 2, 3)

PSD sensor 2

RX

PSD sensor 6

PSD sensor 4

PSD sensor 5

Figure 2: Mobile robot with six PSD sensors.

(1)

yɺ r

r ω

Y v1

Wheel
Wheel 1

φɺr

R

(3)

X

L

v2

(4)

Wheel 3xɺ r

v3

Figure 3: Kinematic model of the mobile robot.

By using the relation given by Eq. (5), the robot canbe controlled kinematically.
Figure 4 shows the static measurement result of a
PSD sensor, which is the relation between the actual
distance and the digital value obtained from the PSD
sensor. It can be seen that the sensible distance iswithin the range from 10 cm to 90 cm.

Digital value from a PSD sensor

80

Self-Control ModeIn self-control mode, each robot behaves by using
only information obtained from six PSD sensors shownin Fig. 5. The robots have three types of self-control
modes, which are a following mode, an avoidance mode
and a schooling mode. All mobile robots can communicate with a PC server. The PC server switches themode of each robot individually by sending broadcast
commands.

©ISAROB 2011

R

Wheel 2(2)where L is the distance between the center of the robot
and the center of each wheel. Eqs.(2), (3) and (4) lead
to





L
√1
− 13
ẋr
ω1
3
3
1

 ω2  = (5)
 − 13 − √13 L3   ẏr 
r
2
L
ω3
ϕ̇
r
0
3
3

3

PSD sensor 1

PSD sensor 3

where, r is the radius of the wheel. If the positionand orientation vector of the robot, i.e., the origin in
the robot coordinate system ΣR , is given by [x y ϕ]T ,then the velocity is represented by [ẋ ẏ ϕ̇]T . First ofall, following equations are obtained from Fig. 3 [2, 3].
11ẋr = − v1 − v2 + v3
2√
2 √
3
3
ẏr =
v1 −
v2
2
2
1
1
1
ϕ̇r = v1 + v2 + v3
L
L
L

RY

Max. values

70

Min. values

60
50
40
30
20
10
0
0

10

20

30

40

50

Real distance cm

60

70

80

90

Figure 4: Relation between actual distance and digitalvalue measured from a PSD sensor.

388

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

3.3
R

If the schooling mode is set to all mobile robots,
they try to regularly move along the inner of a circular
fence keeping the distance to both the fence and othermobile robots. This mode allows the robots to behave
like carps in a Japanese artiﬁcial circular pond. For
example, when a robot moves counterclockwise alonga circular fence, the following control law is basicallyapplied.

d2(k)Wheel 2

Wheel 1
Sensor 2

d3(k)

d1(k)

Sensor 3

Sensor 1

R

O
Sensor 4

Wheel 3X

Sensor 6

Sensor 5d4(k)ẋr = vd6(k)ϕ̇r = Kϕ {d6 (k) − d1 (k)}3.4

If the following mode is given, mobile robots try tofollow a moving object including other mobile robots
in the sensitive region. Six PSD sensors can independently detect distances di (k) (i = 1, ..., 6) at the discrete time k. di (k) is the value measured by i-th PSD
sensor. If the distance to the nearest object is larger
than a desired value dd , the robot tries to shorten the
distance by generating the velocity given byẋri{dd − min di (k)} (∀di (k) > dd )i∥ẋri ∥

3.4.1

3.2

2
0
2

3
√
− 3
1

4
√
− 3
−1

5
0
−2

(6)

3.4.2

6
√
3
−1

If the avoidance mode is given, each mobile robot
tries to move away from moving objects including
other mobile robots in the sensitive region. When six
PSD sensors simultaneously detect shorter distances
compared to the restriction dd , the robot preferentiallyleaves the nearest object. If the distance to the nearest object is smaller than dd , the robot tries to expand
the space to be dd by generating the velocity given by©ISAROB 2011ẋri
{dd − min di (k)} (∃di (k) < dd )i∥ẋri ∥

Following mode

Avoidance mode

In the avoidance mode, each robot moves aroundrandomly when no object is detected. If a robot detects an object in sensing area, it moves to the reverse
direction to the object. Also, when plural objects aredetected at a time, the nearest one is regarded as an
object and the others are ignored. Desirable and interesting avoidance behaviors were observed from theexperiments. Figure 6 shows an experimental scene.

Avoidance modeẋr = −v

Experiment and discussion

In the following mode, each robot basically stoodstill and looks around. When an object was detectedin the sensing area, the robot tried to follow the object.
In order to overcome the problem of blind spot and toskillfully sense a moving object, PSD sensors 1, 2 and3 shown in Fig. 5 were used. The front of the robot isthe direction of sensor 2. The following was conductedto the direction of sensor 2. Note that the sensors 1
and 3 were assistantly used in order not to lose sightof the object. It was observed from the experimentsthat a robot in following mode could run after otherrobots.

Table 1: Velocity components to move to the directionof each PSD sensor.
1
√
3
1

(9)

In order to evaluate each behavior, three experiments were conducted.

where ẋr = [ẋr ẏr ]T is the translational velocity in
Eq. (5), v is the scalar signifying the magnitude of therobot’s velocity, ẋri = [ẋri ẏri ]T is the vector signifying the direction as shown in Table 1. dd is the desired
distance to the nearest object.

i
ẋri
ẏri

(8)

where Kϕ is the gain which can control the orientationof the robot to be parallel to the inner of the circular
fence.

Following modeẋr = −vẋr2
∥ẋr2 ∥In this case, the orientation is simultaneously controlled byd5(k)Figure 5: Six PSD sensors equally ﬁxed on a mobile
robot.

3.1

Schooling mode

Y

3.4.3

Schooling mode

In the schooling mode, each robot ﬁrst moved to
the front direction to detect a circular fence. After
detecting a circular fence, the robots moved around tothe counterclockwise direction. Seven mobile robots
in schooling mode could move around the fence likecarps. In this case, the robots could keep a distance to

(7)

389

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

PC server

Self-control mode5

BluetoothServer-controlled mode

1

13

Figure 6: Experimental scene of avoidance mode.
Figure 8: Example of server supervisory mode, inwhich robots 1, 2 and 3 are set to server-controlled
mode, also the others are set to self-control mode bya PC server.

When many multiple mobile robots are used to simulate a swarm or a school, the cost with the increaseof the number of robots becomes a serious problem.
Therefore, there is an essential demand to construct
the system at as low cost as possible. In future work,
we plan to consider a server supervisory mode in order that a high level software architecture can be constructed even under the condition of technically andcostly poor hardware platform of each mobile robot.
In server supervisory mode, all mobile robots canbe broadcastly switched self-control mode or servercontrolled mode as shown in Fig. 8.

Figure 7: Experimental scene of schooling mode.

the fence by using the two PSD sensors, nos. 1 and 6.
Figure 7 shows the experimental scene of the schoolingmode, in which nine mobile robots are schooling along
the inner of fence.
It has been conﬁrmed from the simple experimentsthat the actual simulations for following, avoidanceand schooling behaviors can be conducted for multiple
mobile robots.

4

References[1] T. Yamashiro and F. Nagata, “Cooperative Control of Schooling for Multiple Mobile Robots by Using only Information from PSD Sensors,” in Procs.
of the 12th IEEE Hiroshima Section Hiroshima
Student Symposium, (F)–13, 2 pages, 2010 (inJapanese).
[2] K. Watanabe, Y. Shiraishi1, S. G. Tzafestas,
J. Tang and T. Fukuda, “Feedback Control of an
Omnidirectional Autonomous Platform for Mobile
Service Robots,” Journal of Intelligent & Robotic
Systems, vol. 22, nos. 3/4, pp. 315–330, 1998.

Conclusions

The objective of this study is to construct an attractive system by using multiple mobile robots. First ofall, multiple mobile robots with six PSD (Position Sensitive Detector) sensors have been designed. The PSD
sensor is a kind of photo sensors. A control system
has been considered to realize such a swarm behavior
as Ligia exotica by using only the information of PSD
sensors. Experimental results have shown interesting
behaviors of multiple mobile robots such as following,
avoidance and schooling. The proposed system was
intriguingly demonstrated to high school students in
OPEN CAMPUS 2010 held in Tokyo University of Science, Yamaguchi and was very instructive to the students.

©ISAROB 2011[3] J. Tang, K. Watanabe, K. Kuribayashi and Y. Shiraishi, “Autonomous Control for an Omnidirectional Mobile Robot with the Orthogonal-Wheel
Assembly,” Journal of the Robotics Society ofJapan, vol. 17, no. 1, pp. 51–60, 1999 (in Japanese).

390The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Control of Movement on Stairs for a Cleaning Robot
Takahisa Kakudou, Keigo Watanabe, and Isaku Nagai
Department of Intelligent Mechanical Systems, Graduate Schoolof Natural Science and Technology, Okayama University
3-1-1 Tsushima-naka, Kita-ku, Okayama, 700-8530, Japan
Tel & Fax: +81-86-251-8064
Email: t.kakudou@usm.sys.okayama-u.ac.jp, {watanabe, in}@sys.okayama-u.ac.jp
Abstract: An autonomous cleaning robot is proposed so as to move on all floors including stairs in a building. In
human living environments, it is often the case that the cleaning area is a three-dimensional space such as a high-risebuilding. However, many of cleaning robots cannot clean and move on stairs, because they are not considered to moveon places between floors. The proposed cleaning robot possesses L-shaped legs on the both sides of the body frameof a rectangular solid. The robot climbs down stairs by rotating the body so that the top and bottom sides of it may
be reversed using L-shaped legs. In this paper, the mechanism and its control method are described for translational
movement on stairs.
Keywords: Cleaning robot, Stairs, Translational movement.

I. INTRODUCTION
Recently, various robots that support and act for thework have been developed to be utilized in various fields.
One of such robots is the autonomous cleaning robot [1],[2]. The automation of cleaning by robots reduces laborsand saves energy for a cleaning task, so that there is anincreasing need for it in large areas such as stations andairports.
In human living environments, it is often the case thatthe cleaning area is a three-dimensional space such asa high-rise building. However, many of cleaning robots
are not considered to move on places between floors.
Tajima et al. [3], [4] have developed a robotic system inwhich the cleaning robot cooperated with the elevator toclean floors in a high-rise building. The cleaning robot
was equipped with an optical transmitter to communicate
with the elevator. The elevator accepted requests fromthe robot, opened the door and moved to the designatedfloor carrying the robot. However, this system did notconsider the cleaning of stairs. Also, the several types ofvehicle using crawlers, wheels and legs were proposed
to move on stairs [5], [6]. However, those vehicles were
not assumed to turn and keep posture level on the tread
board of stairs during moving on stairs.
The objective of this study is to automate of cleaning
in a three-dimensional space to develop a cleaning robot
which can move on stairs. A cleaning robot has been
already proposed, where its structure was divided into
two mechanisms for climbing down stairs and translational movement [7]. The proposed cleaning robot
climbed down stairs by rotating the body so that the
top and bottom sides of body may be reversed using Lshaped legs which are attached on the both sides of the
body frame of a rectangular solid. It was also confirmedthat the robot was able to climb down stairs keeping astable posture through a simulation. In this paper, themechanism and a control method are further described©ISAROB 2011

Fig. 1.

Outline of stair cleaning by robot

for translational movement on stairs in detail.

II. CONCEPT OF CLEANING ROBOTFOR STAIRSIn this study, stair movement is assumed to climb
down stairs. As shown in Fig. 1, whenever the robot
moves on step of stairs, it needs to move in the perpendicular direction to a direction of movement. In thefollowings, several items are considered to design acleaning robot for stairs:
The cleaning from the upper floor to the lower floor
is efficient for the stairs cleaning.
• The size and shape of the robot are based on theform of stairs for cleaning each step of stairs.
• The simple mechanism of a cleaning robot for stairs
is desirable because of control design and weightsaving.
• A cleaning area is a flat surface in indoor environment.
•

391

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig. 4.

Fig. 2.

Mechanism for climbing down stairs

Mechanism for translational movement

1. Mechanism for Climbing Down Stairs
Fig. 2 shows the proposed robot for climbing down
stairs. The robot consists of the body frame of a rectangular solid and four L-shaped legs, where the leg has
two degrees-of-freedom. The center of gravity (CG) of
the robot is set to the center of the body. The posture of
the robot remains in stable, if the CG of the robot is kept
within the support polygon. Also it is assumed that thisrobot does not cause the displacement in y-axis direction
for climbing down stairs, because the motion in a pairof L-shaped legs fixed in a shaft is synchronous. Fig. 3
shows the algorithm of stair climbing down stairs. The
following is the relation between climbing motions andthe number in the figure.
First, the robot moves to the edge of stairs in Fig. 3(1)to Fig. 3(2). The robot is located in the thickness of leg
forward from the edge of stairs in Fig. 3(2). Secondly,rotate legs by 180 degrees in Fig. 3(2) to Fig. 3(3). Third,
move the CG of the robot within the support polygon
using the legs for stability in Fig. 3(3) to Fig. 3(5). The
support polygon is to be a convex region surrounded by
the sole of leg contacting with floor. Fourth, rotate thebody keeping the CG of the robot within the support
polygon in Fig. 3(5) to Fig. 3(7). Finally, bend thelegs not so as to contact with the floor on the way oftranslational movement in Fig. 3(7) to Fig. 3(8).
2. Mechanism for Translational Movement
It is assumed that a cleaning robot is used in the indoor
environment that is flat such as wooden floor or tile
floor. The robot performs translational movement using
wheels. The corner of the robot may collide with the walland the riser of stairs when turning around, because therobot shape is a rectangular solid. Therefore an omnidirectional mobility is adopted to move and turn aroundkeeping a posture.
For some of omni-directional mobile mechanisms
with wheels, there are mechanisms with omni-wheel,
mecanum wheels, etc. Since the robot climbs down stairs
by rotating the body so that the upper and lower sidesof body may be reversed, two mechanisms for moving
are attached to the top and bottom of the robot. To(1) → (2) : Move to an dege of stairs.
(2) → (3) : Rotate legs by 180 degrees.
(3) → (5) : Move the CG of the robot to the support polygon.
(5) → (7) : Rotate the body keeping the CG of the robot in
the support polygon.
(7) → (8) : Bend legs for translational movement.
Fig. 3.

©ISAROB 2011

Algorithm of climbing down stairs using L-shaped legs

392

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig. 5.

Kinematic model of two-wheel-drive mechanism
Fig. 6.

reduce the robot weight, it is desirable to use the fewest
possible actuators. Fig. 4 shows the proposed mechanism
for translating on stair treads. The mechanism consists offour ball-casters and two driving wheels that are attached
on a circular plate with a free joint. This locomotive
mechanism is attached to the top and the bottom of therobot body. Also the robot bends legs not so as to contact
with the floor during translational movement.
Fig. 5 shows the kinematic model of the mechanism
for translational movement. The velocity V and theangular velocity ω of the robot are given by
⎡ r


V
⎢ 2
=⎣ r
ω
2d

r ⎤
2 ⎥
r ⎦
−
2d

ahead. Wall-following motion is of moving along an
obstacle when having an obstacle collision.
Spiral motion and wall-reflection motion are unsuitable for the stair cleaning, because one step of stairs isnarrow and it takes time to complete the cleaning. On thecontrary, parallel motion and wall-following motion aresuitable for the stair cleaning, because the shape of stepis a rectangular form. In this study, the parallel motion
will be adopted.
2. Range sensor

θ̇R

The range sensor is used for the robot to recognize andface stairs. A position sensitive detector (PSD) SHAPE
GP2D120 is used as the range sensor. Table 1 shows theelectro-optical characteristics of the PSD.

(1)

θ̇L

where θR and θL are the rotational angles for the left
and right wheels, r is the wheel radius, 2d is the distance
between wheels. The kinematic model is given by
⎡ r

⎡

⎤
ẋ
⎣ ẏ ⎦
θ̇

=

sin θ

⎢ 2
⎢ r
⎢
⎢ cos θ⎢ 2⎣r2d

Basic motion for cleaning robot

⎤
r
sin θ
⎥
2
⎥
r
⎥
cos θ ⎥
2
⎥
r ⎦
−
2d

θ̇R
θ̇L

Table 1.

Electro-optical characteristics of the PSD

Parameter
Operation supply voltage
Average dussupation current
Distance measuring range
Measurement period
Output terminal voltage

Symbol
Vcc
Icc
L
T
Vo

Value
4.5 ∼ 5.533
40 ∼ 300
38.3
0.25 ∼ 0.55

Unit
[V]
[mA]
[mm]
[ms]
[V]

(2)
The PSD outputs the value that converts a one-way
distance to an object into DC voltage. Also, the PSD hasless influence on the color of reflective objects and thereflectivity, because it is an optical sensor by applying
a triangulation method. In addition, the PSD has lessinfluence on the outside light, because the operational
environment is assumed to be indoor in this study.

where x and y are the position of the robot, and θ is theturning angle of a circular plate.

III. LOCOMOTION CONTROL
1. Locomotion Process

IV. OPERETIONAL CHECKThe locomotion of commonly-marketed cleaning
robots is classified into four basic motions [8], [9] asshown in Fig. 6. Parallel motion is a repeated motion
combining advance and 90-degree turns. Spiral motion
is of moving toward outside on a spiral path. Wallreflection motion is of changing the direction at random
when having an obstacle collision and going straight©ISAROB 2011

An operational check of the robot was conducted withstopping and turning actions in the edge of a step forrecognizing stairs.
1. Mobile Robot
Fig. 7 shows the topview of the mobile robot usedin experiment. Table 2 shows its specification, where

393

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig. 8. Relationship between distance to a reflective object and A/Dconverted value of output voltage
Fig. 7.

Topview of mobile tobot

V. CONCLUSION
Table 2.

Specification of mobile robot

Prameter
Width
LengthVehicle height
Mobile velocity

Value
400
250
20
15

A cleaning robot to climb down stairs has been developed for cleaning three-dimension space. In particular,a mechanism and a locomotion control method were
proposed for the robot to clean and climb down stairs.
The operational check of the robot was conducted forrecognizing stairs. In the result, it was confirmed that itneeds to change the layout of PSDs and control to slow
down near the edge of stairs.
As future work, a stair cleaning robot will be developed to solve the above problem, together with making
a real robot.

Unit
[mm]
[mm]
[mm]
[cm/s]

the robot consists of two driving wheels and four bollcasters. Three PSDs were attached to the downward. The
robot is located in the thickness of leg forward from theedge of stairs with two PSDs attached on both sides of
the robot. The thickness of leg is assumed to be 40 mm.
The DC voltage is converted from the analog information
to the digital information by using an A/D converter of
H8/3664 microcomputer, where the resolution of the A/D
converter is 10 bits with 5 V. The motor can be controlled
in normal and reverse rotation, stop and braking by using
a motor driver.

REFERENCES[1] Tribelhoron B and Dodds Z (2007), Evaluating the Roomba: A
Low-cost, Ubiquitous Platform for Robotics Research and Educations, Proc. of the IEEE International Conference on Robotics andAutomation, pp. 1393–1399.
[2] Aoyama H, Tajima S, Yokota K, Ozaki K, and Yamamoto S(1998), Development of Floor Cleaning Robots (in Japanese), Jour.
of the Robotics Society of Japan, Vol. 16, No. 1, pp. 57–64.
[3] Tajima S, Aoyama H, Seki T, Ishikawa K, Yokota K, Ozaki K, andYamamoto S (2004), Development of Robotic Floor Cleaning System for High-Rise Buildings (in Japanese), Jour. of the Robotics
Society of Japan, Vol. 22, No. 5, pp. 595–602.
[4] Aoyama H, Ishimura S, Nishihara I, Ishikawa K, Seki J, Wadasako
T, Satumi Y, and Adachi Y (2009), The Development of the Cleaning Device of the Office Building Cleaning Robot (in Japanese),
Proc. of the JSME Conference on Robotics and Mechanics, P1B14.
[5] Matsumoto O, Kajita S, Saigo M, and Tani K (1998), Dynamic Trajector Control of Passing Over Stairs by a Biped Type
Leg-wheeled Robot with Nominal Reference of Static Gait (inJapanese), Jour. of the Robotics Society of Japan, Vol. 16, No. 6,
pp. 868–875.
[6] Yuan J and Hirose S (2005), Actualization of Safe and Stable Stair
Climbing and Three-Dimensional Locomotion for Wheelchair,Proc. of the IEEE/RSJ International Conference on Intelligent
Robots and Systems, pp. 2391–2396.
[7] Kakudou T, Nagai I, and Watanabe K (2010), A Cleaning Robot
for Stairs and the Simukation of Stair Movement, in Proc. of the
13th International Conference on Climbing and Walking Robots
and the Support Techologies for Moblie Machines, pp. 1306–1313.
[8] Katsuki Y, Yamamoto M, and Ikeda T (2008), Motion Primitives and Their Sweeping Efficiency in Sweeping Algorithm(in Japanese), Proc. of the JSME Conference on Robotics andMechanics, 2P1-I10.
[9] Matsumoto K, Ogata H, Torige A, and Matsumura D (2008),
Comparison of Simulation Result with Real Sweeping Robot(in Japanese), Proc. of the JSME Conference on Robotics andMechanics, 1A1-D22.

2. Condition
The size of stairs is assumed as follows: the widthis 1200 mm, the tread is 270 mm, and the rise is 180
mm. A threshold was defined to recognize stairs with
300 in A/D converted value as shown in Fig. 8. The
robot recognized stairs when the threshold was less thanor equal to 300.
In stop actions using all PSDs, the robot stops if
any one of PSDs reacts. In turning actions using PSDs
attached on both sides, if one of PSDs reacts, then the
robot turns in the direction so that the other PSDs react.
3. Result and Consideration
As a result, it is confirmed that the robot stopped
and turned to recognize stairs using PSDs. However,there was a case that the ball-caster had fallen fromstairs before stopping the motor. This problem will beovercome by changing the layout and attached angel of
PSDs for securing the distance enough to brake. Also,
the robot will be controlled to slow down near the edgeof stairs.

©ISAROB 2011394

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Jumping rhythm generator by CPG for a multi-legged robot
Masaaki Ikeda
Department of Advanced Systems Control Engineering,
Saga University,
1 Honjomachi, Saga 840-8502, Japan
Email : 09539002@edu.cc.saga-u.ac.jp,
Kiyotaka Izumi
Department of Mechanical Engineering,
Saga University,
1 Honjomachi, Saga 840-8502, Japan
FAX: +81-952-28-8587
Email: izumi@me.saga-u.ac.jp

Keigo Watanabe
Department of Intelligent Mechanical Systems,
Okayama University,
1-1-1 Tsushima-naka, Kita-ku,
Okayama-shi 700-8530, Japan
Email: watanabe@sys.okayama-u.ac.jp

Abstract: Three features of a legged robot are listed as the discrete disposition of supported legs, the flexible posture withoutchanging disposition of supported legs, and the obstacle avoidance with a three dimensional behavior. The maximum
height of an avoidable obstacle is defined by the leg mechanism of a robot. The capability of obstacle avoidance isimproved by a jumping motion. In this paper, we discuss the rhythmic jumping of a multi-legged robot using the central
pattern generator (CPG). Authors propose the construction method of CPG for six legged robot, in which each leg has acompressed spring. The eﬀectiveness of the present method is illustrated by some simulations.
Keywords:

Legged robot, Jumping robot, Central pattern generator, Robot simulator

I.

Introduction

ing is possible. However, the point that the obstacle ofthe height exceeding the excursion of a leg cannot be
overcome as a subject of this feature can be considered.
Movement by a jump or flight can be considered to thissubject.

In recent years, working places of robots are expanding from factories to outdoors and homes, etc. Required
tasks of mobile robots are rescue operations in dangerous
places such as a disaster area and the scene of an accident, and assistance of human at a home. Mobilities ona rough terrain of a robot are demanded to be high. Mobile mechanisms which have high energy eﬃciency area wheel type and a crawler type. Those mechanisms areable to travel on the terrain which is secured a continuous
contacting surface, however, the mobility of those mechanisms is almost low on the rough terrain which has big
bumps. The legged mechanism has a high capability of
movement on a rough terrain because of some features
which are a discrete disposition of supported legs, theflexible posture without changing the disposition of supported legs, high degree of freedom, etc. Many legged
type robots have been developed with several concepts
which are biomimetic machines, and improvements ofmobile capability and adaptation capability against for
any ground conditions.
Moreover, it is listed to one of the features of a legged
robot ranging over an obstacle that three-dimensional
obstacle avoidance behavior of getting over and mov-

©ISAROB 2011

It is shown clearly that a living thing controls periodic
movement which was autonomously adapted to environment by making feeling feed back to the central patterngenerator (CPG) of the neural circuit which exists in aspine. CPG is applied to a legged robot and researchwhich controls an autonomous periodic motion by feeding back the information acquired from the sensor attached to the robot to the mathematical model of CPG
to environment like walk behavior is done[1], [2].
In this paper, a jump is adopted as a method of easing
restriction of the avoidance operation by the mechanismin the obstacle avoidance of a legged robot, and it thinksof the mobile robot of six leg types which has a jumpmechanism. Moreover, CPG generates the desired value
for a jump mechanism, and a periodic jump is enabled.
The eﬀectiveness of the proposal approach is examinedfrom the result of the simulation using OpenHRP3 whichis the distributed component type robot simulator.

395

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011h( h max = 0 . 05 [ m ])Fig. 2: Slide mechanisms for jump
Fig. 1: Jumping six-legged robot
s1

N1

N2

s2

s3

N3

N4

s4

s5

N5

N6

s6

II. Jumping Six-Legged Robot
Although it is in the number of legs of a legged robot
variously, in this paper, we adopt a six-legged robot, because this robot can always keep static stability. In order to perform posture maintenance and walk movement,
there are three joints in each leg of a robot. It has a mechanism of the direct acting furthermore slid to each leg apical portion as a jump mechanism for leg them. Therefore,
as for the jumping-of-flow-control-valve legged robot to
examine, since it is an existing six-legged robot, 4 degreeof freedom is a robot which has 24 degree of freedom in
total at each leg. The target six jumping-of-flow-controlvalve robot models, each link length, and the leg number
which were used by the kinetics simulator are shown in
Fig. 1. The line extended on a robot in Fig. 1 expresses
perpendicular above, makes this a z axis, and makes an
x-axis the line extended in the direction of this side of arobot. Further Axis, Direction which makes an axis and aright-hand system It is considered as an axis. Moreover,
let the field which is in sight to the front with Fig. 1 bea front face of a robot. Saw from the robot, and become
an odd number on left-hand side, it is made to turn intoeven numbers on right-hand side, and the leg number of
each leg set the leg number of the front left-hand side leg
to Leg1.
The total mass of a robot is 12 kg. The maximum
height of the axial direction of the robot in the postureof Fig. 1 is set to 0.35 m. The sliding mechanism fora jump of the leg point is shown in Fig. 2. The sliding
mechanism for a jump is slid in +0.05 m and -0.05 m on
the basis of the leg point. In addition, the leg used eachleg as the same mechanism.

III.

inhibitory synapse
excitatory synapse

Fig. 3: Network for CPG

When two or more CPG neurons exist, it is known thatthe phenomenon called drawing in with the passage oftime will generate the output cycle signal of each neuron by defining the coupling coeﬃcient between neurons. Although there were some mathematical models
proposed so far in CPG, in this paper, Matsuoka oscillator with a comparatively simple relation of a parameter
was used[3]. Matsuoka model was following formula.
Tr△yi = g(ui ) (g(ui ) = max(0, ui ))d fi
Ta+ fi = yidt(1)

Where, t is time variable, i, j is number of neuron. T r
is rise time constant, ui is membrane potential of neuron
body, ai j (= 0 for i , j and = 0 for i = j) is a weightof inhibitory synaptic connection from j-th neuron to thei-th neuron. yi is a firing rate or output of the neuron, sis an impulse rate of input. b is the parameter that deter-

Jumping Rhythm Generator

CPG is a kind of a neuron model and generates the periodic signal it is supposed that it is deeply related to theperiodical activity of a living thing of the periodic signal.

©ISAROB 2011n∑dui
+ ui = −ai j y j + si − b fi
dt
j=1

396

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

0.9

Force τ ι [N]

0

z[m]

0.6

0.3

0

0

5
Time t [s]

−200
τ1
τ2
τ3
τ4
τ5
τ6

10
−400

0

5

10

Time t [s]Fig. 4: The vertical posision of the robot

Fig. 5: Driving forces of soles
mines the steady-state firing rate for a constant input. f
is adaptation variable, T a is adaptation time constant.
The composition of the CPG network by inhibition
binding of six neurons used in this paper is shown in
Fig. 3. The output value y from each neuron of the constituted CPG network is carried out as a next desired valueof the sliding mechanism for a jump of the leg number i
as shown in following equation.
hoi = −yi

was set to a12 = a13 = 2.5,a21 = a24 = 2.5, a31 = a34 =a35 = 2.5, a42 = a43 = a46 = 2.5, a53 = a56 = 2.5 anda64 = a65 = 2.5. The control gain used for the force
calculation in the case of a jump was set to P = 4500 and
D = 1.

2.

(2)

The force outputted to each sliding mechanism isshown in Fig. 5. The orbit of the body of a robot is shownin Fig. 4.
The output of each CPG neuron is shown in Fig. 6.
Fig. 6 is put in order by neuron numerical order from thetop. By the constituted CPG network, Phases of the first,
the fourth and the fifth neurons is same each other. Similarly, outputs of the second, the third and sixth neurons
have same phase. Moreover, a phase shift from which thephase angle of such combination obtains the maximum
by turns is seen.
The number of times of a jump is because the shakeof a robot was not fully settled, so few things are not
outputting force compared with the number of times ofan output of CPG. Conversely, the force currently outputted calculates the output of CPG when conditions are
fulfilled. When the output of Fig. 5 and the orbit of Fig. 4
are measured, it turns out that it has jumped immediately after outputting force. However, it turns out thatthe height proportional to the output of force has not necessarily come out.
This is because slope of the force outputted since force
is outputted when slope of the CPG output of the phaseangle of another side is negative when one of the two is
0 among the outputs by two kinds of phase angles also
becomes negative. On the contrary, the height to which
one output is proportional to force since slope of the force
outputted when slope of the CPG output of the phase angle of another side is positive also just becomes by 0 is
obtained. When a CPG output is larger, it turns out that

However, it is less than a value with x of the robot centerof gravity, and the acceleration of y axial direction, Whenthe acceleration of an axial direction approaches gravitational acceleration enough, it thinks that the shake of arobot was fully settled, and it limits to when the shakeof a robot is fully settled, and a CPG output is used as adesired value. The output of the sliding mechanism fora jump is calculated by the following formula using thedesired value acquired from CPG.
τi = P(hoi − hi ) + D(voi − ḣi )(3)Where, i is number of the leg, hi is the displacement of theslide joint, voi is objective velocity (Now, voi = 0). P is
the proportional gain, D is the diﬀerential gain. Thus, theforce for a jump is generated with the generated period.

IV. Simulation
1.

Setting up of parameters

It is referred to as T r = 0.3, si = 0.25, b = 10.5, T a =0.2 and each parameter given to the Matsuoka oscillatorof CPG used for the simulation is an initial value of themembrane potential u of each neuron. It carried out andthe initial value of u1 = u4 = u5 = 0, u2 = u3 = u6 =0.05 and a fatigue state f was set to f1 = f4 = f5 = 0,f2 = f3 = f6 = 0.05. Coupling load of the CPG network©ISAROB 2011

Results

397

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 20110.06

the jump to which it is higher to output force is carriedout. Moreover, in order that the direction whose slope ofthe output of CPG the CPG output of two kinds of phaseangles is both positive at the time or more of zero at thetime of the output of force may work strongly, it turns outthat the height proportional to force is obtained.

Output y 1

0.04

0.02

0

V. Conclusions

0

5Time t [s]

10

0

5
Time t [s]

10

0

5
Time t [s]

10

0

5
Time t [s]

10

0

5
Time t [s]

10

0

5
Time t [s]

10

0.06

It is checked that a robot jumps periodically by using
the network composed of CPG neurons which are modeling with Matsuoka osillator. The robot has six legs which
are equipped jump mechanism. In the CPG network ofthis paper, the height of the jump at each time changed
unintentionally. This feeds back the force sensor of arobot, and the information on a gyro sensor to CPG, andis expected that a suitable CPG output to which the heightof a jump becomes fixed is obtained by presuming time
until a posture is stabilized from the flight duration andlanding of a robot.

Output y 2

0.04

0.02

0
0.06

Output y 3

0.04

References

0.02[1] Ijspreet AJ (2008), Central pattern generators for locomotion control in animals and robots: A review. Neural Networks 21:642-653

0[2] Liu GL, Habib MK, Watanabe K, Izumi K (2008), Central
pattern generators based on Matsuoka oscillators for the
locomotion of biped robots. Artificial Life and Robotics
12(1-2):264-269

0.06

Output y 4

0.04[3] Matsuoka K (1987), Mechanisms of Frequency and pattern control in the neural rhythm generators. Biological
Cybernetics 56(5-6):345-353

0.02

0
0.06

Output y 5

0.04

0.02

0
0.06

Output y 6

0.04

0.02

0

Fig. 6: Outputs of neurons©ISAROB 2011

398

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A Bearing-Only Localization Solved by an Unscented Rauch-Tung-Striebel Smoothing
Saifudin Razali, Keigo Watanabe, and Shoichi Maeyama
Department of Intelligent Mechanical Systems,
Graduate School of Natural Science and Technology, Okayama University,
3-1-1 Tsushima-naka, Kita-ku, Okayama 700-8530, Japan
Tel & Fax:+81-86-251-8064
E-mail: saifudin@usm.sys.okayama-u.ac.jp, {watanabe, maeyama}@sys.okayama-u.ac.jp
Abstract: The unscented Kalman filter (UKF) has become an alternative in nonlinear estimation problems to overcome
the limitation of Taylor series linearization used by the extended Kalman filter (EKF). It uses a deterministic sampling
approach known as sigma points to propagate nonlinear systems and has been discussed in many literature. However,a nonlinear smoothing problem has received less attention than the filtering problem. Therefore, in this article we
examine an unscented smoother based on Rauch-Tung-Striebel form for discrete-time dynamic systems. This smootherhas advantages available in unscented transformation over approximation by Taylor expansion as well as its benefit
in derivative free. This smoothing technique has been implemented and evaluated through a bearing-only localization
problem.
Keywords: Unscented transformation, Rauch-Tung-Striebel smoother, Bearing-only localization problem.

I. INTRODUCTION

position of the prior covariance, where n is the state
dimension. These sigma points are propagated throughthe true nonlinear function, without approximation, andthen a weighted mean and covariance is taken. Thisapproach results in approximations that are accurate to
the third order Taylor series expansion for Gaussian
inputs in all nonlinearities.
However, the nonlinear smoothing problem has received less attention than the filtering problem in theliterature. Therefore, in this article we investigate theunscented smoother based on Rauch-Tung-Striebel form[6], [7] for discrete-time dynamic systems studied bySärkkä [8] and Saifudin et al. [9], [10]. This smoothertakes a benefit over unscented transformation to the limitation of Taylor approximation as well as its derivative
free advantages. To evaluate the performance of thissmoother, the algorithm is applied for a bearing-only
localization problem. In what follows, note that we will
use the abbreviations URTSS for the unscented RauchTung-Striebel smoother.
The structure of this paper is as follows: In section 2
we briefly describe the derivation of unscented RauchTung-Striebel smoother and its summary of an implemented algorithm. A bearing-only localization problem
is presented in section 3 as an application example ofthis algorithm, as well as discussions on the simulation
results. The paper is concluded in section 4.

The nonlinear filtering problem has been deeply
studied and various methods are provided in literature. Among them, the most useful ones are the extended Kalman Filter (EKF), the ensemble Kalman Filter(EnKF), the unscented Kalman Filter (UKF), and theParticle Filter (PF). Historically, the EKF is still themost widely adopted approach to solve the nonlinear
estimation problem. It is based on the assumption that
nonlinear system dynamics can be accurately modeledby a first-order Taylor series expansion as proved by van
der Merwe [1]. The EnKF introduced by Evensen [2] isa reduced rank filter which propagates the states throughnonlinearity and updates a relatively small ensemble ofsamples from which an assumed Gaussian distribution
captures the main characteristics in the uncertainty. The
PF also uses a sampling approach to estimate the higherorder moments of the posterior probability distribution
by propagating and updating a number of particles, but
without assuming Gaussian statistics as explained by
Arulampalam et al. [3].
The UKF, which is a derivative free alternative to
EKF, overcomes the differentiation problem by using adeterministic sampling approach demonstrated by Julier
and Uhlmann [4] and Wan and van der Merwe [5]. The
state distribution is represented using a minimal set ofcarefully chosen sample points, called sigma points. This
technique is used to linearize a nonlinear function of arandom variable through a linear regression between npoints drawn from the prior distribution of the randomvariable. Since we are considering the spread of therandom variable during linearization, the technique tends
to be more accurate than the Taylor series linearization
used in the EKF, particularly in the presence of strong
nonlinearities as proved by van der Merwe [1]. The 2n+1
sigma points, are chosen based on a square-root decom-©ISAROB 2011

II. UNSCENTED
RAUCH-TUNG-STRIEBELSMOOTHER
Consider a state space model of the form,
xk = Fk−1 (xk−1 , uk−1 , wk−1 )yk = Hk (xk , vk )

399

(1)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011e w denote the parts of thee x and X
where Xk,ik,iaugmented sigma point i, which correspond to xk
and wk , respectively.
3) Compute the predicted mean m−
k+1 , the predicted−covariance Pk+1
and the cross-covariance Ck+1 :
X (m)
e−
m−
Wi−1 Xk+1 =k+1,iwhere xk ∈ Rn is the state, yk ∈ Rm is themeasurement at time tk , uk−1 is the control action,
wk−1 ∼ N (0, Qk−1 ) is the Gaussian process noise,vk ∼ N (0, Rk ) is the Gaussian measurement noise,
Fk−1 (·) is the process model function and Hk (·) is themeasurement model function. The time step k runs from0 to T and at time step 0 there is no measurement, only
the prior distribution x0 ∼ N (m0 , P0 ).
The purpose of the smoothing algorithm is to find approximations to the smoothing distributions p (xk | y1:T )for all k = 1, 2, . . . , T . The approximations are chosen
to be Gaussian:p (xk | y1:T ) ∼ N (xk | msk , Pks ) .

i

−
Pk+1
=

i



(c)
−
e−
Wi−1 X
−
m
k+1,i
k+1

T

−
e−
× X
−
m
k+1,i
k+1

X (c) xe − mkWi−1 X
Ck+1 =
k,i

(2)

i


T
−
e−
× X
−
m
k+1,i
k+1

The optimal smoothing equations of the model canbe written in two options as mentioned by Klaas etal. [11], named as two filter smoother and forwardbackward smoother. For the purpose of deriving theRauch-Tung-Striebel form of smoother, the forwardbackward smoothing will be used and it can be written
as follows:(m)where the definitions of the weights Wi−1 and(c)
Wi−1 are the same as in [5].
4) Compute the smoother gain Dk , the smoothedmean msk and the covariance Pks : − −1
Dk = Ck+1 Pk+1

msk = mk + Dk msk+1 − m−
k+1
 T
 s
−
Dk
Pks = Pk + Dk Pk+1
− Pk+1p (xk | y1:T ) = p (xk | y1:k )Z
p (xk+1 | xk ) p (xk+1 | y1:T )dxk+1
×p (xk+1 | y1:k )(3)

The above procedure is a recursion, which can be used
for computing the smoothing distribution of step k fromthe smoothing distribution of time step k+1. Because thesmoothing distribution and filtering distribution of thelast time step T are the same, we have msT = mT , PTs =PT , and thus the recursion can be used for computing
the smoothing distributions of all time steps by startingfrom the last step k = T and proceeding backwards tothe initial step k = 0.

where p (xk | y1:k ) is the filtering distribution of the time
step k and p (xk+1 | y1:k ) is the predicted distributionof the time step k + 1, which can be computed by theprediction step of the optimal filtering. The smoothing
recursion is started from last time step k = T andproceeded backwards in time.
From Eq. (3), the Rauch-Tung-Striebel smoother canbe derived as shown by Särkkä [8]. Assumed that the(approximate) mean and covariance of the filtering distributions

III. AN EXAMPLE APPLICATIONIn this section we consider the problem of bearing
only localization as used by Bailey [12].

p (xk | y1:k ) ≈ N (xk | mk , Pk )for the model in Eq. (1) have been computed by theunscented Kalman filter or a similar method.
Further assume that the smoothing distribution of time
step k + 1 is known and Gaussiansp (xk+1 | y1:T ) ≈ N xk+1 | msk+1 , Pk+1.

1. Process and measurement state
The discrete time vehicle state is given by:
xk = f (xk−1 , uk−1 , wk−1 )(4)Twhere xk = [xk yk φk ] are the vehicle position coordinates and its orientation in time step k , respectively.
Tuk−1 = [V G] is control action in which V is the vehicle velocity, and G is the vehicle steering angle. wk−1
is a zero-mean Gaussian process noise with covariance
Q.
This vehicle is equipped with a range and bearing
sensor. It can sense an object bounding in ±30 degreesemi-circle with the maximum range of 30 meter. Only
bearing measurement data will be used in this example.
The measurement equation is as follows:This smoothing algorithm can be summaried as following steps:
1) Form the matrix of sigma points of the augmentedT
random variable x
ek = xTk wkTsuch that

q
q 
√
e
e
Pk − Pek
Xk = [mek ··· me k] + c 0





mk
Pk 0
e
where mek =and Pk =.
00 Qk2) Propagate the sigma points through the dynamic
model


e−
ex ew
X
i = 1, . . . , 2n + 1
k+1,i = Fk Xk,i , Xk,i ,©ISAROB 2011X

zk = h (xk , f i ) + vk

−1 fi,y − yk
) − φk + vk
= tan (
fi,x − xk

400

(5)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 1.

180
Start
True
Landmark

y-coordinate [m]160

Simulation setup

140

Parameter

Description

Value

Unit

120

V

Velocity3

m/s
radian100
80

G

Steering angle

0.05

WB

Wheel-base

4

m

σV

Standard deviation of V

0.3

m/s

σG

Standard deviation of G

0.05

radianσBStandard deviation of bearing

0.09

radian

60
40
20
0
-20

180
0

20

40

60

80

100

120

140

160

Start
True
UKF
URTSS
Landmark

160

x-coordinate [m]140True trajectory and landmark.

120

x-coordinate [m]Fig. 1.

where xk , yk and φk are the vehicle position coordinates
and its orientation in time step k, respectively. f i is alandmark feature available at time when the sensor takes
a measurement. This landmark feature is assumed to bestatic and represented as a Cartesian coordinate system
as (fi,x ,fi,y ). vk is assumed as zero-mean Gaussian whitemeasurement noise with covariance R.

100
80
60
40
20
0
0

20

Fig. 2.

Fig. 1 shows the landmark and true vehicle trajectory
setup for this simulation. The vehicle starts at a known
location (20 m, 20 m, −0.8 rad) and travels with anominal speed and a steering angle of 3 m/s and 0.05 rad,respectively. The nominal control values are corrupted
with Gaussian noise with standard deviations 0.3 m/s and0.05 radian, respectively for each 0.5 s sampling interval.
The sensor takes bearing measurement and its value
is assumed to be corrupted with Gaussian noise withstandard deviation 0.09 radian. All simulation parameters
and their values are shown in Table 1.
The initial conditions for the filter are set tox̂0 = x0

=

and

40

60

80

100

120

140



10−10
P0 =  00

the error values produced by the UKF. This situation also
can been seen in rms vehicle position errors as shownin Fig 6. However, the vehicle orientation rms errors
for both methods did not show a significant difference.
Futhermore, we took root mean square values for 1000
Monte Carlo runs. The results of filtering and smoothing
estimation in x-axis, y-axis, and φ are shown in Table 2
and it is proved that the URTSS has a better performanceover the UKF.

IV. CONCLUSIONIn this paper, an unscented Rauch-Tung-Striebelsmoother (URTSS) has been applied to a bearing-only
localization problem and its performance has been also
evaluated in simulations. It was assumed that the vehicle
was equipped with a range and bearing sensor. To
compare the performance of both UKF and URTSS, therms errors were calculated. It was then found the URTSS
has a better performance over the UKF.

20
20 
−0.8
0
10−10
0

Estimates by UKF and URTSS.


0
0 
10−10

which basically means that the vehicle initial position
and its orientation are known.
3. Result and discussion
Fig. 2 shows the result of both UKF and URTSS. For
more reliable result, we calculated the root mean square(rms) error of x-axis, y-axis, φ and also its position forevery iteration step and they are shown in Figs. 3, 4, 5
and 6 respectively. Figs. 3 and 4 show that the rms errorsof the URTSS are always comparable to or lower than

©ISAROB 2011160

x-coordinate [m]2. Simulation setup

401

Table 2.

RMS errors

MethodxRM SEyRM SEφRM SE

U KF

0.87300.8361

0.0941

U RT SS

0.74380.73670.0771

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

4
UKF
URTSS

4
UKF
URTSS

3.5

3.5

3

RMS error [m]RMS error [m]

3

2.5

2

2.5
2
1.5

1.5

1
1

0.5
0.5

0
0
0
0

100

200

300

400

500

k [step]
50

100

150

200

250

300

350

400

450

500

k [step]

Fig. 3.

Fig. 6.

RMS vehicle position error estimates by UKF and URTSS.

RMS error of x-axis.

REFERENCES[1] van der Merwe R (2004), Sigma point Kalman filter for probabilistic inference in dynamic state-space models. PhD Thesis,Oregon Health & Science University, Portland.
[2] Evensen G (2003), The ensemble Kalman filter: Theoreticalformulation and practical implementation. Ocean Dynamics, 53:
343–367.
[3] Arulampalam MS , Maskell S, Gordon N, and Clapp T (2002),
A tutorial on particle filters for online nonlinear/non-Gaussian
Bayesian tracking. IEEE Trans. on Signal Processing, 50(2): 174–
188.
[4] Julier SJ and Uhlmann JK (2004), Unscented filtering and nonlinear estimation. Proceedings of the IEEE, 92(3): 401–422.
[5] Wan E and van der Merwe R (2004), The Unscented Kalman
Filter, New York: Wiley.
[6] Rauch HE, Tung F and Striebel CT (1965), Maximum likelihood
estimates of linear dynamic systems. AIAA Journal 3(8): 1445–
1450.
[7] Rauch HE (1963), Solutions to linear smoothing problem. IEEE
Trans. on Automatic Control 8(4): 371–372.
[8] Särkkä S (2008), Unscented Rauch-Tung-Striebel smoother. IEEE
Trans. on Automatic Control AC-53(3): 845–849.
[9] Saifudin R, Watanabe K, Maeyama S and Izumi K (2010),
An unscented Rauch-Tung-Striebel smoother for a bearing only
tracking problem. Int. Conf. Control, Automation and Systems
2010, Gyeonggi-do, Korea.
[10] Saifudin R, Watanabe K, Maeyama S and Izumi K (2010), An
unscented Rauch-Tung-Striebel smoother for a vehicle tracking
problem. Int. Conf. on Soft Computing and Intelligent Systems
and Int. Sympsium on Advanced Intelligent Systems, Okayama,
Japan.
[11] Klaas M, Briers M, de Freitas N, Doucet A, Maskell S, and Lang
D (2006), Fast particle smoothing: If I had a million particles.
Proceedings of ICML, 25–29.
[12] Bailey T (2003), Constrained initialization for bearing only
SLAM. IEEE Int. Conf. on Robotics and Automation (2): 1966–
1977.

3.5
UKF
URTSS
3

RMS error [m]

2.5

2

1.5

1

0.5

0
0

50

100

150

200

250

300

350

400

450

500

k [step]

Fig. 4.

RMS error of y-axis.

7
UKF
URTSS
6

RMS error [rad]

5

4

3

2

1

0
0

50

100

150

200

250

300

350

400

450

500

k [step]

Fig. 5.

©ISAROB 2011

RMS error of φ-axis.

402

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Trajectory Tracking Control for Nonholonomic Mobile Robots
by an Image-Based Approach
Tatsuya Kato, Keigo Watanabe, and Shoichi Maeyama
Department of Intelligent Mechanical Systems,
Graduate School of Natural Science and Technology, Okayama University,
3-1-1 Tsushima-naka, Kita-ku, Okayama 700-8530, Japan
Tel & Fax: +81-86-251-8064
E-mail: t.kato@usm.sys.okayama-u.ac.jp, {watanabe, maeyama}@sys.okayama-u.ac.jp
Abstract:
The vision-based control that uses cameras for observing a robot environment has been researched widely. Especially,a method called image-based control has high robust properties, because it can control a target on an image plane
coordinate without using a robot position. Kurashiki et al. have already studied on an image-based control methodthat can realize a robust trajectory tracking. Although their objective was to control a nonholonomic mobile robot, theproblem setting used there was to be little affected from the nonholonomic constraint. Further more, there were unclear
points in deriving a control law. In this paper, such unclear points are explained from an geometric relation and otherproblem settings, which are clearly affected by a nonholonomic constraint, are proposed.
Keywords: Vision-based control, Image-based control, Trajectory tracking, nonholonomic mobile robot.

I. INTRODUCTION
Unmanned vehicles such as automated driving cars require high robustness against any disturbances for safety.
On the other hand, with the popularization of inexpensive
cameras, vision-based control has been researched in
the domain of robot control. Vision-based control isclassified roughly into two methods that are called a“position-based” method and an “image-based” method[1], [2], [3], [4]. In the position-based method, the control errors are calculated from the position and the poseof the camera estimated from captured images. Although
the method can control these states directly, the camera
calibration is very important to the state estimation. The
image-based method does not consider the position of
the camera. The control errors are calculated on thecoordinate which is attached directly to the captured
2D image. The control input is determined according
to the control errors such as the amount of features, thelocation of the target on the image coordinate and soon. In general, the image-based method is known to berobust not only camera but also robot calibration errors.
The purpose of our research is controlling mobile
robots with the image-based method, because it has
higher robustness than the position-based method. As
the earlier study of the image-based method, Kurashiki[5] developed a system consisting of a nonholonomic
robot and a camera so that the robot can track the line
drawn on the floor, with controlling the gradient andthe intercept of the line on the captured image to their
desired values. However, a nonholonomic constraint does
not affect its problem setting. Additionally, there is a
mistake of the derivation of a control low. In this paper,
these two problems are explained and the control low
is checked with a simulation experiment. Then, a new
problem setting which is influenced by a nonholonomic©ISAROB 2011

Fig. 1.

Definition of coordinates

constraint is described.

II. PROBLEM SETTING
Fig. 1 shows the environment of the trajectory tracking
system. A camera is attached on the robot to observe thetarget line drawn on the floor. The objective of the control is that the robot tracks the target line autonomously
based on the captured image.
1. Coordinates
The world coordinate is set such that x-axis is along
the target line and y-axis is perpendicular to the xaxis shown in Fig. 1. The v-u coordinate, whose origin
corresponds to the center of the captured image plane,is attached on the image plane as shown in Fig. 2. For
simplification, the camera is assumed to be equipped
at the center of the robot with its downward direction.
Thus, the origin of v-u coordinate corresponds to theposition of the robot (x, y) on the world coordinate.
An anticlockwise rotation is to be positive for the angle
between target line and u-axis θ (i. e., θ has a negativevalue in Fig. 2).

403

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig. 3.

Fig. 2.

2. Question of Earlier Research

Coordinate of the image plane

Kurashiki et al. derived the relationship between y andc3 as follows:h(7)
y = u23 + v32 · sign(c3 c1 ) ·f

In general, the equation of a straight line on v-u
coordinate is denoted as follows:
a1 u + a2 v + a3 = 0

where the point of (v3 , u3 ) is the nearest point to theorigin of v-u coordinate on the target line. These v3 andu3 are calculated by the following equations:c3(8)
v3 = − 2c1 + 1c1 c3(9)u3 = − 2c1 + 1(1)

Assuming that this strait line is not parallel to v-axis, a2
is not 0. Thus, Eq. (1) can be divided by a2 to obtain
Eq. (2),


a1
a3
(2)
c1 u + v + c3 = 0, c1 = , c3 =a2
a2

Although Kurashiki et al. said that Eq. (5) was able to be
derived with Eqs. (7), (8) and (9), the signum function
still remains. Thus, it is necessary to split the case wherec3 c1 is positive or negative, but there is no explanation
about it. Additionally, when c1 > 0 and c3 < 0 as shownin Fig. 2, Eq. (7) gives a negative value, though y is
positive. Thus, Eq. (7) seems to be a wrong relationshipequation. However, note that somehow they derived aright relationship given in Eq. (5).

where c1 is a factor related to the gradient of the line
and c3 is the reversal sign of v-coordinate value at theintersection of the line and v-axis. Thus, the parametersof the target line are c1 and c3 . The relationship between
c1 and θ is written as follows:
 π
π
(3)
c1 = − tan θ, − < θ <2
2
On the other hand, the distance y between the target line
and the robot should converge to 0, to track the target
line. On the world coordinate, it is just as the y valueof the robot position (x, y). The relationship between c3
and y is obtained geometrically such as
y = −c3 cos θ ·hfIII. ROBOT MODEL
In this paper, a robot is to be a two-wheel independent
driven type shown in Fig. 3. The position of the robot is(x, y) on the world coordinate. The pose of the robot isthe angle θ between the direction of forward movement
and x-axis. The kinematic model of this robot is denoted
by
⎡ ⎤ ⎡
⎤
x
cos θ 0  
d ⎣ ⎦ ⎣sy = sin θ 0⎦
(10)
ω
dt
θ
0
1

(4)

where f is a focal length of a camera and h is analtitude of a camera position. Then the above equationis rewritten as follows:
yf
1
·
(5)
c3 = −cos θ hThus, controlling c1 and c3 to zero on the image plane
is equivalent to tracking the target line on the world
coordinate. The observing equation to obtain c1 and c3
from the captured image is written as follows:
 


1
c1
v1 − v2=(6)c3
u2 − u1 −u2 v1 + u1 v2

where s = ẋ2 + ẏ 2 is the translational velocity andω = θ̇ is the angular velocity.

IV. CONTROLLER
In this section, a controller is designed based on
Liapunov’s theory. Since an image-based method is
proposed, a control target is not a robot but coefficientsof the target line, i. e., parameters c1 and c3 on the image
plane. The goal of the control is to be the convergenceof these values to 0.

where the points of (v1 , u1 ) and (v2 , u2 ) are arbitrary
points on the target line on the captured image. Assuming that the target line is not parallel with v-axis,
it follows that u2 − u1 = 0.

©ISAROB 2011

Robot model

404

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

To derive a control low, Eqs. (3) and (5) are differentiated with respect to time and they are rearranged toobtain    2

fs 0
d c1
c +1
=
− 1
ω
(11)
c1 c3
dt c3
h c1
The next equation is one candidate of a Liapunov function:
K 1 2 K3 2
V =
c +
c
(12)
2 1
2 3
where K1 and K3 are positive gains. Differentiating itwith respect to time givesfs
dV=
K3 c1 c3 − c1 {K1 (c22 + 1) + K3 c23 }ω (13)dthAssume that the input value ω is taken as follows:fs
K3 c3 + K2 c1 (14)
ω = {K1 (c21 + 1) + K3 c23 }−1hFig. 4.

Simulation result

where K2 is a positive gain. Then, substituting Eq. (14)into Eq. (13) yields
dV= −K2 c21 ≤ 0
dt

(15)

Using the Barbalat’s lemma [6], it is proved that V̇ → 0
and c1 → 0. Also, note that

Fig. 5. Situation when the y-axis is given the upper and lower limitsfs
K3 c3 = 0 (16)hThus, assuming that s = 0, it is seen that c3 → 0, sothat the convergence to a desired state is ensured.
lim c˙1 = − lim (K1 + K3 c23 )−1

t→∞

t→∞

1. Problem Setting 1: Specifying Direction
The target line is to be two colored to specify the
robot direction as shown in Fig. 5. Since the problem
setting explained previously assumes that the pose of
the robot is within −π/2 < θ < π/2, the robot cannot
turn around. When applying any controllers to realrobots, specifying the direction of the robot movement
is useful for widespread purposes. Additionally, since
the controller designed in this paper shows an overshootshown in Fig. 5, the robot needs to use any crosscut
motion by limiting y-axis value.

V. SIMULATION
A simulation experiment is conducted to test thedesigned controller. The initial state of a robot is set
to (x, y, θ) = (0 [m], −0.71 [m], −0.78 [rad]) so as
to obtain (c1 , c3 ) = (1, 1). The function of a target line
is to be y = 0 [m]. The focal length of a camera is f
= 1 [m] and its altitude from the floor is h = 1 [m].
The translational velocity is fixed as s = 0.5 [m/s]. The
control gains are set as (K1 , K2 , K3 ) = (0.1, 3, 10).
Fig. 4 shows the experimental result. It is confirmedthat controlling c1 and c3 to 0 was able to be accomplished by tracking the target line.

2. Problem Setting 2: Specifying Endpoint
The controller explained previously makes the robot
with nonholonomic features track a target line by ignoring the value of x. This problem setting is given theend point of a target line. The goal of this control is
to position the end point to the center of image plane.
The forementioned controller cannot accomplish such an
objective because the controller ignores x value and thetranslational velocity is set to be constant. Thus, it needs
to set the translational velocity as a variable, instead ofsetting it as a constant.

VI. CONSIDERATION
Although the mobile robot has nonholonomic characteristics, the problem setting is not affected by suchfeatures. In the problem setting explained previously,
the robot tracks its target line by controlling only yand θ, without controlling x. Since the states to becontrolled are only two, it need not use any crosscut ofsteering. Therefore, it is necessary to set other problems
in which there exist influences due to nonholonomic
characteristics.

VIII. CONCLUSION
In this paper, the paper given in Kurashiki et al. [5]has been questioned and a designed controller has beenchecked on a simulation experiment. It was confirmed
from the simulation experiment that the robot can trace
a target line with the designed controller. Two problem
settings affected by nonholonomic characteristics wereVII. OTHER PROBLEM SETTINGS
In what follows, two problem settings affected bynonholonomic characteristics are proposed.

©ISAROB 2011405

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig. 6.

Situation when the target line has an end point

also proposed to demonstrate the ability of the current image-based control method. In the future, suitable
controllers will be designed and checked for the two
proposed problems.

REFERENCES[1] Hutchinson S, Hager G, and Corke P (1996), A tutorial on visual
servo control, IEEE Trans. on Robotics and Automation, vol. 12,
no. 5, pp. 651–670.
[2] Chaumette F and Hutchinson S (2006), Visual Servo Control,
Part I: Basic Approaches, IEEE Robotics and Automation Magazine, vol. 13, no. 4, pp. 82–90.
[3] Chaumette F and Hutchinson S (2007), Visual Servo Control,
Part II: Advanced Approaches, IEEE Robotics and Automation
Magazine, vol. 14, no. 1, pp. 109–118.
[4] Malis E (2002), Survey of vision-based control, presented at the
In European Naval Ship Design, Captain Computer IV Forum,
ENSIETA, Brest, France.
[5] Kurashiki K, Fukao T, and Osuka K (2008), Image-Based Robust
Trajectory Tracking Control of a Nonholonomic Mobile Robot(in Japanese), The 26th Annual Conference of The Robotics.
[6] Slotine EJJ and Li W (1991), Applied Nonlinear Control, Prentice Hall, pp. 122–126.

©ISAROB 2011406

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A Nonholonomic Control Method for Stabilizing an X4-AUV
Zainah Md. Zain, Keigo Watanabe, Kiyotaka Izumi† , and Isaku Nagai
Department of Intelligent Mechanical Systems,
Division of Industrial Innovation Sciences,
Graduate School of Natural Science and Technology, Okayama University
3-1-1 Tsushima-naka, Kita-ku, Okayama 700-8530, Japan
Tel & Fax: +81-86-251-8064
E-mail: zainah@usm.sys.okayama-u.ac.jp, {watanabe, in}@sys.okayama-u.ac.jp†
Department of Mechanical Engineering,
Graduate School of Science and Engineering, Saga University
1-Honjomachi, Saga 840-8502, Japan
Tel: +81-952-28-8696, Fax: +81-952-28-8587
E-mail: izumi@me.saga-u.ac.jp
Abstract: A nonholonomic control method is considered for stabilizing all attitudes and positions (x, y or z) of an
X4-AUV with four thrusters and six degrees-of-freedom (DOF), in which the positions were stabilized according to
the Lyapunov stability theory. The derived model is dynamically unstable, so a sequential nonlinear control strategy isimplemented for the X4-AUV, composed of translational and rotational subsystems. A controller for the translational
subsystem stabilizes one position out of x-, y-, and z-coordinates, whereas controllers for the rotational subsystems
generate the desired roll, pitch and yaw angles. Thus, the rotational controllers stabilize all the attitudes of the X4-AUV
at a desired (x-, y- or z-) position of the vehicle.
Keywords: AUV, Underactuated control system, Nonholonomic systems.

I. INTRODUCTION

translational and rotational subsystems. The controller
for the translational subsystem stabilizes the position andthe controllers for the rotational subsystems generate thedesired roll, pitch and yaw angles.

Nowadays control problems of underactuated vehicles
motivate the development of new design methodologies in nonlinear control. The control system for suchvehicles has fewer number of control inputs than thenumber of generalized state variables to be controlled.
Hence, the dynamical equations of the vehicle exhibit socalled second-order nonholonomic constraints, i.e. nonintegrable conditions imposed on the acceleration in one
or more DOFs, because the vehicle lacks capability tocommand instantaneous accelerations in these directionsof the configuration space [1]. As pointed in a celebrated
paper of Brockett in1983, such nonholonomic systems
cannot be stabilized by usual smooth, time-invariant,and state feedback control algorithms. For the control
of underactuated underwater systems, many researches
proposed nonlinear feedback control [2], among manyothers. Nonholonomic property of a mechanical systems
can be seen in underactuated systems, even though theconnection between nonholonomic control systems andunderactuated systems is not completely understood. To
control nonholonomic systems, many control approaches
have been studied. Kolmanovsky and Clamroch present
a vast literature and overview on nonholonomic control
in their work [1].
In this paper, we present a model of an underactuated X4-AUV with 6-DOF and four control inputs andpropose a control scheme based on Lyapunov approachto stabilize all attitudes and positions of the vehicle.
A sequential nonlinear control strategy is implemented
for the derived 6-DOF vehicle model, constituted of©ISAROB 2011

II. DYNAMICAL MODEL
1. Coordinate System of AUV
In order to describe the underwater vehicle’s motion, a
special reference frame must be established. There have
two coordinate systems: i.e., inertial coordinate system(or fixed coordinate system) and motion coordinate system (or body-fixed coordinate system). The coordinate
frame {E} is composed of the orthogonal axes {Ex Ey
Ez } and is called as an inertial frame. This frame iscommonly placed at a fixed place on Earth. The axes
Ex and Ey form a horizontal plane and Ez has the
direction of the gravity field. The body fixed frame {B}is composed of the orthonormal axes {X, Y, Z} andattached to the vehicle. The body axes, two of whichcoincide with principle axes of inertia of the vehicles,
are defined in Fossen [3] as follows:
X is the longitudinal axis (directed from aft to fore)
Y is the transverse axis (directed to starboard)
Z is the normal axis (directed from top to bottom)
Fig. 1 shows the coordinate systems of AUV, which
consist of a right-hand inertial frame {E} in which the
downward vertical direction is to be positive and righthand body frame {B}.
Letting ξ = [x y z]T denote the mass center of the
body in the inertial frame, defining the rotational anglesof X-, Y- and Z-axis as η = [φ θ ψ]T , the rotational

407

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Ex

{E}

From the assumption of the balance between the buoyancy and the gravity, i.e., the potential energy U = 0,
the Lagrangian can be written as

Ey

ξ

Ez

L=T −U= Ttrans + TrotφXθThe resultant dynamical model for the X4-AUV is summarized by [5]{B}Y

m1 ẍ = cos θ cos ψu1
m2 ÿ = cos θ sin ψu1

ψZ
Fig. 1.

(9)(10)

m3 z̈ = − sin θu1

Coordinate systems of AUV(11)

Ix φ̈ = θ̇ψ̇(Iy − Iz ) + u2
Iy θ̈ = φ̇ψ̇(Iz − Ix ) − Jt ψ̇Ω + lu3
Iz ψ̈ = φ̇θ̇(Ix − Iy ) + Jt θ̇Ω + lu4

matrix R from the body frame {B} to the inertial frame{E} can be reduced to:Here, the values of m1 , m2 , m3 , Ix , Iy , and Iz are
defined as in (5) and (6). u1 , u2 , u3 , u4 are the control
inputs for the translational (x, y and z-axis) motion, theroll (φ-axis) motion, the pitch (θ-axis) motion, and yaw(ψ-axis) motion, respectively. Defining that b is a thrust
factor, d is a drag factor, taken from τMi = dω 2i , Ω, u1 ,
u2 , u3 and u4 are given by⎤cφsθcψ + sφsψcφsθsψ − sφcψ ⎦cφcθ
(1)
where cα denotes cos α and sα is sin α.
⎡cθcψ
R = ⎣ cθsψ−sθsφsθcψ − cφsψsφsθsψ + cφcψsφcθ

Ω = (ω2 + ω4 − ω1 − ω3 )2. Derivation of a Dynamical Modelu 1 = f1 + f2 + f3 + f4
= b ω12 + ω22 + ω32 + ω42 2
u2 = d −ω2 − ω42 + ω12 + ω33
u3 = f1 − f3 = b ω12 − ω32u4 = f2 − f4 = b ω22 − ω42

Following a Lagrangian method, this section describes
the dynamic model of the X4-AUV with the assumptionof balance between buoyancy and gravity. The kinetic
energy formula is
T = Ttrans + Trot(2)

where Ttrans and Trot are the translational kinetic energy
and the rotational kinetic energy, which are given by

III. STABILIZATION CONTROL FOR
X4-AUV1 T(3)ξ̇ M ξ̇
2
1
Trot = η̇ T J η̇(4)2
in which M is the total mass matrix of the body, andJ is the total inertia matrix of the body. From thecharacteristics of added mass, they can be written as
Ttrans =M = diag (m1 , m2 , m3 ) = mb I + Mf

(5)

J = diag (Ix , Iy , Iz ) = Jb + Jf

(6)

The model (11), developed in the previous section,
can be rewritten in a state-space form Ẋ = f (X, U ) byintroducing X = (x1 . . . x12 )T ∈ 12 as state vector of
the system as follows:
x1 = xx2 = ẋ1 = ẋx3 = yx4 = ẋ3 = ẏx5 = zx6 = ẋ5 = ż

Here, mb is a mass of the vehicle, Jb is an inertia matrixof the vehicle and I is a 3 × 3 identity matrix. Letting ρ
denote a density of the fluid and using the formulationof the added mass and inertia [4] under the assumptionof r1 = 5r and r2 = r3 = r where r1 , r2 and r3 are thelengths of the semi axes of the ellipsoidal vehicle, theadded mass matrix Mf and the added inertia matrix Jf
are obtained by

x10 = ẋ9 = θ̇
x11 = ψMf = diag (0.394ρπr3 , 5.96ρπr3 , 5.96ρπr3 )(7)x12 = ẋ11 = ψ̇Jf = diag (0, 24.2648ρπr5, 24.2648ρπr5)(8)©ISAROB 2011(12)

x7 = φx8 = ẋ7 = φ̇x9 = θ

where the inputs U = (u1 . . . u4 )T ∈ 4 .

408

(13)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Translational
subsystem

Angularsubsystem

φ
u2 
u 
 3
u4 

Fig. 2.

φ

u1

θ
θ
ψ
ψ

φ 
θ 
 
ψ 

from the other subsystem. A stabilization of the X4-AUV
angles is considered for Xαd = (xd7 , 0, xd9 , 0, xd11 , 0)T .
Let us consider the Lyapunov function V (Xα ) that is
positive definite around the desired position Xαd :xx

V (Xα ) =yyz

The time derivative of (15), V̇ = (∇V )T fα , in the caseof X4-AUV (Iy = Iz ) is drastically reduced to:zV̇ = x8 (x7 − xd7 ) + x10 (x9 − xd9 ) + x12 (x11 − xd11 )
l
l
1
(16)
+ x8 u2 + x10 u3 + x12 u4
Ix
Iy
Iz

Connections of rotational and translational subsystems

By simply choosing:From (11) and (13) we obtain:
⎞
⎛
x2
1
⎟
⎜
(cos θ cos ψ) m1 u1
⎟
⎜
⎟
⎜
x4
⎟
⎜
1
⎟
⎜
(cos θ sin ψ) m2 u1
⎟
⎜
⎟
⎜
x6
⎟
⎜
1
⎟
⎜
(− sin θ) m3 u1⎟⎜
f (X, U ) = ⎜⎟ (14)x8⎟⎜
Iy −Iz⎟⎜1x10 x12 ( Ix ) + Ix u2
⎟
⎜
⎟
⎜
x10
⎟
⎜
⎟
⎜
Jt
l
x
⎟
⎜ x8 x12 ( IzI−I
)
−
x
Ω
+
u
12
3
Iy
Iy
y
⎟
⎜
⎠
⎝
x12
Ix −IyJtlx8 x10 ( Iz ) + Iz x10 Ω + Iz u4

u2 = −Ix (x7 − xd7 ) − k1 x8
Iy
u3 = − (x9 − xd9 ) − k2 x10
lIz
u4 = − (x11 − xd11 ) − k3 x12l(17)with k1 , k2 and k3 of positive constants, we obtain for(16):V̇ = −

1ll
k1 x28 − k2 x210 − k3 x212
Ix
Iy
Iz

(18)

which is only negative semi-definite. By Lyapunov theorem the simple stability for equilibrium is now ensured.
By Lasalle invariance theorem we can ensure also thatstarting from a level curve of the Lyapunov function
defined in (15) where V (Xα ) is constant, the stateevolution is constrained inside the region bounded by
the level curve [6]. This is very useful when trying
to avoid particular configuration. It is simply necessary
to start with a level curve not containing these points
and apply the previous defined control. We can also
ensure the asymptotic stability by applying the Lasalle
theorem because the maximum invariance set of (angularrotations) subsystem under control (17) contained in theset S = {XαS ∈ 6 : V̇ X S =0 } is restricted only to theαequilibrium point [6].
If V (Xα ) > 0 when (Xα = Xαd ) and V̇ (Xα ) ≤0 (Xα = Xαd ), the system is asymptotically stable. In
the case V̇ (Xα ) ≡ 0, the solution of state equation is
given by only Xα = Xαd .
By the latter consideration we can ensure an asymptotical stability starting from a point in a set around theequilibrium. To ensure the global stability, it is sufficientthat limXα →∞ V (Xα ) = ∞, which is our case.

It is interesting to note that in the dynamics of thelatter system how the angles and their time derivatives
do not depend on the translational components, whereas
the translations only depend on the angle. It can ideallyimagine the overall system described by (14) as constructed by two subsystems: i.e., the angular rotations
and the linear translations (see Fig. 2). The subsystemof the angular rotations has the restriction Xα of X tothe last 6 components as state, which is regarded as theroll, pitch, yaw and their time derivatives. The dynamics
related to these variables are described by fα (X, U )corresponding to the last 6 components of the mapping(14). Observe that the mapping fα (X, U ) is a function
only of Xα and of (u2 , u3 , u4 )T , and does not dependon the translational components. On the other hand, thetranslational subsystem (with state XΔ ) consists of thefirst 6 elements of the state X, which are the x, y, z andtheir time derivatives, where the dynamics are describedby the first 6 rows fΔ (X, U ) of the mapping (14).
The translational subsystem mapping fΔ (X, U ) is notindependent of the angular variables but depends only on
the roll, pitch and yaw and not on their time derivatives.
Note that these discussions are the same as those for a
Quadrotor studied in [6].

2. Translation Controller
Let us consider the simple task for the X4-AUV to
be translated to a particular position x = xd , y = y d
and z = z d . The dynamics of the x, y and z-position
are described by lines 1 and 2, 3 and 4, and 5 and 6 insystem (14), i.e.,

1. Rotation Control
Consider first the control for the subsystem of theangular rotations because of its complete independence©ISAROB 20111
1
1
(x7 − xd7 )2 + x28 + (x9 − xd9 )2
2
2
2
1 2
1
1
d 2
+ x10 + (x11 − x11 ) + x212
(15)
2
2
2

409

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

x-position:


y-position:z-position:

ẋ1
ẋ2

ẋ3
ẋ4






x2
u1
(cos x9 cos x11 ) m
1

=


ẋ5
ẋ6


=

x4
u1
(cos x9 sin x11 ) m
2




=

x6
u1
(− sin x9 ) m
3

IV. DISCUSSIONS

A nonlinear control strategy is implemented to stabilize the X4-AUV. The position and angles of the X4AUV are stabilized by using control input u1 , u2 , u3
and u4 respectively. PD controllers were introduced forcontrolling each orientation angle such that

(19)

(20)

ui+1 = kpi e2i+5 + kdi ė2i+5 ,(21)

m1 û1
cos x9 cos x11(22)

m2 û2
cos x9 sin x11(23)−m3 û3
sin x9(24)

V. CONCLUSION
In this paper, a nonholonomic control method forstabilizing all attitudes and positions (x, y or z) of anunderactuated X4-AUV, with four thrusters and 6-DOF
has been presented. The controller design was separated
into two parts: the rotational and translational dynamicsrelated controller designs. The stabilization strategy isbased on the Lyapunov stability theory. For the future
work, an underactuated controller will be constructed by
combining such three types of controllers to realize anunderactuated control system.

y-position:
u1 =z-position:
u1 =where û1 , û2 and û3 are additional terms. By this partial
feedback linearization [7], (19), (20) and (21) become
x-position:   
ẋ1
x2
=
(25)
ẋ2
û1


or

y-position:




or

z-position:

or

ė1
ė2



ẋ3
ẋ4

ė3
ė4




=

e2
−û1




=



ẋ5
ẋ6

ė5
ė6





x4
û2








=

x6
û3

[1] Kolmanovsky I and McClamroch NH (1995), Developments innonholonomic control problems. IEEE Control Systems Magazine, 15(6): 20–36[2] Nakamura Y and Savant S (1991), Nonholonomic motion control
of an autonomous underwater vehicle. IEEE/RSJ Int. Workshop
Intelligent Robots and Systems, 1254–1259[3] Fossen TI (1994), Guidance and control of ocean vehicles, John
Wiley & Sons Ltd.
[4] Leonardo NE (1997), Stability of a bottom-heavy underwater
vehicle. Automatica, 33(1): 331–346[5] Okamura K (2009), Position and attitude control for an autonomous underwater robot using a manifold theory. Master
Thesis, Saga University.
[6] Bouabdallah S, Murrieri P and Siegwart R (2005), Towards
autonomous indoor micro VTOL. Autonomous Robots, 18(2):171–183[7] Fantoni I, Lozano R and Spong MW (2000), Energy based controlof the pendubot. IEEE Trans. on Automatic Control, 45(4): 725–
729
[8] Zain ZM, Watanabe K, Danjo T, Izumi K and Nagai I (2010),
Stabilization control for an X4-AUV. Proc. of the 3rd International Conference on Underwater System Technology: Theory
and Applications (USYS ’10).
[9] Zain ZM, Watanabe K, Nagai I and Izumi K (2010), The
stabilization control of a position and all attitudes for an X4-AUV.
Proc. of the 5th International Conference on Soft Computing
and Intelligent Systems and 11th International Symposium on
Advanced Intelligent Systems (SCIS & ISIS 2010).


(26)

(27)

e4
−û2

=

REFERENCES



=


(28)



e6
−û3

(29)

(30)

where ei  xdi − xi , i = 1, . . . , 6. Adopting a simple
linear state feedback stabilization law û1 = k4 e1 + k5 e2 ,û2 = k6 e3 + k7 e4 and û3 = k8 e5 + k9 e6 we can stabilize
the position by placing the poles of the subsystem in any
position in the complex left half plane.

©ISAROB 2011(31)

where kp1 ≡ Ix , kp2 ≡ Iy /l, kp3 = Ix /l, kd1 = k1 ,kd2 = k2 , and kd3 = k3 .
The angles and their time derivatives of rotational
subsystem do not depend on translation components,
whereas the translations depend on the angles. Ideally,
it can be imagined as two subsystems: the angularrotations and the linear translations. Due to its complete
independence from the other subsystem, the angularrotation-related subsystem is tuned first. The rotational
control keeps a 3D orientation of the X4-AUV to thedesired state and the translational control moves thevehicle to the desired position. The controllers have been
implemented on MATLAB and the simulation results forstabilizing the X4-AUV can be found in [8] and [9].



By the previous considerations in the control for thesubsystem of the angular rotations, we ensure that starting from an initial condition where V (Xα ) < π/2,
the angles and their velocities are constrained in this
hypersphere of 6 . In this case cos x9 cos x11 = 0,
cos x9 sin x11 = 0 and − sin x9 = 0 for all the trajectories of the system under the previous control law.
Systems (19), (20) and (21) can be linearized by simply
compensating the weighted force by
x-position:u1 =i = 1, 2, 3

410

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

The inevitability of the bio-molecules:
five nitrogenous bases and twenty amino acids
K.Hashimoto, H.Inoue and K.Naitoh
Waseda University, Faculty of Science and Engineering, 3-4-1 Ookubo, Shinjuku, Tokyo, 169-8555 Japan(Tel : 81-3-5286-3265)(Email address:k-naito@waseda.jp)

Abstract: Living beings meta-stable between unstable and neutral-stable conditions use only five types of nitrogenous
bases and twenty amino acids selected naturally. The unified momentum theory derived at the triple point of thedeterministic Boltzmann equation, the stochastic Langevin equation, and the indeterminacy principle [K. naitoh,
Artificial Life Robotics, 2010] reveals the reason why the molecular weights of the twenty types of amino acids show athreefold variation between 240 of cysteine as the maximum and 75 of glycine as the minimum, whereas that of purinesand pyrimidines among nitrogenous bases varies by only about 1.5 times, although the variation principle for theenergy conservation law cannot do so. Let us take a higher order of the Taylor series for the unified momentum
equation describing the deformation motions of biological molecules. Even-numbered terms such as the second andfourth ones show no other quasi-stable size ratios. However, odd-numbered terms result in other quasi-stable ratios.
The threefold variation of amino acids will come from the third term of the Taylor series.
Keywords: Asymmetry, Cyto-fluid dynamics

I. INTRODUCTION
II. CYTO-FLUID DYNAMICS THE
ORYBASED ON the γ-ε EQUATION

Many types of amino acids and bases are possible in
the artificial pools of molecules, but living beings metastable between unstable and neutral-stable conditions
use only five types of nitrogenous bases and twenty
amino acids selected naturally. [1-6]Life can exist only when fully surrounded by watermolecules. Thus, the meta-stability of biologicalmolecules such as nitrogenous bases, nucleic acids,
amino acids, and proteins in living beings should beanalyzed by considering the flows of water molecules.
Our previous model based on traditional fluid
dynamics [7-9] has qualitatively revealed the reason
why living beings employ only five nitrogenous bases,
i.e., two purines of A and G and three pyrimidines of T,
C, and U, and also why purines and pyrimidines have asize ratio of about 1.5. However, the model was withsome gratuitous assumptions and also analyzed by the
first order of accuracy, which led only to qualitative
results. Consequently, a qualitative analysis did notreveal the reason why the molecular weights of thetwenty types of amino acids show a threefold variation
between 240 of cysteine as the maximum and 75 ofglycine as the minimum and also why proteins are morevarious.
In this report, a unified momentum theory is extended
from our previous model as a result of statisticalmechanics [9] and the theory of adding mass [10]. A
higher order of analysis based on the momentum
conservation law reveals the size varieties of the fivebases and twenty amino acids. The present report
reveals several mysteries underlying the bio-molecularsystems.

©ISAROB 2011

First, we define a parcel as a flexible spheroid having
two long and short radii of a (t ) and b(t ) dependent
on time t and constructed of a bio-molecule such as abase or an amino acid that is surrounded by watermolecules. (A parcel consists of a bio-molecule such as
nitrogenous base or amino acid, water molecules
hydrated with the bio-molecule, and the immerse massdue to adding mass effect while impulsive start and stopof deformations occur [8, 9].)The parcel size is proportional to that of biologicalmolecules such as nitrogenous base or amino acid.
The parcel becomes a sphere of the radiusrd (= [ab 2 ]1 / 3 ) under an equilibrium condition. The
deformation rate γ (t ) is defined as a (t ) / b(t ) , whilea sphere without deformation corresponds to γ = 1 .
We assume that the flow field inside the parcel is thepotential flow.
We derive a theory for describing the deformation andmotions of two connected spheroid parcels having two
equilibrium radii of rd 1 and rd 2 and two deformation

rates of γ k [ k = 1, 2] , while the size ratio of the twoparcels is defined by ε = rd 1 / rd 2 . We model the one-dimensional relative motion between the two parcels,
nonlinear convections inside the parcels, the interfacial
force at the parcel surface due to forces interacting
between molecules, and collisions with water molecules

411

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

outside the parcels. The interfacial force is evaluated in

the form of σ / r where m and σ are constants and r
is the curvature of parcel surface. [7] Several types offorces such as van der Waals force, coulomb force, andsurface tension can be explained by varying m. Here,the relation m = 1 implies the surface tension of liquid.
The mean density of the parcels is ρ L .
Moreover, the scale for averaging, i.e., the minimum
scale representing the phenomenon, will be smaller thanthat in continuum mechanics and will be between theatomic scale and the size of the bio-molecule. Thus, thissmall averaging window applied to the Boltzmann
equation [9] leads to a weak indeterminacy of physicalquantities such as deformation and density because ofmolecular discontinuity.
Next, we derive the momentum equation describing
the relation between a dimensionless deformation rate
γ k (≡ ak / bk [k = 1, 2]) of each parcel dependent onmdimensionless time t k =Fig.1. Two parcels connecting

III. FIRST ORDER OF ANALYSIS [6]8σt [k = 1, 2] andρ L rdk 2+ m

We then define the deviation from a sphere as y i ,which is equal to γ i − 1 . Taking the first order ofthe size ratio of the two parcels of ε = rd 1 / rd 2 . The

approximation in the Taylor

stochastic governing equation having indeterminacy canbe described as

− 4ε

d
2
2
−1 / 3−4 / 3
γ j )2+ [ ε 2 + m E0iγ j B0 j − ε 2 + m E0iγ j ](
3
9
dt j
5 2
5 2
2
2
−1 / 3− m−1 / 3− m+ (−ε − ε + εE0 jγ j )C0i γ i 3 3 + ε 2 + m E0iγ j C0 j γ j 3 3 } / Det33+ δ st[for i = 1, 2. j = 1, 2. i ≠ j ]asymmetric ratio of 3 3 around 1.5 ( ε =3) makes thesecond term zero. The size ratios of 1.00 andapproximately 1.50 can be described by the unified
number of the n-th root of n.
We define a system as being quasi-stable when only
one term on the right-hand side of the differential
equation system governing the phenomenon (Eq. (2)) iszero. Life is relatively quasi-stable because d2yi/dt2
becomes smaller when the size ratio of connected
3

2

2
2
1 γk −2−1 / 3−1 / 3
Det = −ε − ε 4 + ε 4 E0iγ i + ε E0 j γ j , B0 k =,3
3
3γ k γ k 2 − 1 / 2
7/3γ
3 2γ k − 1 / γ k − γ k, and E0 k = 3 2 k[for k = 1, 2]
8
γ k 2 −1/ 2
γ k −1/ 2

parcels takes the values of ε =1 or ε =3. An importantclue for clarifying the inevitability and variety of
biological molecules is the concept of quasi-stability(meta-stability) weaker than neutral stability. This isbecause biological systems are essentially neither
neutrally stable nor absolutely stable owing to thepotential for a brush with death and also since people
can live over 50 years while maintaining the mysterious
shape of the body. Thus, a new stability concept
between the neutrally stable and unstable conditions isnecessary for a meta-stable system of life, i.e., quasistability. [6-9]Free nitrogenous bases in water often seem to be a3

where the parameter δ st denotes random fluctuation.
The long derivation of Eq. (1) is in Ref. 6 confirmedby the referees, although only the stochastic term δ st is
not in Ref. 6.

©ISAROB 2011

(2)
'

(1)

m

 dy j  + 12ε 1+ m m y j ] /[3(ε 3 + 1)] dt 
jwhere the parameter δ st denotes random fluctuation.
A symmetric ratio of 1.0 ( ε =1) makes the first term on
the right-hand side of the equation zero, while an

with

m

1+ m

+ δ ' st ,

4

2m

2

2

d2
d
2
2
γ = {[(−ε − ε 4 + εE0 jγ j −1 / 3 ) B0i + ε 2 + m E0iεγ i − 4 / 3 ](γ i )2
2 i
9
3
dti
dti

C0 k =

series leads to dy 
2d yi= [− (3 − ε 3 − 2ε 2 + m ) i  + 3(3 − ε 3 )m yi
23
dt i
 dt i 
2

412

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

column of two identical bases such as adenine-adenine,
which are not in a two-dimensional plane. There are
multimeric complexes such as that of guanines. Then, it
is well known that identical base pairs of ε =1 are often
in RNA, while asymmetric base pairs such as theas the third order of approximation.
2
3
The three terms of a31 y1 , a32 y1 , a36 y1 come from thesurface tension, whereas the later terms of b31 , b32 y1 , b35 y12come from convection.
Here, we define

Watson-Crick type of about ε =3 are used in DNA.
(Fig. 1)As Eqs. (1) and (2) show a slightly vague solution forthe phenomenon, this indeterminacy also implies thatvariations of molecular sizes are possible in a limited
range. This indeterminacy will permit the possibility of3

a31 = 0 or b31 = 0

as the quasistable condition, while
a32 = 0, a36 = 0, b32 = 0, or b35 = 0

sizes around 1: 3 3 , i.e., two types of base pairs suchas A-T and G-C.

is semi-quasistable.

IV. HIGHER-ORDER OF ANALYSIS

Table 1 shows the size ratios of parcels, which bringquasistable and semi-quasistable conditions.
The semi-quasistable ratio of about 1: 3.6 willcorrespond to a threefold variation between 240 of
cysteine as the maximum and 75 of glycine as theminimum, while the other semi-quasistable ones ofabout 1: 1.27 and 1:1.35 lead to some variations of basepairs such as A-T and G-C.

Higher order of the Taylor seriesf (1) (0 )f (2 ) (0 ) 2 f (3) (0 ) 3y+y +y + .......
1!
2!
3!

f ( y ) = f (0 ) +(3)Can be applied to Eq. (1).
The third order of the Taylor series brings




2
2
2
3
2
3
a31 y1 + a32 y1 + a33 y1 y2 + a34 y1 y2 + a35 y1 y2 + a36 y1 + a37 y2 + a38 y2 + a39 y2 2d 2 y1 1 
2
2  dy1 =
+ b31 + b32 y1 + b33 y2 + b34 y1 y2 + b35 y1 + b36 y2 2dτ 1 m2  dτ 1 



2




2
2 dy 2+ c31 + c32 y1 + c33 y1 y2 + c34 y1 + c35 y2 + c36 y2  dτ 
 2



(

)

(

)

Table 1.Size ratios of parcels (The third order ofapproximation type)surfaceconvecti
Stability
tension termve term(4)

Quasistable
Semiquasistable
Semiquasistable

with

(

)

)( ) a = 34 (ε − 3)(ε + 34ε + 33)
a = 12ε (3ε − 8ε − 2ε − 8ε − 5) a = −8ε (2ε + 7ε + 6ε − 57ε − 28)
a = ε (− 111ε + 336ε + 426ε + 80ε + 25)
1
a = (ε − 3)(ε − 42ε − 171) a = 36ε (ε + 1)
2
a = −3ε (37ε + 42ε + 5) a = ε (383ε + 134ε + 7 ) b = 6(ε − 1)(ε + 1)
32
b = −2(5ε + 5ε − 21ε − 21)b = 32(ε + ε )b = − ε (5ε + 21)
3
m3 = 9 ε 3 + 1
2

33

a31 = −9 ε 3 − 3 ε 2 + 1

6

2

35

(

3

4

3

36

32

(

)

c33 =

2

32 2 6ε 6ε − 15ε 3 − 53()

c35 = 32ε 2 2ε 6 + 3ε 3 + 1

4

3

3

)

)

3

6

3

31

3

3

34

(

16
b36 = − ε 3 21ε 3 + 5
3

(

(

3

3:2
3.6:1

1.35:1

VII. CONCLUSION)

Higher-order of analysis for the cyto-fluid dynamic
theory reveals the size ratios of 1:1 and about 2:3 fornitrogenous bases and also the ratios over 2:3 for aminoacids..

))

We can consider only the disturbance for parcel1,
because the system of two parcels is mathematicallysymmetric. Then, we get2
d 2 y1 1 2
3
2  dy1  


=
+
+
+
+
+
a
y
a
y
a
y
b
b
y
b
y
 31 1 32 1 36 1

31
32 1
35 1 2dτ 1 m2  dτ 1   (5)©ISAROB 2011

1.27:1

2

V. QUASI-STABILITY EXTENDED

(

3:2

3

)

(

1:1

2

3

4 2
ε − 9ε 6 + 6ε 3 + 15
316
c34 = ε 2 ε 6 + 3ε 3 − 14316
c36 = − ε 2 47ε 6 + 41ε 3 + 10
3

c32 =

(

6

6

6

33

8
b35 = 5ε 9 + 9ε 6 − 27ε 3 − 63
3

(

2

39

3

c31 = −12 ε 3 + 1

2

37

3

6

2

3

3

3

6

9

32

34

4

6

2

38

3

6

2

3:2

)

413

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

REFERENCES[1]Miller S.L, (1953) A production of amino acids underpossible primitive earth conditions. Science, 117: 528529.
[2]Oro J and Kimball A.P, (1961) Synthesis of Purines
Under Possible Primitive Earth Conditions. I. Adenine
from Hydrogen Cyanide, Arch. Biochem. Biophys. 94:
217−227.
[3]Yanagawa H, (1991) How Life Originated. TBS
Britannica. 1-466.
[4]Ricardo A and Szostak J.W, (2009) Origin of Lifeon Earth. Scientific American.
[5]Watson J.D, Hopkins N.H, Roberts J.W, etal. (1987) Molecular Biology of the Gene, Fourth
edition: The Benjamin/Cummings Publishing
Company.
[6]Naitoh K, (2001) Cyto-fluid Dynamic Theory, Japan
Journal of Industrial and Applied Mathematics, 18-1:
75-105.
[7]Naitoh K, (2003) Macroscopic kinetic model on theorigins of biological information. Trans. of JIAM:
115-137.
[8]NaitohK,(2008)
Stochastic Determinism
underlying Life, Artificial Life and Robotics, Springer,
13: 10-17.
[9]Naitoh K, (2010) Onto-biology: clarifying the
spatiotemporal structure, Artificial Life and Robotics,
Springer, 15: 117-127.
[10]Lamb H, (1945) Hydrodynamics, 6th ed. New York:
Dover.

©ISAROB 2011414

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Engine for brain development: Similarity between engine and brain
H. Kawanobe and K. Naitoh
Waseda University, Faculty of Science and Engineering, 3-4-1 Ookubo, Shinjuku, Tokyo, 169-8555 Japan(Tel : 81-3-5286-3265)(Email address:k-naito@waseda.jp)

Abstract: Observations of the development of the human brain show the bones of the skull become increasingly larger
over the neck, and a lot of soup like fluid for generating brain cells enters the skull from the body. This process isessentially similar to the intake process of an internal combustion engine and also that the soup for generating flexible
brain cells can be essentially modeled by using the Navier-Stokes equation. The flow of air during the intake process ofa piston engine and fluid flow in the brain, including water as the main component, can be approximated as being
incompressible. In this report, we will examine the spatial patterns of convexoconcave of brain and blood vessels in
details.
Keywords: Brain shape, Blood vessels, Morphogenesis, Engine, Fluid dynamics

because the human neck looks like the intake port thatfunctions as the throat of an engine (Fig.1).
The soup for generating flexible brain cells can beessentially modeled by using the fluid dynamics forflow in an engine. The flow of air during the intake
process of a piston engine and fluid flow in the brain,including water as the main component, can beapproximated as being incompressible. (The intake
process of an engine at low engine speeds isincompressible, although the compression process isstrongly compressible with time using the zero-Machnumber regime.) [2,3]I. CEREBRAL DEVELOPMENT [1]Observations of the development of the human brain
with a high-speed camera reveal that the bones of theskull become increasingly larger over the neck, and a lotof soup-like fluid for generating flexible cells, including
nerves and synapses, enters the skull from the body.
(a) Brain volume increasing

II. NAVIER-STOKES EQUATION [1]
We employ the incompressible Navier-Stokes
equation [ 4 ] as the basic governing equation that
describes the flexible dynamic motion of a lot of souplike fluid for generating brain cells. [1-3] The equation
can be written as(b) Volume increasing during the intake process of apiston engine

∂u i
+
∂t

∑

uj

j

∂u i
1 ∂p
=−
+ν
ρ ∂x i
∂x j

i

，，，

j

2
j

where ui ρ p and ν denote the fluid velocity,
fluid density, pressure, and kinetic viscosity coefficient,
respectively, while xi i=1,2,3 denotes Cartesian
coordinates in three dimensional space. Equation 1 iscomputed by the finite difference method using a higher
order of accuracy. [ 1 , 2 , 3 ] The details of thecomputational method are given in the references.

Fig.1 Topological similarity of the brain and anengine[1]This process is essentially similar to the intake
process of an internal combustion engine, because thevolume of an engine cylinder, which increases
according to the descent of the piston, corresponds
geometrically to the development of the skull, and also

©ISAROB 2011

∑ ∂x

（１）

∂u i

∑ ∂x = 0
i

∂ 2ui

415

（

）

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

field. The convexoconcave shapes inside the brain in
Fig.3a are similar to those in Fig.3b.

III. COMPUTATIONAL RESULTS
Figure 2a shows an image of the human brain
obtained with magnetic resonance imaging (MRI) at ahorizontal cross section including the two eyes, whileFig.2b is the result of the corresponding computation
performed with the Navier-Stokes equation. Red lines inFig. 2 show that the convexoconcave shapes inside thebrain are similar to the flow structure in engine.
(a)Figure 4a shows an image of the human brain
obtained with magnetic resonance imaging (MRI) at thecross section different from those of Figs. 2 and 3, while
Fig.4b shows the vorticity surface computed in theengine. In this figure we can see two spheres
representing the eyeballs and a cylinder representing the
nose.

(b)

(a)

Fig.2 Comparison of brain MRI and engine flow
(j-k cross section)(a) Magnetic resonance imaging (MRI) of brain [5](b) Computational result at an engine speed of 1400 rpm
(a)

(b)

Fig.4 Comparison of brain MRI and vorticity in engine( i-j cross section)(a) Magnetic resonance imaging (MRI) of brain [5](b) Computational result at an engine speed of 1400 rpm(b)
Figure 5 shows the instantaneous stream lines
computed at a very low engine speed of 1.4 r.p.m. A
comparison with Fig.2 indicates that the flow pattern is
independent of the engine speed.

Fig.3 Comparison of brain MRI and engine flow
(i-k cross section)(a) Magnetic resonance imaging (MRI) of brain [5](b) Computational result at an engine speed of 1400 rpm
Figure 3a shows an image of the human brain
obtained with magnetic resonance imaging (MRI) at a
center cross section, while Fig.3b is the result of thecorresponding computation performed with the NavierStokes equation for engine. Figure 3b is inclined inorder to compare with the MRI image and engine flow

©ISAROB 2011

Fig.5 Instantaneous stream lines at a low engine speedof 1.4 r.p.m

416

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Figure 7 shows the blood vessels in brain and thecorresponding path lines in engine observed from threedifferent directions. Each blood vessel can be also
observed in the path lines for in engine at close
positions, while the direction of flows are correct bothfor blood vessels and path lines in engine.

Those computations were made at a crank angleof 72°after top dead center (ATDC) in the intake
process, because at that point the cylinder volumeof the piston engine considered here is about thesame as that of the skull. (Fig.3)
Figure 6 shows the categorization of blood vessels
in brain. In this figure, red line indicates external carotid
artery, green line indicates middle cerebral artery, blueline indicates internal carotid artery, and yellow line
indicates vertebral artery. We’ll examine these fourmain blood vessels.

External carotid artery

IV. CONCLUSION
In this paper, we can see the similarity between theconvexoconcave forms inside the human brain and theflow structure of engine stream lines. In addition, thevorticity surface represents the eyeballs and nose that
corresponds to those in the MRI imaging. Finally, we
can conclude there is a similarity between blood vessels
in human brain and engine path lines.

Vertebral arteryAcknowledgments The author sincerely thanks
Mr.Keiji Matsuda of the Neuroscience ResearchInstitute, AIST, Tsukuba, Japan, for permission to re-use
the MRI images of the brain. Sincere thanks are also
due Otsuka Phamaceutical for permission to re-use three
dimensional images of blood vessels in brain. And thisarticle is part of the outcome of research performed
under Waseda University Grant for special research
project (2010B-155).

Internal carotid artery
Middle cerebral artery

Fig.6 Categorization of blood vessels in brain [6](a) Blood vessels in brain [6](b) Computational result at an engine speed of 1.4r.p.m
Fig.7 Comparison of blood vessels in brain and engine path lines observed from the i direction©ISAROB 2011

417

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

REFERENCES[1] Naitoh K (2008) Engine for cerebral development.
Vol. 13, pp.18-21. J. of Artificial Life and Robotics.
[2] Naitoh K , Kaneko Y, Iwata K (2004) Cycleresolved large eddy simulation of an actual 4-valve
engine based on a quasi-gridless approach.SAE paper
2004-01-3006[3] Naitoh K , Kuwahara K (1992) Large eddy
simulation and direct simulation of compressible
turbulence and combusting flows in engines based on
the BI-SCALES method. Fluid Dyn Res 10:299-325[4] Tatsumi T (1998) Fluid dynamics (in Japanese).
Baifukan[5] http://riodb.ibase.aist.go.jp/brain/welcome.html (alsoin Matsuda K, Oishi T, Higo N (2007) S128, P1-k22
Web-based MRI brain image database system. Neurosci
Res 58(suppl 1) (ISSN0618-0102))[6]
http://www.otsuka-elibrary.jp/library/support/101/252©ISAROB 2011

418

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Spatial surface wave spread network from Ambient Noise
correlation
Z. Zheng, T. Fujiwara, N. Sakurai, K. Yoshizawa and K. Yamasaki
Tokyo University of Information Science, 265-8501 Japan(TEL:81-43-236-4652; FAX:81-43-236-4652)(zhengzy@edu.tuis.ac.jp)

Abstract: We computed the correlations of the ambient noise correlation of 66 stations of F-NET(Broadband Seismic Network JAPAN) that were apart from tens to hundreds kilometers. By stacking
cross-correlation functions of ambient noise, we can extract the coherent part of each two stations.
Through quantitative analysis of the magnitude of these extracted pluses of ambient noise correlationfrom any two stations, we construct a network that may reflect the main surface wave spread characters inspatial field.
Keywords: seismic noise, complex network, tomography, correlation function.

I.

INTRODUCTIONFirst, the “one-bit” method (3) was used to remove
earthquakes and other contaminants from seismic

Campillo and paul Campillo [1] found in 2003 that the

waveform data. We divide the wave of each day of a

stacking cross-correlations function of codas is similar

station into 1440-s-long segments with overlap of 300 s.

the Green function tensor. Then many researches about
the cross-correlations function of seismic waves andnetwork method were carried Bensen et al [3], Shapiro etal [4], Zheng et al [5], Shiraishi et al [6] and Weaver[7].
Recently, some research indicated that these ambient
noise correlations may concern with big earthquakes
Xu[2]. In our research we analyzed the big area in space
and time scale, found that the cross-correlations based on
ambient noise may effect by big earthquakes. The big
earthquakes have a effect that destroy the stability ofcross-correlation in a wide area.

II. DATA AND METHOD
Fig. 1. The F-NET stations that was used in thisresearch. (66 stations)

We used the 2003 66 stations of F-net data in 1HZ.

©ISAROB 2011

419

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011precisely when the big earthquake occurred. It

Then we computed the cross-correlations between
thesetruncated

signals,

next

each

indicate one possibility for the earthquake forecast.

pairs’

cross-correlations function were stacked to give thesignals. The samples were show in Fig. 2. When SNR >30, we may define a link between this two stations. All
links construct a network as Fig. 3, her we call thisnetworks is “seismic ambient noise network”.

III. RESULTS and DISCUSSION

From Fig. 2 we can found that the correlation function
peaks are changed in different day. Tough we don’t know
the reason of these dynamical changes, in our research
we could show that the big earthquake may give someinfluence to these links. In order to quantitatively show
this influence, we sum the links number of “seismic
ambient noise network” of each day in 2003, the links
numbers graph was show in Fig.4 , from this graph we
can found that when each big earthquake (except
aftershock) is occurred the links number will go down
sharply. The second earthquake in September is anFig. 2. The sample correlation functions of FUK

aftershock.

and NAA stations. Each one indicate one days
correlation, this graph shows 3 continues days

VI. CONCLUSION

correlations. The blue vertical lines show theIn our research, we show the Ambient Noise

predict maximum and the minimum group arrival

correlations in the big spatial scale as a network. The

times.

network show a very dynamics characters. We checked
the total links number in each day of the network.
The result shows that the number would down

©ISAROB 2011

420

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig. 3. The example network maps of 6 days in January 2003. The links are changed dynamics.

©ISAROB 2011421

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Ambient Seismic Noise. Science 307: 1615-1618[5] Zheng S, Sun X, Song X et al. (2008), Surface wave
tomography of China from ambient seismic noisecorrelation. Geochemistry Geophysics Geosystems 9[6] Shiraishi K, Matsuoka T, and Kawanaka T (2008),
Review of Seismic Interferometry. Jour. of Geography
117:863-869[7] Weaver R (2005), Information from Seismic Noise.
Science 307:1568-1567

Fig. 4. The total links numbers in each day of 2003.
The red vertical lines indicate the main earthquakes
bigger than magnitude 6.5 in Japan.

Acknowledgment
Thanks for the support of Grant-in-Aid of Research
project for the sustainable development of economic andsocial structure dependent on the environment in eastern
Asia.

REFERENCES[1] Campillo M and Paul A (2003), Long-Range
Correlations in the Diffuse Seismic Coda. Science 299:
547-549[2] Xu Z and Song X (2009), Temporal changes of
surface wave velocity associated with major Sumatra
earthquakes from ambient noise correlation. PNAS 106:
14207-14212[3] Bensen G, Ritzwoller M, Barmin M, et al. (2007),
Processing seismic ambient noise data to obtain reliablebroad-band surface wave dispersion measurements.
Geophys. J. Int. 169:1239-1260[4] Shapiro N, Campillo M, Stehly L et al. (2005),
High-Resolution©ISAROB 2011

Surface-Wave

Tomography

from

422

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Effect of Hot Summer against Environment which was induced by Extra
Economical Demand via Japanese Ecological Footprint
K. Yamasaki, T. Fujiwara, N. Sakurai, K. Yoshizawa, Z. Zheng
Tokyo University of Information Sciences, 4-1 Onaridai, Wakaba-ku, Chiba,Japan(Tel&Fax : 81-43-236-4653)(yamasaki@rsch.tuis.ac.jp)

Abstract: The Ecological Footprint (EF) indictor which had been presented by Wackernagel and Rees in 1990's has
attracted larger attention as a numerical assessment value for sustainable development on Earth. In this paper, weanalyze the effect of hot summer against environment which was induced by extra demand via Japanese EF based on
47 prefecture's Input-Output table. The electricity sector and the gas & heat supply sector are affected by hightemperature within 48 industry sectors in this Input-Output table. The extra demand of the electricity sector increases inthe EF through the increase of CO2 emission in the same prefecture and the same sector. The effect of the extra
demand loss of the gas & heat supply sector is not strong but is affected opposite direction. The extra demand(demandloss) does not affect other prefectures and other sectors so much.
Keywords: Ecological Footprint, Input-Output table, Induced analysis,

globe.
EF is calculated for an area on the basis of consumption
in the area. That is EF of the imported product does notbelong to the nation where the product is produced but
belongs to the nation where the product is consumed.

I. INTRODUCTION
We have to pay some money to buy food , clothes andelectricity etc. for daily life. Although, the air forbreathing is free and we live without being aware of theenvironmental services such as "regulation of CO2/O2
balance", "greenhouse gas regulation" and "flood
control". R. Costanza and his team calculated the main
17 ecosystem services we received in 1997. These
values (most of which is outside the market) is
estimated US$33 trillion per year and it's comparable toglobal world economy of US$38 trillion per year onpurchasing price basis. For sustainable life market priceis not enough as the metric and we need more effective
numerical value. One of the candidate is EF. The
essential part is to convert nation's basic consumption to
the nation's virtual area which is necessary to produce it.
For example, we consume agricultural products to keep
our life, and the consumption is converted to croplandarea, and finally our total consumption is compared tobio capacity of the globe.

EF  

yN ,

 l ( ) N l , ( ) i  ij .... k C

(1)Here C is the weight of the consumed product  ofwhich we want to know the effect,  ij is the weight ofthe product i to produce unit weight of product j, l ( ) is the equivalence factor of land type l ( ) for the
product α *,  N ,l ( ) is yield factor of nation N and land
type l ( ) , yN , is the average weight of the productfrom unit area in the nation N. And the sums are
executed in all paths from product  to product αthrough intermediate products i, j... Using the matrix-1｝= １－  , andrepresentation, ｛１＋＋＋＋EF = λ[1- χ]-1 C

II. INDUCED ANALYSIS OF EF(2)here λ is diagonal matrix of  l ( ) N ,l ( ) / yN , .
But usually it is difficult to know  ij of all products'
pairs [i, j] without double counting. Sometimes we wantto measure the effect of consumption toward othersectors or toward other regions, not only on economic
effect but also on environmental effect. Input-Output
table enables to calculate such influences by induced
analysis.

EF is represented by total area of six land types,"Cropland", "Forest", "Grazing land", "Fishing ground" ,"Carbon uptake land" and "Built-up land". Each area iscalculated to reproduce our final consumption asnecessity area for "the supplies of crop", "the supplies
of wood", "the supplies of livestock feed" , "the supplies
of fish", "absorption of CO2 emission" and "house,factory, road etc.". And it is compared to the real area
(bio capacity) on the globe. Actually Japan needs 2.5
times area of Japan to keep our lifestyle, and wholeworld people need more area than whole area of the©ISAROB 2011

1

*The equivalence factors and the yield factors are defined by GFN
and represent the differences of productivity between different land
types and between different nations.

423

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

EF = λ[1- χ]-1 C  λ f u eClmultiplied by CO2 emission unit requirement(tCO2/million yen).

(3)

In eq.(3), C is a demand vector of sectors expressedby amount of money, χ is the coefficient matrix and[1- χ]-1 is the Leontief Inverse Matrix of the InputOutput table. λ is a kind of induced value added
matrix in Input-Output table analysis.
Additional term λ fuel C represents CO2 emission fromthe direct fuel use of final consumption.
For example, in case of agriculture sector, yN , is thetotal amount of agricultural product divided by the total
area of the Cropland of the considering region N.
 l ( ) is the equivalence factor of Cropland,  N ,l ( ) isyield factor of the region N and Cropland **.

3.

There is another method to calculate the matrix λ .
GFN calculates nation's EFl of each land type l every
year and these EFl s represent the area after considering
the productivity. If EFl s were distributed to each sectorα(They are called direct EFs), λ is able to replace bythem. The merit of this method is induced EFs areable to compare to other nations' EFs through EFs
calculated by GFN.

We use "Monthly Family Budget Survey"
calculated by Japanese government[4] to get theextra consumption for the hot summer. We
calculate the correlation between temperature andconsumption of each 48 sector in July in main
cities in Japan. We found that the electricity
consumption of 11 years has positive correlation to
the temperatures, and that the gas & heat supply
consumption has negative correlation.

4.

We calculate induced demand using eq.(4) andcalculate induced EF using eq.(5). The elements of
C are the extra consumption of the electricity
sector and the extra loss consumption of the gas &heat supply sector. We calculate the data of the hotsummer in July/2010. We also calculate the data of
the cold summer July/2003 for comparison. When
we use the competitive import type Input-Output
table, eq.(3) must be deformed to eq .(5) for
separating imported products from domestic
products.

Table 2. Land types and corresponding sectors
1 EF of Cropland
Agriculture sectors
2 EF of Forest
Forestry sectors
3 EF of Grazing land
Agriculture sectors
4 EF of Fishing groundFishery sectors
5 EF of Carbon uptake land All sectors
Final consumption
6 EF of Built-up land
All sectors

III. THE EFFECT OF HOT SUMMER
1.

We use "Multi-regional Input-Output table for 47
Prefectures and 48 industry sectors in Japan" in2000 [1]. First we transform this Chenery-Moses
type Input-Output tables of 47 prefectures to a
Isard type table[2], which includes 2256x2256
matrix.
Table 1. a part of the Isard type Input-Output table.

2.

Next we allocate the EF calculated by GFN toeach sector of each prefecture. EFs are allocated inproportion to the total of each sector according to
the Table 2. We divide EF of Carbon uptake land
into two parts in proportion to the total, one is forintermediate industry and the other is for final
consumption. Using 3EID table[3], we allocate
Carbon uptake land EF of intermediate industry inproportion to the total (million yen) of each sector©ISAROB 2011

Figure.1.Correlations (consumption and temperature)**If the region is not a nation, we use  N ,l ( ) of the nation's factor
instead.
424

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

W = [1-[1- M]χ]-1[1- M]C(4)

Acknowledgements

EF = λ[1-[1- M]χ]-1[1- M]C  λ fuel C(5)

The study was supported by research project fora sustainable development of economic andsocial structure dependent on the environmentof the eastern coast of Asia. We thank for not
only financial support but also helpful discussionof the project members.

Here M is import ratio diagonal matrix whose
elements indicate the ratio of imported products intotal products.
5.

The last term of eq.(5) represents CO2 emission
from the direct fuel use of final consumption.
Using 3EID table, we calculate the remaining CO2
emission ratios (t-CO2/million yen) in energy
sectors and Carbon uptake land EF of final
consumption is allocated in proportion to the total
multiplied by these ratios.

REFERENCES[1] Hitomi K. & B. Pongsun(2008), CRIEPI Research
Report, No Y07035[2] Okamoto N.(2002), Asian Input-output Series,
JETRO, No. 61, pp.1-20. (in Japanese)[3] 3EID:http://wwwcger.nies.go.jp/publication/D031/jpn/index_j.htm[4] http://www.estat.go.jp/SG1/estat/OtherList.do?bid=000000330002&
cycode=1[5] Simmons C., et al.( 2006), WWF One Planet
Business Methodology Report V12.
http://old.bestfootforward.com/OPB.html[6] Iha K., Y. Wada (2009), Environmentally Extended
Input-Output Analysis (EFIO) for Ecological Footprint
and its Application: Japan as a Case Study.
[7] Yoshizawa K., T. Fujiwara, Z. Zheng , R. Shiba, N.
Sakurai, K. Yamasaki(2010), Network Analysis ofJapanese Ecological Footprint based on Input-Output
Table for the 47 Prefectures in Japan, Proceedings of
FOOT PRINT FORUN 2010[8] Fujiwara T., K. Yoshizawa, N. Sakurai, K. Yamasaki(2010), The Analysis of Ecological Footprint using
Multi-regional Input Output Table for 47 Prefectures inJapan, Proceedings of 38th Annual Meeting ofEnvironmental Systems Research 2010, pp.239-243. (inJapanese)

IV. RESULT
Figure.2 shows the effect from extra demand ofelectricity/gas respectively in 47 prefectures in 2010.
Three lines show extra household consumption, induced
production and the extra use of the "Carbon uptake
land". In July 2010, The hot summer already began in
the east part of Japan, but the heavy rainy season still
stayed in the west part of Japan. So high demand ofelectricity is clear in east part of Japan. The highdemand does not affect other prefectures' production somuch except Fukushima where there is nuclear plant ofTokyo Electric Power Company. The effect from gas
consumption is not so clear. Figure.3 shows the effectfrom extra demand of electricity/gas respectively in 48
sectors. The electricity consumption induces the
production of Coke & Petroleum, Finance & insurance
and Other business services. But it looks as if
production of the electricity does not induce theincrease of CO2 emission in other sectors, because theproduction of the electricity produces carbon dioxide inhigher ratio per money than other sectors.
Figure. 4 (Figure. 5) shows the effect from extra(loss)
demand of electricity/gas in 47 prefectures(in 48
industry sectors) respectively in cold summer in 2003.
The results are almost opposite to the hot summer in2010.

V. DISCUSSION
The extra demand(demand loss) does not affect otherprefectures and other sectors so much. It comes fromcharacteristic of these industries. But the true effect of
the hot summer is thought to have spread widely invarious industry sectors. Input-output table of 47
prefecture includes only 48 sectors and there is no
Input-output table with much sectors in all prefectureslevel.
we have to give up the analysis of theprefectures level and use Input-output table with more
sectors to improve the study.

©ISAROB 2011

425

*
+
*
+

*
+

©ISAROB 2011
*
+
*
+

+*

**
* * * * + +* ++* * +++++ ++* +* +*+* *household consumption
induced production
Carbon uptake land
+*
*
+ +
+
*
+++ *
+* +* +* *++* * *+++** * *+*+* +*household consumption
induced production
Carbon uptake land

+*
+*

+ +
+* ++
* * +++* * + +* ++***** * * * +* * ++* * ++++*+*

7/ 2010 induced production by electricity consumption

+
*

+
+
++++
+ ++ ++++ +
+
+ +
+++
* * * * ++++++* * * * * * * +* * +++++++++++* * * * * * * * * * * +* * +* +* * * * * +* +++* * * * ++**********

7/ 2010 induced production by gass consumption

household consumption
induced production
Carbon uptake land+++++++++++++++++++++++++* * * * * * * * * * * * * * * * * * * * * * * * * +* +* ++++
* * * * ++* * ++++++++++++++**************+*

Figure 3426
consumption & productionv(million yen)
−20000
20000
60000
+*

*
+

Figure 2

+
+

*
*

+

+

*

*

Figure 5

7/2010 induced production by gass consumption
7/2003 induced production by gass consumption++++ +* ++* +* +* +* * + +* +*+* + +* * +* +++ ** * + ++ +* +* +*+ +* * * +* * +* +* + +* * +** * +*
+* *
+*
*
* **
+* *
*
+
+ +
+
+
+*
*
*
+
+

*
+
household consumption
induced production
Carbon uptake land+*+++* ++ + ++ + +++ +* * ++ +++ +* +* * + + +* +* +++ + ++++++++ + ++++++* * * * * *** + ******************* * * +* * * * * * *
+*
*

+++
* * * +* +++++* * * * * +* +* +* +* +++++++++++* * * * * * * * * * * +* +* +* +* +* * +* +* +* +++* * * +* ++* * ++* * * ++++
* * * * +*++Carbon uptake land
induced production
household consumption+*++++++++++++* * * * * * * * * * * * +* ++++++++++++* * * * * * * * * * * * +* +* ++++
* * * * +* +++++++++++++++***************

Carbon uptake land
induced production
household consumption−0.0020.002
0.006
Ecological Footprint (global hectare)household consumption
induced production
Carbon uptake land

−0.0020.002
0.006
Ecological Footprint (global hectare)+* + +* +* + + ++++* * * * *
*
*
*
+ *
+* +

*
+

−0.030 −0.020 −0.010
0.000
Ecological Footprint (global hectare)

Hokukuikaido
Aomori
Iwate
Miyagi
Akita
Yamagata
Fukushima
Ibaragi
Totigi
Gunma
Saitama
Chiba
Tokyo
Kanagawa
NiigataToyamaIshikawa
Fukui
Yamanashi
NaganoGifu
Shizuoka
Aichi
Mie
Shiga
Kyoto
Osaka
HyougoNara
Wakayama
Tottori
Shimane
Okayama
Hiroshima
Yamaguchi
Tokushima
KagawaEhime
Kouchi
Saga
Nagasaki
Kumamoto
Oita
MiyazakiKagoshima
Okinawa
Hokukuikaido+*

Hokukuikaido
Aomori
Iwate
Miyagi
Akita
Yamagata
Fukushima
Ibaragi
Totigi
Gunma
Saitama
Chiba
Tokyo
Kanagawa
NiigataToyamaIshikawa
Fukui
Yamanashi
NaganoGifu
Shizuoka
Aichi
Mie
Shiga
Kyoto
Osaka
HyougoNara
Wakayama
Tottori
Shimane
Okayama
Hiroshima
Yamaguchi
Tokushima
KagawaEhime
Kouchi
Saga
Nagasaki
Kumamoto
Oita
MiyazakiKagoshima
Okinawa
Hokukuikaido**+−0.030 −0.020 −0.010
0.000
Ecological Footprint (global hectare)

*
+
+

+

Agriculture
Forestry
Fishing
MiningFood&Beverages
TextilesClothes
Wood
Furniture
Pulp&Paper
Printing&Publishing
Chemicals
Coke&Petroleum
Plastics
RubberPottery
Iron&SteelNon−ferrousmetals
Metalproducts
Machinery
ElectricalmachineryTransportequipment
Precisioninstruments
OtherManufactures
Recycling
Construction
Electricity
Gas&Steam
Water&Waste
Wholesale&Retailtrade
Finance&Insurance
RealestateTransport
Telecommunications
Broadcasting
Publicadmin
Education
Research
Health&Insurance
Publicservices
Advertisement
Rentingproducts
OtherBusinessservices
Amusement
Restaurants
HotelsOtherPersonalservices
Other

*

consumption & productionv(million yen)
−20000
20000
60000

+

−0.0020.002
0.006
Ecological Footprint (global hectare)

7/2010 induced production by electricity consumption

Agriculture
Forestry
Fishing
MiningFood&Beverages
TextilesClothes
Wood
Furniture
Pulp&Paper
Printing&Publishing
Chemicals
Coke&Petroleum
Plastics
RubberPottery
Iron&SteelNon−ferrousmetals
Metalproducts
Machinery
ElectricalmachineryTransportequipment
Precisioninstruments
OtherManufactures
Recycling
Construction
Electricity
Gas&Steam
Water&Waste
Wholesale&Retailtrade
Finance&Insurance
RealestateTransport
Telecommunications
Broadcasting
Publicadmin
Education
Research
Health&Insurance
Publicservices
Advertisement
Rentingproducts
OtherBusinessservices
Amusement
Restaurants
HotelsOtherPersonalservices
Other+* + + + + +* ++* * * * * +* * +**+household consumption
induced production
Carbon uptake landconsumption & production (million yen)−300000−1500000+*+* ++* * +* +* +*+* +* **+

−0.0020.002
0.006
Ecological Footprint (global hectare)

Hokukuikaido
Aomori
Iwate
Miyagi
Akita
Yamagata
Fukushima
Ibaragi
Totigi
Gunma
Saitama
Chiba
Tokyo
Kanagawa
NiigataToyamaIshikawa
Fukui
Yamanashi
NaganoGifu
Shizuoka
Aichi
Mie
Shiga
Kyoto
Osaka
HyougoNara
Wakayama
Tottori
Shimane
Okayama
Hiroshima
Yamaguchi
Tokushima
KagawaEhime
Kouchi
Saga
Nagasaki
Kumamoto
Oita
MiyazakiKagoshima
Okinawa
Hokukuikaido

consumption & productionv(million yen)
−20000
20000
60000

*
+

0.000
0.010
0.020
0.030
Ecological Footprint (global hectare)

Hokukuikaido
Aomori
Iwate
Miyagi
Akita
Yamagata
Fukushima
Ibaragi
Totigi
Gunma
Saitama
Chiba
Tokyo
Kanagawa
NiigataToyamaIshikawa
Fukui
Yamanashi
NaganoGifu
Shizuoka
Aichi
Mie
Shiga
Kyoto
Osaka
HyougoNara
Wakayama
Tottori
Shimane
Okayama
Hiroshima
Yamaguchi
Tokushima
KagawaEhime
Kouchi
Saga
Nagasaki
Kumamoto
Oita
MiyazakiKagoshima
Okinawa
Hokukuikaido

consumption & productionv(million yen)
−20000
20000
60000

+ +*
+

0.000
0.010
0.020
0.030
Ecological Footprint (global hectare)

Agriculture
Forestry
Fishing
MiningFood&Beverages
TextilesClothes
Wood
Furniture
Pulp&Paper
Printing&Publishing
Chemicals
Coke&Petroleum
Plastics
RubberPottery
Iron&SteelNon−ferrousmetals
Metalproducts
Machinery
ElectricalmachineryTransportequipment
Precisioninstruments
OtherManufactures
Recycling
Construction
Electricity
Gas&Steam
Water&Waste
Wholesale&Retailtrade
Finance&Insurance
RealestateTransport
Telecommunications
Broadcasting
Publicadmin
Education
Research
Health&Insurance
Publicservices
Advertisement
Rentingproducts
OtherBusinessservices
Amusement
Restaurants
HotelsOtherPersonalservices
Other

consumption & production (million yen)
0
100000 200000 300000*

consumption & production (million yen)−300000−150000
0

Agriculture
Forestry
Fishing
MiningFood&Beverages
TextilesClothes
Wood
Furniture
Pulp&Paper
Printing&Publishing
Chemicals
Coke&Petroleum
Plastics
RubberPottery
Iron&SteelNon−ferrousmetals
Metalproducts
Machinery
ElectricalmachineryTransportequipment
Precisioninstruments
OtherManufactures
Recycling
Construction
Electricity
Gas&Steam
Water&Waste
Wholesale&Retailtrade
Finance&Insurance
RealestateTransport
Telecommunications
Broadcasting
Publicadmin
Education
Research
Health&Insurance
Publicservices
Advertisement
Rentingproducts
OtherBusinessservices
Amusement
Restaurants
HotelsOtherPersonalservices
Other

consumption & production (million yen)
0
100000 200000 300000

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

7/2003 induced production by electricity consumption

Figure 4

7/ 2003 induced production by electricity consumption

7/ 2003 induced production by gass consumption

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Blood flow velocity waveforms in the middle cerebral arteryat rest and during exercise
T. Matsuo, S. Watanabe, M. Sorimachi, M. Kanda, Y. Ohta, and #T. Takahashi
Kanagawa Institute of Technology, Shimo-ogino 1030, Atsugi, Kanagawa 243-0292, Japan.
(Tel : 81-46-291-3279; Fax : 81-46-291-3262)(Email address : matuo@rm.kanagawa-it.ac.jp)#
Asahikawa Medical College, Midorigaoka Higashi 2-1-1-1, Asahikawa 078-8510, Japan.
Abstract: The blood flow velocities in the middle cerebral artery were measured under steady-state and incremental
cycle exercises using the transcranial Doppler ultrasound velocimeter. The peak-systolic velocity was found to risemarkedly under exercise, while the end-diastolic velocity tended to keep a resting value. The fluctuation of velocity andresistance index were calculated in order to evaluate the hemodynamic load on the vessel wall. They also increased
markedly under exercise. Such hemodynamic changes in activity might be important in understanding the genesis ofvascular diseases as well as physiology of cerebral circulation.
Keywords: Transcranial Doppler, Ultrasound, Middle cerebral artery, Cerebral Circulation, Blood velocity

cerebrovascular, or respiratory disease. All subjects
were fully informed about the procedures, risk, andbenefits of the study, and written consent was obtained
from all subjects before the study. This study wasapproved by the university institutional board.

I. INTRODUCTION
Since the brain tissues are extremely vulnerable interms of oxygen deficiency, an adequate and continuous
supply of oxygenated blood is essential for the functionof the brain. The blood flow in the middle cerebral
artery (MCA) maintains about 80% of the flow volume
arriving at the cerebral hemisphere. Since the blood
velocity waveforms are considered to be closely related
to the flow disturbances caused by arterial diseases suchas stenosis and cerebral aneurysms, blood velocity
measurements using transcranial Doppler (TCD)
sonography are widely conducted in clinicalexamination [1][2].
Most of the clinical data of transcranial Doppler havebeen measured under resting condition. It should,
however, be noted that the blood flow would be easily
disturbed by daily activity such as work, exercise andposture change, as well as changes in environmental
factors such as ambient temperature. In the present
study, we measured velocity waveforms in the MCA
during cycle exercise in order to understand the
dynamics of MCA blood flow caused by physicalexercise. Velocity waveforms might contain information
concerning the homodynamic load on the vessel wallsuch as shear stress and velocity fluctuation which areconsidered to play an important role of the genesis anddevelopment of arterial diseases [2].

2. Experimental design
Each subject was required to keep his body in an
upright, seated position on an electromagneticallybraked cycle ergometer during rest and exercise periods.
Two kinks of exercises were conducted as follows;(A) Steady-state exercise;
The work rate was maintained at 100 W for males
and at 50 W for females for 15 minutes exercise. During
exercise, the subjects pedaled at a constant rate of 60
rpm paced by a metronome. The exercise intensity
corresponds to “moderate exercise”.
(B) Incremental exercise;During 15 minutes exercise, the work rate wasincreased from 20 to 100 W by a 20 W increment stepfor males, and from 10 to 50 W by a 10 W step forfemales. The duration of each step was 3 minutes withkeeping a pedaling rate of 60 rpm.
3. Measurement of blood velocity and pressure

II. METHOD
1. Subjects
Six healthy young male and 6 healthy young
female volunteers participated in the present study.
None of the subjects had any history of cardiovascular,©ISAROB 2011

Fig.1

427

Position for Doppler examination

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

2. Hemodynamic data during steady-state exercise
Figure 3 shows physiological quantities (blood
velocity, blood pressure and heart rate) measured at restand during steady-state exercise. The measured values
given in the graph are average for 12 volunteers (6females and 6 males). Peak velocity MCAVs, systolicblood pressure, and heart rate rise immediately after
onset of exercise. On the contrary, the diastolic pressureand end-diastolic velocity MCAVd were found to show
no statistically significant change during exercise. It isalso noted that the heart rate tends to slightly increaseduring exercise, while MCAVs and systolic pressure
show slight inclination to decrease.

The blood flow velocity in the right middle cerebral
artery was obtained using a 2-MHz pulsed Doppler
ultrasound system (Intra-view, Rimed, Israel). Its
pulsed probe was located over the temporal bone andthe Doppler signal was optimized through a change in
the insonation angle. The probe attached to the skull ata fixed angle was held using a headgear with an
adjustable positioning system. The blood flow velocity
was assessed for 6-10 consecutive cardiac cycle
recorded during the last minute before and during
exercise. Peak-systolic, end-diastolic, and mean bloodflow velocity in MCA were represented by MCAVs,
MCAVd, and MCAVm, respectively.
The blood pressure in the brachial artery was
sphygmomanometrically measured using a pneumatic
arm cuff which was held at heart level. Mean arterial
blood pressure (MAP) was calculated as diastolic bloodpressure (DBP) plus one-third pulse pressure [3].

III.

MCAVs (cm/s)
MCAVd (cm/s)
Systoloc pressure(mmHg)
Diastolic pressure (mmHg)
Heart Rate (beats/min)

160

RESULTS AND DISCUSSION

120

1. Velocity waveforms
Rest

80

Exercise

Peak-systolic
40

Exercise
0Fig. 2

4

8

12

16

20

Fig. 3 Physiological quantities measured at rest andduring steady-state exercise.

Measured velocity waveforms in the MCAIn Fig. 2, an example of the obtained velocity
waveforms is shown, where the waveform in theexercise is compared with that in the rest. As can benoted in the left panel, the waveform shows substantial
variation of flow velocity during one cardiac cycle. The
velocity rises sharply at initial systolic phase of heart(peak-systolic velocity), and decreases gradually during
diastole. The flow waveform is characterized by a highforward flow at end-diastole, which is as much as half
peak-systolic velocity. This feature is characteristic ofblood flow in intracranial arteries, and contrasts with therelatively low diastolic flow component in the external
carotid artery. This difference is ascribed to themarkedly low flow resistance in the cerebral vasculaturecompared with that in the territory of the external
carotid artery [1].
It should be also noted that the peak-systolicvelocity (MCAVs) markedly increased during exercise,
while the end-diastolic velocity (MCAVd) showed atrivial change. Accordingly, the difference between
MCAVs and MCAVd was increased.

©ISAROB 20110Time (min)End-diastolic

3. Hemodynamic data during incremental exercise

Fig. 4 Physiological quantities measured at restand during incremental exercise.

428

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

exercise intensity increases. Such VF variations as afunction of exercise intensity are quite similar to the
MCAVs variations shown in Figs. 2 and 3.

The measured physiological quantities underincremental exercise are shown in Fig. 4. The MCAVs
gradually increases as the degree of exercise increases.
After 15 minutes, corresponding work rate being 100 W
for male and 50 W for female, MCAVs reaches a value(130 m/s) comparable to that in the steady-state exercise.
Also the systolic pressure and heart rate are on theincrease. As in the steady-state exercise, both MCAVd
and diastolic pressures show no significant change
during exercise.

5. Resistance index
Pourcelot [4] introduced the “resistance index” toDoppler sonography as a means to characterize theperipheral resistance in the cerebral circulation.
Resistance index (RI) is defined as following equation,

RI =5. Fluctuation in MCA velocity

This index describes the ratio of peak-systolicvelocity to the end-diastolic velocity. For the normal
common carotid artery, RI is reported to be between
0.55 and 0.75. In arteries that supply high-resistancemusculocutaneous beds, RI is known to be greater than
0.75 [4]. The arteries that supply the brain have RI
index less than 0.75. A low RI value is also seen poststenotically in cases where flow resistance is decreased
by reactive dilatation of blood vessels [1].
In Figs. 7 and 8, are shown the obtained RI values
during exercise, in which results for males and females
are indicated individually. Under steady-state exercise
shown in Fig.7, although the variance of the data areconsiderably large, RI values tend to increase rather
gradually from 0.57 to 0.74 in males and from 0.53 to
0.67 in females, compared with the fluctuation index
shown in Fig. 6. These values of RI can be wellcompared with the previous data mentioned above.

€

Fig. 5

Indices used in waveform analysis

In order to see the variation in velocity due toexercise, we calculated velocity fluctuation index (VF)for measured velocity waveforms, which is defined as,n∑ (Vi − MCAVm ) 2
VF = i =1n€

MCAVs − MCAVd
MCAVd
=1− (
) ····(2).
MCAVs
MCAVs

·········(1).

Here, Vi: velocity at time ti, MCAVm: mean velocity
averaged over one cardiac cycle, and n: number ofmeasured velocity data for one cardiac cycle. This index
is corresponding to the standard deviation in statistics.

0.8
0.7

25

0.6

20
15

0.4

VF_incremental
VF_steady state

10

Exercise
0

3

6

9

steady-state exercise

0.3
0.2

5
0

RI_maleRI_female

0.5

0

3

6

9

12

15

18

21

Time (min)
12

15

18

Fig. 7

21

Resistance index during steady-state exercise

Time (min.)As shown in Fig. 8, in males, the RI values inincremental exercise vary increasingly from 0.57 to 0.69
as increase of exercise intensity. Although an increasing
trend is seen also in females, the data points show rather
scattering. The reason for the difference between female
and male is not clear at present, and sophisticatedexperiment would be necessary.
The index RI is not only a function of flow
resistance but also influenced by vascular compliance. It

Fig. 6 Velocity fluctuations（VF）of MCA bloodflow under incremental and steady-state exercises.
Figure 6 compares VF values calculated for steadystate exercise with those for incremental exercise.
Velocity fluctuations in steady-state exercise are wellconstant during exercise. The VF values under exercise
were found to be about 1.8 times larger than those atrest. In the incremental exercise, VF increases as the©ISAROB 2011429

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011is generally known that the velocity waveform changes
considerably with aging result from increased resistancedue to cerebral atherosclerosis, and decreased
compliance of vessel wall [1]. The waveforms of elder
subjects were reported to be characterized by high RI
values and steep decrease of velocity in diastole. The
waveform change form rest to exercise shown in Fig. 2
seems to be similar to that due to aging.

200

r=0.513p<0.001

MCAVs
160
120
80
40
40

80

120

160

Heart rate (beats/min.)Fig. 10. Correlation between MCAVs and heart rate

IV.

Fig.8

The velocity waveforms in the middle cerebral
artery in young volunteers were measured undermoderate-intensity cycle exercise. The peak systolicvelocity MCAVs and the end-diastolic velocity MCAVd
were obtained from measured waveforms. Calculated
hemodynamic variables, velocity fluctuation VF andresistance index RI, showed remarkable increase under
exercise. The waveforms in exercise seemed to be
similar to those of elder people at rest. A preliminary
experiment of high intensity exercise showed nonlinearchanges of MCAVs, MCAVd, VF and RI, indicating thatthe influence of exercise on cerebral blood flow is more
complicated than thought before. However, the numberof volunteers was still limited in the present study andthe results are thought to be affected by individual
differences in physiological variables. Therefore, further
study is required to derive a concrete conclusion.
Simultaneous measurements of MCA blood flow
waveforms and arterial blood pressure waveforms areunder way, since these measurements would provide
much information on the regulation of cerebral
circulation.

Resistance index during incremental exercise

We also measured velocity waveforms during strong
exercise with intensities from 120 to 150 W (data notshown). It was found that above 120 W of exercise
intensity, the MCAVs values were almost constant, while
MCAVd values were slightly decreased, resulting in thatboth velocity fluctuation VF and resistance index RI
were increased.
5. Correlation between hemodynamic variables
Measured hemodynamic variables such as blood
velocities, arterial pressures, and heart rate were
subjected to correlation analysis each other. Among theresults, the relationship between MCAVs and systolicblood pressure (SBP), and that between MCAVs andheart rate HR were found to show statisticallysignificant correlation as shown in Figs. 9 and 10. The
significant correlation between MCAVs and systolic
arterial pressure might be supported by the results
indicated in Figs. 2 and 3. Although the data are notgiven here, mean MCA blood velocity MCAVm showedsignificant correlation with mean arterial pressure andheart rate.

REFERENCES[1] G-Michael von Reuturn and Hans Joachim von
B u e d i n g n (1993), Ultrasound Diagnosis of
Cerebrovascular Disease, Georg-Thieme Verlag
Stuttgart·Newe York.
[2] Hennerici M G and Stephen P M (2001),
Cerebravascular ultrasound: theory, practice andfuture developments, Cambridge Univ. Press New
York.
[3] Watanabe S, Matsuo T, Sorimachi M, et al.
(2009), Comparison of blood flow velocity in the
middle cerebral artery between men and women at
rest and during exercise. Therapeutic Research,
30(4), 537-544.
[4] Pourcelot L (1976), Diagnostic ultrasound for
cerebral vascular disease, in Donald JS, Levi S
edits: Present and Future of Diagnostic Ultrasound.
Kooyker Rotterdam.

Fig. 9. Correlation between MCAVs and systoricarterial pressure©ISAROB 2011

SUMMARY

430

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Two-Level Time-Series Clustering for Satellite Data Analysis
Ayahiko Niimi
Faculty of Systems Information Science
Future University Hakodate
116-2 Kamedanakano-cho, Hakodate-shi,
Hokkaido 041-8655, JAPAN(niimi@fun.ac.jp)

Takehiro Yamaguchi
Graduate School of Systems Information Sciences
Future University Hakodate
116-2 Kamedanakano-cho, Hakodate-shi,
Hokkaido 041-8655, JAPAN(g2109046@fun.ac.jp)

Osamu Konishi
Faculty of Systems Information Science
Future University Hakodate
116-2 Kamedanakano-cho, Hakodate-shi,
Hokkaido 041-8655, JAPAN(okonishi@fun.ac.jp)

Abstract: In this paper, we propose a method for finding the frequent occurrence patterns and the frequent occurrencetime-series change patterns from the observational data of a weather-monitoring satellite. The observational data of theweather-monitoring satellite are temporal and spatial large-scale data. However, to analyze this large amount of data
incurs a high calculation cost. Therefore, we propose parallel computation when the frequent occurrence pattern andthe frequent occurrence time-series change pattern are extracted at the Artificial Life and Robotics conference (AROB)
2010. In this paper, we apply the proposed system to Moderate Resolution Imaging Spectroradiometer (MODIS) data anddiscuss its results.
Keywords: distributed processing, clustering, frequent occurrence pattern extraction, satellite data, data stream

I.

INTRODUCTION

veloped to deal eﬃciently with large-scale data streams [1,2, 3, 4, 5, 6, 7].
An example of a data stream for this technology isdata from satellites. Satellite data are used for various
purposes, such as land-cover classification and forecasts[8] and marine information analysis [9, 10, 11, 12, 13].
However, satellite data treated up to now as static data.
Therefore, much computing time was required to analyze a large amount of data. In this paper, we propose
a method to solve this problem by using distributed processing.
At the Artificial Life and Robotics conference (AROB)
2010, we propose a method for finding the frequent occurrence patterns and the frequent occurrence time-serieschange patterns from the observational data of a weathermonitoring satellite [14]. The observational data of theweather-monitoring satellites are temporal and spatial largescale data. Various uses are possible such as forecasting
marine resources by analyzing satellite data. However,there is an issue with respect to the calculation cost to
analyze a large amount of data. Thus, we propose to use
parallel computation when the frequent occurrence pattern and the frequent occurrence time-series change pattern are extracted in this paper.

In our network society, the development of information processing enables us to collect and utilize massive
amounts of data; and data mining has gained attention
as a technology to discover new knowledge and patterns.
But those data are changing continuous, and new types oflarge-scale data have emerged. For example, records offinancial and distributional transactions, telecommunications records, and network access logs are typical data
streams. The term data stream suggests that the temporally changing, massive amounts of data records thatare generated, accumulated, and consumed are looked on
as flow of data (stream). In the real world, the requirement has been growing to elicit information from thoselarge data streams whenever we need information. At
first glance, data mining seems to be eﬀective; but a data
stream has the following dynamic properties:
1. massive amounts of data are
2. coming over a high-speed stream,
3. temporally changing, and
4. continue to arrive permanently;
and data mining is intended for static data, not a stream.
Therefore, data stream mining technology has been de-©ISAROB 2011431

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Our proposed method is as follows. First, to extract
the frequent occurrence pattern from satellite data, thenecessary marine information is acquired by using thefilter from satellite data. Next, the extracted marine data
undergo clustering to merge similar data and are labeled.
As a result, similar data are brought together for data witha spatial extension. The labeling data are re-clustered tothe data group and re-labeled according to the degree ofsimilarity between labels. As a result, similar data arebrought together for data with a time extension. Finally,frequent events are extracted as the frequent occurrence
pattern from the labeling data. Moreover, the frequent
occurrence of the time-series pattern can be extracted as
rules by detecing the change in the labeling data group.
However, it takes computing time to analyze long-term
data. Therefore, by dividing data and integrating the results, we propose to shorten the computing time by parallel computation of clustering and the frequent occurrenceof the time-series pattern rule extraction. As for clustering and the frequent occurrence of the time-series change
pattern extraction, parallel computation is possible by dividing data. The shorter computing time can be expectedby division degree because each algorithm never influences the parallel calculation. Each algorithm was examined with regard to whether it was possible to it make parallel. Because clustering and extracting change patterns
can be applied in parallel computing, we constructed thesystem with clustering of marine information and the extraction of the change pattern.
At AROB 2010, the frequent occurrence pattern andthe frequent occurrence of the time-series change pattern of the sea surface temperature are extracted by using the sea surface temperature data, with the weathermonitoring satellite providing verification. In this paper,
we use Moderate Resolution Imaging Spectroradiometer(MODIS) data, which includes the sea surface temperature (SST) and the concentration of chlorophyll-a (chl-a)
to verify our proposed system and discuss its results.

II.

Data

Data

Data

Clustering

Clustering

Clustering

Clustering

ClusteringLabeling

Pattern Extraction

Pattern Extraction

Pattern Extraction

Frequent PatternFig. 1: Flowchart of proposed system
between the labels.
4. Finally, the frequent events are extracted as the frequent occurrence pattern from the data.
In the first clustering, similar data are brought together for data with a spatial extension. In the second
clustering, similar data are brought together for data witha time extension. Moreover, the frequent occurrence of
the Time-series pattern can be extracted as rules by detecting the change in the labeling data group. However, ittakes computing time to analyze long-term data. Therefore, by dividing data and integrating the results, we propose to shorten the computing time by parallel computation of clustering and the frequent occurrence of thetime-series pattern rule extraction. As for clustering andthe frequent occurrence of the time-series change pattern
extraction, parallel computation is possible by dividing
data. The shorter computing time can be expected bydivision degree because each algorithm never influences
the parallel calculation. Each algorithm is examined withregard to whether it was possible to make parallel. Because clustering and extracting of change patterns can beapplied in parallel computing, we constructed the systemwith clustering of marine information and the extractionof the change pattern.

THE PROPOSED METHODAt AROB 2010, we propose a method for finding thefrequent occurrence patterns and the frequent occurrencetime-series change patterns from the observational dataof a weather-monitoring satellite [14].
Fig. 1 shows a flowchart of the proposed system.
The flow of the algorithm is shown below.
1. First, to extract the frequent occurrence pattern fromsatellite data, necessary marine information is acquired by using the filter from satellite data.
2. Next, the extracted marine data undergo clustering
to merge similar data and are labeled.
3. The labeling data are re-clustered to the data groupand re-labeled according to the degree of similarity©ISAROB 2011

Data

III. DETAILS OF SATELLITE DATAAt AROB 2010, we acquire the data of Meteorological Satellite Center as satellite data. We use the observation monthly report of the Meteorological Satellite Cen-

432

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

ter for the experiment [15]. The observation monthly
report can be obtained on CD-ROM. These CD-ROMs
contain the monthly report of observation data derived
from Multi-functional Transport Satellite (MTSAT-1R)
and the polar orbital meteorological satellite from National Oceanic and Atmospheric Administration (NOAA).
For the problem of forecasting the hot spot of marine
products using satellite data, the sea surface temperature,
the chlorophyll, and the flow of the ocean are often targeted. Because only the sea surface temperature data are
included in the observational data, sea surface temperature is used in the experiment. Ten-day mean sea surface
temperature consists of grid points arrayed every one degree latitude and longitude covering the area from 50 degrees North to 50 degrees South and 90 degrees East to
170 degrees West. We use data from March 2009 to May
2009 because the regression equation for SST was updated on March 1, 2009.
In this paper, we use MODIS data, which includes
Sea Surgace Temperature(SST) and the concentration of
Chlorophyl-a(Chl-a) to verify our proposed system. MODISis a key instrument aboard the Terra (Earth Observing
System (EOS) AM) and Aqua (EOS PM) satellites. Terras orbit around the Earth is timed so that it passes from
north to south across the equator in the morning, while
Aqua passes south to north over the equator in the afternoon. Terra MODIS and Aqua MODIS view the entire
Earth s surface every 1 to 2 days, acquiring data in 36
spectral bands, or groups of wavelengths [16, 17, 18, 19].
We use MODIS data from the Academic Frontier Promotion Center of Tokyo University of Information Sciences.
We used the publicly available data of SST and chl-a.
The public data contain 30 days composite data, 5 days
composite, and daily composite by PNG format, FLAT
format, and HDF format. We convert from HDF data to
CSV data by HDF utility commands. We use Hakodateera data from January 2005 to September 2010.

IV. EXPERIMENTS
In this paper, we try to extract the frequent occurrence pattern and the frequent occurrence of the timeseries change pattern using SST data and chl-a with theweather-monitoring satellite for verification.
We use SST and chl-a data from MODIS data. We
use linear interpolation with the data at a time before andafter the missing value. The input data use nine attributes
in which the data of eight neighborhoods are added to thedata of a certain point. We already described the data indetail in section III..
We use the InTrigger platform of the information explosion project [20]. InTrigger is a distributed platform
for information technology research for the Information
Explosion Era. It is a cluster of clusters distributed across©ISAROB 2011433

Japan. Weka is used for clustering [21].
The first step clusters data; and the eﬀect of the distributed surrounding is examined. The result is being reasoned now.

V. CONCLUSIONSWe proposed a method for finding the frequent occurrence patterns and the frequent occurrence time-serieschange patterns from the observational data of a weathermonitoring satellite at AROB 2010. The observational
data of the weather-monitoring satellite are temporal andspatial large-scale data. Various uses are possible such as
forecasting marine resources by analyzing satellite data.
However, there is a problem with respect to calculation
cost to analyze a large amount of data. Therefore, we proposed parallel computation when the frequent occurrence
pattern and the frequent occurrence time-series change
pattern are extracted. In this paper, we applied the proposed system to MODIS data, which includes the sea surface temperature and the concentration of chlorophyl-a toverify our proposed system.

REFERENCES[1] Martin H. C. L., Zhang, N., Anil, K. J. (2004), Nonlinear Manifold Learning For Data Stream. In Proc.
SIAM International Conference for Data Mining,
pp.34–44[2] Jain, A., Zhang, Z., Chang, E. Y. (2006), Adaptive
non-linear clustering in data streams. CIKM ’06:
Proceedings of the 15th ACM international conference on Information and knowledge management,
Arlington, Virginia, USA, pp.122–131[3] Graf, H. P., Cosatto, E., Bottou, L., Durdanovic,
I., Vapnik, V. (2005), Parallel support vector machines: The cascade svm. In Advances in Neural
Information Processing Systems, pp.521–528[4] Zhang, Y., Jin, X. (2006), An Automatic Construction and Organization Strategy for Ensemble Learning on Data Streams. SIGMOD Record, Vol.35,
No.3, pp.28–33[5] Wang, H., Fan, W., Yu, P. S., Han, J. (2003), Mining Concept-Drifting Data Streams Using Ensemble Classifiers. SIGKDD’03, pp.226–235[6] Yamaguchi, T., Niimi, A. (2009), Community
Graph Sequence with Sequence Data of Network
Structured Data. 5th International Workshop on
Computational Intelligence & Applications (IWCIA2009), Hiroshima, Japan, pp.196–201[7] Minegishi, T., Ise, M., Niimi, A., Konishi, O.
(2009), Extension of Decision Tree Algorithm for

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Stream Data Mining Using Real Data. 5th International Workshop on Computational Intelligence& Applications (IWCIA2009), Hiroshima, Japan,
pp.208–212[8] Yamaguchi, T., Noguchi, Y., Ichimura, T., Mackin,
K.J. (2009), Applying Cluster Ensemble to Adaptive Tree Structured Clustering. 5th International
Workshop on Computational Intelligence & Applications (IWCIA2009), Hiroshima, Japan, pp.186–
191[9] Mustapha, M. A., Saitoh, S. (2008), Observationsof sea ice interannual varitions and spring bloom
occurrences at the Japanese scallop farming area in
the Okhotsk Sea using satellite imageries. Estuarine, Coastal and Shelf Science, 77, pp.577–588[10] Zainuddin, M., Kiyofuji, H., Saitoh, K., Saitoh, S.
(2006), Using multi-sensor satellite remote sensing
and catch data to detect ocean hot spots for albacore (Thunnus alalunga) in the northwestern North
Pacific. Deep-Sea Research II, 53, pp.419–431[11] Iida, T., Saitoh, S. (2007), Temporal and spatial variability of chlorophyll concentrations in the
Bering Sea using empirical orthogonal function(EOF) analysis of remote sensing data. Deep-Sea
Research II, 54, pp.2657–2671[12] Radiarta, I. N., Saitoh, S. (2008), Satellite-derived
measurements of spatial and temporal chlorophylla variability in Funka Bay, southwestern Hokkaido,
Japan. Estuarine, Coastal and Shelf Science, 79,
pp.400–408[13] Zainuddin, M., Saitoh, K., Saitoh, S. (2008), Albacore (Thunnus alalunga) fishing ground in relation
to oceanographic conditions in the western North
Pacific Ocean using remotely sensed satellite data.
Fisheries Oceanography, Vol.17, No.2, pp.61–73[14] Niimi, A., Yamaguchi, T., Konishi, O. (2010),
Parallel Computing Method of Extraction of Frequent Occurrence Pattern of Sea Surface Temperature from Satellite Data. International Symposium
on Artificial Life and Robotics (AROB 15th ’10),
Beppu, Oita, Japan: 4 pages (in CD-ROM)[15] Meteorological Satellite Center Monthly Report,
Meteorological Satellite Center.
[16] MODIS Website, http://modis.gsfc.nasa.
gov/[17] MODIS Near Real Time Data, Japan Aerospace
Exploration Agency, http://kuroshio.eorc.
jaxa.jp/ADEOS/mod nrt/
[18] Earth Observation Research Center (EORC), Japan
Aerospace Exploration Agency, http://www.
eorc.jaxa.jp/index.php[19] Academic Frontier Promotion Center ofTokyo University of Information Sciences.

©ISAROB 2011

http://www.frontier.tuis.ac.jp/modis/
frontier/index.html[20] InTrigger, https://www.intrigger.jp/wiki/
index.php/InTrigger[21] M. Hall, E. Frank, G. Holmes, B. Pfahringer, P.
Reutemann, H. Ian. (2009), The WEKA Data Mining Software. SIGKDD Explorations, Volume 11,
Issue 1

434

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

PSP Practice Support System Using Defect Types based on
phenomenon
Daisuke Yamaguchi1, Ayahiko Niimi2, Fumiyo Katayama1 and Muneo Takahashi1
1:Toin University of Yokohama, 1614 Kurogane-cho, Aoba-ku, Yokohama, JAPAN
2:Future University-Hakodate, 116-2 Kamedanakano-cho, Hakodate, Hokkaido, JAPANyamaguti@intlab.toin.ac.jp

Abstract: In this paper, we propose the PSP Practice Support System using Defect Types based on
Phenomenon. This system can transmit programming to specific human among many software
processes using a Multiagent technology. The system is also synthesized to do parallel and cooperative
proposing internally. Applying the proposed method to a personal process-removing task, a flexible
programming for quality of software. Software developments depend on information, which is possible
to collection of personal process. Agent planning has get use working data on user action and othercommunication. Therefore collection of all user data is necessary for agent learning. Agent studies thebest transmission programming, planning and quality according to the makes planning in the personal
process.
Keywords: Multi-Agent System, Personal Software Process, Software Engineering, Artificial Intelligence

I. INTRODUCTION

II. Intelligent Agents and Multiagent System

Software architecture has emerged as an important
sub discipline of software engineering [1]. PSP support
system is built using this. Moreover, We think that thedata inputted can acquire software development processby sorting out using a user action record table [2].
In this paper, the PSP practice support system using
Defect Types based on Phenomenon. This PSP practices
Support System based on Multiagent Techniques [3].
Generally, software process data is complicated, andwhen building a support system using such data including some action time, the calculation with expression isdifficult in many cases. Then, the PSP systems [2] configuration from a data pattern is effective using the
Machine Learning who is excellent in pattern recognition to such a problem.
The system is also synthesized to do parallel and cooperative proposing internally. Applying the proposed
method to a personal process-removing task, a flexible
programming for quality of software. Software developments depend on information, which is possible tocollection of personal process. Agent planning has get
use working data on user action and other communication. Therefore collection of all user data is necessary
for agent learning. Agent studies the best transmission
programming, planning and quality according to themakes planning in the personal process.

Artificial Intelligence (AI) has made great strides incomputational problem solving using explicitly represented knowledge extracted from the task. If we continue to use explicitly represented knowledge exclusively for computational problem solving, we may
never computationally accomplish a level of problem
solving performance equal to humans. From this idea,
the paper describes the development of a multiagent
system that can be used to support the assessment ofdesign performance in the cellular automata model.
Agents represent objects or people with their own behavior, and take the structure of cellular automata lattice.
Intelligent agents and multiagent systems are one of
the most important emerging technologies in computerscience today [4]. The advent of multiagent systems hasbrought together many disciplines in an effort to build
distributed, intelligent, and robust applications. They
have given us a new way to look at distributed systems
and provided a path to more robust intelligent applications.
Multiagent systems deal with coordinating intelligent behavior among a collection of autonomous agents.
Emphasis is placed on how the agents coordinate theirknowledge, goals, skills, and plans jointly to take action
or to solve problems. Constructing the multiagent systems is difficult [5,6]. They have all the problems of
traditional distributed and concurrent systems plus the©ISAROB 2011435

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

additional difficulties that arise from flexibility requirements and sophisticated interactions.

design. Team Software Process exists over the PSP hierarchies.

III. Personal Software Process
The Personal Software Process (PSP) is a selfimprovement process that helps you to control, manage,and improve the way you work. It is a structured
framework of forms, guidelines, and procedures fordeveloping software [2]. Properly used, the PSP
provides the data you need to make and meet commitments, and it makes the routine elements of your job
more predictable and efficient.
The PSP's sole purpose is to help you improve yoursoftware engineering skills. It is a powerful tool thatyou can use in many ways. For example, it will help you
manage your work, assess your talents, and build yourskills. It can help you to make better plans, to precisely
track your performance, and to measure the quality ofyour products. Whether you design programs, develop
requirements, write documentation, or maintain existing
software, the PSP can help you to do better work.
Rather than using one approach for every job, youneed an array of tools and methods and the practiced
skills to use them properly. The PSP provides the data
and analysis techniques you need to determine which
technologies and methods work best for you.PSP write
several program using the evolving process shown Figure 1.
The PSP also provides a framework for understanding why you make errors and how best to find, fix, andprevent them. You can determine the quality of yourreviews, the defect types you typically miss, and thequality methods that are most effective for you.
After you have practiced the exercises in this book,
you will be able to decide what methods to use andwhen to use them. You will also know how to define,measure, and analyze your own process. Then, as you
gain experience, you can enhance your process to takeadvantage of any newly developed tools and methods.
The PSP is not a magical answer to all of your software engineering problems, but it can help you identify
where and how you can improve. However, you must
make the improvements yourself. PSP0 and PSP0.1
hierarchy include introduces process discipline andmeasurement. PSP1 and PSP1.1 hierarchy include introduces estimating and planning. PSP2 and PSP2.1
hierarchy include Introduces quality management and©ISAROB 2011

Fig.1. PSP Process Evolution

IV. The PSP Practice support system using
Multiagent
In this section, we study combined as it occurs ingenetic Techniques into agent learner [2]. We used as atool for searching wide and complex solution space inIntelligent Agent learns data. Intelligent agent using
complex techniques of related research. Multiagent isstate in a filed shown Figure 2.

Fig.2. Support System Communication of Multiagent

436

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Figure 2 depicts the Agent Communication Moduleand shared Information Data. The Agent make filed inorder to share information data from Agent communication filed. These fields include other Learner kept inIntelligent Agent shown Figure 3.
Figure shows the Agent between communication
modules in other communicate method. In this case,Intelligent Agent supports the PSP time and size measures record to user manipulation data. Intelligent AgentController selects Agent Information Data Share (AiDS) or Agent Information Data Delivery (AiD-D) [3].

Total
In the PSP, engineers use the time recording log tomeasure the time spent in each process phase. In this log,
they note the time they started working on a task, thetime when they stopped the task, and any interruption
time. For example, an interruption would be a phone
call, a brief break, or someone interrupting to ask aquestion. By tracking time precisely, engineers track theeffort actually spent on the project tasks. Since interruption time is essentially random, ignoring these times
would add a large random error into the time data andreduce estimating accuracy.
Since the time it takes to develop a product is largely determined by the size of that product, when using
the PSP, engineers first estimate the sizes of the products they plan to develop. Then, when they are done,
they measure the sizes of the products they produced.
This provides the engineers with the size data they needto make accurate size estimates. However, for these data
to be useful, the size measure must correlate with thedevelopment time for the product. While a line of code
(LOC) is the principal PSP size measure, any size
measure can be used that provides a reasonable correlation between development time and product size. It
should also permit automated measurement of actual
product size.
So, This any measure record to support agent consider with using this Agent Learner expanded of PSP
support. A person engaging in a person who experienced PSP and software development for many years is
not very worried about a form record-keeping work.
Record keeping is vague, and what is performed of aperson pressed by a work still increases. Necessity toperform automatically is important in a soldier, remission of an activity and process assay to record an activity precisely.
Therefore I record all activities, and a support system shares the documentary information, and Intelligent
Agent examines to whether be content which documentary information to shows personal characteristic of difference with another person.

Fig.3. The Configuration of Agent Module
Other Learner support anything AI techniques of input data. Intelligent Agent has made combined thesetechniques into the Machine Learning. Machine Learning include same function of standard algorithm using
user analyses data. These techniques supported by analysis data in time sheet that retrieval of start and end
point.
Table 1 shows the PSP record form Time Measuresand Size Measures [2].
Table 1. The Sample data scale of Program
size categories No.X
Plan
Results Accumulation
Base

70

100

+100

Added

5

35

+35

Modified03+3DeletedNew andchangedRe-used

0

8

+8

0

0

0

20

40

+40

New Re-used

0

20

+20

©ISAROB 2011

V. Improvement of Software Estimate Efficiency Centered Multiagent
In this section, explain improvement of Software Estimate used to Multiagent internal Agent Learner for

437

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

VI. CONCLUSIONIntelligent Agent. Multiagent connects in other Intelligent Agents. Hence, that Intelligent Agent put the Agent
Learner on necessary thoughts in Multiagent [7] .

In this research we were analyses PSP Practice Support System used to phenomenon of working data. We
were able to viewpoint different searching user experience data. We create agent learner data in phenomenon
data beside with user working analyses and wrote PSP
Practice phenomenon error code type.
For future works, we will consider methods quick
running of agent learner in data of error type phenomenon and user experience. We try to delete user missing
work date filter on experience data. We consider to that
delete missing work filter on error type phenomenon
error type.
Future versions of PSP Practice Support System
User need be conscious of rewrite error type code orfact process from error message of compiler and executer to test pattern data. But, This model will show at
the system in a more natural, unscripted scenario, involving multiple parts in addition to other forms ofprocess and error type phenomenon.

1. Software Estimate Design of Agent Learner
The Software Design Estimate kept in Intelligent
Agent. Figure 4 shows the Agent internal Data, PSP
database and user logs connection modules in othercommunication method. In this case, Intelligent Agent
supports the PSP time and size (LOC: Line of Code)measures record to user manipulation data. Intelligent
Agent used to learning Control on internal database for
AiD.

REFERENCES[1] Paul Clements, Felix Bachmann, Len Bass, David
Garlan, James Ivers, Reed Little, Paulo Merson, RobertNord, Judith Stafford (2010), Documenting Software
Architectures SECOUND WDITION, Addison-Wesley[2] Watts S. Humphrey (2005), PSP - A Self Improvement Process for Software Engineers, Addison-Wesley[3] Daisuke Yamaguchi, Ayahiko Niimi and Muneo
Takahashi (2010), Improvement of a Software Estimate
Efficiency Centered PSP Practice Support System Using Multiagent Techniques, The Fifteenth International
Symposium on Artificial Life and Robotics 2010,
pp.857-860[4] Weiss, G. (1999), Multiagent Systems, A Modern Approach to Distributed Artificial Intelligence,
the MIT Press[5] Abul, O., Polat, F., and Alhajj, R. (2000), Multiagent Reinforcement Learning Using Function Approximation, IEEE Transaction on systems, man, andcybernetics-part c: application and reviews, Vol. 30, No.
4, pp. 485-497.
[6] Khosla, R., and Dillon, T. (1997), Engineering Intelligent Hybrid Multi-Agent Systems, Kluwer Academic
Publishers[7] Carles Sierra, JohnThangarajah, Lin Padgham andMichael Winikoff (2007), Designing Institutional MultiAgent Systems, AOSE 2006, LNCS 4405, PP.84-103

Fig.4. The AiD Data transferred from Agent
Learner to action logs
2. The Estimating Probe Method of Agent Software Design
The Probe Method guides user in using historical
data to make estimates. With estimated proxy size E,Intelligent Agent can calculate the projected program
size P and did total estimate development time. The
parameters β 0 and β 1 are used in the following equation to calculate projected added modified size:
Projected Added & Modfied size P = β 0 + β1 *Ε

(1)When two sets of data are strongly related, Intelligent Agent can use the linear regression method to represent that relationship.€This means that linear regression is often appropriate. The parameters β 0 and β 1
are calculated from user historical data.

©ISAROB 2011

438

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Development and Evaluation ofSatellite Image Data Analysis Infrastructure
Akihiro Nakamura*, Jong Geol Park**, Kotaro Matsushita**
Kenneth J. Mackin**, Eiji Nunohiro***: Graduate School of Tokyo University of Information Sciences**: Tokyo University of Information Sciences(Tel: 81-43-236-1292; Fax : 81-43-236-1292)(Email address: nunohiro@rsch.tuis.ac.jp)

Abstract: Tokyo University of Information Sciences (TUIS) receives Moderate Resolution Imaging Spectroradiometer(MODIS) data and provides the processed data to universities and research institutes as part of the academic frontier
project. One of the major fields of the research using MODIS data is the analysis on change of environment. We are
currently developing applications to analyze the environmental changes. These applications run on our satellite image
data analysis system implemented in a parallel distributed system and a database server. When using satellite data, one
common problem is the interference of clouds. In order to remove the interference of clouds, the standard solution is tocreate a composite data of the same regions during a selected time span, and to patch together data not covered byclouds to create a clear image. We introduced a piece processing algorithm, which separates one satellite image data
into many small pieces of image data, making it quicker and easier to analyze and process the time-series satellite data.
In this research, we implemented the piece processing and composite processing algorithms in order to increase theanalysis speed within the satellite image database. We tested the proposed processing and verified the effectiveness fortarget applications.
Keywords: Satellite image data, MODIS, Composite.

I. INTRODUCTION

II. SATELLITE IMAGE DATA ANALYSISSYSTEM

The Academic Frontier Project at Tokyo Universityof Information Sciences (TUIS) is promoting researchon “the sustainable development of economic and social
structure dependent on the environment in eastern Asia”.
As part of this project, TUIS receives and processes
Moderate Resolution Imaging Spectroradiometer(MODIS) satellite data, one of the sensors equipped byNASA's Terra and Aqua satellites, to provide to
universities and research institutes. One of the major
research aims of this project is the analysis on change ofenvironment. For this research, we are developing asatellite image data analysis system to analyze theenvironmental changes and its effects[1][2][3]. This
paper reports on a) an overview of the satellite data
analysis system and MODIS data, b) the composite
processing algorithm in order to improve theperformance and accuracy of the data analysis, and c)
the proposed piece processing algorithm to improve thecomposite processing speed and data utilization
efficiency for analytical applications.

©ISAROB 2011

1. System Overview
This system works as the main platform of the total
satellite image data analysis service, maintains thesatellite image data, and implements satellite image data
analysis applications. The user can access the systemover the network via Web browser or other web
applications. Users can request for data analysis, checkthe results and download satellite image data throughthe system. The analysis applications currently being
developed are 1) search for fire regions in forests andfields, 2) search for similar image data, 3)
spatiotemporal analysis of land cover changes of east
Asia, 4) creation of east Asian disaster map, 5) Timeseries NDVI prediction using auto-regression analysis,
and 6) statistical and soft computing analysis for land
cover estimation.
Figure 1 shows the system configuration. Thissystem is consists of a Web server, distributed computer
nodes, and a database server. The user accesses the Web
server with a Web browser, following the guidance on
the accessed page. The role of the Web server is

439

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

accepting requests by web users, execution of specified
analysis application at compute nodes, and creating thefinal Web page of results. The role of the distributed
computer nodes are running the analysis applications,and running the piece processing and composite
processing algorithm. The role of the database is the
management of the original satellite image data and theresult data of the composite processing, and piece
processing operations.
The satellite image data is extremely large, anddirect manipulation of the large data leads to a high loadon both the processor and network. In order to improve
the throughput and turnaround time of data processing,in the proposed system the computer nodes are
configured from high performance servers and multiple
PC clusters, implemented as a parallel distributed
system. The various analysis applications and piece
processing and composite processing operations are runon this distributed computer nodes.

TUIS receive MODIS data using a tracking antenna.
The reception area is between 105 to 180 longitude eastand 15 to 65 latitude north. From the received MODIS
data, the proposed system uses the standard MODIS
products of MOD02, MOD03, and MOD09.
The Level 1A, Level 1B, geolocation and cloudmask products and the Higher-level MODIS land, ocean
and atmosphere products are produced by the MODIS
Adaptive Processing System.
MOD02 (Level-1B) is Calibrated Geolocation Data
Set. The Level 1B data set contains calibrated andgeolocated at-aperture radiances for 36 bands generated
form MODIS Level 1A sensor counts. The radiances are
in W/(m2-µm-sr). In addition, reflectance may be
determined for solar reflective bands (bands 1-19, 26)
through knowledge of the solar irradiance.
MOD03 is Geolocation Data Set. The MODIS
Geolocation product contains geodetic coordinates, andsolar and satellite zenith, and azimuth angle for each 1
km sample. These data are provided as a companion
data set to the Level 1B calibrated radiances and theLevel 2 data sets to enable further processing.
MODIS Surface Reflectance product (MOD09) is
computed from the MODIS Level 1B land bands 1 to 7
at 500m resolution. The product is an estimate of thesurface spectral reflectance of each band as it would
have been measured at ground level if there were no
atmospheric scattering or absorption.

Compute nodes

Web serverIII. COMPOSITE PROCESSING

Database
Figure 1 System configuration

1. Overview
When analyzing large area satellite data, one
common problem is noise caused by cloud interference.
In order to remove the interference of clouds, thestandard solution is to create a composite data of thesame regions during a selected time span, and to patch
together data not covered by clouds to create a clear
image.
The selected time span for the composite processdepends on research objective and region. For example,
when analyzing the seasonal variation of vegetation, it
is common to use a short 8 day composite. However,when the composite time span is short, there are lessdata to select from, which leads to a higher possibilityof cloud interference remaining in the composite data.

2. MODIS
A key instrument aboard Terra (EOS AM) and Aqua
(EOS PM), two satellites of the NASA-centeredinternational Earth Observing System project is MODIS(Moderate Resolution Imaging Spectroradiometer).
Terra was launched on December 18, 1999, and Aqua
was launched on May 4, 2002. MODIS has a viewing
swath width of 2,330 km, with a fast orbit covering theentire Earth surface every one to two days. Terra’s orbit
crosses the equator from north to south in the morning,
and aqua passes the equator from north to south in theafternoon. The MODIS detectors measure 36 spectral
bands between 0.405 and 14.385µm, acquiring the dataat three spatial resolutions, 250m, 500m, and 1,000m,
depending on the spectral bands.[4]©ISAROB 2011

440

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

By piece processing the original large satellite image
data, out-of-range data can be eliminated, reducing
database size.
Satellite data analysis applications which use thesatellite data will also have less data to analyze if outof-range data is initially removed, leading to improved
analysis throughput.
When analyzing satellite data, it is common to focus
on specific target areas. For example in the case ofJapan, it would be common for researchers to not use allof received MODIS data but limit the region to, forexample, northern region (Hokkaido), northeast region(Tohoku), or Central region (Kanto). Piece processing isefficient for selecting limited regions from the total
MODIS reception area, making it easier for researchers
to select the exact regions necessary, as well as
decreasing the analysis computational cost.
The same merit applies to composite data processing.
Researchers requiring composite data can request for
the exact regions and time span, and with the reduced
process area, improve the request turnaround time ofon-the-fly composite data processing.

Figure 2 Image of composite processing

2. Effect
Traditionally, it has been the responsibility of theresearcher analyzing the satellite data to process thelarge satellite data and create a composite image forindividual research use. Creating a web service toprovide systematic composite data processing usinglarge scale computing resource over a parallel
distributed configuration, will be valuable forresearchers.

IV. PIECE PROCESSING
1. Overview
Satellite image data taken every day from slightly
different orbits must be adjusted using some a map
projection (e.g. WGS84). For MODIS data, the sensor
data (MOD02 or MOD09) are resampled using map
coordinate information (MOD03). When a particular
map coordinate is out-of-bounds for a given satellite
sensor scan, the pixel data for the particular coordinate
will be defined an unused parameter value (65000 for
MOD02, -10000 for MOD09), as seen in the black areain Figure 3. This leads to large portions of the processed
image file being wasted to hold unused data, leading to
unnecessary load on the database. Furthermore, analysis
applications must process the large image data holding
unnecessary data, leading to significant deterioration of
calculation time.

Figure 3 Image of piece processing

V. CONCLUSION
In this paper, a composite processing and piece
processing algorithm for effective satellite data handlingis proposed. The implemented system is currently underevaluation. It is planned to apply the proposed data
preprocess to all of the archived satellite data and open
for use by users. For future works, improvement ofthroughput performance for real-time composite
processing will be researched.

2. Effect
In this research, we define “piece processing” as theprocess of separating a large satellite image data into
small squares or “pieces” of equal latitude and longitude
sizes.

©ISAROB 2011

ACKNOWLEDGEMENT
This research is supported by the Research projectof Tokyo University of Information Sciences for the441

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011sustainable development of economic and social
structure dependent on the environment in eastern Asia.

REFERENCES[1] Eiji Nunohiro, Jong Geol Park, Kenneth J. Mac
kin, et al, “Development of satellite data analysis system”, Proceedings of the 5th International Confere
nce on Information (2009), pp.143-146[2] Eiji Nunohiro, Kei Katayama, Kenneth J. Mackin, et al, “Forest and Field Fire Search System using MODIS Data”, Journal of Advanced Computation
al Intelligence and Intelligent Informatics (JACⅢ 2007), vol.11 No.8, pp.1043 1048
[3] Kotaro Matsushita, Kei Katayama, Kenneth J.
Mackin, et al, “Development of Satellite Image Sea
rching using Distributed Genetic Algorithm with No
rmalized Correlation”, The twelfth International Sym
posium on Artificial Life and Robotics, Proceeding
Index GS12-7, (AROB 12th’ 2007)[4] National Aeronautics and Space Administration(NASA), MODIS Web, http://modis.gsfc.nasa.gov/©ISAROB 2011

442

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Application of Neural Network Swarm Optimization
for Paddy Field Classification from Remote Sensing Data
Kazuma Mori*, Takashi Yamaguchi*, Jong Geol Park** and Kenneth J. Mackin** Department of Information Systems, Tokyo University of Information Sciences,
4-1Onaridai, Wakaba-ku, Chiba, 265-8501 Japan** Department of Environmental Information, Tokyo University of Information Science,
4-1Onaridai, Wakaba-ku, Chiba, 265-8501 Japan(Tel:81-43-236-1329; Fax:81-43-236-1329)(s07122km@edu.tuis.ac.jp)

Abstract: Monitoring changes in paddy area is important for economic and environment research since rice is staple
food in Asia, and paddy agriculture is a major cropping system in Asia. Recently, remote sensing is used actively toobserve the change of paddy area. However, monitoring paddy area by remote sensing is difficult due to the temporalchanges of paddy and difference of spatiotemporal characteristics of paddy agriculture between countries or regions. In
our previous research using MLP and spatiotemporal satellite sensor data, the proposed classifier yielded 90.8% correctclassification rate. In this paper, we proposed a cooperative learning method using PSO as the global search method and
MLP as the local search method in order to improve the classification accuracy for practical use.
Keywords: multi-layered perceptron, particle swarm optimization, cooperative learning, classification, remote sensing.

Thus the teaching signal in this research is modified tousing paddy area ratio. Through computer simulation,
we investigated the effectiveness of the proposedcooperative learning method and application to paddy
field classification using moderate resolution sensor
data.

I. INTRODUCTION
Monitoring changes in paddy area is important foreconomic and environment research since rice is staple
food in Asia, and paddy agriculture is thus a major
cropping system in Asia. Recently, remote sensing is
actively used to observe the change of paddy area.
However, monitoring paddy area by remote sensing isdifficult due to the temporal changes of paddy anddifference of spatiotemporal characteristics of paddy
agriculture between countries or regions. To solve thisproblem, we proposed creating a paddy field classifier
by machine learning. Our aim is to automatically create
localized classifiers for targeted countries and regions.
In our previous research using multi-layered
perceptron (MLP) and spatiotemporal satellite sensor
data, the proposed classifier yielded 90.8% correctclassification rate [1]. However, it is necessary tofurther improve accuracy for practical use. One of the
known weaknesses of MLP training is the probability offalling into the local optimum causing decline ofaccuracy. For solving this problem, we proposed acooperative learning method using particle swarm
optimization (PSO) for introducing perturbation to the
local search of multiple MLPs in order to improve theaccuracy.
In this paper, we applied the proposed artificial
neural networks to paddy field classification using
moderate resolution sensor data that includes
spatiotemporal information. In previous research [2],the teaching signal was either paddy or non-paddy forthe input, making it difficult to assess the accuracy of
training. On the other hand, promising results for paddy
estimation had been reported using paddy area ratio [3].

©ISAROB 2011

II. METHOD
1. Neural Network Swarm Optimization
In this paper, we proposed a cooperative learning
method in multiple MLPs. The proposed method repeats
local search and global search in order to find a global
optimum in a serialized weight vector space of MLP.
Behavior of the proposed method is shown in Figure
1. First, each MLP (acting as a particle in PSO) searches
the neighborhood from its initial position by
back-propagation. The search space is a serialized
weight vector space of MLP (A in Figure 1). After the
back-propagation process, weight values are changed by
global search mechanism of PSO (B in Figure 1). The
search is repeated from the neighborhood search using
the new initial position (updating by the previous PSO
search) for each MLP particle (C in Figure 1).
Figure 2 shows a flow chart of the proposed method.
Training of MLP is started after the initialization ofweight values of each MLP. Training of each MLP is
continued to time step τ where τ is a parameter defining
the training time step of MLP. Training of PSO is started
and continued to time step σ in training process of MLP
where σ is a parameter that decide training time step of
PSO. Training of MLP and PSO are repeated till the
final termination condition is satisfied.

443

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

(A) Local search by MLP

Finally, the MLP classifier with the global best
weight values is selected as the final training result.
The detailed algorithm is described below.
2. Multi-Layered Perceptron
MLP is a type of artificial neural network that canapproximate complex functions by machine leaning [4].
In this research a standard three-layered MLP isused. The structure of three-layered MLP is shown in
Figure 3.

(B) Global search by PSO(C) Local search by MLP again
Figure 3. Structure of three-layered MLP
MLP consists of an input layer, a hidden layer, andan output layer. The input layer has h neurons and a bias
neuron. The hidden layer has l neurons and a bias
neuron. The output layer has m neurons. Each neuron isconnected with every neuron in the next layer and eachconnection has a weight value.
Let xd = {xd1,xd2,…,xdh} be the dth input vector and td= {td1,td2,…,tdm} be the dth teaching signal vector whered = 1,2,…,n and n is the number of samples in training
data set. Let S = {sd} be the training set where sd =
{xd,td} for the dth sample.
When input signal xi is given, the output of jthhidden layer neuron zj is calculated by xi and wji wherewji is the weight value between ith input layer neuron
and jth hidden layer neuron by equation (1)Figure 1. Outline of the proposed methodhz j = f (∑ w ji xi )

(1)

i =0

where i = 0 , 1 , … , h ; j = 0 , 1 , … , l ; f is t h eactivation function and x0 is the output of bias neuron
in the input layer and always outputs 1.
The output of kth output layer neuron yk is calculated
by zj and wkj where wkj is the weight value between kth
output layer neuron and jth hidden layer neuron byequation (2)lyk = f (∑ wkj z j )

(2)

j =0

where k = 1,2,…,m and z0 is the output of bias neuron in
the hidden layer and always outputs 1.

Figure 2. Flow chart of the proposed method©ISAROB 2011444

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

For the activation function, a sigmoid function iscommonly used. The sigmoid function is defined byequation (3)f (X ) =1
1 + exp(− aX )u, u is a random number vector from 0 to φ1 or φ2, ω is
× is Hadamard product.
momentum rate and ○Let index set of particle be P = {1,2,…,q}. In
updating local best step, if fitness value of pth particle
in generation u is better than fitness value of pth particle
local best in generation u-1, then plup is set to rup, else
plup is set to plu-1p. In updating global best step, index
for gb P is update to index that fitness value of local
best is best in all particle in generation u.
In addition, search space for PSO is a weight vector
space of MLP where the weight vector is serialized
weight values in MLP. Thus current position r isdefined as follow in the proposed method.
r = {w10 ,..., w ji ,..., wlh , w10 ,..., wkj ,..., wml } (9)(3)

where a is a user defined parameter.
In the proposed method, back-propagation is used
for the training method [4]. Back-propagation is basedon a steepest descent method. Updating weight values
by back-propagation is shown in equation (4) andequation (5).
Let ∆wji(s) be the weight modification of ith input
layer neuron and jth hidden layer neuron at time step s,and ∆wkj(s) be weight modification of kth output layer
neuron and jth hidden layer neuron at time step s∆w ji ( s ) = −λ ⋅∂E+ µ ⋅ ∆w ji ( s − 1)
∂w ji

(4)

∆wkj ( s ) = −λ ⋅∂E+ µ ⋅ ∆wkj ( s − 1)
∂wkj

(5)

The best position in search space is the position
where mean square error between the output and theactual answer for each data is smallest. In other words
the fitness function of PSO is error function in MLP.

III. EXPERIMENT

where λ is leaning rate, µ is momentum rate and E is themean square error.
E is derived by the error between output yk andteaching signal tdk as shown in equation (6)E=

1 n m d∑∑ (tk − ykd ) 2
2n d =1 k =1

1. Application to MODIS Data
In this research, we used MODIS (Moderate
Resolution Imaging Spectoradiometer) satellite datacollected at Tokyo University of Information Sciences,
Japan, for remote sensing data. MODIS data can be
widely used for the remote sensing in land, sea andatmosphere research since MODIS data contains 36
bands. In addition, MODIS observes the same area inhigh frequency because MODIS satellites have a fast
orbit compared to high-resolution satellites. Therefore
MODIS is effective for monitoring annual cycle of apaddy area.
In this research, spatiotemporal MODIS band 1 (red),band 2 (infra red) and band 6 (short wave infra red) data
were used similar to previous research [2].
In previous research [2], input signal consisting ofmonthly band data from January to December (3 bands
x 12 months inputs) was used. The band data of eachmonth was derived from 1-month composite MODIS
sensor data of 500m resolution. For the teaching signal,
we constructed a land-truth data derived from 1km
resolution mesh land-use data from digital national land
information provided by the Japanese Ministry of Land,
Infrastructure, Transport and Tourism (JMLIT). Thisvalue is paddy or non-paddy {1,0}.
In this research, the input signal consists of 4 pixels
refraction values of 500m resolution at thecorresponding 1km square (4 pixels x 3 bands x 12
months inputs). The value of input is normalized to[0,1]. 4 Neighboring 500m MODIS pixels are merged toa 1km square. The 1km square merged teaching signal
is the estimated paddy area ratio. The value of teaching
signal is [0,1]. Therefore, it is difficult to approximate
for an unknown function in comparison with past
research data, and a difference of the capability forclassification appeared.

(6)

where tdk is the teaching signal for the output of kth
output layer neuron in dth training data and ydk is theoutput of kth output layer neuron in dth training data.
3. Cooperative Learning Method using Particle
Swarm Optimization
For the global search method, PSO, a type of
population-based search algorithm in evolutionary
computation [5], was used in the proposed method. PSO
searches for an optimum value in a vector space using
multiple particles, similar to other population based
search algorithm. For this research, PSO was selected infavor of other population based search methods such as
GA, with the expectation that the fast conversion rate of
PSO would be advantageous in combination with MLP
since the learning in multiple MLPs requires longcalculation time.
The particle searches for an optimum by updating
current position according to particle velocity. In
generation u, updating pth particle velocity vup andcurrent position rup is shown as equation (7) andequation (8)vup+1 ←ω⋅ vup +u(0,φ1) ⊗(plup −rpu ) +u(0,φ2 ) ⊗(plugb −rpu ) (7)r pu +1 ← r pu + v up+1

(8)

where p = 1,2,…,q; q is the number of particles, plup is
the position of best fitness for the pth particle (local
best) vector in generation u, plugb is position of best
fitness in all particles (global best) vector in generation©ISAROB 2011445

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

In addition, the answer value of the test data and theoutput of classifier are divided into paddy or non-paddy
by a threshold θ for classification. The threshold θ isdefined by equation (10)

1 if o > θg (o) = 
0 otherwize

IV. CONCLUSION
In this paper, we proposed applying a cooperative
learning method where multiple MLP are used as PSO
particles, and weight values of MLP is affected withtraining of PSO for improving the classifier accuracy.
Experiment result showed the accuracy of proposed
NNSO improved in comparison with previous NNSO.
However, the dimensions of the weight values wereincreased. The number of the neuron in the input layer
and the hidden layer were fixed in the experiment, andunnecessary weight training may be exists in theprocess of training. We plan to investigate the structure
which can optimize the number of the dimensions
automatically.
In addition, the proposed NNSO yields higher
accuracy than the MLP-bagging. The proposed method
creates a classifier using only 1 MLP with weight values
set to the global best, and is advantageous in that the
proposed method requires less memory in comparison
with commonly used bagging method.
In this research, the classifier output was a binary
value of paddy or non-paddy, but the accuracy was
improved by using a continuous value of paddy area
ration for classifier training. The experiment results
showed an advantage of the proposed method in
approximation capability. For future works, we plan toinvestigate the methods to apply the proposed method toestimate paddy area ratio for a given input area.

(10)

where o is yb that is the final output in bth test data or tb
is the answer value for the output in bth test data, b =1,2,…,c and c is the number of samples in test data set.
In this experiment, the parameters were defined asfollows. Leaning rate of MLP λ is set to 0.1, momentum
rate of MLP µ is set to 0.05, input size h is set to 144,
hidden size l is set to 144, output size m is set to 1,
particle size q is set to 50, momentum rate of PSO ω is
set to 0.9, upper limit of random number φ1 and φ2 is set
to 2.0, training time step of MLP τ is set to the numberof sample x 10, training time step of PSO σ is set to 10,
threshold θ is set to 0.4. In addition, the training of
proposed method ends when training loop of MLP and
PSO are repeated 50 times. The training step and thetest step were repeated 10 times, each time changing theinitial values and training data and test data, and theaverage accuracy for the 10 trials were used to evaluate
the calcification accuracy.
2. Experiment Result
In this experiment, we evaluated classification
accuracy by using the proposed paddy classifier. For
evaluating classification accuracy, data set was divided
into 2 disjoint subsets, training data set and test data set,by using random sampling from the north region of
Chiba, Japan. The number of test data set was 10% of
the number of training data set.
Table 1 shows classification accuracy of a) proposed
paddy classifier with input signal of 1km square(proposed NNSO), b) previous method with input signal
consisting of monthly band data from January to
December (previous NNSO), c) and MLP using bagging
with input signal of 1km square (MLP-Bagging).
Experiment result showed improved an accuracy of
proposed NNSO compared with previous NNSO. In
addition, total accuracy of proposed NNSO improved
by about 0.3% compared to total accuracy of the
MLP-bagging. Notably, as much as 9% improvement
over paddy accuracy was seen. In this experiment, theproblem is more difficult since the teaching signal is acontinuous value. The experimental results showed thatthe proposed method has better function approximation
capability for continuous values.

ACKNOWLEDGMENTS
This research was supported by the Research projectof Tokyo University of Information Sciences for thesustainable development of economic and social
structure dependent on the environment in eastern Asia.

REFERENCES[1] T.Yamaguchi, K.Kishida, E.Nunohiro, J.G.Park,
K.J.Mackin, K.Hara, K.Matsushita and I.Harada(2010),
Artificial Neural Networks Paddy Field Classifier using
Spatiotemporal Remote Sensing Data. Artificial Lifeand Robotics, vol.15, pp.221-224[2] K.Mori, T.Yamaguchi, J.G.Park and K.J.
Mackin(2010), Cooperative Learning Artificial Neural
Networks using Particle Swarm Optimization for Paddy
Field Classification from Remote Sensing Data (inJapanese). Proceedings of the Fuzzy System
Symposium, vol.26, pp.642-645[3] W.Takeuchi and Y.Yasuoka(2006), Mapping of
fractional coverage of paddy fields over China using
MODIS 250m data (in Japanese). Journal of The
Remote Sensing Society of Japan, vol.26, pp.213-223[4] D.E.Rumelhart, G.E.Hinton and R.J.Williams(1986),
Learning representations by backpropagating errors.
Nature, vol.323, pp.533-536[5] R.Poli, J.Kennedy and T.Blackwell(2007), Particle
swarm optimization An overview. Swarm Intelligence,
vol.1, no.1, pp.33-57, Springer

Table 1. The comparison of classification accuracy
in proposed paddy classifier.
Paddy Non-Paddy
Total
Proposed NNSO
0.889
0.751
0.946
Previous NNSO
0.867
0.752
0.907
MLP-Bagging
0.886
0.659
0.932

©ISAROB 2011

446

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A Systemic Payoff in a Self-Repairing Network
Masahiro Tokumitsu1 and Yoshiteru Ishida2
1

Department of Electrical and Information Engineering,2
Department of Computer Science and Engineering,
Toyohashi University of Technology, Tempakucho, Toyohashi, Aichi, 441-8580, Japan(Tel : +81-53-244-6895; Fax : +81-53-244-6873)(tokumitsu@sys.cs.tut.ac.jp, ishida@cs.tut.ac.jp)

Abstract: Cooperation among agents is a crucial problem in autonomous distributed systems composed of selfish
agents pursuing their own profits. An earlier study of a self-repairing network revealed that a systemic payoff enabled
to make the selfish agents cooperate with other agents. The systemic payoff is a payoff mechanism that sums up not
only the agent’s own payoff but also neighborhood’s payoff. In the systemic payoff, the distance effect between theagents has not been studied yet. This paper considers the systemic payoff that involves the distance effect among theagents. We study the effectiveness of the proposed mechanism for the network performance by computer simulations.
Keywords: Autonomous distributed systems, self-repairing network, selfish agents, kin selection, game theory.

other agents in the self-repairing network. Moreover
those studies reported that the systemic payoff wassimilar to kin selection [9]. The systemic payoff sums
up not only its own payoff but also the neighborhood’s
payoff connected. Finally, those studies concluded thatthe agents with the systemic payoff improved the
network performance.
The earlier studies [7, 8] of the self-repairing
network have not deeply mentioned a distance effectamong the agents in the systemic payoff. In an
information network, the agents are connectedaccording to a network structure and distance. This
paper considers that a cooperation factor needs toinclude the distance effect. The earlier studies have notconsidered the network performance caused by the
distance effect of the systemic payoff.
This paper deals with the systemic payoff involving
the distance effect between the agents in the selfrepairing network. In this assumption, the agents areconnected with any distance and connection weight. A
connection weight of the systemic payoff represents
strength of the relationship among the agents and isdifferent according to the distance among them. We
study the performance of the proposed systemic payoffby computer simulations.

I. INTRODUCTION
Autonomous distributed systems are composed ofselfish agents pursuing their own profits. In theautonomous distributed systems, selfish agents need tocooperate with other agents because collective selfish
acts of the selfish agents would lead the systems toabsorbed states. Studies on selfish routing reported that
if agents route their traffic selfishly then the network
would show a poor performance [1, 2].
Cooperation is a crucial issue in the autonomous
distributed systems. Cooperation mechanisms forpreventing the worst performance are investigated incongestion games. The studies [3, 4] introduced thecooperation factor to the agents in which the factor
elicits the altruistic behaviors by a tunable parameter.
In evolutionary game theory, several studies for theevolution of cooperation have been investigated by apayoff mechanism and related to the present paper. An
earlier [5] investigates effects of a neighborhood sizeand connectivity in spatial games because spatial
structures affect cooperation among individuals.
In the self-repairing network, cooperation is also an
important problem in order to maintain the agents [6, 7].
The self-repairing network is a model in which theagents repair other agents mutually [8]. In the selfrepairing network, to bring out cooperation among theagents has been studied using spatial strategies and thepayoff mechanism.
Earlier studies [7, 8] revealed that the systemic
payoff is capable of making the agents cooperate with©ISAROB 2011

II. MODEL
1. Self-Repairing Network Model
We model the self-repairing network by a Spatial
Prisoner’s Dilemma [5, 6, 7]. The agents make theirdecision either repair or not repair. The abnormal agents

447

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Success
RepairNormalNormalAbnormal

FailureAbnormal(a) Repair by normal agents
Success
RepairAbnormalFig. 2. Simple illustration of the systemic payoffwith the distance effect. The agents with the samecolor are located at the same radius from the central(yellow color) agent.

NormalNormalFailureAbnormalassume that the normal agents become abnormal by aspontaneous failure. We denote the failure rate by λ .
Each agent has the maximum available resources
Rmax . The agents consume their resources Rc forevery repairing. The agents assign their remained
resources as the available resources to their own task.
The remained resources of the abnormal agents arealways evaluated as empty resources because they do
not work well due to their state.
The agents update their strategies to the strategies
that earn the highest payoff in the eight neighbors. The
strategy update of the agents is done with strategy
update cycle S . The agents sum up their payoff by thesystemic payoff mechanism in the agent simulations.
The strategy update error occurs when the agents update
their strategies. The strategy update errors make theagents switch to other strategies. This mechanism
contributes to prevent the local minima of the network
performance. We denote the rate of the strategy update
error by µ .

(b) Repair by abnormal agents
Fig. 1 Repair scheme of the self-repairing network.
The blue and red circles indicate the normal andabnormal agents respectively.
will spread if no agents repair other agents. However,
the agents are reluctant to consume their resources. Thissituation can be modeled by the Spatial Prisoner’s
Dilemma.
Each agent has binary states: normal or abnormal.
We assume that the agents cannot know their own states
and the states of other agents. Each agent is placed ateach cell in a square lattice network. Basically,
interactions of the agents are restricted in eight neighbor
agents(Mooreneighborhood).
The
Moore
neighborhood of this assumption corresponds to a radiusr = 1 for the interactions. The agents will repair otheragents in the Moore neighborhood. However, the agentsare able to communicate for the payoff with the eightneighbors besides the other agents outside of the Moore
neighborhood ( r = 1 ).
Each agent determines the next action: repair or not
repair. The agents determine their action based on thestrategies. The agents choose either All-C or All-D
strategies. The All-C strategy always repairs otheragents, while the All-D does not repair.
Fig. 1 shows a repair scheme of the self-repairing
network. The repair success rate is different by thestates of the agents. We denote the repair success rate ofand β
the normal and abnormal agents by α
respectively. We assume to simplify the model that the
repair by the normal agents is always successful( α = 1 ). The repaired agent becomes normal if therepairing by abnormal agents is successful ( ! = 0.1 )otherwise the repaired agents become abnormal. We©ISAROB 2011

2. Systemic Payoff
This paper considers the systemic payoff that involves
the distance effect of the relationship between the agents.
Environmental effects of the neighbor agents reflect tothe agents by their payoff. The agents collectinformation from the weighted payoff of the otheragents. The weight of the connection strength between
the agents is different according to the distance between
them. The payoffs of the agents are weighted withstrength of the connection between the agents. The
agents will obtain not only local information (neighbor
agents) but also global information (outer of the nearest
neighbor agents) from the payoff. The systemic payoff
allows gathering local and global information from theneighbor agents through their payoff.

448

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011(a) r = 1,2(b) r = 2,3Fig. 3. Averaged resources distribution where the connection weights of each radius are varied. The color bars
indicate the averaged resources of each cell for the connection weight pairs.
In the systemic payoff, how much the agents involve
the payoff of neighbor agents is a crucial issue becauseof the neighborhood’s payoff affects the decision
making of the agents. This paper defines the systemic
payoff in the square lattice network (Fig. 2). Let aij
and R(aij ) denote the agent and its payoff located at(i, j ) in the square lattice network. We denote theconnection weight of the systemic payoff from the agent
with distance r by wr . In this paper, we assume thatthe radius of the systemic payoff for involving theneighborhood’s payoff is less equal than five. Let
denote the summed payoff of the agent aij by
Rt (aij ) . Let denote the set of the agents on the radiusr from the agent aij by Ar (aij ) . Therefore, thetotal payoff of the agents aij is expressed as follows:

Rt (aij ) = R(aij ) + "" w R(a )rijTable 1. Parameters for computer simulations.

(1)

combination of the radiuses. In simulations, we give theagents two pairs of the connection weights
corresponding to each radius. Those pairs of theconnection weights are r =1, 2 and r = 2,3 . Wechange the connection weight of each radius assimulation parameters from 0.0 to 1.0.
Fig. 3 shows the averaged resources distribution for
the fixed radius pairs. The averaged resources are
sufficiently kept in both cases where either connection
weight is larger than 0.4. Fig.3 (b) shows that the
averaged resources drop a little than the case that thepayoff interaction is restricted to r =1, 2 (Fig. 3 (a)).
The connection weight between the agents should be
adjusted strongly, because the weak relationshipbetween them would cause the bad performance.

r axy !Ar (aij )The total payoff of the agents will change by the
combination of the connection weights of the systemic
payoff. We investigate the relationship between theconnection weight of the systemic payoff and the
network performance.

III. SIMULATIONS
We obtain the simulation results from computer
simulations, and then average the data by the number of
the trial counts. The computer simulations use theparameters show in Table. 1. We assume that the agentschange their connection weight corresponding to theradius of that agent in the calculation of the systemic
payoff.

2. Performance for Single Radius
We evaluate the network performance for the systemic
payoff gathering the payoff from the agents on single
radius. We change the connection weight wr for the

1. Performance for Connection Weight Pair
For the first simulations, we consider that the agents
gather their own payoff from the agents located on the©ISAROB 2011

449

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

VI. CONCLUSIONS
We investigated the systemic payoff involving the
distance effect that changes the connection weighaccording to the distance between the agents. Our
simulations showed that the sufficient large connection
weight could lead the agents to cooperation, and then
network performance improves. Furthermore, the agentscan perform well where the agents allow only obtaining
local information of the neighbor agents. We consider
that the systemic payoff involving the distance effectwould support for designing and constructing theautonomous distributed systems.

Fig. 4 Averaged resources for the connection weight
where the agents obtain from the other agents on
single radius.
radius r of the agents. The connection weights
excluding the radius r are set to zero.
In Fig. 4, the network performance is worse for all
radiuses where the connection weight is less than 0.4.
However, the network performance improves as theconnection weight grows to the large value. The impactof the connection weight appears when it is set to high,and then the agents choose to the repair action. Fromthese results, the network performance decreases as the
distance between the agents becomes longer.

ACKNOWLEGEMENT
This work was supported by Global COE Program“Frontiers of Intelligent Sensing,” from the ministry of
Education, Culture, Sports, Science and Technology.

REFERENCES[1] Roughgarden T, Tardos E (2002) How bad is selfish
routing? Journal of the ACM 49: 236–259.
[2] Koutsoupias E, Papadimitriou CH (2009) Worstcase equilibria. Computer Science Review 3: 65-69.
[3] Chen PA, Kempe D (2008) Altruism, selfishness,and spite in traffic routing. In: EC ’08: Proceedings of
the 9th ACM conference on Electronic commerce. New
York, NY, USA: ACM, pp. 140–149.
[4] Azad AP, Altman E, Azouzi RE (2009) Fromaltruism to non-cooperation in routing games. In:
Proceedings of Networking and Electronic Commerce
Research Conference.
[5] Ifti M, Killingback T, Doebeli M (2004) Effects ofneighbourhood size and connectivity on the spatial
continuous prisoner’s dilemma. Journal of Theoretical
Biology 231: 97 - 106.
[6] Oohashi M, Ishida Y (2007) A Game Theoretic
Approach to Regulating Mutual Repairing in a SelfRepairing Network, Springer, Netherlands, chapter
Innovative Algorithms and Techniques in Automation,
Industrial Electronics and Telecommunications. pp.
281–286.
[7] Ishida Y, Tokumitsu M (2008) Asymmetricinteractions between cooperators and defectors forcontrolling self-repairing. Lecture Notes in Computer
Science 5179: 440–447.
[8] Ishida Y (2005) A critical phenomenon in a selfrepair network by mutual copying. Lecture Notes in
Computer Science 3682: 86–92.
[9] Brembs B (2001) Encyclopedia of Genetics,
Academic Press, chapter Hamilton’s Theory. pp. 906–
910.

IV. DISCUSSIONSAgent simulations revealed that the connection weight
representing the degree of the relationship between theagents is the dominant parameter for the network
performance. From the results, the sufficient large
connection weight between the agents can keep the highaveraged resources. We think that the strong
relationship between the agents in the systemic payoff
could support the other agents by repairing because of
the sufficient large connection weight could elicit
cooperation among the agents.
In computer network, agents interconnect with eachother with a distance and structures. It is possible thatmessages from neighbors to an agent will lose by link
failures. In that case, the agents need to determine theirdecisions based on the neighborhood’s payoff. The
agent simulations demonstrate that the agents are able tokeep the high averaged resources where either
connection weight of the pairs is small. These results
imply that the systemic payoff involving the distanceeffect would impact to the decisions of the agents
appropriately and have the robustness for theenvironmental changes.

©ISAROB 2011

450

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Effects of a membrane formation in Spatial Prisoner’s Dilemma
Yuji Katsumata and Yoshiteru Ishida
Department of Electronic and Information Engineering,
Toyohashi University of Technology,
Tempaku, Toyohashi, Aichi, 441-8580, Japan(Tel : +81-532-44-6577)(Email katsumata@sys.cs.tut.ac.jp)

Abstract: A cooperative relationship has been developed among individuals. However, an altruistic behavior has little advantage against selfish behavior in the sense of rational terms. Each individual chooses a selfish behavior pursuing their own payoff then the altruistic behavior will vanish. Earlier studies proposed the mechanisms based on game theory w
hich explains the problem of the difference between the theoretical prediction and observation. Furthermore, those studi
es also considered the mechanisms of protecting a cooperators cluster in a spatial prisoner’s dilemma involving spatial s
trategies and a spatial generosity, although did not analyze rigorously effects of the membrane for the cooperators. In th
is paper, we report the quantitative effect of membrane for protecting the cooperators from the exploitation of the defectors.
Keywords: Spatial Prisoner’s Dilemma, Spatial Strategy, cooperation, membrane formation

I. INTRODUCTION

II. MODEL

Cooperation is basic components of animals andsociety. And a cooperative relationship has beendeveloped among individuals. However, an altruistic
behavior has little advantage against a selfish behavior
in the sense of rational terms. In stead of an altruistic
behavior, each individual should choose a selfish
behavior pursuing their profit. The mechanisms areproposed by earlier studies for explain the deference
between the theoretical consideration and observational
results [2, 3]. The some proposed mechanisms are basedon the game theory, especially Prisoner’s Dilemma
model [4-7].
In the model we proposed that is involving spatial
strategy and spatial generosity [8], we observed
membrane formation as a mechanism for protects
cooperators form invasion of defectors [9]. Furthermore,
we revealed the condition for constructing membrane.
The constructing membrane only depends upon thespatial generosity k [10]. However, we have notanalyzed rigorously effects of the membrane on thecooperators.
In this paper, we report the quantitative effect ofmembrane on protecting the cooperators form the
exploitation of the defection.

©ISAROB 2011

1. Prisoner’s Dilemma
The Prisoner’s dilemma (PD) is a fundamental
model of game theory. It played just once by two
players have two behavioral options: C (Cooperation),or D (defection). The players decide behavior
simultaneously whether to cooperate or to defect. If
both players cooperate, both players receive payoff R(reward), whereas if both players defect, both players
receive payoff P (punishment). If a player cooperates
and an opponent player defects, the cooperator receives
payoff S (sucker) and the defector receives payoff T(temptation) where the payoffs must satisfy T > R > P >S. If an opponent player chooses behavior whatevercooperation or defection, the other should choose
defection, because it is better than choses cooperation.
Therefore, the both players always choose defection andreceive payoff P lower than that when both choose
cooperation.
The Iterated Prisoner’s Dilemma (IPD) is temporal
expansion model of PD. In IPD, PD is carried out
repeatedly. Many strategies have been proposed in IPD.
Axelrod reported tit-for-tat (TFT) strategy [1]. In
Axelrod’s round-robin tournaments, TFT strategy was
the best strategy. TFT strategy is consists in playing C in
the first round and from then on chose action whateverchosen by other player in the previous round. TFT
strategy contains temporal generosity as an element.

451

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

2. Spatial Prisoner’s Dilemma
Spatial prisoner’s dilemma is spatio-temporalversion of PD. Our model generalized SPD byintroducing spatial strategy. Each player placed at eachlattice of the two-dimensional lattice. Each player hasan action and a strategy, and receives a score. Spatial
strategy determines the next action dependent on thespatial pattern of actions in the neighbors. Each player
plays PD with the neighbors, and changes its strategy to
the strategy that earns the highest total score among theneighbors. Table 1 is the Payoff Matrix of PD. In our
simulations, R, S, T, and P are respectively set to 1, 0, b
(1 < b < 2, a bias for defectors) and 0 in simulations
below following the Nowak-May’s simulations.
Table 1. The Payoff Matrix of the Prisoner’s Dilem
ma Game. R, S, T, P are payoff for player 1. (1 <b < 2)
Player 2

Player 1

C
D

C

D

1
b

0
0

strategy can be regarded as a spatial version of TFT
where k indicates the spatial version of the generosity.

Fig.1. An example of spatial strategy. In this situation, k (spatial generosity) is 6 and neighborhood is
Moore neighborhood.

III MEMBRANE FORMATION AND MEM
BRANE INDEX
We simulated interaction between All-D (always D)
vs. k-D instead of All-D vs. All-C (Nowak-May’s
simulation). In All-D vs. k-D simulation, we alreadyobserved the membrane as a mechanism for protects
cooperation from invasion of defectors of All-D. The
membrane is composed of only D players of k-D. Fig.2
shows an example of the membrane formation.
Although we can understand that the membrane protects
cooperators from invasion of defectors intuitively, theeffects of the membrane on cooperators have not been
investigated mathematically.

Our SPD model is done in the following way with nplayers.
1. Initial phase: the action and the strategy of eachplayer are determined randomly.
2. Renewal of action: the next action will be
determined by player’s strategy based on theneighbors’ actions and the player’s own action.
3. Calculate score: the score for each player iscalculated by summing up all the scores
received from PD with neighbor players anditself (self-interaction involved to makecompare to the Nowak-May model), and thescore added to the current player’s score.
4. Renewal of strategy: the nest strategy will bechosen from the strategy with the highest scoreamong the neighbors including the player itself.

Fig.2. The membrane formation generated by SPD
simulation. Black cells indicate All-D players. White and gray cells indicate C and D players of k-D.
In this snapshot, k (spatial-generosity) is 6. The C
clusters are covered by the membrane (gray color).
This snapshot shows typical membrane.
Therefore we consider a membrane index toinvestigate quantitative effects of the membrane
formation on cooperators. The membrane index means
how much membranes protect cooperators frominvasion of defectors. The following is definition of theplayer as a membrane.
1. The player is D player of k-D.
2. The player is playing PD game with C player.

3. Spatial Strategy and spatial generosity
The next action will be determined based on thepattern of neighbor’s actions. However, the pattern ofneighbor’s action is a lot. For simplicity, we restrict
ourselves to a “totalistic spatial strategy” that only
depends upon the number of D (defection) of theneighbors, not on their positions. To represent a strategy,
let l be the number of the D action of the neighbors
excluding the player itself. We define k-D strategy thatchose action D if l ≧ k and C otherwise. This k-D©ISAROB 2011452

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 20113.

The interaction between All-D players and C
playersisnothingwithinplayer’s
neighborhood.
In other words, we define the membrane in terms of
the function protecting cooperators from invasion of thedefectors. Fig.3 shows the membrane that satisfied adefinition.
A B C D E
1percentage of the D players of k-D that are effective as amembrane. If the membrane indexes per player are one,all of D players of k-D are action effectively as amembrane.

2
3
4
5
Fig.3. In the Moore neighbors, the center cell (C3player) is a membrane that satisfied a definition. Bl
ack cells indicate All-D players. White and gray ce
lls indicate C and D of k-D. If the immediate left
cell of center (B3 player) was white (Cooperator), t
he center cell did not satisfy a definition of the membrane. Because All-D players and C player (immediate left cell of center cell) will be interaction eac
h other among center cell’s neighbors, the 3rd defin
ition of the membrane could not satisfy.

Fig.4. The time evolution of the average payoff ofthe player C. The 6-D earns highest score among other k-D strategies.

IV SIMULATION
We simulate to investigate quantitative effect ofmembrane on cooperators with the following
parameters list in Table 2.
Table 2. Parameters list for simulations.
Parameter
Description
Value
Name
L×L
Size of lattice
500 × 500
N
Number of players
250,000
T
Number of steps
1000
r
Neighborhood radius
1bBias for defectors of the
1.800001
payoff matrix in Table.1

Fig.5. The time evolution of the membrane indexes
per D player of k-D.

In our simulation, the membrane is formed withincertain scope of k [10].
Fig. 4 plots the time evolution of the average scoreof the C players when the k varies. The 6-D earns
highest score among other k-D strategies. We canconsider that the cooperators construct a cluster if thescore per player is high. Because, if the cooperators
have not constructed a cluster, the cooperators would be
exploited by defectors, and the average payoff of the C
players would become low.
Fig. 5 plots the time evolution of the membrane
indexes divided by the number of D of k-D. It means©ISAROB 2011

Fig.6 The time evolution of frequency of C players
when k varies. The fraction of 6-D is the highest
among this three strategies. The fraction of 5-D isthe lowest, because the 5-D cluster could not expand although the cluster is protected by the membrane.

453

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig.6 plots the time evolution of frequency of C
players. In 5-D vs. All-D simulation, the cooperators
have been remained in a small amount. In 6-D or 7-D vs.
All-D simulations, the cooperators have been remainedin a relative large amount.
We calculate coefficient of correlation between
average score of C players and membrane indexes per
players to investigate effect of the membrane oncooperators.
Table 3 shows the coefficient of correlation between
average score of C players and membrane indexes per D
players of k-D for each All-D vs. k-D simulation.
Table 3. The coefficient of correlation between average score of C players and membrane indexes per
D players of k-D.
k value
Coefficient of correlation
5-0.7119
60.812
70.9549membrane indexes have a strong correlation with theaverage score of C players when certain scope of k. If
the membrane expands, the coefficient of correlation
will become positive; otherwise the coefficient ofcorrelation will become negative.
We investigate an effect of membrane oncooperators when Moore neighbors hood. We
investigated the simulation in the Moore neighborhood;
however we need to investigate the simulation in the
Neumann neighborhood in future to compare bothneighborhoods.

ACKNOWLIDGEMENTS
This work was supported in part by Global COE
Program “Frontiers of Intelligent Sensing” from the
Ministry of Education, Culture, Sports, Science andTechnology, Japan.

REFERENCES[1] Axelod, R (1984), The Evolution of Cooperation,
Basic Books, New York.
[2] Nowak, M.A. and May, R.M. (1992), Evolution
ary games and chaos, Nature, vol.359, 826—829[3] Nowak, M.A. and May, R.M. (1993), The spati
al dilemmas of evolution, Int. J. Bufurc. Chaos, vol.
3, 35—78[4] Nowak, M.A. (2006), Five Rules for the Evolut
ion of Cooperation, Science, vol.314, 1560—1563[5] Ohtsuki, H., Hauert, C., Lieberman, E. and No
wak, M.A. (2006) Nature, vol.441, 502—505[6] Hauert, C. and Doeveli, M. (2004), Spatial stru
cture often inhibits the evolution of cooperation in
the spatial Snowdrift game, Nature, vol.428, 643—6
46[7] McNamara, J.M., Barta, Z. and Houston A.I. (2004), Variation in behavior promotes cooperation in
the Prisoner’s Dilemma game, Nature, vol.428, 745—748[8] Ishida, Y. and Mori, T. (2005), Spatial strategies
in a generalized spatial prisoner’s dilemma, J. Arti
ficial Life and Robotics, vol.9, 139—143[9] Ishida, Y. and Katsumata, Y. (2008), A Note on
Space-Time Interplay through Generosity in a
Membrane Formation with Spatial Prisoner's Dilemma.
KES 2008, vol.5179, 448—455[10] Katsumata, Y. and Ishida, Y. (2008), On a Me
mbrane Formation in a Spatio-Temporally Generaliz
ed Prisoners Dilemma, ACRI 2008, vol.5191, 60—6
6

Consequence of calculation of correlation
coefficient, we get interesting results. In 5-D vs. All-D
simulation, the average score of C players have anegative correlation with the membrane indexes. By
contrast, in the simulations of 6-D and 7-D, the averagescore of C players have a strong positive correlation
with the membrane indexes.
We considered that this consequence is related to theconditions the membrane expands. Only 5-D constructs
membrane that can not expand among three strategies.
Therefore, the amounts of the cooperators do notincrease. Thereby a lot of D players of k-D do notsatisfy the 2nd condition of the definition that themembrane should interact with cooperators. Hence,
although the cooperators construct a cluster and themembrane protects cooperators from defectors, themembrane indexes lower. Because cooperators earnhigh score by a cluster whereas the membrane indexes
lower, the negative correlation occurs.

VI. CONCLUSION
We investigate that membrane formation one of amechanism for protects cooperation from invasion of
defectors. The membrane formation has been reported
in SPD, however the quantitative effects have notinvestigated.
We defined membrane index to investigate themembrane. And we denoted the quantitative effect ofmembrane on cooperators by membrane index. The©ISAROB 2011454

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Extracting probabilistic cellular automata rules from spatio-temporal patterns andanalyzing features of these rules
Takuya Ueda1 and Yoshiteru Ishida2
1
Department of Electronic and Information Engineering2
Department of Computer Science and Engineering
Toyohashi University of Technology, Toyohashi, Aichi, Japan 441-8580
Abstract: This research deals with one of the inverse
problem. It is estimating rules or strategies of generating spatio-temporal patterns which is generated by natural phenomena or social phenomena. We try to consolidate identifying method and evaluation method to clarify
generative mechanism. In this research, mainly, we use
probabilistic cellular automata (PCA) to describe generative mechanism. And we restrict spatio-temporal patterns
to ASEP patterns. In particular, we discuss conservationof mass in ASEP model from spatio-temporal patterns.
Keywords: Probabilistic Cellular Automata (PCA),
Reverse problem, Spatio-temporal patterns, ASEP

1patterns to target more complex mechanisms. The
previous method targets on spatio-temporal patterns
which is generated by one dimensional and straightforward rules of CA. However, we deal with characteristic patterns (e.g. patterns of traﬃc jams, diﬀusion ofmatters and other physical phenomena) which satisfy
conservation of mass. In fact, we succeeded to develop
a method to discover the conservation of mass fromthese patterns.
Section 2 states deﬁnitions and notations used inthis paper. Section 3 represents the method of thegenerative mechanism identiﬁcation in the previous
study, and its problem. Section 4 expands the previous method for patterns of conservation of mass, andapplies to the ASEP model ones.

Introduction

Natural phenomena including physical and biological ones present us inexhaustible amount of spatial
patterns: patterns of ice crystal, cloud, coastal railroad, forest ﬁre, leaf arrangement, shells, and butterﬂies, to name only a few. Social phenomena also generate spatial patterns (e.g. traﬃc jams). They are a
few compared to natural one.
Constructing physical or mathematical models are
known as the way of research for understanding thegenerative mechanisms of these spatial patterns. On
the other hand, Richards studied extracting cellular
automaton rules directly from experimental data [1].
They dealt with spatial patterns of dendrites formed
by N H4 Br. They searched the space of rules which
is a set of probabilistic CA (PCA) rules as possiblemodels for spatio-temporal patterns, with a learningalgorithm. Ichise proposed a general and theoretical method for identifying a generative mechanism of
spatio-temporal patterns in CA frameworks [2]. They
restricted rules to one dimensional and straightforward
ones. Hence Richards’s and Ichise’s researches are oneof the reverse problems for traditional ones.
In this paper we expand the Ichise’s method which
can identify generative mechanism of spatio-temporal©ISAROB 2011

2

Definitions and Notations

After von Neumann used cellular automata (CA) in
his designing self-reproducing automata [3], not only
deterministic cellular automata (DCA) (e.g. [4, 5, 6])
but also probabilistic cellular automata (PCA) (e.g.
[7]) have been studied extensively. Cellular automaton consists of cells arranged in a d-dimensional lattice where d is a natural number. Each cell is an automaton which has a certain number of states; whose
inputs are the state of neighbor cells; and the output
is the state of the cell itself. In this paper, we restrict
ourselves to the case of binary state: 0 and 1 and onedimensional lattice with periodic boundary condition
where each cell has two neighbor cells: right and left.
sti denotes the state of the cell i at the time step t andits state at the next time step deﬁned in equation (1).
st+1
= f (nti )i(1)

nti is states of the neighbor cells of the cell i at time
step t and deﬁned in equation (2) with neighborhood
radius r.
nti = (sti−r , · · · , sti , · · · , sti+r )

455

(2)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

f : N → S is a mapping and called “transition rule”.
S is the set of state and N (= S 2r+1 ) is the set of theneighbor cells state. Where equation (3) represents
the graph f ∗ of f , lf (∈ f ∗ ) is called “local rule of f ”.
Also, l ∈ N × S is called “local rule”.

Table 3: An example of spatio-temporal pattern whenm = 5 and T = 1.
c0 (1, 0, 1, 0, 0)
c1 (1, 0, 0, 1, 0)

f ∗ = {lf |lf = (n, s), n ∈ N, s = f (n)}Table 4: The identiﬁed probabilistic rule from thespatio-temporal pattern of Table 3.

(3)

nti
t+1
P (si = 1|nti )When each cell changes state stochastically, theprobability is described by equation (4).

(1, 0, 1)0(1, 0, 0)1(0, 1, 0)0.5(0, 0, 1)0

P (s|n)Where,∀n ∈ N∑P (s|n) = 1

(4)

4

s∈S

The condition of the cells is expressed by c ∈ S mwhere m is the number of cells and ct indicates the condition at time step t. Then CT = (c0 , c1 , · · · , cT ) represents a spatio-temporal pattern with T time steps.

3
3.1

4.1

Generative Mechanisms Identification
Deterministic Mechanismspp

Figure 1: ASEP model. Each cell can be occupied byonly single car. Only if there is not a car to front, eachcar goes forward at each time step with probability p.

Table 1: An example of spatio-temporal pattern whenm = 5 and T = 1.
c0 (0, 0, 1, 0, 0)
c1 (0, 1, 0, 1, 0)

Figure 2 shows the spatio-temporal pattern which
generated by ASEP. Clumps of black cells indicatethat the traﬃc jams are occurring.

Table 2: The identiﬁed deterministic rule from thespatio-temporal pattern of Table 1.
nti(1, 0, 0) (0, 1, 0) (0, 0, 1) (0, 0, 0)
st+1
0
1
0
1
i

4.2

Problem of Previous Method

The previous method has a problem when thespatio-temporal pattern satisﬁes the conservation ofmass. Because the previous method gives the ruleof straightforward CA (such as deﬁned in Section 2).
In general, straightforward CA changes states synchronously and cannot generate patterns which satisfy
the conservation of mass [9].
Table 5,6 show the rules identiﬁed from Figure 2, 3.
Figure 2 is diﬀerent from Figure 3. Figure 2 satisfy theconservation of mass and Figure 3 is not. However, theProbabilistic Mechanisms

If given CT is generated by probabilistic rule, weneed to estimate probabilistic distribution of local rulefrom it. Hence we calculate the occurrence ratios ofeach local rule. For example, Table 3 is a spatiotemporal pattern which is generated by the probabilistic rule and Table 4 is the probabilistic distribution of
local rules of it.

©ISAROB 2011

ASEP and its Patterns

ASEP (Asymmetical Simple Exclusion Process) [8]is known as the traﬃc jam model which satisfy theconservation of mass. ASEP is also one of PCA. Eachcar is arranged on the cell and goes forward at eachtime step with probability p if there is not any car to
front (e.g. Figure 1). Each cell has binary state, 1
indicate existing a car and State 0 is not existing.

We consider to identify the transition rule f froma spatio-temporal pattern CT . Scanning the spatiotemporal pattern CT gives the local rules f ∗ = {lf },and we get the transition rule f . For example, thespatio-temporal pattern such as Table 1 is given, we
can identify the transition rule of Table 2.

3.2

Conservation of Mass Patterns Identification

456

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Figure 2: The spatio-temporal pattern which generated by ASEP with p = 0.8. The black cells indicate 1
and the white cells are 0. The vertical axis represents
time step and the horizontal is space.

Figure 3: The spatio-temporal pattern which generated by PCA without the conservation of mass. The
black cells indicate 1 and the white cells are 0. The
vertical axis represents time step and the horizontal isspace.

rules are remarkably similar. Because the rule of Table
5 lose the conservation of mass. Hence we need expand
the previous method in order to solve the problem.

In addition, the spatio-temporal pattern generatedby ASEP (such as Figure 2) also satisﬁes equation (6)where ∗ is the wild card: 0 or 1 and lit = (nti , st+1).
i
When a car goes forward, equation (6) means that
other car does not suddenly appear and the car does
not either suddenly disappear.

Table 5: The identiﬁed probabilistic rule from Figure
2.
nti
t+1
P (si = 1|nti )(0,1,1)1.00(0,1,0)0.18(0,0,1)0.00(0,0,0)0.00n= 1|nti )P (st+1i(1,1,1)1.00(1,1,0)0.19(1,0,1)0.81(1,0,0)0.80

tlit = ((∗, 1, 0), 0) → li+1
= ((1, 0, ∗), 1)tt= ((1, 0, ∗), 0)
li = ((∗, 1, 0), 1) → li+1

Table 6: The identiﬁed probabilistic rule from Figure
3.
nti
t+1
P (si = 1|nti )ntit+1
P (si = 1|nti )

4.3(0,1,1)1.00(0,1,0)0.18(0,0,1)0.00(0,0,0)0.00(1,1,1)1.00(1,1,0)0.19(1,0,1)0.80(1,0,0)0.80

We consider a pattern and assume that the patternsatisﬁes equation (6). Also we assume that the ruleof Table 7 identiﬁed from the pattern by the previous
method. Of course, the rule does not satisfy the conservation of mass. However we can transform the ruleto the hierarchical one. The hierarchical rule satisﬁes
the conservation of mass.
Table 8,9 show the hierarchical rule which transformed from the rule of Table 7. In the hierarchicalrule, the state of cell is described as sti = (ati , bti ).
In the case of ASEP with probability p, the hierarchical rule is represented by equation (7). Whereati indicates whether there is a car on the cell and bti
indicates whether a car goes forward. The hierarchical rule consists of the two phase. The ﬁrst phase isdeciding that the car on the cell goes forward withprobability p (such as Table 8). The second phase isactually moving the car on the cell to next cell basedon the decision of the ﬁrst phase (such as 9).

Identification

In order not to lose the conservation of mass whenidentifying the transition rule, investigating whether
the spatio-temporal pattern satisﬁes the conservationof mass is necessary. Equation (5) is the necessary
and suﬃcient condition for satisfying the conservationof mass. Equation (5) shows that the sum of variationof mass of cells is 0 at every time step.
T−2 m−1∑∑(st+1− sti ) = 0i(5)t=0 i=0©ISAROB 2011

(6)

457

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

t+ 12

t+ 21

t+ 21

si

= (ai

bi

= ati (1 − ati+1 )σp

t+ 12

, bi

Finally, we redeﬁned the ASEP rule as the hierarchicalrule.
)

Acknowledgments

t+ 1= (ati , bti ) = (at+1, bi 2 )
i
t+ 1
t+ 1
= (1 − bi−12 )(1 − bi 2 )ati
t+ 1
t+ 1
+bi−12 (1 − bi 2 )ati−1 (1 − ati )

sti
at+1
i

{

(7)

0 (with probability 1 − p)
1 (with probability p)

σp =This work was supported in part by Global COE
Program “Frontiers of Intelligent Sensing” from the
Ministry of Education, Culture, Sports, Science andTechnology, Japan.

(8)

References
Table 7: The straightforward probabilistic rule.
nti
P (st+1= 1|nti )i(0,1,1)p3(0,1,0)p2(0,0,1)p1(0,0,0)p0

nti
t+1
P (si = 1|nti )(1,1,1)p7(1,1,0)p6(1,0,1)p5(1,0,0)p4[1] Richards F, Meyerb T, Packard N (1990) Extracting cellular automaton rules directly from experimental data. Physica D: Nonlinear Phenomena 45:
189–202.
[2] Ichise Y, Ishida Y (2008) Reverse engineering spatial patterns in cellular automata. Artiﬁcial Lifeand Robotics 13: 172–175.

Table 8: The ﬁrst phase of the transformed hierarchical rule from the rule of Table 7.
nti(0,1,1)(0,1,0)(0,0,1)(0,0,0)

0

p2

0

0

(1,1,1)(1,1,0)(1,0,1)(1,0,0)

0

p6

0

0

1
t+ 2

= 1|nti )
nti

P (bi

t+ 1
P (bi 2

= 1|nti )[3] Von Neumann J, Burks A, et al. (1966) Theory ofself-reproducing automata. Univ. of Illinois Press
Urbana, IL.
[4] Wolfram S (1984) Universality and complexity incellular automata. Physica D: Nonlinear Phenomena 10: 1–35.

Table 9: The second phase of the transformed hierarchical rule from the rule of Table
7. t tt(ai−1 , ai , ai+1 )

1
t+ 2

)

P (at+1
|ni
i

)

t+ 1

(bi−12 , bi

t+ 1
2

(0,0)
(0,1)
(1,0)
(1,1)

(0,1,1)(0,1,0)(0,0,1)(0,0,0)

p3
-

1
0
-

p1
-

p0
-

[5] Packard N, Wolfram S (1985) Two-dimensional cellular automata. Journal of Statistical Physics 38:
901–946.
[6] Langton C, et al. (1992) Life at the edge of chaos.
Artiﬁcial life II 10: 41–91.

(ati−1 , ati , ati+1 )(1,1,1)(1,1,0)(1,0,1)(1,0,0)

(0,0)
(0,1)
(1,0)
(1,1)

p7
-

1
0
-

0
1
-

0
1
-

t+ 1

(bi−12 , bi

t+ 1
2

)

t+ 1
P (at+1|ni 2 )i5[7] Domany E, Kinzel W (1984) Equivalence of cellular automata to Ising models and directed percolation. Physical review letters 53: 311–314.
[8] Rajewsky N, Santen L, Schadschneider A, Schreckenberg M (1998) The asymmetric exclusion process: Comparison of update procedures. Journalof statistical physics 92: 151–194.
[9] Suzudo T (2004) Spatial pattern formation in
asynchronous cellular automata with mass conservation. Physica A: Statistical Mechanics and its
Applications 343: 185–200.

Conclusions

We addressed the problem of previous method. The
problem is losing the conservation of mass which isone of the hidden rule in the spatio-temporal patternwhen identifying the rule. We solved the problem byexpanding the method to identifying the hierarchicalrule. The expanded method can investigate the conservation of mass from the spatio-temporal patterns.

©ISAROB 2011458

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Emergence of Observable Rules in a Spatial Game System*

Kouji Harada and **Yoshiteru Ishida* **Toyohashi University of Technology
1-1, Tenpaku, Toyohashi-shi, Aichi 441-8585, Japan(Tel : 81-532-44-6885; Fax : 81-532-44-6873){harada, ishida}@cs.tut.ac.jp

Abstract: The present study focuses on player’s strategies observed from outside in our original spatial game iterated
by players, each of which placed in each lattice site on a two dimensional square lattice. A particularity of the game liesin the point that a player's strategy is not preliminary given, but constructed dynamically in response to a spatial patternon player's actions. This means the strategy can evolve in time. However simulations bring us unexpected results.
Actually all of strategies observed from outside did not evolve and they were fixed in time. This paper enumerates allof the observed strategies in detail and examines their characteristics.
Keywords: Spatial game, Inverse problem, External observersuggests that a model which enables to consistently
explain the observation results from outside is notalways true one.

I. INTRODUCTION
It is very important for many scientific fields
ranging from science, medical science to engineering to
extrapolate inputs or rules hidden from outside byknowing outputs such as observational results. This kindof problem is called the “inverse problem”. Richard etal. [1] try to extract a cellular automaton rule to produce
a spatiotemporal pattern of given experimental data byemploying the genetic algorithm. Ueda and Ishida [2]deal with one dimensional cellular automata andexamine to extrapolate a rule generating given
spatiotemporal patterns of cell states. The present study
addresses an inverse problem with a spatial game byplayers placed in a two dimensional square lattice spaceand discusses whether it is possible to extrapolate
player's strategy from spatiotemporal patterns on
player's actions. In our model, each player observes
actions of its neighborhood players, and builds its own
strategy by applying the observed actions for a strategybuilding rule and determines its action for the next
round game. We suppose an external observer watchinga game from outside cannot know what strategy eachplayer applies for determining its action, but canobserve actions of all players in each round. Under thissetting, the present study discusses a possibility that aplayer's strategy extrapolated through a global
observation by the external observer chimes perfectlywith the real player's strategy, and presents a disaccord
case: although each player's strategy actually evolves ineach game round, the external observer observes any
player obeys a unique steady strategy. This result©ISAROB 2011

II. SPATIAL GAME MODEL
Our study considers a spatial game on a twodimensional square lattice. Each player is placed oneach lattice site. To designate each lattice site, ahorizontal axis with a suffix j and a vertical one with asuffix i (i,j=0,1,…,L-1) are prepared. We let a player toselect one of two actions: 0, 1. A symbol, Ai,jr denotes
the player (i,j)'s action at the game round r, and asymbol Si,jr denotes the player (i,j)'s strategy at the gameround r. Our model adopts so-called "spatial" strategy.
The spatial strategy determines a player's action fromthe total number of players taking an action "1" in its
eight neighborhood players called the Moore neighbor.
Figure 1 shows an example of the player (i,j)'s
spatial strategy at the round r+1. A symbol "k" is thetotal number of players taking an action "1" in the
Moore neighbor of (i,j) site. A symbol "A" is the player(i,j)'s action at the round r+1. In this example, As kamounts to 5, the strategy determines the player (i,j)'s
action at the round r+1 becomes 0.
The original of the proposed spatial game [3] lies inthat a determination relationship between a strategy andan action is not one-way but mutual way: a strategy
determines an action as well as actions determine astrategy. This implies to destroy the original boundary
between the strategy and the action. Gunji [4] tries todestroy a boundary between hierarchies on a level of a459

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

coarse graining by introducing an internal observer
enabling only local observation.
We introduce a strategy building rule to build astrategy based on a spatial configuration of player's
actions. The strategy-building rule defines thecorrespondence between k and player's action A as
follows.
When k is 0, let an action A to be Ari-1, j-1.
When k is 1, let an action A to be Ari-1, ,j.
When k is 2, let an action A to be Ari-1, j+1.
When k is 3, let an action A to be Ari, j+1.
When k is 4, let an action A to be Ari+1, j+1.
When k is 5, let an action A to be Ari+1, j.
When k is 6, let an action A to be Ari+1, j-1.
When k is 7, let an action A to be Ari, j-1.
When k is 8, let an action A to be Ari, j.

ExampleIII. SIMULATION RESULTS
This study chooses the lattice-size L to be its
minimal value: three, hence considers the spatial gameon a two dimensional 3x3 square lattice. The number ofplayers is nine. A boundary condition of the squarelattice space is periodic.
1. PERIODIC ATTRACTORSThe previous study [3] clarified every observed
game becomes periodic. Types of the observed period
are period-1 (fixed point), period-3, period-6 andperiod-9. Some initial bit configurations converge into aperiodic game, hence some periodic games are“attractive” in other words, attractor. The number of
attractors with a certain period: period-1, -3, -6 and -9 is2, 15, 1 and 2, respectively.
2. EXTERNALLY OBSERVED STRATEGIESThis section shows an odd discrepancy that eachplayer's strategy dynamically evolves in response withchange of a bit spatial pattern of its Moore neighbor;however a strategy extrapolated by the external observeris irrelevant with the spatial bit pattern thus steady. Thissection also mentions that interestingly the external
observer sees a player have a memory from its behavior.

k 0 1 2 3 4 5 6 7 8Sir, +j 1

A 1 0 1 1 1 0 1 0 1

k : the total number of players taking “1” in the Moore neighbor of (i.j)
A: player(i,j)’s action at the round r+1

j
i

1

0

1

0

1

1

1

0

1

k =5
0

PERIOD-3 GAME
Strategies observed from outside in the 3-period
games are generally described as a “copy” strategy ofmemorizing a player's action in the previous round andusing its action in the next round. Figure 2 exhibits an
example of a strategy observed from outside in a period3 game, each player memorizes an action in theprevious round of a certain player placed in the ‘right’side for its own site and the player takes the memorized
action in the next round. In the other period-3 game,each player memorizes an action in the previous roundof the player placed in not the ‘right’ but the ‘bottom
left’ for its own site and uses the memorized action. It isinteresting that in the period-3 games, the external
observer must see any player have one bit memory tomemorize one bit action.

Moore neighbor

At the round r

At the round r + 1

Fig.1. An example of how to build a strategy
Figure 1 shows a process to build the strategy Si, j r+1
from a spatial configuration of player (i,j)'s Moore
neighbor through the strategy-building rule.
The spatial game proceeds as stated follows. As an
initial condition, all players must preliminarily
determine their actions. One round game consists of thetwo steps:
1. Each player estimates the quantity k from player's
actions in its Moore neighbor at the previous round, andbuilds its strategy based on the strategy-building rule.
2. Each player takes the next action corresponding to
the quantity k referring to its strategy.
In addition, this study does not interest in relative
merits between strategies, thus not consider a scoringprocess based on a score table representing comparative
merits between actions.

©ISAROB 2011

460

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

0

1

0

1

0

0

0

0

1

1

1

0

1

0

1

0

1

1

0

0

0

0

0

0

0

0
Memorize

Take an action

0
Memorize

Take an action

1

0

0

0

0

0

0

0

0

1

0

0

1

0

1

0

1

0

0

0

0

0

0

1

0

1

1

1

0

1

0

1

0

0

0

0

0

0

1

0

1

1

1

0

0

0

1

0

0

0

0

0

0

1

0

0

Player (1,1)

Player (1,1)

1

0

0

0

1

1

0

0

1

0

0

0

0

0

0

0

0

0

1

0

0

0

1

0

1

0

1

Fig.2. A strategy observed from outside in a period-3
game
PERIOD-6 GAME
A strategy observed from outside in the period-6
game is the same as the one observed in the period-3
game except memorizing a player's action in not the one
but two rounds before. Figure 3 exhibits an example ofa strategy that each of all players memorizes an actionin the two rounds before of the player placed in the rightside of each own site, and then takes the memorized
action in the next round. In the period-6 game, theexternal observer must see all players have two bits
memory.
0

0

1

0

1

1

0

1

0

0

1

0

1

1

0

1

0

0

1

0

0

1

0

1

0

0

1

Fig.4. An example of a strategy in a period 9 gameThe table 1 sums up all of strategies observed fromoutside. The strategies are specified by threeparameters: P, D and T because all strategies observedin the period "P" games are described as a copy strategy
of each player memorizes an action in the "T" round(s)before of a certain player placed in the "D" direction foreach own site and then takes the memorized action. The
rest of the parameters is N which is a number ofdifferent games with a certain strategy.
Table1. All types of strategies observed fromoutside in every periodic game
P

1

0

1

1

0

0

1

1

3

0

0

1

1

0

0

1

1

0

1

1

1

0

0

1

0

0

1

1

Fig.3. An example of a strategy in the priod-6 game
PERIOD-9 GAME
Strategies observed from outside in the period-9
game is the same as those observed in the period-3 and 6 except memorizing a player's action in not the one/two
but three rounds before. Figure 4 exhibits an instance ofa strategy in a period-9 game that each player
memorizes an action in three rounds before of a certain
player placed in the just above for each own site, andthe player takes the memorized action in the next round.
It is interesting that in the 9-period games, the external
observer must see any player have three bits memory.

©ISAROB 2011

T

D

1

6

1

9

3

N

Just above

1

Just below

3

Top right

2

Right

4

Bottom right2

Top left

0

Left

1

Bottom left

2

Right

1

Just above

1

Left

1

The table.1 suggests that in the period-3 games, astrategy to copy an action of player in the top left
position is not observed. It also shows the number ofgames with a strategy to copy an action of a player inthe right side is four, whereas in the case of copying anaction of player in the opposite left side, number ofgames is only one. Like this, in response with thedifference of the parameter D, it features that there is
imbalance on the number of the observed games. Thisasymmetric nature is considered due to the asymmetryof the strategy-building rule.

461

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

IV. DISCUSSIONSIt seems to be odd that the strategies observed fromoutside in the proposed game model do not depend on a
bit pattern of actions in the Moore neighbor although theactual strategy defined in the model depends on it.
Figures 2 and 5 shows the same game from outside andinside respectively. The figure 2 exhibits the player(1,1)'s strategy is just a copy strategy irrelevant with a
bit pattern of its Moore neighbor. On the other hand,
Figure 5 exhibits the player (1,1)'s strategy evolves in
response with a bit pattern of its Moore neighbor. Here
we discuss through this example it is possible thesituation that externally observed strategies do not
evolve but in fact internally defined strategies evolve.
Now it supposes the externally observed strategy in Fig.
2 that all the players memorizes an action in theprevious round of the player in the just below for theirown site and they take their memorized action in thenext round. Then a bit pattern at the round r+1 of theMoore neighbor of a player (i,j) becomes equal with a
bit pattern at the r round of the Moore neighbor of aplayer (i,j+1). Following the strategy-building rule, if apair of bit patterns of the Moore neighbor of the twoplayers, (i,j) and (i,j+1) are the same, their strategies are
the same:

S

1

0

0

0

0

1

1

1

0

1

0

1

0

1

1

0

0

0

0

0

0

0

0

0

Take an action

Observe

Take an action

Player (1,1)

k 0 1 2 3 4 5 6 7 8
A 0 1 0 0 0 0 0 1 1

2
1,1

S

k 0 1 2 3 4 5 6 7 8
A 1 0 0 1 0 0 0 1 0Strategy evolves

Fig.5 Time evolution of the player (1,1)’s strategy
To solve our inverse problem, what will be needed?
We will need more external observational data on
various types of periodic games. However, if we
randomly select an initial spatial bit pattern on player's
actions and starts the game, it must be hard to observe acertain periodic game with a smaller basin size because
a probability of its game being selected becomes lower.
For instance, a basin size of the period-6 game in Fig.3
is zero, thus a probability of the game being observed isvery low: 6/256. Here the basin size of a periodic gameis defined as the number of initial bit patterns on
player's actions which finally transit to its game.
It is necessary to focus on identifying not an
individual player's strategy but a “meta”-rule to build anindividual strategy: the strategy-building rule. Howeveris it possible to discover not individual rules but a metarule which is their origin from the observational data? In
the future work, we have to think this issue.

(2)

REFERENCES

Eqs. (1) and (2) lead to[1] Richards F.C., Meyer T.P., and Packard N.H. (1990)
Extracting cellular automaton rules directly fromexperimental data. Physica D 45:189-202[2] Ueda T and Ishida Y (2009), Reverse Engineering of
Spatiotemporal Patterns in Spatial Prisoner’s Dilemma.
J. of Artificial Life and Robotics 14 4:498-501[3] Harada K and Ishida Y (2010), A Note on
Dynamical Behaviors of a Spatial Game Operated on
Intercrossed Rules. LNAI 6278:637-644[4] Gunji Y.-P (2010) SEIMEI-ICHIGO (in Japanese),
SEIDOSYA Press(3)

In other words, the strategy of the player (i,j)
evolves.
This discussion shows that it is possible thatexternally observed strategies do not evolve but in fact
inner defined strategies evolve.

©ISAROB 20110

Player (1,1)
1
1,1

(1)
It would be valid to suppose that at the round r, a bit
pattern of the Moore neighbor of a player (i,j) does notequal with that of the Moore neighbor of a player (i,j+1). Thus,S ir, +j 1 ≠ S ir, j .

1

observe

S ir, +j 1 = S ir, j +1 .

S ir, j ≠ S ir, j +1

0

462

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

An Artificial Intelligence Membrane to Detect Network Intrusion
Takeshi Okamoto
Department of Information Network and Communication, Kanagawa Institute of Technology
1030, Shimo-ogino, Atsugi, Kanagawa 243-0292 JAPAN
take4@nw.kanagawa-it.ac.jp

Abstract: We propose an artificial intelligence membrane to detect network intrusion, analogous to a biologicalmembrane that prevents viruses from entering cells. This artificial membrane is designed to monitor incoming packets
and to prevent a malicious program code (e.g., a shellcode) from breaking into a stack or heap in a memory. Whilemonitoring incoming TCP packets, the artificial membrane constructs a TCP segment of incoming packets, derives thebyte frequency of the TCP segment, from 0 to 255 bytes, as well as the entropy and size of the segment. Thesefeatures of the segment can be classified by a data mining technique such as a decision tree or neural network. If thedata mining method finds a suspicious byte sequence, the sequence is emulated to ensure that it is just a shellcode. If
the byte sequence is the shellcode, the sequence is dropped. At the same time, an alert is communicated to the systemadministrator. Our experiments examined 7 data mining methods for normal and malicious network traffic. The
malicious traffic included 114 shellcodes, provided by the Metasploit framework and including 10 types ofmetamorphic or polymorphic shellcodes. In addition, real network traffic involving shellcodes was examined. We
found that a random forest method outperformed all the other data mining methods, with a very high detection accuracy,
including a true positive rate of 99.6% and false positive rate of 0.4%.
Keywords: network intrusion detection, malicious software, shellcode, data mining

many patterns that it may be difficult to cover all patterns[9]. Heuristic methods attempt to detect intrusive code
sequences such as consecutive NOP sequences (i.e., NOP
sleds) and sequences that get a program counter (i.e.,
getPC). However, some NOP sleds are polymorphic[9],[10], and intrusive attacks may not get the program
counter. Emulation methods, which emulate incoming
packets as program code, can correctly detect anintrusive program code, but these processes are very
slow. Data mining methods use a classifier, such as adecision tree or neural network, to distinguish between
benign and malicious traffic using the features of
network traffic. Although these methods detect malicious
traffic at a high rate, their false positive rates may behigh.
We propose here an artificial intelligence membrane
to detect network intrusion, analogous to a biologicalmembrane that prevents viruses from entering cells. The
artificial membrane was designed to prevent a malicious
program code (e.g., a shellcode) from breaking into astack or heap in memory. Our experiments examined 7
data mining methods for normal and malicious network
traffic. The malicious traffic included 114 shellcodes,
provided by the Metasploit framework [10], and 10 kindsof metamorphic and polymorphic shellcodes. In addition,
real network traffic involving intrusive network attacks
was examined.

I. INTRODUCTION
Anti-virus systems protect computers and networks
from malicious programs, such as computer viruses andworms, by discriminating between malicious andharmless programs and by removing only the former.
Therefore, anti-virus systems can be considered as acomputer’s immune system.
An innovative method, called a “virus throttle,” [1],has been found to slow and halt high-speed worms
without affecting normal network traffic. We havepreviously proposed a “worm filter” to prevent the
spread of both slow- and high-speed worms [2]. This
worm filter limits the number of unacknowledged
requests, rather than the rate of connections to newcomputers. In addition, we have proposed an immunitybased anomaly detection method to detect worms in
network traffic [3].
All of these methods monitor outgoing packets froman internal network; i.e., they detect internal anomalies.
Other methods are needed to monitor incoming packets
and to detect intrusive attacks. The four types of methods
used to detect intrusive attacks include pattern matching[4], heuristic [5], emulation [6], and data mining [7][8]methods. Since pattern matching methods requiresignatures to detect intrusive attacks, they may miss new
attacks due to an absence of signatures. In addition,
metamorphic and polymorphic codes can produce so©ISAROB 2011463

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Vulnerability
attack code

Start
Shellcode
Control flow

Awaiting a new segment.
NOP

Polymorphic
Shellcode

Returnaddress
The segment
is accepted.

Decoder

Data mining

Encoded shellcode(Encrypted shellcode)
No

Figure 1. Structure of a polymorphic shellcode. The NOP area
contains consecutive NOPs, indicating no operation and control
flowing to the shellcode area. The return address area contains
the address to which the program counter returns (i.e., theaddress of the NOP area). If a vulnerability attack is successful,the program counter will jump to the NOP area and enter theshellcode area.

Suspicious?
Yes
EmulationNo

Shellcode?

II. Network Intrusion
Yes

Most network intrusion attacks are composed of avulnerability attack and a shellcode execution (Figure 1).
The vulnerability attack is used for network intrusion,
following which the shellcode is executed. The shellcode
is a tiny program code for operating anything on thecomputer, such as download of malicious software andits execution.
The appearance of a shellcode is often disguised byan encoder (Figure 1). This type of shellcode is called ametamorphic or polymorphic shellcode. In metamorphic
shellcodes, a set of instructions is replaced by anequivalent set of different instructions, whereas, inpolymorphic shellcodes, a set of instructions is hidden by
encryption (Figure 1). These techniques make suchshellcodes difficult to detect because the appearance of
these shellcodes differ from each other and on eachoccasion. Although signature-based detection works insome cases, polymorphism will eventually defeat suchdetection methods [9].

The segment is dropped, andan alert is generated.
Figure 2. Algorithm of the artificial intelligence membrane. The parameters of the data mining method
are trained in advance.
byte frequency of the TCP segment, from 0 to 255 bytes,
as well as the entropy and size of the segment. Thesefeatures of the segment are classified by a data mining
method such as a decision tree or a multi-layer
perceptron (i.e., a neural network). If the data mining
method identifies a suspicious segment, that segment isemulated to ensure that it includes just a shellcode. The
emulation plays the role of eliminating false positives. If
the segment includes the shellcode, the segment is
dropped. At the same time, the system administrator isalerted.
Emulation is performed by “libemu 0.2.0,” a small
library offering x86 architecture emulation for shellcode
detection [12]. This libemu has been partially modified
for brute force detection of a suspicious byte sequence.
Thus, the modified libemu can detect all x86
architecture-based shellcodes provided by the Metasploit
framework, including all metamorphic and polymorphic
shellcodes. Although emulation is very slow, the entire
performance of the proposed method is not slow because
normal packets are eliminated in advance by the data
mining method and most of the packets would not be
suspicious.

III. AN ARTIFICIAL INTELLIGENCE MEMBRANE TO DETECT NETWORK INTRUSION
To detect shellcodes, we propose an artificial
intelligence membrane to detect network intrusions. The
membrane plays a role similar to a cell membrane that
protects a cell from non-self molecules.
Figure 2 illustrates the algorithm of the artificial
intelligence membrane. The artificial membranemonitors incoming TCP packets, and it constructs a TCP
segment consisted of incoming packets. It then derives a©ISAROB 2011464

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

IV. EVALUATION OF DETECTION ACCURACY

nai” encoder to remove 0x00 byte codes from theoriginal shellcodes.
In addition, the encoders of “alpha mixed” and“alpha upper” can disguise a shellcode as real traffic byrecoding the shellcode in a form that contains bytes
matching the statistical properties of real traffic.
The simulated traffic was examined using WEKA.
Table 1 shows the accuracy of detection of simulated
traffic. The true positive rate (TPR) was defined as therate at which a suspicious segment was correctlyclassified as suspicious, whereas the false positive rate
(FPR) was the rate at which a normal segment wasfalsely classified as suspicious. We found that the
random forest method outperformed all other data
mining methods (Table 1), with a TPR of 99.9% and an
FPR of 0%.
Table 2 shows the accuracy of detection ofmetamorphic and polymorphic shellcodes. The data
mining method used in this experiment was a random
forest method. All detection accuracies were very high,with all metamorphic and polymorphic shellcodes otherthan (4) having a TPR of 100% and an FPR of 0%. In
addition, the polymorphic shellcodes encoded by CLET
had a TPR of 100% and an FPR of 0%, whereas
metamorphic shellcodes encoded by “alpha mixed” and“alpha upper” had a TPR  99.8% and an FPR of 0%,
though the statistical properties of these shellcodes weresimilar to those of normal traffic.

To find the best accuracy of detection of data mining
methods, we evaluated accuracy of detection for 7
methods equipped with Wakaito Environment forKnowledge Acquisition (WEKA) [13]: an instance basedlearner (IBk), decision trees (J48 and random forest),naïve bayes, an inductive rule learner (JRpp), and a
support vector machine using sequential minimal
optimization (SMO). The algorithm of each data mining
method has been described [13]. All parameters of thedata mining methods were default settings of WEKA.
Each evaluation has been used for 10-fold crossvalidation. The data set was randomly divided into 10
subsets, with 9 subsets used for training and 1 for testing.
The process was repeated 10 times for every
combination. This methodology can be used to evaluate
the robustness of a given approach to detecting
shellcodes. Experiments were performed on dual Intel
Xeon E5410 2.33GHz processors with 12 GB RAM. The
operating system was Debian GNU/Linux Squeeze.
1. Evaluation of simulated traffic
Experimental data consisted of normal http trafficand simulated malicious http traffic. The normal traffic
consisted of 544 segments, whereas the malicious traffic
was simulated by combining one normal http segmentwith one shellcode for each of the 114 shellcodes
provided by the Metasploit framework, version 3.5.1.
Other types of malicious traffic included metamorphic or
polymorphic shellcodes encoded by 10 engines:“ADMmutate” [14], “CLET” [10], “alpha mixed,”“alpha upper,” “call4 dword xor,” “context cpuid,”“countdown,” “fnstenv mov,” “jmp call additive,”“shikata ga nai,” with the last 8 engines provided by theMetasploit framework [10].
“CLET” can disguise a shellcode as normal network
traffic by padding bytes close to the statistical propertiesof the normal traffic between the shellcode and the return
address sequence (Figure 1). In this experiment, thepadding size was 500 bytes. Larger padding results incloser byte frequency between normal and malicious
traffic [9], but it also makes it more difficult for theshellcode to control the target computer because the sizeof the buffer is not always sufficient to intrude into astack in a memory. Note that only CLET-encodedshellcodes were encoded in advance by the “shikata ga©ISAROB 2011

2. Evaluation of real traffic
Experimental data were captured from a highinteractive honeypot on VMware Workstation 6.0.3. The
guest operating systems were Microsoft Windows XP
Professional, SP1 and SP2. After extracting segment datafrom the captured data, we examined all the segment
data using the modified libemu, finding 1469 normal
segments and 976 malicious segments includingshellcodes.
To evaluate the performance of the data mining
method, we examined all the above segments using thedata mining methods of WEKA. Table 3 shows theaccuracy of detection and testing time of real traffic.
Again, we found that the random forest method
outperformed all other data mining methods, with a TPRof 99.6% and an FPR of 0.4%. Eventually, there would
be no false positives, because the suspicious segments
can be analyzed by emulation.

465

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 1 Accuracy of detection of simulated traffic (not all shellcodes were encoded). Each value isa weighted average.
IBk
J48
Random forest Multilayer perceptron
JRpp
SMONaïve bayes
TPR
FPR

99.6%
0.7%

99.6%
0.7%

99.9%
0%

99.6%
0.7%

99%
2.7%

99.6%
1.3%

99.3%
3.2%

Table 2. Accuracy of detection of simulated traffic for metamorphic and polymorphic shellcodes. Eachvalue is a weighted average. Shellcodes were encoded by the following engines: (1) “ADMmutate,” (2)“CLET,” (3) alpha mixed, (4) alpha upper, (5) call4 dword xor, (6) context cpuid, (7) countdown, (8)fnstenv mov, (9) jmp call additive, and (10) shikata ga nai.
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
(10)
TPR
FPR

100%
0%

100%
0%

100%
0%

99.8%
0%

100%
0%

100%
0%

100%
0%

100%
0%

100%
0%

100%
0%

Table 3. Accuracy of detection and testing time of real traffic. Each value is a weighted average.
IBk
J48
Random forest Multilayer perceptron
JRpp
SMONaïve Bayes
TPR
FPRTime (sec.)

99.4%
0.6%
0.6695199.3%
0.7%
0.0005499.6%
0.4%
0.0007199%
0.7%
0.2000298.9%
0.8%
0.0030798.5%
1.2%
0.02604[4] Roesch M (1999), Snort: lightweight intrusion
detection for networks. LISA’99, 229 – 238[5] Pasupulati A, Coit J, Levitt K, Wu SF, Li
SH, Kuo JC, Fan KP (2004), Buttercup: on
network-based detection of polymorphic buffer
overflow vulnerabilities. NOMS, 1:235 – 248[6] Polychronakis M, Anagnostakis KG, Markatos EP(2007), Network-level polymorphic shellcode
detection using emulation. Journal in Computer
Virology 2(4): 257 – 274[7] Payer U, Teufl P, Lamberger M (2005), Hybrid
engine for polymorphic shellcode detection. LNC S
3548(200):19 – 31
[8] Masud M, Khan L, Thuraisingham B, Wang X,
Liu P, Zhu S (2008), Detecting remote exploits
using data mining. IFIP, 285:177 – 189[9] Song Y, Locasto ME, Stavrou A, Keromytis AD,
Stolfo SJ, On the infeasibility of modelingpolymorphic shellcode. Proc. of the 14th ACM
CCS’07, 541 – 551[10] Metasploit project (2006), http://www.metasploit.
com/[11] Detristan, T, Ulenspiegel, T, Malcom, Y, Underduk,
M (2003), Polymorphic shellcode engine using
spectrum analysis. Phrack 11(61)[12] Baecher P, Koetter M (2007), libemu. http://libemu.
carnivore.it/[13] Witten IH, Frank E (2005), Data mining: Practicalmachine learning tools and techniques. Morgan
Kaufmann, 2nd edition[14] K2 (2001), ADMmutate. http://www.ktwo.ca/ADM
mutate-0.8.4.tar.gz

V. CONCLUSIONS
We have proposed an artificial intelligencemembrane to detect network intrusion. This membrane isanalogous to a biological membrane, which prevents
viruses from entering cells. Similarly, the artificial
membrane prevents shellcodes from breaking into amemory.
Our experiments indicated that the random forest
method outperformed all other methods. In addition, allmetamorphic and polymorphic shellcodes were detected
at a high rate. For real traffic, the TPR was 99.6% and
the FPR was 0.4%. This high accuracy of detection isconsidered due to the training of both normal andmalicious traffic data [9].
We are currently planning to implement this artificial
intelligence membrane for practical use.

REFERENCES[1] Williamson, MM (2002), Throttling viruses:
restricting propagation to defeat malicious mobile
code. ACSAC Security Conference 2002, 61 – 68
[2] Okamoto T (2005), A worm filter based on thenumber of unacknowledged requests. KES’05,
LNAI 3682:93 – 99
[3] Okamoto T, Ishida Y (2006), Towards animmunity-based anomaly detection system for
network traffic. KES’06, LNAI 4252: 123 – 130©ISAROB 2011

99.2%
0.8%
0.00049466

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A Dynamic Houjin (Square) and a Symmetric Houjin
Y. Tsuzuki and Y. Ishida
Toyohashi University of Technology,
1-1 Hibarigaoka Tenpaku-cho, Toyohashi, Aichi 441-8580, Japan(Tel.: +81-532-44-6895; Fax: +81-532-44-6895)(E-mail: ishida@cs.tut.ac.jp)
Abstract: A Houjin is an n by n square lattice with each cell containing a symbol (such as a number or a letter). Further,
these numbers or letters are designed to exhibit symmetry. For example, a magic square is a Houjin where thesymmetry embedded is that the numbers in each row, column and a center diagonal have an equal sum. This paper
reports a new Houjin: a dynamic Houjin. A dynamic Houjin changes its containing numbers at each time step whilesatisfying the symmetry as the Houjin (the magic square). The dynamic Houjin has a further symmetry in a time
dimension, that is, the sums of the numbers for each cell are identical.
Keywords: Houjin, magic square, Latin square, Euler square, Sudoku.

Section II briefly states the studies of Houjin
focusing on the spatially elaborated ones to contrast
temporally devised ones: the dynamic Houjin. Among
spatial Houjins, symmetric Houjins are used todemonstrate the dynamic Houjin. Section III presents
the dynamic Houjin, giving an example of the one
constructed from 4×4 symmetric Houjin.

I. INTRODUCTION
Houjin in Japanese mathematics means a magic
square (or Lo Shu in Chinese). A Houjin is an n by nsquare lattice with each cell containing a number wherethese numbers satisfy a certain constraint: sums in eachrow, column, and a center diagonal are equal. Moregeneral Houjin can have other symmetries.
Several Houjins (squares) have been studied indiscrete mathematics, typified by Latin squares andGreco-Latin squares (Euler squares). As several names
such as Houjin, Lo Shu, and Squares indicated, they
have been extensively studied in world wide ([1-4] inJapan for example) and for a period of historic scale.
Greco-Latin squares have been used in the experimental
design to make sure all the possible combination of thecontrol factors are involved. As for Latin square, 9x9
two layered latin square are used a puzzle known as
Sudoku [5].
On the other hand, cellular automata (CA) have been
attracting attention as a potential model for complex
systems (e.g., [6]). CA also can be expressed as a squarelattice where each cell can take one state among several
states. One difference between CA and Houjin is thatthe former is a dynamical system, while the latter is astatic one.
In order to bridge between CA and Houjin, thispaper tries a preliminary study to design a dynamic
Houjin that changes by a certain rule while satisfying
the symmetry of Houjin. Since there is much degree of
freedom in a design of the dynamic Houjin, this paper
can present an example of the dynamic Houjin based onsymmetric Houjins.

©ISAROB 2011

II. HOUJIN AS COMPUTATIONAL AND
MATHEMATICAL OBJECTS
Houjin have been discussed as a mathematicalobjects, however, it could be placed as a computational
object. For example, Houjin with two numbers in eachcell may be related to a matching problem: Stable
Marriage Problem (SMP) [7]. An SMP of n men and n
women requires two preference matrices: one indicatingeach man’s preference over the set of n women; andanother indicating each woman’s preference over the setof n men. These two matrices may be combined as a
Houjin with two-tuple of numbers (mij,,wij)where mij isthe preference of man i to woman j and wij that ofwoman j to man i in each cell. A Houjin M={ mij }(W={ wij }) is a half Latin Houjin, since each number
appears once and only once in each row (column).
When these two Houjins are orthogonal the combined
Houjin becomes a Greco-Latin Houjin (Euler Houjin).
1. Recursive Houjin
Recursive Houjin is defined to be a Houjin whose
structure is the same one with another row and column
added or the one deleted. For example, a recursive
Houjin, can be generated by the SMP above. When themost popular man and woman, hence they are mutually
the first order with each other, are added the resulted

467

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Dynamic Houjin is a new type of Houjin thatchanges as time step proceeds satisfying the twoconstrains: it must satisfy the constraint of the initial
Houjin; and it has a constraint along the time dimension
similar to the spatial constraint of the initial Houjin For
example, the dynamic Houjin discussed here satisfies
the constraint satisfied by the magic square: the equal
sum in row, column, and center diagonal at each step;and at some specific time step the sum of every cellmust be identical. We can build an example of 4×4
dynamic Houjin based on 4×4 symmetric Houjin.
When defining dynamic Houjin, an updating rule forgenerating the next Houjin from the current Houjin isrequired. A simple updating rule, for this case is
exchanging the number in a symmetric position in thesquare (Fig. 3). Since the numbers in a symmetricposition of the symmetric Houjin form a complement of
17, this updating rule defines the dynamic Houjin of theperiod two. That is, it turns back the original Houjin in
two steps; and further the numbers in each cell adds up
to 17. We call two Houjins are complement when thenumbers in one Houjin are complement to the numbersin the same cell in another Houjin.

Houjin with two-tuple is the recursive houjin (new row
and column is added at the upper left corner).

1/1

1/1

2/1

1/1

2/1

3/1

1/2

2/2

1/2

2/2

3/2

1/3

2/3

3/3

Fig.1. Recursive Houjin
Houjin with some specific structure resulted from
SMP can be used to study not only SMP (such as theLatin SMP [7]) but also Houjin itself such as the Euler
Square.
2. Symmetric Houjin
The numbers in a Houjin can have symmetry. For
example, the numbers of a pair of cells in a rotationallysymmetric position can be complement. In 4×4 Houjin,
for example, a complement numbers are two numbersthat sum up to 17 such as 1 and 16; 12 and 5. Fig. 2
shows three examples of 4×4 symmetric Houjin.
1 12 15 6

16 5

2 11

1 12 15 6

8 13 10 39

7 14

8 13 10 3

1 12 15 616

8 13 10 3

9

4

14 7

4

9

3 10 13 8

14 7

4

11 2

5 16

6 15 12 1

11 2

5 16

9

Fig.2. Three examples of 4×4 symmetric Houjin

14 7

4

9

11 2

5 16

5

2 11

4 7 14
3 10 13 8
6 15 121Complement numbers are connected by the edge.

S(0) at time=0;
S(1)= S(0)’ at time=1
Fig.3. A dynamic Houjin with period two.

III. DYNAMIC HOUJIN

A dynamic Houjin with the period of 16 can beconsidered when four operators of exchanging rows andcolumns are involved (Fig. 4). We denote a Houjin by anumber n and its complement by n’ and its horizontal(vertical) mirror image nX (nY) (Fig. 5). A Houjin nX iscomplement to nY. When the initial Houjin is S(0) (Fig.
3 left) and the four operators R1, C1, R2 and C2 areapplied four times in this order, the trajectory of the
dynamic Houjin is shown in Fig. 6. The four operators
are depicted by colored arrows.
Note that the Houjin that appears after applying thefour operators twice is the Houjin complement to theinitial Houjin. That is, the Houjins in a symmetricposition in the circle trajectory are complement witheach other. Also, a Houjin nX (nY) can be obtained byapplying the four operators R1R2R1R2 (C1C2C1C2) toa Houjin n.

Dynamic Houjin can be considered as a special typeof three dimensional Houjin whose shapes may notnecessarily be cubic and one dimension is a specifictype: time. An interesting property of the Dynamic
Houjin proposed here satisfies the time sum constant is
the specific number; and its partition is realized at eachcell. Further, in three dimension Houjin of size N, N3
distinct numbers appear, although the dynamic Houjin
proposed here uses N2 distinct numbers and the thenumber set do not change as the time proceeds.
Dynamic Houjin may be placed as a special type ofcellular automata where the next state of each cell does
not necessarily depend on the states of neighbor cells;
rather depend on more global configuration of cellstates. Further, it must satisfy the global constraints suchas the constant sum for several directions (row, column,
center diagonal and pan-diagonal).

©ISAROB 2011

468

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Starting from the same initial Houjin but changing
the orders of the application of four operators, several
trajectories with distinct symmetric Houjins are
obtained (Fig. 7). The trajectory 0-1-9-12-15Y-14X-10X4’-0’-1’-9’-12’-15X-14Y-10Y-4-0 is the same trajectoryas that shown in Fig. 6. Fig. 7 shows a system ofsymmetric Houjins, including all possible trajectorywhen the four operators R1, R2, C1, C2 are applied in
an arbitrary order. It can be observed that this system
may be considered a development chart of torus, since a
torus (not sphere) will be formed by connecting
corresponding Houjins in the boundary.
Another system that includes Houjins nR instead ofn exists where nR is the 90 degree clock-wise rotationof the Houjin n. In this system, nR, nXR, nYR, and n’R
appear instead of n, nX, nY, and n’, respectively.

VI. DISCUSSIONSThe study of a dynamic Houjin suggested the
Houjins themselves would exhibit an external symmetry,
although the studies Houjin mainly focused on theinternal symmetry embedded in the numbers of a
Houjin. For example, the trajectory of the dynamic
Houjin (Fig. 6) indicated Houjins in a symmetricposition form a complement of Houjin, while in asymmetric Houjin the numbers in a pair of numbers in
the cells in a symmetric position form a complement.
Another interesting property of the dynamic Houjin
is that the numbers in the same cell add up to 136 whichis the sum of all the numbers from 1 to 16 (that is
17x16/2). However, which numbers (among 1 to 16)
appears and how many times in a period depend on thecell, suggesting a possible relation to the partition
number.
Further, the dynamic Houjin proposed here is a
specific type whose trajectory proceeds with the same
set of operators applied repeatedly in a fixed order ofoperations; and the trajectory of a period is divided into
two parts in which Houjins appear in a fixed order in thefirst half and then the complement Houjins appear in thereversed order in the last half. Various and more general
trajectory and the way of change (application ofoperators) could be found.

VI. CONCLUSION
An attempt to develop new types of Houjin (square)
has been made with an emphasis that Houjins would
have external symmetry in the relation among Houjins
other than the internal symmetry so far studied.
Further, since Houjin is a mathematical object
similar to numbers, each Houjin could have a specificcharacter similarly to a specific numbers. Composing
new type of Houjin is a design mathematics that isrequired for education in current Engineering andScience.

Fig.4. Four operators to change Houjin

Fig.5. A Houjin n and its horizontal (left) and vertical (right) mirror image nX and nY)©ISAROB 2011469

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig.6. Trajectory of a dynamic Houjin starting fro
m S(0) and four operators R1, C1, R2 and C2 a
re applied in this order.

Fig.7. A system of symmetric Houjins, includin
g all possible trajectory when the four operator

REFERENCES[1] Abe G (1994), Unsolved problems on magic
squares. Discrete Mathematics 127(1-3): 3-13
[2] Shasha DE (2003), Prime Squares. SCIENTIFICAMERICAN June[3] Bell J (2005), An introduction to SDR’s and Latin
squares. Morehead Electronic Journal ofApplicable Mathematics Issue 4 — MATH-2005-0[4] Ishida Y and Kotovsky K (1995), Symmetry
Analysis on Symmetry Cognition on Multi-Level
Figures. Computers & Mathematics with
Applications, 30(7): 93-102[5] Delahaye JP (2006), The Science behind Sudoku.
SCIENTIFIC AMERICAN June 2006
[6] Wolfram S (2002), A New Kind of Science,
Wolfram Media, Inc.
[7] Benjamin AT, Converse C, Krieger HA (1995),
How do I marry thee? Let me count the ways,
Discrete Applied Mathematics©ISAROB 2011

470

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Performance Evaluation of Immunity-based Statistical En-route Filteringin Wireless Sensor Networks
Yuji Watanabe
Graduate School of Natural Sciences, Nagoya City University,
1 Yamanohata, Mizuho-cho, Mizuho-ku, Nagoya 467-8501 Japan(Tel : 81-52-872-5037; Fax : 81-52-872-5037)(yuji@nsc.nagoya-cu.ac.jp)
Abstract: Statistical en-route filtering (SEF) schemes can detect and eliminate false data injection attack in wirelesssensor networks. However, SEF do not address the identification of compromised nodes injecting false reports. In thispaper, we propose an immunity-based SEF to identify compromised nodes and achieve the earlier detection of false
reports. In the proposed scheme, each node has a list of neighborhood and assigns credibility to each neighbor node.
Each node can update the credibility of neighbor node based on success or failure of filtering and communication, andthen use the updated credibility as the probability of next communication. Some simulation results show that theimmunity-based SEF outperforms the original SEF.
Keywords: wireless sensor networks, statistical en-route filtering, immunity-based approach.

and LEDS [7]. However, these schemes do not address
the identification of compromised nodes injecting false
reports. If the compromised nodes are successfully
identified, then neighbor nodes of the compromised
nodes can drop false reports at an earlier stage.
For the detection of fault nodes on networks, animmunity-based diagnostic model [8] has been proposed
inspired by the Jerne's idiotypic network hypothesis [9].
In the diagnosis, each node has the capability of testing
the neighbor nodes, and being tested by the adjacent
others as well. Based on the test outcomes, each nodecalculates its credibility. However, compromised nodes
can not only output bogus test outcomes but also
calculate the credibility at random.
In this paper, we propose an immunity-based SEF to
identify compromised nodes. In the proposed scheme,
each node has a list of neighborhood and assigns
credibility to each neighbor node. Each node can not
only update the credibility of neighbor node based on
success or failure of filtering and communication but
also use the updated credibility as the probability ofnext communication. We carry out some simulations toevaluate the performance of the proposed scheme. Some
results show that the immunity-based SEF outperforms
the original SEF.

I. INTRODUCTION
In the last decade, wireless sensor networks have
paid much attention because of the popularization of
sensor nodes that are smaller, cheaper, and more
intelligent [1]. In large-sized wireless sensor networks
including a lot of sensor nodes, a detected event report
can be sent to base station (user) using multi-hop
communication where intermediate nodes forward thereport. Wireless sensor networks may also be deployedin potentially hostile environment, so that the issue of
security must be addressed. Attackers can compromise
sensor nodes to inject false data reports of non-existingor bogus events using the compromised nodes. Such an
attack is called false data injection attack [2]. The attack
may cause not only false alarms but also the depletionof the limited energy of the nodes forwarding thesereports to the base station.
Several research efforts [2-7] have proposed
schemes to overcome such attack. The statistical enroute filtering (SEF) scheme [3] can probabilisticallyfilter out false reports en-route in the dense deploymentof large sensor networks. In SEF, assuming that thesame event can be detected by multiple nodes,
forwarding nodes along the way to base station canstatistically detect false reports. SEF has achieved theearly detection of false data reports with low
computation and communication overhead. There areseveral revised en-route filtering schemes, for example,
the dynamic en-route filtering [4], the multipath enroute filtering [5], the ticket-based en-route filtering [6],©ISAROB 2011

II. SENSOR NETWORK MODEL
Following the previous studies on SEF, we also
consider a large sensor network composed of a lot of
sensor nodes and a base station which is a datacollection center. We further assume that the sensor

471

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 20115)

The CH collects all the {i, Mi}s from the detecting
nodes and randomly chooses T MACs fromdistinct partitions. This set of multiple MACs acts
as the proof that the report is legitimate. Then the
CH sends the final report attached T key indices
and T MACs like {𝐸, 𝑖1 , 𝑀𝑖1 , 𝑖2 , 𝑀𝑖2 , … , 𝑖 𝑇 , 𝑀𝑖𝑇 }toward the BS.
Fig.1 illustrates the example of the key assignmentand report generation in SEF. In this figure, the BS has a
global key pool of N = 12 keys divided into n = 4
partitions, each of which has m = 3 keys. Each sensor
node randomly picks k = 2 secret keys from one
partition of the key pool. After each detecting nodeendorses the event report by producing a keyed MAC
using one of its stored 2 keys, the CH collects all the
MACs from the detecting nodes and attaches randomly
selected T = 3 MACs, that is, M2, M9 and M10 to theevent report E.

nodes are deployed in high density, so that an event(sensing target) can be detected by multiple surrounding
nodes. Because it is useless for each of the detecting
nodes to send the event report (e.g., the location, thetime, and the type of event) to the base station, one ofthem is elected as the cluster head. The cluster headcollects and summarizes all the received event reports,
and forward a synthesized report toward the base station.
The report potentially traverses a large number of hops.
We assume that the attacker can compromise a node
to obtain the security information installed in the node.
Once compromised, the node can be used to inject false
data reports of bogus events. However, we consider theattacker cannot defeat the base station because the basestation has powerful security. Furthermore, this paper
does not focus on various other attacks, for instance,
false negative attacks and Dos attacks, by thecompromised node.

III. STATISTICAL EN-ROUTE FILTERING(SEF) [3]SEF can probabilistically filter out false reports enroute. SEF exploits collective decision-making by
multiple detecting nodes and collective false detection
by multiple forwarding nodes. SEF consists of threemajor components: (1) key assignment and report
generation, (2) en-route filtering, and (3) base station
verification.

Fig.1. Example of the key assignment and report
generation in SEF with 12 keys, 4 partitions, 3 keys ineach partition, 2 keys in each node, and 3 MACs
attached to event report.

1. Key assignment and report generation
The process of key assignment and report generation
is as follows:
1) The base station (BS) maintains a global key poolof N secret keys {𝐾𝑖 , 0 ≤ 𝑖 ≤ 𝑁 − 1}, divided into
n non-overlapping partitions. Each partition has m
keys. In other words, 𝑁 = 𝑚 𝑛.
2) Before each sensor node is deployed, it stores
randomly chosen k (k < m) keys from a randomly
selected partition in the key pool.
3) When an event appears, multiple surrounding
nodes can detect the event. A cluster head (CH) iselected from the detecting nodes to generate theevent report.
4) Each of the nodes that detected the event generates
a keyed message authentication code (MAC) Mi
using the event report E and randomly selected Ki,
one of its k stored keys. Each detecting node thensends {i, Mi}, the key index and the MAC, to the
CH. Ki is secret while Mi is public.

©ISAROB 2011

2. En-route filtering
In en-route filtering process, intermediate
forwarding nodes verify the correctness of the MACs
probabilistically and drop a report with forged MACs
en-route. The en-route filtering process is as follows:
1) Since a legitimate report carries exactly T MACs
produced by T keys of distinct partitions, a reportwith less than T MACs or more than one MACs in
the same partition is dropped.
2) Because of the randomized key assignment, eachforwarding node has certain probability to possess
one of the keys that are used to produce the T
MACs. If forwarding node finds out that it has oneof the T keys in the report, it reproduces the MAC
using its stored key and compares the result withthe corresponding MAC attached in the report. If

472

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

the attached MAC is different from the reproduced
one, the report is dropped.
3) When intermediate node does not have any of the
T keys, the node forwards the report to the nexthop.
The key assignment ensures that each node canproduce only partial proof for a report. A single
compromised node has to forge MACs to assemble aseemingly complete proof in order to forward false
reports. In Fig.2, since a malicious node has 2 keys fromonly partition 1, it needs to forge the other 2 MACs, M9and M10. The report with the forged MACs is dropped
because the correctness of the MACs can be verified by
the intermediate node with K10.

malicious nodes can not only output bogus test
outcomes but also calculate the credibility at random.
Therefore, we propose an immunity-based SEF
scheme to identify compromised nodes. In the proposed
scheme, each node has a list of neighborhood andassigns a state variable 𝑅 ∈ [0, 1] indicating credibilityof neighbor to each neighbor node. Note that each node
does not have its own credibility. Node j updates thecredibility Rji of the previous neighbor node i sending
the event report based on its filtering result and the
reply from next neighbor node k as follows:
𝑅𝑗𝑖 (𝑡 + 1) =if node 𝑗 receives the reply⎧ 𝑅𝑗𝑖 (𝑡) + ∆𝑠 from next node 𝑘⎪⎪
if node 𝑗 does not receives
𝑅𝑗𝑖 (𝑡) − ∆𝑓the reply from next node 𝑘⎨⎪
if node 𝑗 drops the report⎪
𝑅 (𝑡) − ∆𝑑using SEF
⎩ 𝑗𝑖

(1)

The initial value of credibility Rji(0) is 1. If
credibility Rji(t) is over 1 (under 0), it is set to 1 (0). The
values of the parameters ∆𝑠 , ∆𝑓 and ∆𝑑 should bechosen through mathematical analysis and simulation.
For example, in Fig.3, node i increases thecredibility Rih of the previous node h because the reply
from next node j can be received. However, node jdecreases the credibility Rji of the previous node i
because next node k drops the event report using SEF
and does not reply to node j. Since node k filters out thereport by itself, the credibility Rkj of the previous node jalso decreases.

Fig.2. Case that a false report with forged MACs from amalicious node is dropped by the intermediate
forwarding node.
3. Base station verification
Due to the statistical nature of the detection
mechanism, a few bogus reports with invalid MACs
may escape en-route filtering and reach the BS. In basestation verification process, the BS further verifies thecorrectness of each MAC and eliminates false reportsthat elude en-route filtering.

IV. PROPOSED IMMUNITY-BASED SEF
The original and revised SEFs do not deal with theidentification of compromised nodes injecting false
reports. Simple trace back is futile if the compromised
nodes tell a lie that the false reports are received fromthe other nodes. To detect fault nodes in networks, theimmunity-based diagnostic model [8] is a promising
approach. In the diagnosis, each node has the capabilityof testing the neighbor nodes, and being tested by theadjacent others as well. Based on the test outcomes,
each node calculates its own credibility. However,©ISAROB 2011

Fig.3. An immunity-based SEF scheme for identifying
compromised nodes.
Only the credibility update process cannot achieve
the identification of compromised nodes. For instance,in Fig.3, if node h is compromised, false reports are still
forwarded toward node k. Therefore each node uses the
updated credibility as the probability of next
communication. In the same example, node i has

473

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

adversely higher probability of receiving the report fromcompromised node h because of the increase of thecredibility Rih. However, since node j has lower
probability of receiving the report from node i, node i
may fail to communicate with node j at next stage, andthen the credibility Rih of the previous node h in theneighbors list of node i decreases. Although thecredibility Rkj of the previous node j in the neighbors listof node k decreases at first, if node j sends legitimate
reports received from the other previous nodes to node k,
the credibility Rkj can be recovered. By iterating thecredibility update and the communication based on the
updated credibility, our scheme will be expected toinhibit neighbor nodes of compromised nodes fromforwarding false reports.

Fig.4. Percentage of dropped false reports as a functionof the number of forwarding nodes for immunity-based
approach and SEF, respectively.

ACKNOWLEDGEMENTS

V. SIMULATION RESULTS

This work was partly supported by a Grant-in-Aid
for Scientific Research (C) (22500063) from Japan
Society for the Promotion of Science.

We carry out some simulations to evaluate theperformance of the proposed scheme. Simulation
conditions are the same as [3]: 340 nodes are uniformly
distributed in a field which size is 200 x 20 m2. One
base station and one event source sit in opposite ends ofthe field, with about 100 hops in between. The BS has a
global key pool of 1000 keys divided into 10 partitions,each of which has 100 keys. Each node has 50 keys, and
5 MACs are attached to event report. The results areaveraged over 10 network topologies.
Fig.4 shows the percentage of dropped false reports
as a function of the number of forwarding nodes for
immunity-based approach (∆𝑠 , = ∆𝑓 = ∆𝑑 =0.02) and
SEF, respectively in case that one node is compromisedand 1000 bogus reports are sent by the compromised
node. Results show that as false reports are forwarded,
more and more reports are dropped: about 100% bogus
reports are detected within 20 forwarding nodes for bothmethods. Furthermore, about 65% false reports are
dropped by the original SEF within 5 intermediate
nodes, while about 73% reports are filtered out by the
immunity-based SEF. We confirm that the immunitybased SEF can achieve the earlier detection of false
reports than the original SEF.

REFERENCES[1] Yick J, Mukherjee B, Ghosal D (2008), Wireless Sensor Network Survey. Computer Networks 52:
2292-2330[2] Zhu S, Setia S, Jajodia S, Ning P (2004), An I
nterleaved Hop-by-Hop Authentication Scheme for F
iltering of Injected False Data in Sensor Networks.
IEEE Symposium on Security and Privacy, 259-271[3] Ye F, Luo H, Lu S, Zhang L (2005), Statistical
En-Route Filtering of Injected False Data in Sensor Networks. IEEE Journal on Selected Areas in Co
mmunications 23(4):839-850[4] Yu Z, Guan Y (2006), A Dynamic En-route Sc
heme for Filtering False Data Injection in WirelessSensor Networks. Proceedings of the 25th IEEE Co
nference on Computer Communications, 1-12
[5] Kim MS, Cho TH (2007), A Multipath En-Route Filtering Method for Dropping Reports in Sensor
Networks. IEICE Transactions on Information and S
ystems E90-D(12):2108-2109[6] Krauß C, Schneider M, Bayarou K, Eckert C(2007), STEF: A Secure Ticket-Based En-route Filtering Scheme for Wireless Sensor Networks. 2nd IE
EE International Conference on Availability, Reliabil
ity and Security, 310-317[7] Ren K, Lou W, Zhang Y (2008), LEDS: Provid
ing Location-aware End-to-end Data Security in Wir
eless Sensor Networks. IEEE Transactions on Mobile Computing 7(5):585-598[8] Ishida Y (1990), Fully Distributed Diagnosis by
PDP Learning Algorithm: Towards Immune Networ
k PDP Model. Proceedings of International Joint Co
nference on Neural Networks, 777-782[9] Jerne N (1973), The Immune System. Scientific
American 229(1):52-60

VI. CONCLUSION
In this paper, we proposed an immunity-based SEF
scheme for identifying compromised nodes in wirelesssensor networks. Some results show that the immunitybased SEF outperforms the original SEF. In future, theproposed scheme will be additionally combined withother security mechanisms for higher security level.

©ISAROB 2011474

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Improvement of Immunity-Based Diagnosis for a Motherboard
Haruki Shida1, Takeshi Okamoto1 and Yoshiteru Ishida2
Department of Information Network and Communication, Kanagawa Institute of Technology,
1030, Shimo-ogino, Atsugi, Kanagawa 243-0292, Japan2
Department of Knowledge-Based Information Engineering, Toyohashi University of Technology,
1-1, Tempaku, Toyohashi, Aichi 441-8580, Japan
1

Abstract: We have utilized immunity-based diagnosis to detect abnormal behavior of components on a motherboard.
The immunity-based diagnostic model monitors voltages of some components, CPU temperatures, and fan speeds.
After simulating the abnormal behaviors of some components on the motherboard, we assessed the ability of theimmunity-based diagnostic model to detect these abnormalities. To improve the diagnostic accuracy of the model,
which can be decreased by isolated nodes, we used multiple diagnostic networks to connect isolated nodes to a
network or other isolated nodes. This simulation showed that the immunity-based diagnostic model containing multiple diagnostic networks was an effective method for detecting abnormal behavior of components on the motherboard.
Keywords: Immunity-based system, fault diagnosis, sensor network, motherboard, immune network

the immunity-based diagnostic model outperformed astand-alone diagnostic model. In the second experiment,
which compared a fully-connected network with acorrelation-based network for mutually testing thecredibility of sensors, we found that the correlationbased network had greater diagnostic accuracy in all test
cases. In addition, we utilized a hybrid model,
consisting of the stand-alone and immunity-based
diagnostic models, to diagnose nodes connected to the
network and isolated from the network. We found,however, that the accuracy of hybrid diagnosis for
isolated nodes was dependent on the stand-alone
diagnostic model. These isolated nodes could decrease
the diagnostic accuracy of the hybrid model. In thispaper, we sought to improve diagnostic accuracy ofmultiple diagnostic networks by connecting the isolated
nodes with one of the networks.

I. INTRODUCTION
The prevalence of technology for cloud computing
has increased the demand for data centers that provide
such cloud computing. Each server in these data centersmust therefore be available for data processing and datatransmission. To maintain system availability, it isimportant to detect abnormalities during their earlystages, before system failure.
The simplest way of diagnosing abnormalities
consists of evaluating each component individually bycomparing the output value of its sensor with apredetermined threshold value. However, it is difficult
to identify the abnormal component using this method[1]. Another method of diagnosis uses an immunitybased diagnostic model [2-5], which was derived
primarily from the concept of an immune network [6].
In this diagnostic model, mutual tests are performed
among nodes and the dynamic propagation of active
states. This diagnostic model has been used to diagnose
node faults in processing plants [7], to the selfmonitoring/self-repairing in distributed intrusion
detection systems [3], and to sensor-based diagnostics
for automobile engines [4].
We previously applied immunity-based diagnosis tothe detection of abnormal behaviors of components on amotherboard [8]. After simulating the abnormal
behaviors of some components on the motherboard, we
evaluated the ability of this model to diagnose
abnormalities of components of motherboard sensors in
two experiments. In the first experiment, we found that

©ISAROB 2011

II. Embedded Sensors on the Motherboard
Since a motherboard has multiple sensors, including
voltage, temperature, and fan speed sensors,abnormalities on the motherboard can be detected bymonitoring these sensors. We therefore used sensor
output values for diagnosis of the motherboard.
We collected sensor output values on a server from
July 27 to September 18. The specifications of theserver are shown in Table 1. The average airtemperature during that period was 25.3 °C, ranging
from 20.1°C to 32.8°C. Data were collected using
lm_sensors, a hardware health monitoring package for475

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Linux that allows information to be obtained fromtemperature, voltage, and fan speed sensors.
Table 1. Server specifications
Motherboard

Supermicro® X7DVL-I

OS

Debian GUN/Linux 5.0

Kernel

2.6.26-2-amd64

∑

,
0,
1,
1,
0,

with libesensors version 3.0.2

CPUIntel® Xeon E5410 2.33GHz×2

Power supply

Thermaltake Toughpower 700w

Fan

XFan model: RDM8025B×2
Gantle Typhoon D0925C12B2AP×2ADDA CFX-120S

CPU1
Core2

ComponentCPU
temperature
Core2
temperature

Range

Mean

1, if one of evaluation from i to j or j to i exists,
if neither evaluation from i to j nor j to i exists,(3)

if a balance formula between sensors and is satisfied,
if a balance formula between sensors and is not satisfied,
if there is no balance formula between sensors and .

(4)

SensorBalance formulaStandard

CPU1-Core2|CPU1-Core2| ≤ 26

deviation

CPU1-VCoreA|CPU1-VCoreA×25| ≤ 20

CPU1-Vbat|CPU1-Vbat×9| ≤ 18

CPU1-Fan5|CPU1-Fan5/34| ≤ 18

11.00-48.00(°C)

18.68

4.55

35.00-72.00(°C)

42.79

4.45

Core2-VCoreA|Core2-VcoreA×45.5| ≤ 28

VcoreACoreA voltage

1.11-1.19(V)

1.121

0.007

|Core2-Vbat×16| ≤ 20

Vbat

Internal
battery voltage

Core2-Vbat

3.23-3.26(V)

3.237

0.009

Core2-Fan5|Core2-Fan5/19| ≤ 21

Fan5

Fan speed

1012-1044(RPM)

1034

5.021

III. Immunity-Based Diagnostic Model
The immunity-based diagnostic model has thefeature of a dynamic network, in which diagnoses are
performed by mutually testing nodes (i.e., sensors) andby dynamically propagating their active states. In thispaper, the targets of the immunity-based diagnosis arecomponents with a sensor embedded on a motherboard.
Each sensor can test linked sensors and can be tested bylinked sensors. Each sensor is assigned a state variableindicating its credibility.
The initial value of credibility (0) is 1. The aim ofdiagnosis is to decrease the credibility of all abnormal
sensors. That is, according to this model, if thecredibility of a sensor is below a threshold value, that
sensor is considered abnormal.
When the value of credibility
is between 0 and 1,
the model is called a gray model, reflecting theambiguous nature of credibility. The gray model can be
expressed by the equation:©ISAROB 2011(2)

In the right-hand side of Equation (1), the first termis the sum of evaluations from other nodes for node i.
The second term is an inhibition term that maintains
ambiguous states of credibility. In this model,
equilibrium points satisfy the equation(t)=∑. Thusmonotonically reflects the valueof ∑. If ∑
is close to 0, thenisclose to 0.5. The balance formulas were determined by
calculating the relationships of the output values of thesensors (Table 3).
Table 3. Balance formulas between sensors

After collecting the output values from all 29
sensors on the motherboard, we calculated thecorrelation coefficients of all sensors. We observed
correlations involving 5 sensors (Table 2), and wetherefore used these 5 sensors for evaluation.
Table 2. Sensors used for evaluation and range of sensor output values
Sensor(1)

Wherelm-sensors version 3.0.2

Module,

VCoreA-Vbat|VCoreA-Vbat/2.8| ≤ 0.05

VCoreA-Fan5|VCoreA-Fan5/893| ≤ 0.07

Vbat-Fan5|Vbat-Fan5/316| ≤ 0.07

IV. Evaluation of the immunity-based
diagnosis for motherboard sensors
We evaluated the immunity-based diagnostic model
for motherboard sensors by a simulation, using the four
test cases shown in Table 4.
Test cases 1 and 2 assumed that the speeds of Fan5
were far outside the range shown in Table 2. A
significant decrease in Fan speed (test case 1) would
therefore cause the CPU temperature to rise, with theoverheated CPU causing the server to crash. Conversely,a significant increase in Fan speed (test case 2) would
waste power and decrease the life span of the Fan.
Therefore, test cases 1 and 2 represent abnormal
conditions.
Test cases 3 and 4 assumed that the output values ofthe sensors were slightly out of the range shown in
Table 2. Test case 3 assumed that the speed of Fan5 was

476

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011indicates a correct decision, whereas an accuracy of “X”
indicates an incorrect decision.
The diagnostic model correctly identified theabnormal Fan5 in test cases 1 and 2, and did not falsely
identify abnormalities in test cases 3 and 4. However,this diagnostic model could not correctly diagnose theisolated sensor, because the credibility of the isolated
VcoreA sensor was always 0.50.

slightly higher than that shown in Table 2, but that Fan5
was not abnormal. Test case 4 assumed that thetemperature of CPU1 was slightly higher than thatshown in Table 2, but that CPU1 was not abnormal.
Temperatures outside the range are not always abnormal,
because these temperatures depend on room
temperature. Therefore, test cases 3 and 4 represent
normal conditions.
Table 4. Test cases
Sensor output value

Case

CPU1

Core2

VcoreA

Vbat

Fan5

70
9
14
50

65
35
35
60

1.12
1.12
1.12
1.12

3.23
3.23
3.23
3.23

200
2000
1050
1020

1
2
3
4

2. Multiple diagnostic networks
We hypothesized that utilizing multiple diagnostic
networks, in which isolated nodes are connected to a
network or another isolated node, would approve
diagnostic accuracy.
All combinations of the multiple networks used for
immunity-based diagnosis are shown in Figure 2. Eachevaluation was based on the four test cases shown in
Table 4. The diagnostic accuracy of all multiple
networks is shown in Table 6.
In Table 6, a diagnostic accuracy of “P” indicatesthat the diagnostic model could not identify theabnormal component, although it detected multiple
abnormalities.

State
AbnormalAbnormal
NormalNormal

1. Correlation-based network
We previously described the construction of acorrelation-based network [8], using the correlation
coefficients shown in Figure 1. In the model presented
here, we removed a weakly correlated network from afully-connected network, forming a correlation-based
network, because these connections may be unreliable
for mutually testing the credibility of their sensors.
Table 5 shows the results of correlation-based networks.
Each value is a sensor credibility, i.e.,of Equation
(2).
CPU1

0.90

0.43

0.54

0.62

0.62

Core2

Fan5

0.01

0.55
0.40
VcoreA

0.52
0.00

Vbat

Fig. 1. Correlation-based network
Table 5. Results of a correlation-based network
Test
case

Credibility
Decision

Accuracy

0.00

X

O

0.87

0.00

X

O

0.88

0.98

O

O

0.87

0.67

O

O

CPU1

Core2

VcoreA

Vbat

Fan5

1

0.87

0.97

0.50

0.87

2

0.87

0.97

0.50

3

0.98

0.99

0.50

4

0.67

0.95

0.50

In Table 5, we assumed that a component on themotherboard was abnormal if its credibility was lessthan 0.1. A diagnostic decision of “O” indicates anabsence of abnormality, whereas a diagnostic decisionof “X” indicates an abnormality. An accuracy of “O”©ISAROB 2011

Fig. 2 Multiple diagnostic networks

477

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 6. Diagnostic accuracy of multiple networks

V. CONCLUSION

Test
case

(A)

(B)

(C)

(D)

(E)

(F)

(G)

(H)

(I)

(J)

1

O

X

O

X

X

O

O

X

P

X

2

O

X

O

X

X

O

O

O

X

X

3

O

O

O

O

O

O

O

X

O

O

4

O

X

O

O

O

O

O

O

X

O

We applied immunity-based diagnosis to thedetection of abnormal behaviors of components on amotherboard. We simulated the abnormal behaviors ofsome components on the motherboard, and weevaluated all the combinations of the diagnostic
networks. We showed that diagnostic accuracy depends
on the size of the network and the correlation between
nodes of the network. In addition, we showed that theimmunity-based diagnostic model with multiple
diagnostic networks was an effective method for
detecting abnormal behavior of components on themotherboard.
In future, we will attempt to determine therelationships among diagnostic network topologies andcorrelation between nodes, and to improve the accuracyof the diagnostic model.

We found that diagnostic models (A), (C), (F) and(G) made correct decisions, whereas the otherdiagnostic models made incorrect decisions.
In test cases 1, 2 and 3, each of the diagnostic
networks (A), (C), (F) and (G) consisted of 3 sensors
including Fan5. In contrast, the other diagnostic
networks either consisted of 2 sensors including Fan5 orwere weakly correlated networks. In test case 4, alldiagnostic networks other than (B) and (I) showed
results similar to those of CPU1.
For example, Table 7 shows the successful results ofdiagnostic network (C), and Table 8 shows theunsuccessful results of diagnostic network (I).
Table 7. Results of diagnostic model (C)Credibility

Test
case

CPU1

Core2

VcoreA

Vbat

Fan5

1

0.640

0.640

0.659

0.659

2

0.640

0.640

0.659

0.659

3

0.844

0.844

0.659

4

0.385

0.683

0.659

Decision

Accuracy

0.021

X

O

0.021

X

O

0.659

0.844

O

O

0.659

0.385

O

O

REFERENCES[1] Tanaka T, Kawazu T, Kanda S (2003), Computerassisted Diagnostic System Applied with ANFIS.
Biomedical Fuzzy System Association 5(1):49-54[2] Ishida Y (1996), An immune network approach tosensor-based diagnosis by self-organization. Comples
Systems Publication 10:73-90[3] Watanabe Y, Ishida Y (2003), Immunity-based
Approaches for Self-monitoring in Distributed Intrusion
Detection System. Knowledge-Based Intelligent
Information and Engineering Systems (KES'2003)
2774(2):503-510[4] Ishida Y (2006), Designing an Immunity-Based
Sensor Network for Sensor-based diagnosis of
Automobile Engines. Lecture Notes Computer Science
4252:146-153[5] Watanabe Y, Ishida Y (2003), Mutual tests among
agents in distributed intrusion detection systems using
immunity-based diagnosis. Proc. of AROB 8th ’03:682685[6] Jerne N K (1973), The immune system. ScientificAmrecian, 229(1):52-60[7] Ishida Y (2004), Immunity-Based Systems: A
Design Perspective. Springer-Verlag[8] Shida H, Okamoto T, Ishida Y (2010), Evaluation of
Immunity-Based Diagnosis for a Motherboard.
Knowledge-BasedIntelligent
Information
andEngineering Systems (KES'2010) 6278:628-636

Table 8. Results of diagnostic model (I)
Test
case

Credibility
CPU1

Core2

VcoreA

Vbat

Fan5

Decision

Accuracy

1

0.021

0.293

0.640

0.640

0.293

X

P

2

0.385

0.293

0.683

0.385

0.293

O

X

3

0.844

0.659

0.844

0.844

0.659

O

O

4

0.021

0.659

0.640

0.640

0.659

X

X

The diagnostic model in Table 7 misidentified thenormal CPU1 in test case 1, and misidentified theabnormal Fan5 in test case 2. These results indicate thatan immunity-based diagnostic model could not diagnose
sensors on a weakly correlated network consisting of 2
sensors.
In test case 4 of Table 8, the diagnostic network
misidentified the normal CPU1 due to a weakcorrelation network, although CPU1 belongs to thediagnostic network consisting of 3 sensors.
This simulation showed that diagnostic accuracy
depends on the size of the network and the correlation
between nodes.

©ISAROB 2011

478

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Visualization of keystroke data and its interpretation
T. Samura1, K. Tani1 and Y. Ishida2
1

Akashi National College of Technology,679-3 Nishioka, Uozumi-cho, Akashi, Hyogo 674-8501, Japan(Tel.: +81-78-946-6079; Fax: +81-78-946-6138)(E-mail: samura@akashi.ac.jp)
2

Toyohashi University of Technology,
1-1 Hibarigaoka Tenpaku-cho, Toyohashi, Aichi 441-8580, Japan(Tel.: +81-532-44-6895; Fax: +81-532-44-6895)(E-mail: ishida@cs.tut.ac.jp)

Abstract: Keystroke data from keyboard input is time-series data and follows a fixed pattern. However, it is difficult to
intuitively grasp the meaning of the data. In this study, we investigated visualization methods for keystroke data, which
is a form of biometric information. We extracted feature indices from the keystroke timing between consonant-vowel
letter pairs in Japanese text input and propose three visualization methods: two consonant-vowel doughnut methods, aconsonant-vowel matrix method and a keyboard layout method. These patterns of visualization are expected to be
useful in analyzing personal characteristics and authenticating users
Keywords: Visualization, Keystroke dynamics, Information security, Man-machine interface.

that is able to collect keystroke data from a large
number of participants in a single experiment. The
system uses typing support software that was familiar tothe participants, thereby lowering effects related to
unfamiliarity and tension. Figure 1 shows a screenshotof the software interface used in this study.
The document display screen allows participants
who are skilled typists to input text while viewing theJapanese text displayed in the upper row. Less skilled
typists can type while viewing the Latin alphabet text
displayed in the middle row. Latin alphabet text isremoved from the screen as it is typed, allowing
confirmation of mistyped characters. The top row
displays the number of keystrokes, the number of errors,
and the amount of time remaining. Because thisexperiment focuses only on Latin alphabet input
keystroke, by design Latin character input is notconverted into Japanese kanji characters.
While participants are typing, browser-embedded
JavaScript code records character input, key press times,
and key release times. Times are recorded using UNIX
times (millisecond precision). Recorded data was sent toa server using Ajax.

I. INTRODUCTION
Computers are essential in today's informationsociety, and with the increased frequency of using
keyboards, applications that utilize keystroke data are
expected to lead to innovations in various fields.
Keystroke data is regarded as a form of biometric
information from which various personal characteristics
can be extracted [1–3]. However, keystroke data is timeseries data and difficult to understand intuitively. In thisstudy, we aim to develop methods for visualizing
keystroke data in order to extract various personal
characteristics.
Prior research on the visualization of keystroke data
is limited. Neumann et al. [4] have proposed a method
for visualizing input data in order to make signaturesand postcard-like art for e-mail and other forms ofInternet communication.
In this study, we propose methods for visualizing
keystroke data obtained from free typing of Japanese
text in order to reveal personal characteristics.

II. Typing Data Collecting System
This section describes the keystroke data collection
system[5]. In this study, we used a web-based system©ISAROB 2011

479

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig. 2. Visualization pattern based on the absolutetransition time method (fast typing)
Fig. 1. Screenshot of interface of keystroke datacollecting system.

III. CONSONANT-VOWEL DOUGHNUT
METHODSIn this section, we propose two consonant-vowel
doughnut methods: an absolute transition time method
and a relative transition time method, the latter of which
is based on the time variance from the average transition
time of all participants.
3.1 ABSOLUTE TRANSITION TIME METHOD
Fig. 3. Visualization pattern based on the absolutetransition time method (slow typing)

The visualization procedure is as follows:
1. Define incoming alphabetic characters as nodes.
Exclude the letters "l", "q", "v" and "x", whichhave a low frequency of occurrence.
2. Arrange the letters in a circular doughnut
arrangement, with vowels on the inner circle andconsonants on the outer circle. Indicate
consonant-vowel letter pairs (hiragana) thatappear three times or more using edges.
3. Display the color-coded edges, as indicated inTable 1, on the basis of the transition time T
between pressing a key and the next key.
Table 1. Edge colors according to transition time T
Transition Time T [ms]Color
T < 70Red
70 < T < 100
Peach
100 < T < 130
Yellow
130 < T < 160Green
160 < T < 190Blue
190 < T
Not colored©ISAROB 2011

Fig. 4. Visualization pattern based on the absolutetransition time method (Hepburn Romanization, "ji"typed quickly)
Figure 2 shows a visualization pattern for aparticipant who typed Japanese text. It can be seen thatthe typing speed of this participant was fast, as manyedges are colored red and peach. On the other hand,Fig. 3 shows many blue edges, indicating that thetyping speed of the participant was slow. In Fig. 4, theedge for "ji" indicates a fast typing speed for this pair,from which we can ascertain that the participant usedthe Hepburn system of Romanization.

480

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

3.2 RELATIVE TRANSITION TIME METHOD
Here, we focus the time variance ΔT (=T - T ) between
targeted transition time T and the average transition timeof all participants, T . Edges from consonants to vowels
are color-coded for only three conditions: fast, averageand slow, based on the time variance ΔT. No color isshown for other conditions.
The visualization procedure is as follows:
1. Calculate the average transition time T of allparticipants.
2. Seek the time variance ΔT between transition time
T of a participant and the average transition time ofall participants, T .
3. Draw color-coded edges from consonants tovowels according to Table 2.

Fig. 6. Visualization pattern based on the relativetransition time method (slow typing)

Table 2. Edge colors according to time variance ΔTTime Variance ΔT [ms]ΔT < －60 (fast)－5 < ΔT < 5 (average)
60 < ΔT (slow)
Other（5 ms < |ΔT| < 60 ms)

Color
Green
Black
Yellow
Not
coloredIV. CONSONANT-VOWEL MATRIXMETHODThe visualization procedure is as follows:
For hiragana input of consonant-vowel pairs, placeconsonants of the first letter in columns (vertical)
and vowels of the second letter in rows
(horizontal). Exclude "c", "f", "j", "l", "p", "q", "v"and "x" because of their low frequency ofoccurrence.
2. Prepare two keystroke data sets for input by aparticipant. Obtain variances in the transition timefor each single letter. Consonant-vowel letter pairs(hiragana) that appear three times or more aretargeted for both data sets. Letters with small time
variance are colored dark red and letters with large
time variance are colored light red. Thus, thefigure is configured to be displayed entirely in dark
red if files are input by the same participant, anddisplayed in white and light red when the files are
input by different participants.

1.

Fig. 5. Visualization pattern based on relative transition
time method (fast typing)

Figure 7 shows an image of when a participant typed
three different texts. The participant’s identity can be
confirmed because most of the image is colored dark
red. Meanwhile, Fig. 8 shows an image comparing text
typed by a different participant. In most cases, typing
differs between participants, and thus the image is
generally displayed in light red.

Figures 5 and 6 show examples of visualization
patterns created using this method. As all edges for theparticipant in Fig. 5 are colored green, it is indicatedthat the typing speed of the participant is faster the
average. Meanwhile, in Fig. 6, many edges for theparticipant are colored yellow and black, indicating thatthe typing speed of the participant is slower than the
average. However, only the edge from "y" to "u" iscolored green, which shows that the participant has a
habit such as typing "y" with the left index finger. In
addition, only black edges or no edges at all are seen fortyping of "n" to "n" by all participants, indicating that
there is no difference in the input time of the transition
time among any participants.

©ISAROB 2011481

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

VI. CONCLUSIONSIn this study, various methods for visualizing
keystroke data were proposed, and the results showedthat the methods were effective in revealing thecharacteristics of individuals.
In future work, we plan to study changes invisualization images the typing of English text,
programming languages and other documents inaddition to Japanese texts. We also plan to investigate
typing by non-Japanese participants. In addition, we
considered only transition time in this work, andtherefore we would like to consider other keystroketimes as well.

Fig. 7. Visualization pattern based on the consonantvowel matrix method for a certain participant.

REFERENCES[1] Samura T. & Nishimura H. (2009a), Personal
Identification by Keystroke Dynamics in Japanese Free
Text Typing. Transactions of the Institute of Systems,
Control and Information Engineers (in Japanese).
22(4):145-153[2] Samura T. & Nishimura H. (2009b), Keystroke
Timing Analysis for Individual Identification in
Japanese Free Text Typing, Proceedings of ICROSSICE International Joint Conference 2009. 3166-3170[3] Samura T. & Nishimura H. (2010), A Hybrid Modelfor Individual Identification Based on Keystroke Data inJapanese Free Text Typing. Transactions of the Societyof Instrument and Control Engineers (in press, inJapanese) 46(11)[4] Neumann P., Tat A., Zuk T. & Carpendale S. (2007),
KeyStrokes:
Personalizing
Typed
Text
with
Visualization. EuroVis 2007:43-50[5] Samura T. (2008), A Web-based Typing Learning
System That Supports Japanese Text. Journal of
Education in the College of Technology (in Japanese)
31:471-476

Fig. 8. Visualization pattern based on the consonantvowel matrix method for different participants.

V. Keyboard layout method
The visualization procedure is as follows:1.
2.

Place nodes in the same layout as the QWERTY
keyboard.
The average time of pressing a letter is displayed
as red circles on the nodes, and the transition time
is displayed as light blue links. Shorter times areshown in darker colors.

Figure 9 shows a visualization pattern based on thismethod. For this method, it is verified that a similar
network is created even if different texts are typed. In
addition, Fig. 9 shows that movements are more activeon the right side of the keyboard, which suggests theparticipant is right-handed.

Fig. 9. Visualization pattern based on the keyboard
layout method©ISAROB 2011482

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Extraction of learning point by visualization of skill
Shihoko Kamisato*, Yukihiro Mori*, Nobuhiro Yamashiro*, Kentaro Noguchi* and Yoshiteru Ishida***Information and comunication technology
Okinawa national college of technology
Nago 905-2192 Japan**Department of Knowledge-Based Information Engineering
Toyohashi University of technology.
Toyohashi 441-8580 Japan(Tel : 81-980-55-4145) *(Email: kamisato@okinawa-ct.ac.jp, ac104610@edu.okinawa-ct.ac.jp)* (Email:ishida@cs.tut.ac.jp) **(Email: ac094609@edu.okinawa-ct.ac.jp,knoguchi@okinawa-ct.ac.jp) *Abstract: Experiments are a valuable tool in reinforcing important concepts in engineering students. In prece
dence research, we considered the technical skill education imparted during engineering experiments and analy
ze eye and arm movements of the teachers and students in the experiments. Purpose of this research is impro
vement of engineering experiment by visualization of skills. We visualized the teaching materials movie of theteacher and student. Furthermore, it made clear a weak point of engineering experiment using principal component analysis and protocol analysis. As a result it was able to make sure of having weak point consciousnes
s with default setting of device and relation of connection of device.
Keywords: visualization, skill, experiments education, learning point, principal component analysis,
protocol analysis.

Purpose of this research is improvement of engineeringexperiment by visualization of skills. We visualize theteaching materials movie of the teacher and student.
Furthermore, it makes clear a weak point of engineeringexperiment using principal component analysis andprotocol analysis.

I. ITRODUCTIO
Experiments are a valuable tool in reinforcing
importantconcepts
inengineering
students.
Numerous trials for improvement of engineeringexperiment and training by linking experiment methodwith lectures are documented in literature [1]-[3]. There
are some students who get the knack of experiment, buton the other hand there are many students who cannot
get it. A student getting the knack of experimentarranges measuring device simply and understands theoperation in a short time. But for most students thearrangement of measuring device is chaotic andmoreover they need to get used to its operation. In
precedence research, we considered the technical skill
education imparted during engineering experiments andanalyze eye and arm movements of the teachers andstudents in the experiments[4]-[6]. As a result the sign
parameter was derived. The first was consciousness ofexperiment time. The second was the operation ofexperiment device. The third understand of experimentstep [7]. Previously these parameters were not take into
consideration in experimental education, however our
research strongly supports inclusion of these parameterin improvement of educate method.
But a weak point of a student is not clear, and it is not
the experiment improvement which considered it.

©ISAROB 2011

II. VISUALIZATIO OF VIEW
TRASITIOAt first we did visualization of a point of student
experiment for experiment improvement. The
experiment intended for ten students and a teacher.
Subject experiments using view camera and records
view image. Using this image, it did visualization by
showing stationary time and view transition of subject
by directed graph. We compare a student with a teacher
using the result and make clear the difference. A themeof experiment is electric instrument error measurement.
Fig.1 shows the experiment conditions. Node of graphshows stationary time of a gaze point. An edge
represents direction and number of times of viewtransition. It shows view transition graph of a teacher by
fig.2. It shows view transition graph of a student by
fig.3. As for the teacher, pattern along experiment
procedures manual is found. For example, it makes surewith following step, measurement device → power

483

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

supply → measurement device → procedures text.
On the other hand, as for the student, pattern is notestablished. In particular there is much movement to beuseless in measurement device node neighborhood. On
this account even if stationary time gets longer andcompares a teacher, experiment time becomes long.
From this thing, a weak point of a student understands
that it is concerned with measurement device.

Table 1. Question item and symbol
Question
Experiment time
Preparations for lessons of experiment
Reconfirmation of text
Electric wiring of experimentOperation of device
Understanding of principal
Understanding of text description
partIII. EXTRACTIO OF WEAK POITS
In this paper, it makes clear a weak point of a
student in measurement experiment and reflects it in
improvement.

symbol
T
P
R
W
M
U
Tx

Figure 4 shows contribution ratio of each principal
component. In this research, it paid attention to it to thefourth principal component. In this research, we paid
attention to the fourth principal component. Table 2
shows factor loading. The first principal component
explains total 43%. In the first principal component, it is
the point that electric wiring and operation device arewatched by with interest.

1. Extraction by principal component analysis
In addition, we searched whether a student was
conscious of a weak point. In addition, we searched
whether a student was conscious of a weak point. Usingquestionnaire for experiment, it did self-evaluating
about element of experiment such as wiring or
operation of device. Table 1 shows abbreviation to use
by question item and this report of questionnaire. The
experiment of 8 themes was analyzed using principal
component analysis.

Regulated
power supply

Teacher’s viewtransition

95[s]

Cable
39[s]

Resistor
7

11

54[s]

18

Experimenttext

Measurement device
189[s]

90[s]7Fig. 2 Transition graph of a teacher

Regulated
power supply

Student’s viewtransition
cable

78[s]

Resister

Cable

Resistor

30[s]

13[s]

7

Regulated
Power supply

29

12

0’23
4’27

1’03

2’33

Measurement
device354[s]

text
Amperemeter

23

Experimenttext

130[s]
22

Fig.1 The condition of experiment©ISAROB 2011

Fig. 3 Transition graph of a student

484

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

other

Table 3. Job and abbreviation

Pri.4

job

Pri.1
Pri.3
Pri.2
Fig. 4 Contribution ratio of each principal component

abbreviation

Reads text

TXT

Electric wiring

WC

Device operation

SET

RecordREC

Reading of value

GM / TES

Table 2. Factor loading of principal component
Item / principal

Pri.1

Pri.2

Pri.3

Pri.4

Experiment time

-0.09

0.14

-0.09

-0.03

Preparations

-0.13

0.36

-0.39

0.78

Re-verification of text

-0.09

-0.76

-0.34

0.42

Electric wiring

-0.69

-0.31

-0.01

-0.13

Operation of device

-0.64

-0.17

0.06

0.16

Understanding of principal

-0.12

0.17

-0.07

-0.13

Understanding of text
description

-0.27

0.81

0.57

-0.23

The speech production protocol was classed in action
instructions, situation instructions and self speech
production. The subject intended for five students of abeginner. It recorded a condition of the experiment
which measured each subject. Subject sent own action
in experiment on a voice and explained it. The analysis
divided it into 22 steps as it was shown experiment toprocedures text. Each step paid attention to it by the
number of protocol manifestation. It shows abbreviationand job in table 3. It shows association of activity
abbreviation and description representing step in table 3.
As the situation which represented a weak point ofexperiment, it examined two situations. One is thesituation which self speech production increases. The
others are the situation that productive time gets longer.
On these points, we take it as a weak point of subject.
Fig.5 shows the thing which counted number of self
speech production every step. This is average of fivesubjects. In the job that manifestation of protocol isfound, a big thing of value is SET 1, WC1 and TES3.
Furthermore, it shows the productive time which eachstep took by fig.6. This is difference of productive timeof a student and a teacher by each step. This calculates
average every step.

As a result it shows that there is difference whether astudent is weak with it. From this thing, it is suggested
by improvement of experiment that we should focus ourattention on device operation and electric wiring. It was
important that understanding of text was the fourthprincipal component from the second. It was relatedwith reconfirmation of text. As a result of experiment,
there is not good judgment with a little knowledge andexperience at once. Therefore we think that device
operation, electric wiring and text appear as the element
which it is important of experiment.

2. Extraction by protocol analysis
Device operation and electric wiring understood thething that it was important in improvement from viewtransition and result of principal component analysis.
But concrete weak point and remedy are not clear.
However, we cannot make clear a concrete weak points
and remedies. Furthermore, detailed analysis becomes
necessary. There we use protocol analysis in order tospecify weak point in experiment. Protocol is a gesturein an action of subject and record of speech production.
Analysis of protocol is used for a problem point inaction and detection of the cause. It considers causewith the frequency. By this report, it classed the speech
production protocol which it acquired.

©ISAROB 2011

Fig. 5 The number of speech production every step485

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

the student who was inexperienced in experiment.
Necessity to support the weak point by learning before
the fact was suggested.

ACKOWLEDGMETS
A part of this research was supported by educational
cooperation project in Toyohashi University oftechnology.
Fig. 6 Difference of productive time of a student anda teacher by each step

REFERECES[1] M. Fuse, M. Suzuki, N. Memoto, T. Michikami andS. Ozawa, “Development of video on demand eLearning system for general physics,” vol. 29, pp.
439-444 Mar. 2006.
[2] K. Masuda, and S. Chiba, “A method of motivationin the engineering education to third-year students
and its effects,” vol. 29. pp. 337-342, Mar. 2006.
[3] K. Kondo, K. Yamamoto, M. Okuno, S. Suzuki andF. Yamada, “Some approaches to improvement ofstudent experiment of department of electrical andelectric engineering and results of questionnaires tostudents on these approches,” vol. 27. pp.341-346,
Mar. 2004.
[4] O. Higa, R. Suzuki, K. Noguchi, S. Kamisato, S.
Nozaki, T. Satake and S. Higa, “Relationship ofeyes of student and learning level in engineeringexperiment,” Forum on information technology(FIT2006), vol. 5, pp.439-440, Sep. 2006.
[5] K. Noguchi, S. Kamisato, O. Higa, S. Higa, S.
Nozaki, T. Satake, R. Suzuki, “Consideration ofeffective education technique in engineeringexperiment,” E2-1. Nov. 2006.
[6] S. Kamisato, K. Noguchi, O. Higa, S. Higa, S.
Nozaki, T. Satake, R. Suzuki,, “Relationshipbetween operation of device and arm trajectory inengineering experiment,”
7th SICE system
integration division annual conference (si2006),
1D3-2, Dec. 2006.
[7] S. Kamisato, K. Noguchi, S. Nozaki, A. Okuda and R.
Suzuki “Kaizen of experimental education method
focused on evaluation devise operation,” 4thIEEE International conference on mechatronics,
ThA1-A-1, 2007.
[8] K. Noguchi, S. Kamisato, and R. Suzuki “Activity oftechnical skill education based on motion analysis,”39th ASEE/IEEE Frontiers in Education conference(FIE2009), W1E, 2009.

Work having difference of time is SET 1 and WC1. As aresult it can suppose that the first electric wiring andinstrument setup is the job which is a weak point with a
lot of subject.
3. Consideration about learning point
We specified a weak point of a student. But it has tolook for concrete cause for improvement. We focused
our attention on protocol in electric wiring job ofsubject. At first, as for the confirmed weak point, speech
production explaining relation of connection of device
is not found. The second draws up circuit with layout
completely the same as circuit diagram. The third
identifies circuit diagram every one place of electric
wiring. In the subject that electric wiring was not a
weak point, the speech production which explained
relation of connection was found. In addition, the circuit
which it drew up was not the same as layout of circuit
diagram. As a result they suggest that they use circuit
diagram in order to make sure of relation of connection.
This inclination was not found in case of weak point
electric wiring. To the student who is inexperienced inexperiment, it is important that confirmation of device
connection. Furthermore, when learning effects rise by
supporting it on default setting of device, it is expected.

IV. COCLUSIO
Purpose of this research is improvement ofengineering experiment by visualization of skills. We
visualized the teaching materials movie of the teacher
and student. Furthermore, we identified a weak point ofa student using principal component analysis protocol
analysis. As a result it was supposed that the firstelectric wiring was the job which was a weak point witha lot of subject. In particular inclination was found to©ISAROB 2011486

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Prediction of Electron Flux Environment at Geosynchronous Orbit
using Neural Network Technique
K. Kitamura, Y. Nakamura,Tokuyama College of Technology, Gakuendai Shunan, Yamaguchi, 745-8181, Japan(Tel : 81-824-29-6271)(kitamura@tokuyama.ac.jp)
M. Tokumitsu, Y. Ishida,
Toyohashi University of Technology, Hibarigaoka, Toyohashi, Aichi, 441-8580, Japan(tokumitsu@sys.cs.tut.ac.jp, ishida@tutkie.tut.ac.jp)
S Watari,
National Institute of Information and Communications Technology, Koganei,Tokyo, 184-8795, Japan(watari@nict.go.jp)
Abstract: In this study a neural network technique is adopted to a prediction of the electron flux at the geosynchronous
orbit using several solar wind data obtained by ACE spacecraft and magnetic variations observed on the ground as
input parameters. The parameter tuning for back-propagation leaning method is attempted to the feed-forward neuralnetwork. As a result, the prediction using the combined data of solar wind and ground magnetic data shows the highest
prediction efficiency of 0.61, which is enough to adapt the actual use of the space environment prediction.
Keywords: Neural Network. Spacecraft, Internal charging

by the disturbance of the solarwind (that is the highspeed plasma stream flowing out from the sun).
Some investigator tried to predict the electron flux
variations by the linear prediction filter using theobserved space environment data [1][2]. In these studies,
the accuracy of the prediction was not enough to adopt
the actual operations, though enhancement itself waswell reproduced in 24hours-later predictions.
Fukata et al. [3] first attempted the prediction of theelectron flux variations using the neural network model.
This model well predicted the variation of the electron
flux during the disturbance period of the spaceenvironment. However, this model was developed by
the statistical learning using only disturbed-days data,so that the transition from the quiet days to disturbed
days (commencement of the electron flux enhancement)
was not reproduced well.
The objective of this study is to establish theprediction system to be applied to an actual space
operation. We first attempt the prediction of the highenergy electron flux enhancement by means of theneural network using the much amount of the dataobtained by the spacecraft and ground network
observations. Then we validate the accuracy of theprediction by using the prediction efficiencies (PE) for
the various combinations of the input parameters.

I. INTRODUCTION
From 1950’s many spacecrafts have been launched
into the near earth space and over 300 artificial satellites
are operated as the significant infrastructure today.
However such satellites are exposed in the severenvironment on their orbits. In particular, the orbit in therange from 2 to 7 RE (Earth Radii) which include thegeosynchronous orbit is well known as “Radiation Belt”
filled with the high-energy particles. The high-energy(>106 eV) electrons are thought to be a cause of internal
charging which give rise to the serious troubles on theelectric circuit onboard the spacecraft.
In order to avoid the significant problems on thesatellite systems, it is important to predict the spaceenvironment especially for the high-energy particles.
The physical element process of the high-energy
electrons flux variations has not been understood wellthough much number of observations have been
conducted by many investigators. Thus, it is difficult topredict the electron flux variations by the computer
simulation based on the theoretical models, so that somestudies of the electron flux predictions using theempirical models based on the statistical analysis wereattempted. In the previous observations, it is wellknown that the electron flux shows the large
enhancement during the magnetic storm which is driven©ISAROB 2011487

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

II. DATA SET

magnetic observatories used in calculation for the Dst
index are not at high latitude but at low latitude, the Dst
index is generally utilized for the definitive scale of themagnetic storms (which is major electromagnetic
disturbances in the space environment). The highenergy (>2MeV) electron flux (E) at thegeosynchronous orbit is observed by the GOES 10
satellite operated by National Oceanic and Atmospheric
Administration (NOAA). In the preparation for the
analysis, we removed the error data from the hourly
data for each observed data in the interval from 1998 to2006, and got 74376 samples for each hourly data.

The electron flux enhancement generally occurs
during the magnetic storms which have some precursor
variations in the other monitored data.

III. Neural Network
In this study, since the output data of the model is
24-hours-later prediction, the output data could not
physically affect the past data used as the input
parameter in actual causality. Thus, we adopted thefeed-forward neural network model with the backpropagation leaning method to predict the 24-hourslater electron flux variations. Fig.2 represents theschematic illustration of the network model used in thisstudy. The network consists from arbitrary number of
middle layers which also consist from arbitrary numberof neurons.

Fig1. Example data of the electron flux enhancement
The bottom panel of Fig.1shows the electron flux
variations for 27days in 2005. The horizontal axis is
Day of year (Jan. 1 = 1 and Dec. 31 = 365). The
electron flux increases in 121 and 136 days and exceeds
the horizontal dashed line which indicates the alert level
for the spacecraft’s interference. On the other hand, theobvious precursor could be seen (in the vertical dashedlines), that is, the increasing of the solarwind velocity(top panel) and AE index (third panel) precede theelectron flux enhancement by 2days. The Dst index(second panel) shows the sudden decreasing preceding
the electron flux enhancements.
In this study, we use the solarwind data which isobserved by Advanced Composition Explorer (ACE)space craft. The dataset of the solarwind consists of thevelocity (Vsw), north-south component of the magnetic
field (Bz) and 3-days integration of epsilon parameter( (which is calculated from the velocity andmagnetic field and is consistent with theelectromagnetic poynting flux from solar wind to theearth). The (Auroral Electrojet) AE index is determinedfrom the magnetic field variations observed on the highlatitude ground observatories and is proxy of the
Auroral activity due to the solarwind disturbances. The
Disturbed field during Storm Time (Dst) index is also
determined by the ground magnetic variations. Since the©ISAROB 2011

Fig.2 schema of neural network
All the data obtained from 1998 to 2006 werenormalized in the range from -1 to 1 to be used as input
parameters. In each neuron, input parameters areweighted with an appropriate weight and a sum of theweighted input is send to the transfer function oftransient sigmoid. As a result, one output data can beobtained through the reiteration of above mentioned
scheme. Then output parameter ( 𝑂𝑃 ) for the input
parameters with the arbitrary pattern (P) is comparedwith the observed 24-hours-later electron flux ( 𝑇𝑃 )488

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

which is supervised data, and validate the following
error function (E).

the number of neurons and middle layers is in theorderly manner. The maximum PE of 0.61 is shown
under the condition that number of neurons is more than
6 and number of middle layers is less than 4. This resultmeans that the prediction of the electron flux shows thebest performance using the both data observed in thespace and on the ground.

1
𝐸 = ∑ (𝑂𝑃 − 𝑇𝑃 )22 𝑃
The appropriate weight for each neuron is determinedby the steepest descend method to minimize E. In thisanalysis the reiterating calculation stopped under thecondition that the error function E reaches less than 0.01.

IV. Result of the analysis
In the training process, various combinations of
input parameters were attempted to evaluate theaccuracy of the prediction. In order to quantitatively
evaluate the accuracy of the prediction, we calculated
the prediction efficiency (PE) which was adopted by
NOAA Space Weather Prediction Center (SWPC) [4] as
𝑃𝐸 = 1 −
𝑉𝐴𝑅 =

𝑀𝑆𝐸
𝑉𝐴𝑅

𝑁
1
∑ (𝑥𝑖 − 𝑥̅ 𝑖 )2𝑁
𝑖=1

Fig3. Result of the network turning for 3 cases

𝑁
1
𝑀𝑆𝐸 = ∑ (𝑓𝑖 − 𝑥̅𝑖 )2
𝑁
𝑖=1

For the case 3, the comparison between thepredicted electron flux and observed electron flux isshown in Fig4. The enhancement of the electron flux iswell predicted in entire variations though predicted line
(solid line) sometimes shows the over estimationcomparing to the observed line (dotted line).

where 𝑥𝑖 and 𝑓𝑖 are observed and predicted values ofelectron flux, respectively. Thus PE is based on themean square error which is normalized by variance ofthe observed values.
We classified the input parameters into three cases,that are, (1)data basically obtained in the space (E, Vsw,
Bz, , UT), (2)data basically obtained on the ground (E,Dst, AE, UT), (3)combination data obtained both in thespace and on the ground (E, Vsw, Dst, AE, UT), here UTmeans Universal Time. For each case, the training was
conducted using the data from 1998 to 2006 except
2003. It is known that the solarwind had been much
disturbed in 2003 due to the large coronal hole appearedon the surface of Sun, so that we attempted thevalidation of the prediction to calculation of PE for thepredicted electron flux with the observed data in 2003.
We attempted the various combination of thenumber of middle layers and neurons for above threecases. The result is shown in Fig3. In cases 1 and 2, therelationship between the number of neurons and middle
layers are not clear and the maximum PE is less than
0.58. On other hand, in case 3, the dependence of PE on©ISAROB 2011

Fig4. Comparison between the predicted and observed
variations

489

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

ACKNOWLEDGEMENT

The both predicted and observed values for all data
in 2003 are shown in the scatter plot of Fig4 to validate
the prediction accuracy in more details. The vertical andhorizontal axes are the observed and predicted values,
respectively. The plots are fairly scattered in thecondition that the flux is less than 4, which means thepredicted values sometimes deviate from the observed
values. In the condition that flux is more than 4, theplots become to concentrate on the dashed line, though
the distribution of plots shows the overestimate
tendency of ~10%. In terms of the application of theelectron flux prediction to the actual space operation,~10 % of the overestimation could be acceptable to
avoid the risk, by contrast the underestimation of thepredictions connotes a significant risk for real
operations.

The solarwind data obtained by ACE spacecraft andthe high-energy electron flux data are provided from
National Oceanic and Atmospheric Administration(NOAA). The Dst and AE indices are provided fromWorld Data Center (WDC) for Geomagnetism, Kyoto.

REFERENCES[1] Baker DN, McPherron RL, Cayton TE et al.
(1990), Linear prediction filter analysis of relativisti
c electron properties at 6.6 RE, J. Geophys. Res, 95,
15133-15140[2] Koga S, Koshiishi H, Matsumoto H et al. (2006), Prediction of electron flux variations at radiation
belt, JAXA-SP-05, P24 (in Japanese)[3] Fukata M, Taguchi S, Obara T (2002), Neuralnetwork prediction of relativistic electrons at
geosynchronous orbit during the storm recovery phase:
effects of recurring, Annales Geophysicae, Vol.20, 947951.
[4] NOAA/SWPC (2008), Relativistic Electron Forecast
Model Documentation, http://www.swpc.noaa.gov/refm/doc/REFMDoc.html

Fig5. Scatter plot of the predicted and observed
valuesVI. CONCLUSION
In the present study, we attempted the high-energy
electron flux variations using the feed-forward neuralnetwork with back-propagation learning method. We
could summarize the present study as follows. (1) The
maximum PE shows 0.61 with input parameters
obtained both in the space and on the ground. (2) The
accuracy of the prediction increases with increasing anamount of the electron flux and tends to be an
overestimation of ~10%. These results indicate that thepresent neural network model could be adopted in thereal space environment forecast operation.

©ISAROB 2011490

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A Diagrammatic Classification in a Combinatorial Problem:
The Case of a Stable Marriage Problem
T. Hayashi, Y. Hata and Y. Ishida
Toyohashi University of Technology,
1-1 Hibarigaoka Tenpaku-cho, Toyohashi, Aichi 441-8580, Japan(Tel.: +81-532-44-6895; Fax: +81-532-44-6895)(E-mail: ishida@cs.tut.ac.jp)
Abstract: The Stable Marriage Problem (SMP) is a combinatorial problem to find stable matching between n womenand n men given a complete preference list of men over women and vice versa. An instance of SMP can be expressedby a bipartite graph with multiple (weighted) edges. By rearranging the graph, we use a diagram that involves several
constraints to visualize several symmetries. By the diagram, all the instances of the size three SMP (three women andthree men) are classified. The classification may be supported by the fact that the same class has the same stablematching.
Keywords: Graph equivalence, diagrammatic classification, stable marriage problem, stable matching, decompositi
on, bipartite graph, stable marriage graph.

I. INTRODUCTION

Section II presents the stable marriage problem andits graphical representation. Section III defines
equivalence of instances of SMP through graphhomomorphism (equivalence). Section III presents aclassification of SMP by diagrams (bipartite graph andmarriage graph).

After the proposal of the matching problem by Galeand Sharpley [1], many matching problems including
stable marriage problem (SMP) [2] have been studied
extensively. Stable marriage problem is a combinatorial
problem to find stable matching between n women andn men given a complete preference list of men over
women and vice versa. Stable marriage problem has
several variants such as a job assignment problem,
roommate problem varying the assumption on the set ofmembers to be matched.
Although the stable marriage problem has too strong
assumptions (such as complete list of preference) to be
applied to practical problems, symmetries embedded in
the problem keep us being attracted. Thus, we tried tovisualize the problem by network visualization tool [3].
This paper further focuses the decomposability of the
problem by involving diagrams such as a bipartite graphand a stable marriage graph [4, 5].
As a preliminary example, we presented aclassification of indecomposable structure of size three(three men by three women) SMP. Decomposition of the
problem may be a first thing to do in tackling a complex
problem. For SMP, component decomposition [2] andweak (but applicable to the preference matrix)
decomposition [6] have been studied. In classifying thesize three SMP, we tried to generalize a concept ofmutual infatuation (a pair of man and woman who rank
the partner first). A cycle of fist rank relation plays animportant role in classifying the SMP.

©ISAROB 2011

II. STABLE MARRIAGE PROBLEM
The Stable Marriage Problem (SMP) assumes n
women and n men each of them has an ordered
preference list (or a ranking) without tie to the opposite
sex. As in the example shown in Fig. 1, the men m2 hasan ordered preference list (w3, w2, w1) or a ranking (3, 2,1), which means m2 likes w3 best, and he prefers w3 to
w2, w2 to w1. That is, there is an injection (one to one,but not necessarily onto) mapping from a set of women(men) to an element of permutation group of size n suchas shown in the ranking by each person (Fig. 1). We will
use a graph that extracts a specific rank (such as the firstpreference) in classification.
Under the above assumptions, SMP seeks for thecomplete matching between n women and n men (abijection from n women to n men), which satisfies
stability. The stability requires the concept of blocking
pair. Two pairs (mi, wp) and (mj, wq) are blocked by thepair (mi, wq) if mi, prefers wq to wp and wq prefers mi to
mj. For example, a pair (m2, w3) and (m3, w2) will beblocked by the pair (m2,w2). A complete matching
without being blocked is called stable matching.

491

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

prime ness could be defined on other mathematicalobjects with structures such as instances of Stable
Marriage Problem. With expectation of the importantrole played by “primeness” in the instances of SMP, westudy several type of decomposition in instances of SMP,
and indecomposable structure of them.
Decomposition of SMP had been traced back to thebook by Gusfield and Irving [2], and have been studied[6]. Here, we focus on a decomposability that will benaturally considered by two independence concepts: theconcept of extended individual and an independent pair.
The extended individual is a set of individuals that canbe dealt as if one individual in finding stable matching.
The independent stable pair is the two entities that canbe paired without affecting any other possible pairs, andhence the pair appears in any stable matching. The
decomposition leads to indecomposable structures,
yielding a full classification of SMP with size three.
As an example of extended individuals, consider thecase of SMP with nationality in members. Half of themembers are Japanese and the other half American bothin female and male. Assume any woman (man) prefersman (woman) with the same nationality to man(woman) otherwise. Then Japanese women (men) as
well as American women (men) can be treated as if they
are one woman (man). We use this example also as an
example of trivial decomposition of SMP. In fact, this
SMP is actually two independent SMPs: one between
Japanese women and men and another between
American women and men.
As a trivial example of independent pair, woman andman who mutually rank first can be paired without
affecting any other paring. A nontrivial example would
be the pair who would turn out to be mutual first rank
only after removing the above the trivial independentpair. We call this latter one the independent pair hiddenby the former obvious independent pair.

An instance of SMP can be expressed by a multiple(arcs) bipartite graph in a straightforward way such thatan arc of k-th order from mi, to wj is directed if mi ranks
wj j k-th. Another graphical expression of an instance isthe stable marriage graph [4,5]. For SMP with size n byn, n2 nodes of possible pairs (mi,,wj) are placed in amatrix form. For a row of mi,, an arc from (mi,,wp) to (mj,wq) is placed if mi, prefers wq to wp. Fig. 2 shows astable marriage graph corresponding to the instanceshown in Fig. 1. A stable marriage graph can be
simplified (as in Fig. 2). In a simplified stable marriage
graph, redundant nodes may be abbreviated if it can bederived by a transitivity. For example, if m1 prefers w1 to
w2, and w2 to w3, it follows that m1 prefers w1 to w3hence the corresponding arc is abbreviated.
Rank

Node

1 23

m1

231

m2

2 13

m3

Pair
(Edge)

Node

Rank

w1

123

w2

21 3

w3

231

Fig. 1. An illustration of Stable Marriage Problem withsize three.

Fig. 2. A stable marriage graph (left) and its simplified
one by removing redundant arcs with transitivity (right).

III. EQUIVALENCE OF INSTANCES THROUGH GRAPH

IV. CLASSIFICATION VIA DIAGRAMS
1.

Indecomposable Structure by Bipartite Graph
Trivial example of the indecomposable structure in
SMP is those with size one (Fig. 3). Obvious example
may be the size two (Fig. 4 ), for we need to eliminate
the graph of the size one (otherwise it can be
decomposed to two size one SMPs). In the enumeration,
we do not distinguish the ones that can be mapped byPrime numbers remain to be a mystery not only indiscrete mathematics such as Integer Theory but incontinuous mathematics such as theory of functions.
Prime numbers also plays an important role of an engine
for mathematics and other applied mathematics such as
cryptography.
Prime numbers are defined on the set of integers bymeans of factorization with division operation. The©ISAROB 2011492

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

the label exchange within the same sex, and exchange
the sex with all the members.

Decomposa
ble size 3
SMP

class 1

Indecompo
sable size
3 SMP

class3class 2class4class5class6

Fig.5. Classification of size 3 SMP by a graphhomomorphism in the reduced stable marriage graph.
3.

Classification of Indecomposable SMP
We will further classify three types based on the
bipartite graph of second preference.
Type 0 (Fig. 6) is the simplest, since they can bemapped to class 3, 4, 5, and 6, respectively, by the
number of cycles of length two in the bipartite graph ofsecond preference (Fig. 6).

Fig.3. Indecomposable structure of size one(left) and size two (right). Nodes with different color indicate persons with different
sex, and arcs indicate the fist preference.

Cycles of 2nd
pref.

ReducedSMP graphclass 3Fig. 4. Indecomposable structure of size three
SMP. Arcs of the fist preferences are shown.
They will be called type 0 (above),1 (below, left) and 2 (below, right)

class 4

2.

Indecomposable Structure by Stable Marriage
Graph
Stable marriage graph can be reduced by removing
nodes dominated by a man optimal node or a woman
optimal node, since the pairs corresponding to the nodes
dominated do not appear in any stable matching.
By a graph homomorphism of reduced stable
marriage graph, all the instances of SMP with size three
can be classified to the six classes (Fig. 5). Among them,classes 3, 4, 5 and 6 correspond to indecomposable
structure.
We will further classify the SMP with size three by amultiple (arcs) bipartite graph, since several operations
are required to obtain the reduced stable marriage
graphs, although once obtained they even reveal thestructure of stable matching.

©ISAROB 2011class 5class 6
Fig. 6. Indecomposable structures of type 0,
size three SMP.
Type 1 (Fig. 7) can be divided into five subtypes,
one of which belongs to class 4 (Fig. 5) and the rest toclass 3. In the graph, the arcs (of second preference)
included in a cycles of length four that appears in thefirst preference (Fig. 4) but opposite directions areomitted. It can be observed that when the class is493

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011upgraded from class 3 to 4, the arc is added and thenumber of cycles with length two increases. The graphobtained by adding an arc to the original graph is placed
to the right of the original graph.
Type 2 (Fig. 8) are divided into twenty subtypes.
Among them, twelve subtypes belong to class 3, sevento class 4, and one to class 5. Again, it can be observedthat when the class is upgraded, the arc is added. The
number of cycles with length two increases when theclass is upgraded from 3 to 4 and 4 to 5, however, it is
not true from 3 to 5, although the length 4 cycle with thefirst preference and second preference mixed appears in
the jump from the class 3 to 5.

Indecomposable SMP (size 3 by 3)
Class 3
Class 4
Arcs of 2nd preference

Class5

Indecomposable SMP (size 3 by 3)
Class 3
Class 4
Arcs of 2nd preference

Fig. 7. Indecomposable structures of type 1,
size three SMP.

Fig. 8. Indecomposable structures of type 2,
size three SMP.

V. SUMMARYREFERENCES

We first classified all the instances of the size threestable marriage problem (SMP) into six classes by the
reduced stable marriage graph. Among six classes, fourclasses correspond to indecomposable structure. Then,
we have classified all the instances of indecomposable
stable marriage problem with size three into three types
based only on the bipartite graph of the first preference.
Each type can further be classified to subtypes based on
the second preference structure. By mapping thesestructure to the three classes induced by the reduced
stable marriage graph, it is shown that the cycle in a
bipartite graph plays a curtail role.

[1] Gale D and Shapley LS (1962), College
admissions and the stability of marriage, American
Mathematical Monthly, 69:9-15[2] Gusfield D and Irving RW (1989), The Stable
Marriage Problem: Structure and Algorithm, MIT
Press, London[3] Morizumi Y, Hayashi T, Ishida Y. (2009) A
Network Visualization of Stable Matching in
Stable Marriage Problem, Proc of AROB
[4] M. Balinski and G. Ratier, Of stable marriages andgraphs, and strategy and polytopes. SIAM Review
39 (1997) 575 – 604.
[5] Nemoto T. (2002) Handbook of Applied
Mathematical Programming,
ASAKURASYOTEN, 779-830 (in Japanese)[6] Veklerov E (1989), On the decomposability of thestable marriage problem. BIT Numerical
Mathematics, 29(1): 41-46©ISAROB 2011494

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A Neuro PID control of Power Generation using Low Temperature Gap
Kun-Young Han, Hee-Jae Park, and Hee-Hyol Lee
The Graduate School of Information, Production and Systems, Waseda University, Kitakyusyu, 808-0135, Japan
2-7 Hibikino, Wakamatsu-ku, Kitakyushu, Fukuoka 808-0135, Japan(Email address: kyhan@akane.waseda.jp)

Abstract: Recently, renewable energy is increasingly attractive in solving global problem such as the environmental
pollution and energy shortage. Among varieties of renewable energy resource, power generation using low temperature
gap has received much attention of researchers. However, this system is difficult to control because each of thecomponents of this system, such as heat exchanger, working fluid and turbine, has a dynamic characteristic or nonlinear
factor. In order to overcome this problem, PID controller based on neural network for power generation using low
temperature gap is designed to keep the stable speed of the steam turbine in real environment.
Keywords: Power generation, Evaporator, Turbine, PID control, neural network
the complex systems. In this paper, an improved neuralnetwork, which shows excellent performance fornonlinearity is applied to the power generation using
temperature gap. Initial values of a neuro PID controller
use the parameters obtained from the linearized model.
This paper composed as follows. The power generation
system using low temperature gap and each of thecomponents such as an evaporator and a condenser
based on heat balance equation, and a turbine element
are modeled in Section 2. In the section 3, a traditional
PID control is introduced and a neuro PID controller is
proposed for the power generation using low
temperature.

I. INTRODUCTION
A strong interest in distributed generations using
renewable energy such as wind and solar energy havebeen attracted because the fossil energy will be
exhausted in the future and its use causes theenvironmental issue. Among them, there is somerenewable energy that has not yet been used, such as theheat from hot springs or exhaust heat from factories.
However, the power generation using these unused
renewable energies can only produce a small amount ofheat but also have relatively low heat efficiency.
Moreover, these heat sources are scattered and theirscale are small[1]. Therefore, the suitable control
methods must be developed to overcome these problems
in that capacity and scale.
On the other hand, the power generation using the
low temperature gap used in this paper includes
dynamic characteristics and nonlinear factors. Although
this system contains these complex factors, most of thecontrol algorithms are based on linear models. The
linear models deduced from step responses and impulse
responses are desirable, because they can be identified
in a straightforward manner. In addition, a goal for mostof the applications is to maintain the system at a desired
steady state, so a precisely identified linear model issufficiently accurate in the neighborhood of an
operating point. As this point of view, the power
generation using low temperature gap based on theapproximate linear models are designed in this paper.
PID control is used in process field in generally.
Despite PID controllers are applicable to many
controlled system, it performs poorly in complex system.
Therefore, PID controller cannot be applied directly in©ISAROB 2011

II. Dynamic modeling of power generation
using low temperature gap
The main components of the power system using the
low temperature gap with closed cycle are constructedof the heat exchanger (evaporator and condenser),turbine, and pumps. The simple structure of the power
generation system using the low temperature gap isshown in Fig.1.

Fig.1. Power generation using the low temperature gap
with closed cycle.
2.1 Evaporator model
The evaporator is a shell-and-tube type as shown in
Fig.2. The working fluid is a liquid and moves to the495The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

evaporator by a pump. And then the working fluid isboiled and changed to steam in the evaporator by hotwater.
Modeling assumption
1) Working fluid evaporating pressure is constant instatic state.
2) Latent heat of working fluid keeps no change.
3) Evaporating flow supplies a quantity of heat.
The heat balance equation of hot water, the working
fluid, and supply heat are defined as follows [2].

υ h (∂θ h / ∂x) + ( KU / ωh )(θ h − θ s ) = 0
Gυ (h′′ − h′) = Q

μT =
a1

∫

A1
AE

M E [υ (α T ) ]1/ 2(PE − PT )1/ 2

dx1
υ= K3 () , ( 0 ≤ x1 ≤ 1)
dt
υmax

⎤
dυ
1 ⎡ PE′ − PT′ μT
ξ (t )=− (1 − η )− ηυ 2 ⎥⎢τ T ⎢⎣ PE′ − PT′ υ μT
dt
υ
⎥⎦

(4)
(5)
(6)

where, α T is a area of divergence of the valve, A1/AE isa cross-sectional area of a pipe leading from theevaporator to the turbine, PE is a vapor pressure at theevaporator outlet, PT is a vapor pressure at the turbine
outlet, K3 and a1 are the gain, vmax is a maximum
speed of the turbine blade, ME is a constant parameter,
τ T is a system constant, η is a coefficient of loss, ξ

(1)
(2)

L

Q = KU (θ h − θ s )dx

αT

(3)

0

is a electric load, and PE′ , PT′ , μT are desire values.

where, υ h is a speed of a moving fluid of hot water,θ h is a temperature of hot water, x is a distance, K is aheat transmission coefficient, U is a heating surface area
per unit time, ω is heat capacity per unit time and θ s isa temperature of saturated water vapor .

Fig.3. Block diagram of turbine
Linear approximate turbine model
The linear approximate turbine model is obtained byapproximating the turbine. Fig.4 shows the linear
approximate turbine model, which includes 1st and 2ndorder delay elements and dead time element.

Fig.2. Structure of the evaporator
2.2 Turbine model
The turbine is composed of the steam control valve
and a rotor blade. The block diagram of the turbine isshown in Fig.3. The steam is moved from theevaporator to the turbine blade through the steam
control valve. It adjusts the flow of steam to the turbine
blade. The equations of the turbine systems are given as
follows [3]. Mass rate and a control valve of theworking fluid steam are defined by Eqs.(4),(5),respectively, and the turbine blade speed is defined by
Eq.(6)

Fig.4. Linear approximate turbine model
2.3 Modeling of power generation system as MIMO
systemThe evaporator and the condenser are modeled as amulti-input multi-output model(MIMO) illustrated in
Fig.5.

Fig.5. MIMO system for the Power generation©ISAROB 2011496

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig.7. Block diagram of Power generation system with PID controller

Fig.8. Result of control using PID controller

III. Neuro PID controller
3.1 PID Control
A schematic of a system with a PID controller isshown in Fig.6. The PID controller compares ameasured process value y with a reference value r. The
difference or error e is then processed to calculate a newprocess input u. This input will try to adjust themeasured process value to the desired value. PID
controller is widely used in industrial control systems.

Fig.9. Structure of neuro PID controller
BP network structure
The BP neural network is a multi-layer perception
neural network, which consists of three components:
input layer, hidden layer, and output layer. The output
layer corresponds to the values, which are threeadjustable parameters Kp, Ki, and Kd, as shown Fig.9.
Therefore, each of parameters can be adjusted by self
learning ability of the neural network. And the bestvalues of them can be obtained corresponding to the
outputs of BPNN with a certain optimal control law.
The PID algorithm is given by Eq.(8).
u (t ) = u (t − 1) + Δu (t )Δu (t ) = u (t − 1) + K p (t )(e(t ) − e(t − 1)) + K i (t )e(t )(8)+ K d (t )(e(t ) − 2e(t − 1) + e(t − 2))

Fig.6. Block diagram of a PID controller
PID algorithm is formed as follow,

d
e(t )
(7)
0
dt
where, Kp is proportional gain, Ki is integral gain, Kd isthe differential gain, and e(t) is the error.

∫tu (t ) = K p e(t ) + K i e(τ )dτ + K de(t ) = r (t ) − y (t )where, u is the control value, r is the reference value,
and y is the actual output value. On the other hands, theinputs and outputs of the hidden layer in the neuralnetwork are given by Eq.(9).

3.2 PID tuning
There are several methods for tuning the PID
parameters. The parameters are obtained by using thestep response method and the ultimate sensitivity
method. Fig.7 represents the block diagram of thepower generation system with PID controller, and Fig.8
is a result of control using PID control.

neti2 (t ) =

∑w O
( 2)
ij

(1)
j

j =0

Oi( 2) (t ) = f (neti2 (t )) (i = 1,2, L, Q )Q
netl(3) (t ) =wli(3) Oi( 2) (t )l(3)Ol (t ) = g (neti3 (t )) (l = 1,2,3)

3.3 Neuro PID Controller
The PID controller does not lead to good
performance. In this paper, PID controller with a neuralnetwork is considered, and shown in Fig.9.

©ISAROB 2011

M

∑

497

(9)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig.10. Block diagram of Power generation system with Neuro PID controller

Fig.11. Result of control using Neuro PID controller

Fig.12. Result of control of Neuro controller using the initial value based on linearized model
where, M and Q are the neuron numbers of the input
and hidden layer, respectively, w is the weight, net is the
input, and O is the output. The superscript (1), (2) and(3) denote the input, the hidden, and the output layer. f
and g are activation functions of the hidden layer andthe output layer. Obviously, O(3) is Kp, Ki and Kd.
Parameters of Kp, Ki and Kd, which are obtained fromthe linear approximate model, are used as initial values
in this paper. The weights of BPNN are updated
according to the gradient-descent algorithm.
Δw(t ) = −η[∂E (t ) / ∂w(t )] + αΔw(t − 1)
(10)where, η and α are defined as learning rate and inertia
factor, respectively. The inertial item αΔw(t − 1) is added
to accelerate convergence. The manual differentiation
method is used to increase the accuracy the value of∂E (t ) / ∂w(t ) in Eq.(10). The algorithm of updating theweights for the output layer is given as follows.
Δwli(3) (t ) = αΔwli(3) (t − 1) + ηδ l(3) Oi( 2) (t )∂y (t ) ∂u (t )
δ l(3) = e(t ) sgn()
g& (net l(3) (t ))3∂Δu (t ) ∂Ol

using BPNN is represented in Fig.11 while Kp, Ki, Kd do
not use as initial value. A result of PID control using
BPNN is shown in Fig 12 while the initial values of Kp,
Ki, and Kd are used based on the linearized model.

V. Conclusion
The power generation system, which consists of theevaporator, the condenser, and the turbine are designed.
The conventional PID controller has the advantage ofsimple structure, good stability, easy to engineering
implementation and so on. However, for the complex
system such as the power generation system using low
temperature gap using the traditional PID control ismore difficult to obtain good control performance. This
paper combines the BP neural network and PID control.
The PID parameters from the linear model are used as
initial value in the output layer of BPNN. The neurocontroller proposed is successfully implemented.

REFERENCES(l = 1,2,3)[1] Satomi S, K-Y H, J-S S, et.al (2010) A learning
control of unused energy power generation.
International symposium on AROB 15th.
[2] Hashizume T, Kawai S, Machiyama T (1981)
Experimental studies on the dynamic characteristics ofevaporator in the L.B.M. turbine system. Jpn Soc MechEng C 47(421):1161–1168.
[3] Owens WL (1982) OTEC plant response and control
analysis. ASME J Solar Energy Eng 104:208–215.

(11)
The process to acquire ∂E (t ) / ∂w(t ) is complex andfallible especially when the activation functions arecomplicated or Q is greater than 1. Besides that,
according to Eq.(11), the value of the item ∂y (t ) / ∂u (t )is substituted by its sign function. That is to say, it is set
to be 1 or -1. The impreciseness produced here can becompensated by η. The PID controller with BPNN isapplied as shown in Fig.10, and a result of PID control©ISAROB 2011

498

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

An Efficient Identification Scheme for Nonlinear Polynomial NARX Model
Yu CHENG, Miao YU, Lan WANG and Jinglu HU
Graduate School of Information, Production and Systems, Waseda University
Hibikino 2-7, Wakamatsu, Kitakyushu, Fukuoka, 808-0135, Japan(Tel/Fax : 81-93-692-5271)(chengyu0930@fuji.waseda.jp)

Abstract: Nonlinear polynomial NARX model identification often faces the problem of huge pool of candidate terms,
which makes the evolutionary optimization based identification algorithm work with low efficiency. This paper
proposes an efficient identification scheme with pre-processing to reduce the searching space effectively. Both the input
selection and term selection are implemented to truncate the candidate pool with the help of correlation based orthogonal
forward selection (COFS) algorithm and simplified orthogonal least square (OLS) algorithm, respectively. Then multiobjective evolutionary algorithm (MOEA) is used to identify the polynomial model in a relative small searching space.
Keywords: nonlinear polynomial model identification, input selection, term selection, efficientboth input selection and term selection methods [4],which can be seem as pre-processing for evolutionary
optimization based searching processing. Firstly,
correlation based orthogonal forward search (COFS)
algorithm is applied, which makes the orthogonal input
variable with maximum correlation coefficient of output
select one by one. The final library consisting of all thenecessary variables could be determined according to
the threshold by the user. Although it is consideredsomehow not very accurate, it could exclude most of theredundant inputs thus reduce the original candidate pooleffectively. Then, term selection will be implemented byusing correlation analysis and the simplified OLS
algorithm, hence the searching space could be limited
within small size. At last, MOEA is used to identify thepolynomial model in the reduced space. Simulations areintent to show the effectiveness of the proposed method.
This paper is organized as follows: Section 2 briefly
describes the problem to be solved. Section 3 discusses
the identification scheme in detail. Section 4 provides
numerical simulations to demonstrate effectiveness ofthe new method, and Section 5 presents the conclusions.

I. INTRODUCTION
Recently, nonlinear polynomial NARX (Nonlinear
AutoRegressive with eXogenous inputs) model has
attracted much attention because it has shown great
potential in the ability of approximating nonlinear inputoutput relationship. As a kind of effective approach,evolutionary optimization algorithms, such as genetic
algorithm (GA) and multi-objective evolutionary
algorithm (MOEA) have been commonly used for
identification of nonlinear polynomial NARX model.
However, it is still considered as a difficult task because
the size of candidate terms increase drastically withmaximum time delay of input-output data andnonlinearity of polynomial model [1].
So far, some related research has been devoted tocomplex nonlinear system identification. Ref. [2] claims
that a hierarchical encoding technique is introduced tobe effective for identifying polynomial models withrelatively high-nonlinearity, however, it is just an
improvement of GA, and as told by the authors, theprocess is still time-consuming and easily traps into a
local optimum. Moreover, a two-step scheme forpolynomial NARX model identification has been
proposed in our previous research [3]. It combines
heuristic optimization approach with pre-screening
process, in which the simplified orthogonal least square(OLS) based term selection method is used to formulate
a relative small searching space. However, the selectionof input variables is not considered, which hassignificant influence on the pruning of searching space.
In this paper, an efficient identification scheme is
proposed for nonlinear polynomial NARX model with©ISAROB 2011

II. STATEMENT OF THE PROBLEM
Consider a single-input-single-output (SISO)
nonlinear time-invariant system whose input-output
dynamics is described as=y (t )f ( x (t )) + e(t )(1)x (t ) = [ x1 (t), x2 (t)  , xn (t)]Twhere x(t ) is an n-dimensional input vector, y and edenote an output and white noise vector respectively,
and f () is a nonlinear function.

499

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

In

the

case

of

nonlinear

NARX

model,,

where l1 is the first important input variable selectedfrom the whole library, and the associated orthogonalvariable can be chosen as q1 = xl .

x(t ) =[ y (t − 1), y (t − 2),..., y (t − n y ), u (t − 1), u (t − 2),..., u (t − nu )]where y and u are the system input and output. nu and1

From the second step, every remained input variable
is orthogonalized with all the selected inputs, and eachcorrelation coefficient of output will be calculated.
Assumed there are already m-1 input variables havebeen selected, the m-th significant input is selected fromremaining pool, and orthogonalized with q1 , q 2 ,..., q m−1
as below
xT q
xT q
(4)
q = x − j 1 q −  − j m −1 q .

n y are unknown maximum delays of input and output.
However, many of the input variables are often
redundant and only a subset of them is significant. It is
pointed that there are at least two problems induced [5].
First, the model complexity will increase drastically with
the number of variables. Second, including irrelevant
variables leads to the over-fitting problem, and as aconsequence the model may tend to be oversensitive totraining data and is likely to exhibit poor generalization
properties.

j

t =1

N

i

|

i

N
2
i
i
=t 1 =t 1∑ (x (t ) − x ) ∑ (y(t ) − y )

.

(2)

N

ρi =

2

∑ (y (t ) − y )( y(t ) − y )
t =1

N

i

(6)

i

N
2
i
i
=t 1 =t 1

xi and y represent input and output variable, N is the∑ (y (t ) − y ) ∑ (y(t ) − y )2Therefore, based on the principle of simplicity,
Importance Index 1 should be given by|ρ |(7)
i(1) = Oiiewhere Oi is the order of the i-th term.

length of measurement. The bigger the correlation
coefficient is, the more important the input variable isconsidered.
In order to exclude the influence from other input
variables, orthogonal forward search algorithm is usedand significant input variables will be selected one byone. At the first step, let(3)
l1 = arg max1≤i ≤ n C (xi , y )©ISAROB 2011m −1

2. Term Selection for Identification
Candidate terms formed by all the selected inputs
are still too large if system nonlinearity is high.
Therefore, term selection is needed to reduce thecandidate pool. In this paper, two importance indices are
introduced to evaluate the contribution of each term.
A. Importance Index 1
Importance Index 1 i(1) is used to evaluate thecorrelation of each monomial term to the system output.
Let ρi denotes the correlation coefficient of monomial
term yi (t ) and system output y (t ) , calculated by

1. COFS Based Input Selection
To select significant inputs with big contribution tothe output vector from the whole input space,correlation coefficient could be used to evaluate therelationship between each input variable and the output
vector and denoted as
C ( xi , y ) = |qTm −1q m −1

where D is the subset contains all the remained input
variables. As the index to reflect the importance of an
input variable, ERR is used to represent the contributionof each input to the output vector. The input variables
have very little contribution to the output will beignored. Based on this fact, a threshold is defined whichshould make sure all the necessary inputs will be
included.

It is found that evolutionary optimization based
approaches are very efficient when the size of searching
space is not too large. Based on this fact, input selection
and term selection method are used as pre-processing
for the efficient identification scheme. Then MOEA isapplied to determine a set of significant terms to be
included in the polynomial model with the help of
independent validation data. Readers interested in thedetails of MOEA based identification may refer to ourprevious research [3]. In the following, input selection
and term selection for pre-processing will be discussed.

N

1

q1T q1

Then the orthogonalized input variable which has
maximum correlation with the output is chosen as
following(5)
l j = arg max i∈D C (qi , y )III. IMPLEMENTATION OF EFFICIENTIDENTIFICATION SCHEME∑ (x (t ) − x )( y(t ) − y )jB. Importance Index 2
Instead of recursive manner of the original OLS
method, a simplified OLS algorithm is introduced, inwhich all the terms are orthogonalized in one time, andthe contribution to the output of each orthogonal one is500

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

calculated and denoted as ERRi [3]. The principal of
simplicity is also applied here, and Importance index 2
is given byERR(8)i(2) = Oi ie
Although the two importance indices based term
selection scheme is not very accurate, it could be used
to prune candidate pool efficiently with all the necessary
terms included. Therefore, evolutionary optimization for
identification can work efficiently with a small
searching space.

research [3].
4. Identification Results
In both cases, the thresholds for COFS algorithm are
set as 0.9 and 0.98 after normalization in Example 1
and 2, respectively, then all the input variables selectedsatisfied with the threshold condition are listed in Tab.
1.
Table 1. Input selection for Example 1 and 2
Inputs selected in
Inputs selected inNo.
Example 1
Example 21
y(t-2)u(t-1)
2
3
4
5

IV. NUMERICAL SIMULATIONS
To show efficiency of the proposed identification
scheme, two experiments are simulated in this section,
which are tested with the assumption that the timedelays are unknown thus big values are given.

2. Systems under Study
Example 1: The system is governed by a polynomial
model, described by

Delta
Test
LinearOLSy(t-1),y(t-2),y(t-3),y(t-4),u(t-1),
7u(t-2),u(t-3)
y(t-1),y(t-2),y(t-3),y(t-4),y(t-5),8y(t-7),u(t-1),u(t-2)
y(t-1),y(t-2),y(t-5),y(t-7),y(t-9),
FOS-M
y(t-10),u(t-1),u(t-2),u(t-4),u(t13OD5),u(t-7),u(t-9),u(t-10)
Table 3. Results comparison for Example 2
size
MethodMinimal input subsets−0.5 y (t − 2) + 0.7u (t − 1) y (t − 1)y (t ) =+ 0.6u 2 (t − 2) + 0.2 y 3 (t − 1)
− 0.7u 2 (t − 2) y (t − 2) + e(t ).

Example 2: The system is a nonlinear rational modelstudied by Narendra in 1990y (t=) f [ y (t − 1), y (t − 2), y (t − 3), u (t − 1), u (t − 2)] + e(t )f [ x1 , x2 , x3 , x4 , x5 ] =5COFSy(t-1),y(t-2),y(t-3),u(t-1),u(t-2)
Delta5y(t-1),y(t-2),y(t-3),u(t-1),u(t-2)
Test
Linear20
with all the inputs selected
OLS
FOS-M
20
with all the inputs selectedOD
We can know from the table that COFS method could
get the minimal subset to contain all the true input
variables with the smallest size. However, linear OLS
and FOS-MOD worked not well. In Example 2, with thefurther insight of the rank list, the inputs y(t-1), y(t-3),
u(t-1), u(t-2) ranked top by Linear OLS method,
however, the y(t-2) is ranked at the end of the list,
therefore, this input variable would be easy to lost in
this input selection procedure. What’s more, it isx1 x2 x3 x5 ( x3 − 1) + x4
1 + x22 + x32

Here, e ∈ (0, 0.1) is a white Gaussian noise.
3. Parameter Setting
It is assumed that the time delay for Example 1 and
2 are unknown thus initialized by 10 for both inputs andoutputs (totally 20 input variables contained), which areconsidered big enough. However, when it is assumed
the maximum order of each case is five, it is found thecandidate terms pool is too large for evolutionary
algorithms to search directly. Therefore, COFS based
input selection and simplified OLS based term selection
are implemented, then NSGA-II [6] is applied to extract
all the possible polynomial terms. The details of NSGAII for system identification are from our previous©ISAROB 2011y(t-1)u(t-2)y(t-3)y(t-2)

To make comparison with some other state-of-art
input selection methods, subset from Delta Test
method [7], linear OLS method [4], and FOS-MOD
algorithm [8] are also generated. The minimal subsets
include all the real input variables for each method are
given in Tab. 2 and Tab. 3.
Table 2. Results comparison for Example 1
size
MethodMinimal input subsets
5
COFS
y(t-1),y(t-2),y(t-3),u(t-1),u(t-2)

1. Example Data Sets
In both two examples, 1000 input-output data sets
are sampled for training from each model when thesystems are excited using random input sequences withamplitude between -1.0 and +1.0.

wherey(t-1)y(t-3)u(t-1)u(t-2)

501

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

believable that the method FOS-MOD could not deal
with this case appropriately because all the important
inputs are scattered in the rank list.
Furthermore, the results of term selection are shownin Tab. 4 and Tab. 5.
Table 4. Term selection for Example 1
Selected rank
True model term Initial ranky(t-2)21

u2(t-2)u(t-1)y(t-1)
u2(t-2)y(t-2)
y3(t-1)

39
13
103
45

3
5
10
13

output data is sampled as test data, and the input data is
described as
if t ≤ 500
sin(2π t / 250)
u (t ) = 
0.8sin(2π t / 250) + 0.2sin(2π t / 25) otherwise.
It’s found that the simulation result is as small as 0.1603.

V. CONCLUSIONSIn this paper, our contribution is to introduce anefficient scheme for nonlinear polynomial NARX model
identification. In order to make evolutionary
optimization based identification worked efficiently,
input selection and term selection are implemented tochoose the minimal subset with all the necessary terms
included. As two examples shown, the proposed COFS
algorithm outperforms other input selection methods,
and the pre-processed identification could work
efficiently with a relative small searching space.

Table 5. Term selection for Example 2
True model term
Initial rank Selected rank1u(t-1)
5y2(t-3)u(t-1)
u(t-1)u(t-3)y(t-2)
u(t-3)y2(t-1)y(t-3)
y(t-1)y2(t-3)u(t-2)u(t-3)

87
81
277
483

10
15
261
308

REFERENCES

It can be found that all the true model terms become
more significant after the term selection process. The
ranking value of each term is improved, and theminimal candidate pool is also reduced from 103 to only
13 to contain all the necessary terms in Example 1, andthe similar situation could be found in Example 2, in
which, the candidate pool is pruned from 483 to 308. In
fact, 300 and 500 terms are selected as the searching
space, which is considered big enough to include all thenecessary terms.
In the phase of evolutionary optimization based
system identification, NSGA-II is applied to identify themodel structure. The identified polynomial model for
Example 1 is:(t ) =−0.4989 y (t − 2) + 0.6486 y (t − 1)u (t − 1)yˆˆˆ[1] L. Ljung, System Identification: Theory for the User,
Second Edition, Prentice-Hall PTR, Upper Saddle River,
N.J., 1999.
[2] Z.J. Yang, T. Fujimoto, and K. Kumamaru, A genetic
algorithm approach to identification of nonlinear
polynomial models, in Proc. of the 12th IFAC
Symposium on Identification, Santa Barbara, 2000.
[3] Y. Cheng, L. Wang and J. Hu, "A Two-step Scheme
for Polynomial NARX Model Identification Based on
MOEA with Pre-screening Process", IEEJ Trans. on
Electrical and Electonic Engineering, Vol.6, No.3, 2011.
[4] H. L. WEI, S. A. BILLINGS and J. LIU, “Term andVariable
Selectionor
Non-linear
SystemIdentification ,” INT. J. Control, vol. 77, No. 1, pp. 86–
110, Jan. 2004.
[5] K. Z. Mao and S. A. Billings, “Variable Selection in
Non-linear Systems Modelling,” Mechanical Systems
and Signal Processing, vol. 12, no. 2, pp. 351–366, 1999.
[6] Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal,
and T. Meyarivan, “A Fast and Elitist Multi-objective
Genetic Algorithm: NSGA-II,” IEEE Transaction on
Evolutionary Computation, vol. 6, no. 2, pp. 182–197,
2002.
[7] Emil Eirola, Elia Liitiainen, Amaury Lendasse,
Francesco Corona and Michel Verleysen, “Using the
Delta Test for Variable Selection ,” Proc. of European
Symposium on Artificial Neural Networks, Bruges,
Belgium, April, 2008, pp. 25–30.
[8] H. L. WEI, S. A. BILLINGS “Feature Subset
Selection and Ranking for Data Dimensionality
Reduction ,” IEEE Transactions on Pattern Analysis andMachine Intelligence, vol. 29, no. 1, pp. 162–166, Jan.
2007.

+ 0.6922u 2 (t − 2) + 0.1914 yˆ 3 (t − 1)
− 0.6545 yˆ (t − 2)u (t − 2) 2 .

From the final model it is found that although theirrelevant input variable y(t-3) is selected by COFS, it iseliminated and only true inputs are included in the final
results.
In Example 2, the identified model could be
expressed as:=y (t ) 0.9151u (t − 1) − 0.2998 y 2 (t − 3)u (t − 1)
− 0.4218 y (t − 2)u (t − 1)u (t − 3)
− 0.3855 y 2 (t − 1) y (t − 3)u (t − 3)
+ 0.3563 y (t − 1) y 2 (t − 3)u (t − 2)u (t − 3).

To test the obtained polynomial model, a 800 input-©ISAROB 2011502

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Traffic Signal Control of Multi-Forked Road
Chengyou Cui, MizuKi TakaMura, Heehyol Lee
Graduate School of Information and Systems, Waseda University
2-7 Hibikino, Wakamatsu-ku, Kitakyushu, Fukuka, 808-0135, Japan(tel:81-93-692-5164)(cuichengyou@ruri.waseda.jp)
Abstract: The traffic jam has become more serious at the multi-forked road intersection, and the conventional
pre-timed control is less efficiencies to the congestion problem. In this paper, the new traffic signal control system for
multi-forked road is proposed. Firstly, the cellular automaton (CA) model is used to develop a traffic simulator for
multi-forked road. Next, the stochastic model of traffic jam is built up. In addition, new traffic signal control algorithm
is designed using the optimization technique, Genetic Algorithm (GA). Finally, the effectiveness of the proposed
method is shown using the actual traffic data with traffic simulator.
Keywords: Traffic signal control, Multi-Forked, Bayesian Network, Cellular automaton, Urban micro traffic simulator,
Genetic algorithm
traffic signals timing cause the congestion of a part of theroads at multi-forked road intersection.
In this paper, the stochastic forecasting model [4] andGenetic algorithm (GA) is used to design a real timetraffic signal control system for multi-forked road. Firstly,
the probabilistic distributions of standing vehicles ofroads are predicted using the stochastic forecastingmodel, and then, GA is used to calculate optimal traffic
signal to minimize the probability of the traffic jam. An
urban micro traffic simulator is developed by using
cellular automaton (CA) model. Through the observationof vehicle movements of traffic simulator, the parametersof the GA are determined.

I. INTRODUTION
In recent years, the traffic congestion has become
serious problem with the exponential grown in vehicles.
In the urban area, the road networks are not easy to beextended. In this case, the traffic signal control isconsidered as an effective way to solve the problem. The
traffic signal control can be divided into two classes. One
is the offline (pre-timed) control, the other is online(adaptive) control. In the pre-timed traffic signal control,
Webster’s formula is used to calculate green splits andcycle lengths offline using the historical traffic data ofroad networks. The pre-timed traffic signal control
cannot handle any variation of traffic flows. On the other
hand, the adaptive traffic signal control can overcome
this limitation and this method can adjust the traffic
signals online with changing traffic flows.
The various intelligence techniques such as fuzzy
concept, reinforcement learning agents, and neuralnetworks [1-3] are used to implement the adaptive traffic
signal control. However, as the limitation of thesemethods, the green time and/or the cycle length areincluded as the control target. The signal of the right turnexclusive is not included. And then, the road networks
were not concerned multi-forked road intersection.
Generally, the traffic signal timing of multi-forked roadintersections is more complicated than crossroadintersections or t junction intersection. The inappropriate©ISAROB 2011

II. URBEN TRAFFIC SIMULATOR
The micro traffic simulator can model the movementof individual vehicles on road network. CA is usuallyfaster than any other traffic micro simulators, thecomputational requirements are rather low with respect
to both storage and computation time, making it possible
to simulate large traffic networks on personal computers[5].
The CA traffic movements are based on the SchChmodel (highway traffic model) [6]. The rules of thevehicle movements on the road network are shown as
follows.
1) Input to cell: according to comparison of a random
number (from 0 to1) with set value, a vehicle will be
generated and the direction of travel on the intersection

503

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

will be determined.
2) Speed: the vehicle can accelerate up to maximum
speed (Maximum speed = move 2 cell; 1cell=7m/1 step
[1sec]), when there is no obstacle. According toconditions of the road the speed can be changed
randomly by probability.
3) Intersection: when traffic signal is green, the vehicle
will be allowed to cross the intersection according todirection of travel, and the direction of the travel on next
intersection will be reset.
4) Multi lanes: in multi lanes road, a vehicle can moveto parallel lanes. If the direction of travel is right turn, thevehicle moves to the right turn exclusive lane.
By above rules, we can move the vehicles on the road
network, and the procedure of the simulator is shown in
Fig.1.

vehicles are represented as the nodes.

Fig.2. Multi-forked road and BN model
The probabilistic distribution of the standing vehicles
at k th cycle is obtained by summing over all values ofthe other variables as following,

P( S k ) = ∑∑∑P( S k , S k -1 , I k , Ok )S k -1 I k Ok

Ok = { Ok1 , Ok2 …Okn-1 , Okn }(2)n : number of the derection
With the chain rule, the joint probabilistic distribution isrepresented as the product of the conditional probability.
And then, according to the d-separation, equation (2) canbe represented asP( S k ) = ∑∑∑P( S k -1 ) • P( I k ) • P( Ok )(3)

S k -1 I k Ok

According to equation (3), the probabilistic distributionof the standing vehicles is predicted. Using this model,
the probabilistic distribution of standing vehicles of eachroad can be predicted.

Fig.1. flowchart of simulator

III. STOCHASTIC FOREACASTING MODEL

IV. TRAFFIC SIGNAL CONTROL BY
GENETIC ALGORITHM

In previously published paper [4] we described astochastic forecasting model of traffic jam at crossroads.
The stochastic forecasting model is used to predict
probabilistic distributions of standing vehicles ofmulti-forked road.
A multi-forked road is considered as an example shownin Fig.2. The random variables of inflows Ik, outflows Ok,
standing vehicles Sk on the road 1 have relationship asfollowing:
Sk = Sk-1 + Ik - Ok.
(1)
And then, a Bayesian network model is built up using
this relationship, and the random variables of the inflows
and the outflows of each direction, and the standing©ISAROB 2011

The objective of the proposed system is to reduce thetraffic queue of the congested roads and to maintain the
traffic queue of the other roads in a steady range. At first,
the stochastic forecasting model is applied to predict theprobabilistic distribution of the congested roads. Then,the optimal traffic signal within an adjustable range will
be searched according to minimization for the probabilityof the standing vehicles over a set value Smax. (If thestanding vehicle is bigger than Smax, the road isconsidered as a traffic jam). In the search process, GA isutilized.

504

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

1) Fitness function
For this optimization the fitness function is defined asfollowing;

MIN

i=n

S =∞

i =1

S = S maxF = ∑wi • (A truncation and elitism combination is used in thisstrategy. First, the best chromosome will be cloned to thenext generation. The other chromosomes of next
generation will be created by crossing 50% of the
population.
5) Crossover
Two chromosomes are randomly chosen as a pair ofparent chromosome. And then, at four random points, thegene will be interchanged.
6) Mutation
According to a mutation probability, an individual ischosen to be mutated and the changing point will berandomly chosen.

∑P i ( S k ) )i = number of the roads
S = number of the s tan ding vehicles(4)

S max = set value
wi = weight of the function
and, the P i ( S k ) can be obtained from equation (2). On
the other hand, the probabilistic distribution of the
inflows and the outflows will be changed by different
traffic signal. According to the characteristics of thetraffic flows, an estimating equation is defined asfollowing;

V. SIMULATION
To prove the effectiveness of the proposed system, asimulation was carried out based on the actual data atFukuoka-city of Japan with the micro traffic simulator.
The multi-forked road intersection is shown in Fig.4. In
this intersection, the traffic signals of the direction 1 and
4 are same, and direction 2 and 6 are same. Therefore,
the congested roads 2, 4 and roads 3, 5 are selected to
predict the probabilistic distribution of the standing
vehicles, and to optimize the traffic signals using GA.

P( f k ) = P(( f k / t old ) • t new )f k : the traffic flows of k - th cyclet old : the time of signals of k - th cycle(5)t new : new time of signals
By equation (4) and (5), the fitness value can becalculated.
2) Chromosome encoding
A chromosome is defined as an example shown in Fig.3.
In this multi-forked intersection, the traffic signal can bedivided into six parts. In the chromosome, each traffic
signal is represented using different color, and consistedof some genes. The meaning of each number is time[sec.].

Fig.4. Multi-forked road intersection
The table 1 presents parameters of the fitness function.
The parameters are empirical value by simulation withthe simulator. In the GA, the number of generations set
to 30; population size set to 31; and the mutation
probability is 0.04. During the calculating time of GA,
the traffic signal will be updated at every 5cycle (15minutes).
Table 1. Parameters of fitness function

Fig.3. Chromosome encoding3) Initial population
Chromosomes as an initial population will be generated
randomly within adjustable range of the traffic signal.
4) Selection©ISAROB 2011

505

Road 2

Road 3

Road 4

Road 5

Smax

35

35

110

35

wi

1

0.7

0.9

0.5

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig.5 Progress of the fitness function
The progress of the fitness function shows theminimization by the GA illustrated in Fig.5. The traffic
signals by the proposed method are shown in Fig.6. The
traffic signals of the pre-timed are fixed, but the traffic
signals by the new method are updated with the changing
traffic flows.

Fig.8 Average delay time of each cycle
The total number of the passing vehicles from theintersection is increased from 8,242[cars] to 8,586[cars].

VI. CONCLUSIONSIn this paper, the optimization of the traffic signal
timing by GA and the simulator based on the stochastic
forecasting model were proposed. Through thesimulation with the multi-forked road intersection of anurban area, the effectiveness of the proposed method isshown.

REFERENCES[1]Y.Sazi Murat; Ergun Gedizlioglu(2005)A fuzzy logic
multi-phased signal control model for isolated junctions.
Transportation Research Part C. 13(1)19-36[2]P.G. Balaji; X.German; D.Srinivasan;(2010)Urban
Fig.6 New traffic signals

traffic signal control using reinforcement learning agents.
4(3): 177-188[3]Dipti Srinivasan; Min Chee Choy; Ruey Long Cheu
(2006) Neural Networks for Real-Time Traffic Signal
Control. IEEE Transactions on intelligent Transportation
System.7 (3):261-272[4]CY Cui; JS Shin; HH Lee; (2010)Real-time traffic
singl learning control using BPNN based on predictionsof the probabilistic distribution of standing vehicles.
Artif Life Robotics.15 (1)”59-61
Fig.7 Traffic queue of the roads[5]Javier J.SM; Manuel J.GM; Enrique RR(2010)Traffic

Fig.7 represents the traffic queue of the each road. In
the congested roads 2 and 3, the traffic queue is reducedby the proposed method. In the road 2 and 3, the averageof traffic queue is decreased by 13m and 40 m,
respectively. On the other hand, for the road 4 and 5, thetraffic queue is maintained in a steady range.
By the proposed method, the average delay time isdecreased compare with by the pre-timed, and the results
are shown in Fig.8.

Signal Optimization in”La Almozara” District in©ISAROB 2011

Saragossa Under Congestion Conditions, Using Genetic
Algorithms, Traffic Micro simulation, and Cluster
Computing.

IEEE

Transactionson

intelligent(1992)A

cellular

Transportation System.11 (1):132-141[6]K.NagelandM.Schreckenberg

automaton model for freeway traffic. J.Phys. I France.2(12)2221-2229

506

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Control Design Methods for Platooning in Robot Car
Ryo TAKAKI, Xin ZHAO, Harutoshi OGAI
Waseda University, 2-7 HIBIKINO, WAKAMATSU-KU, KITAKYUSYU CITY, FUKUOKA, 808-0135, Japan(Tel: 81-93-692-5147; Fax: 81-93-692-5147)(Email address: takaki@akane.waseda.jp, zhaoxin_19870121@yahoo.co.jp)

Abstract: Platooning technology is becoming a future task which suggests as a way of reducing carbon dioxid
e emissions and realizing safe driving at a high speed velocity. This paper describes a few control methods f
or vehicle-platooning. The conventional control method improved fuel consumption by shortening the distance
between vehicles. By contrast, the method we proposed improves it by controlling the velocity at the time ofacceleration gently. The velocity is controlled by generating the desired value of inter-vehicular distance corres
ponding to the leading vehicle velocity. Another method which is planned to realize a highly efficient arterial
traffic distribution system includes reducing aerodynamic drag by minimizing the distance between vehicles toallow drafting. In this paper, the two degrees-of-freedom control system is applied for it. These proposed met
hods were evaluated by simulation and some experiments.
Keywords: Platooning, control method, two degrees-of-freedom, distance

Where, Tm is the time constant of the acceleration
response characteristic. The acceleration response
characteristic is controlled by model matching control
system shown in Fig.1 [3]. This control system consistsof model matching compensator, robust compensatorand plant. The model matching compensator generates asignal to match the actual response characteristic andideal response characteristic shown in equation (1).
Furthermore, robust compensator generates a signal toreduce the effect of fluctuation of plant characteristic. In
this research, we defined the time constant as T m=1.0 in
the same way as reference [4].

I. INTRODUCTION
Platooning is considered as one of the innovations in
the automotive industry that aim to improve the safety,
efficiency, mileage, and time of travel of vehicles while
relieving traffic congestion, decreasing pollution andreducing stress for passengers[1]. Also, platooning
makes it possible for vehicles to travel together closely
and safely. This leads to a reduction in the amount of
space used by a number of vehicles on a highway. Thus
more vehicles can use the highway without traffic
congestion [2].
In our research, we use the equipment of robot carwhich is a kind of electric cars carrying the laser range
sensor and CCD camera. By robot car, we can carry out
the research of automation driving and platooning.
In this paper, it designs the model for the robot car
platooning system and develops two degrees-offreedom control system to control the distance ofvehicle-platoon.

uaModel matchingcompensator

+
+

Plant

a

Robustcompensator+-

Fig.1. Robust model matching control system
2.2 Constitution of vehicle-platoon
The constitution of platoon is shown in Fig.2. In thisresearch, we deal with three robot cars as a vehicleplatoon. Each robot car detects distance between two
cars by radar sensor. The variable’s numbers shown in
Fig.2 corresponds to each robot car’s number. For
example, v2 represents the velocity of robot car 2.

II. OUTLINE OF PLATOONING SYSTEM
2.1 Modeling of Robot car
In this research, the response characteristic ofacceleration from target value to actual value is given
by equation (1).

Ga ( s) ©ISAROB 2011

a( s)
1

u a ( s) Tm s  1

(1)

507

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Inter-vehiculardistance:dr2

Inter-vehiculardistance:dr1

Robot car3

Robot car2

Robot car1

position:d3
velocity:v3
acceleration:a3

position:d2
velocity:v2
acceleration:a2

position:d1
velocity:v1
acceleration:a1u (s)vn1 ( s) a ,3n 2s
Tm s  s
Vn+1

Fig.2. Constitution of platoon

ua,n

P(s)

Ga (s)

an 1 vn

3.1 Object of the system
When the leading car moves quickly, it is necessary
to control the velocity of following robot car rapidly forremaining the target inter-vehicular distance. However,the fuel consumption is also increased rapidly. In orderto solve this problem, it is necessary to determine thetarget inter-vehicular distance corresponding to theleading car’s driving situation. Moreover, the control
system which can regulate the inter-vehicular distancewithout influencing ride quality is required.

Gref(s)uff,nFeed forward
compensator
CFF(s)

-

ua,n
-

ufb,n

Controlled
object
P(s)

dm,n

dr,n

dr,n

©ISAROB 2011u a ,n ( s )



(3)

s

a , n  svn1 ( s)

(5)

n 2
GM , n ( s ) F ( s) (6)d ref ,n ( s) s 2  2 nn s  n 2
d m,n ( s)

3.3 Control objective
The block diagram of the controlled object is shownin Fig.4. The transfer function of the control objective
P(s) from target acceleration ua(s) to actual intervehicular distance dr(s) is given by equation (2).

d r ,n ( s )d ref ,n (s)  d ref v,n (s)  d ref a,n (s)3.5 Model matching compensator
The transfer function of the model matchingcompensator GM(s) from inter-vehicular distance oftarget dref(s) to inter-vehicular distance response dm(s) is
given by equation (6).

-Fig.3. Constitution of control system

P( s ) isWhere, h represents inter-vehicular time coefficientand λa represents control parameter. Therefore, dref-v(s)increases as leading car’s speed increases. In my
research, we defined the inter-vehicular time coefficient
as h=2.0 in the same way as reference [5]. When leading
car moves quickly, dref-a(s) also increases quickly. Due
to long target inter-vehicular distance, it is not necessary
for subsequent robot car to accelerate rapidly.

edr,n
+

3.4 Inter-vehicular distance generator of target
Inter-vehicular distance of target dref,n(s)calculated by equation (3).

d ref a ,n ( s)  hFeedbackcompensator
CFB(s)
Model matchingcompensator
GM(s)

dr,n

Where, dref-v(s) represents the element of intervehicular distance of target correspond to leading car’s
velocity and dref-a(s) represents another element of target
inter-vehicular distance correspond to the leading car’s
acceleration. dref-v(s) and dref-a(s) are given by equation(4), (5), respectively.
(4)d ref v,n (s)  h vn 1 (s)Vn+1dref,n1s

Fig.4. Block diagram of control objective

3.2 Constitution of control system
Two degrees-of-freedom control system shown in
Fig.3 is designed to attain the design requirements, [6].
The control system consists of controlled object P(s),target inter-vehicular distance generator Gref(s), model
matching
compensator
GM(s),
feed
forwardcompensator CFF(s) and feedback compensator CFB(s).
And this kind of control system is carried in each robot
car.

Target intervehicular distancegenerator+-sIII. DESIGN OF CONTROL SYSTEM

Vn+1

(2)

Where, ζ represents attenuation coefficient and ω
represents character frequency. F(s) is a filter given byequation (7).

1vn1 (s)  vn (s)s

F ( s) 

508

1
TF s  1(7)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Where, TF represents time constant. In this research,time constant is defined as TF =0.1.

CFB ( s) 

3.6 Feed forward compensator
A block diagram of feed forward control system isshown in Fig.5. In order to match the actual intervehicular distance dr(s) and inter-vehicular distance
response dm(s), it is necessary to equalize the transfer
function of feed forward compensator from target intervehicular distance dref(s) to actual inter-vehiculardistance dr(s) and the transfer function of model
matching compensator. Therefore, transfer function of
feed forward compensator is given by equation (8), (9).

d r ,n ( s)d ref ,n ( s)

 C FF ,n ( s) P( s)  GM ,n ( s)C FF ,n ( s) 2dd,n
dm,n +-P(s)

CFF(s)GM (s)P (s )

CFB(s)++

dr,n

Fig.6. Feedback control system

IV. SIMULATION(9)P( s )

3

In my research, the parameter of sensitivity function
is defined as λS=3.0.

4.1 Simulation condition
Initial position of the vehicle-platoon is shown in
Fig.7. The vehicle-platoon contains one leading robot
car4 and three robot cars 1,2,3 and each initial intervehicular distance is 3[m] and each initial velocity is
0[m/s]. The velocity of robot cars1,2,3 are controlled byproposed control system and the velocity of robot car4
is given as Fig.8. Simulation parameters of each Robot
car are given by Table.1. Under above conditions, Intervehicular distance and velocity of each robot car are
simulated.

Feed forward compensator can control actual intervehicular distance dr(s) similar to inter-vehiculardistance response dm(s). However, due to disturbanceand fluctuation of plant characteristic, feedback
compensator is also necessary.

dref,n3Tm S s 3  3S Tm S  1s 2  S Tm S  3s  S
(12)
s

(8)

GM , n ( s )1  S ( s)S ( s ) P( s )

dr,n
dm,n

dr3 = 3[m]dr2 = 3[m]dr1 = 3[m]Fig.5. Feedforward control system
3.7 Feedback compensator
A block diagram of feedback control system isshown in Fig.6. This compensator controls transfer
characteristic from a disturbance to actual intervehicular distance shown in equation (10).

d d ,n ( s )1
1  C FB ( s) P( s)Robot car1

v4 = 0[m/s]v3 = 0[m/s]v2 = 0[m/s]v1 = 0[m/s]

18
15

(10)
12
9
6
3

3

(11)

0

Where, λS represents a parameter of sensitivity
function. From equation (10) and (11), transfer functionof feedback compensator is calculated as equation (12).

©ISAROB 2011

Robot car2

Fig.7. Initial position of Robot car

Feedback compensator can control disturbance
response without influencing desired value response. In
this research, sensitivity function for disturbance isgiven by equation (11).

 s 

S ( s)  
 S  s 

Robot car3

Velocity[m/s]d r ,n ( s )Robot car4

0

10

20
Time[sec]

30

Fig.8. Speed fluctuation of leading-robot car4

509

40

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table.1:Simulation parameters
Parameters Robot car1 Robot car2 Robot car3
λa
0.65
0.45
0.25
h
2.0
2.0
2.0
ζ
1.5
1.5
1.5
ω
0.7
0.7
0.7

5. CONCLUSION
In our research, it presents a two degrees-of-freedom
control system to control the platooning. The system
controls the velocity of vehicle-platoon smoothly andkeeps the distance of vehicle-platoon accurately.
The future plan is to carry out the simulation ofmodel predictive control and do the comparison. And
we will lead the control system into robot car.

4.2 Simulation results
The calculated value of robot car’s velocity and eachinter-vehicular distance is shown in Fig.9(a), Fig.9(b),respectively. From simulation results, it is verified thatthe velocity of subsequent robot car is controlled moregently than that of leading robot car. This result
indicates that the proposed method can prevent
subsequent following vehicles from accelerating rapidly.
Moreover, Fig.9(b) indicates that actual inter-vehiculardistance dr(s) is controlled accurately to match the intervehicular distance response dm(s).

REFERENCES[1] Using ITS to cut CO2 emissions in half by 2050,“DevelopmentofEnergy-saving
ITS
Technologies,”2009.
[2] Yoshinori YAMAMURA, Yoji SETO and Masao
NAGAI, “Study on a String-stable ACC Using Vehicleto-Vehicle Communication,” Japan Society of
Mechanical Engineers, No.06-7044.
[3] Omae M, Fujioka T, Miyake K (1999), LongitudinalControl for Platoon Treated as One Dynamic System (inJapanese). Transactions of the Japan Society of
Mechanical Engineers, Vol.65, No.640, pp.126-133.
[4] Adachi K (2006), Proposal of a target headway
distance method and applying this method to a car foradaptive cruise control system (in Japanese).
Proceedings of the 15th Transportation and Logistics
Division of The Japan Society of Mechanical Engineers,
No.6-52, pp.23-28.
[5] Sugie T, Fujita M (1999), Introduction to Feedback
Control (in Japanese). CORONA PUBLISHING.

18

Velocity[m/s]

15
12
9
6
Robot car4
Robot car3
Robot car2
Robot car1

3
0

0

10

20
Time[sec]3040(a)Velocity
40Inter-vehicular distance[m]

35
30
25
20
15

dr3
dr2
dr1
dm3
dm2
dm1

10
5
0

0

10

20
Time[sec]

3040(b)Inter-vehicular distanceFig.9. Simulation results©ISAROB 2011

510

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Building of Reverse Logistics Model in Reusable Recovery and
Optimization considering Transportation, Inventory, and Backorder Costs
Jeong-Eun Lee and Hee-Hyol Lee
The Graduate School of Information, Production and Systems, Waseda University(Tel : 81-93-692-5017; Fax : 81-93-692-5021)(leeje@toki.waseda.jp, hlee@waseda.jp)

Abstract: This paper deals with the building of the reusable reverse logistics model considering the decision of thebackorder or the next arrival of goods. The optimization method to minimize the transportation cost and to minimize
the volume of the backorder or the next arrival of goods occurred by the Just in Time delivery of the final delivery stage
between the manufacturer and the processing center is proposed. Through the optimization algorithms using thepriority-based genetic algorithm and the hybrid genetic algorithm, the sub-optimal delivery routes are determined.
Based on the case study of a distilling and sale company in Busan, Korea, the new reverse logistics model in reusable
recovery of empty bottles is built and the effectiveness of the proposed method is verified.
Keywords: Reusable Recovery, Reverse Logistics, Backorder or Next arrival of Goods

the Remanufacturing Recovery considering the delivery
cost from the returning center to the processing center.
And Tang[2] showed the reverse logistics model
considering the disassembly process in the processing
center. However, they haven’t discussed the processing
operation to clean and refill the recovered products. On
the other hand, researchers[3] extended these models,
and represented reverses logistics model of theremanufacturing recovery that the disassembling,cleaning, refilling operations were included in theprocessing center and the transportation cost from thereturning center to the processing center, manufacturing
plant is considered. Moreover, they have built thereverse logistics model considering that a reusable partis delivered to the manufacturer and a part to disposal.
In additions, they have discussed the reverse logistics
model considering the direct route to each processing
operation in processing center, recycling center,
manufacturing plant or disposal as the condition ofrecovered product in the returning center. And theremanufacturing reverse logistics model was built
considered inventory cost before and behind eachprocessing operation which the recovered productthrough the returning center.
Therefore, for optimizing the reverse logistics withuncertainty of amount or occurrence time of therecovery product, to build a model is necessary
considering not only transportation cost but also thedate and the processing of the decision whether waiting
for arrival of an end-of-life product with the unclear

I. INTRODUCTION
For achievement of the resources recycling society
and the low carbon society, the reverse logistics whichtargets the flow from production recovery to
reproduction of end of life products has been received
attention in the logistics field. During tightening
regulation on environment increasingly, the reverse
logistics has been magnified by the following reasons:First, economic effect resulted from the cost reductionof raw materials in manufacturing process. Second, thepropensity to consume changed to the environmentfriendly product. Third, business strategy tried to
improve the image of corporate.
However, the reverse logistics is different from thetraditional forward logistics where new material or partare produced and sold to customer. In the reverse
logistics, it is not only hard to predict the appearing timeor amount of arrivals by the used periods or condition of
the recovered products, but also the recovery routes arecomplex as there are lot of recovery centers. Moreover,even though the recovery products are environmentfriendly, its market is not large yet because of thestereotype of customers who regard the recovery
product as used goods. And the reverse logistics costs
more than the traditional forward logistics to constructand operate the system.
There have been lots of researches on theremanufacturing recovery that processing operation iscomplex and the forms of products are various
Jayaraman[1] represented the reverse logistics model of©ISAROB 2011

511

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011bj: capacity of the processing center j
dK: capacity of the manufacturer K
ri(t): amount of the end-of-life product recovered to thereturning center idK(t): demand of in the manufacturer K
uK: amount of an upper limit of backorder of the
manufacturer K
1c ij: unit cost of the transportation from the returning
center i to the disassembly center j
2
c jK: unit cost of the transportation from the processing
center j to the manufacturer K
3
c jD: unit cost of the transportation from the processing
center j to the disposal D
4c SK: unit cost of the transportation from the supplier S
to the manufacturer K
opc j: unit holding cost of the processing center j
cH1j: unit inventory cost per period at the processing
center j
cH2K: unit inventory cost per period at the manufacturer k
Decision Variables
xij(t): amount shipped from the returning center i to theprocessing center j in the period t
xjK(t): amount shipped from the processing center j tothe manufacturer K in the period t
xjD(t): amount shipped from the processing center j tothe disposal D in the period t
xKB(t): backorder amount of the manufacturer K at theperiod tyHj(t): inventory amount at the processing center j in the
period tzHK(t): inventory amount at manufacturer K in the periodtwK(t): Binary variable equals 1 when the safety
inventory is secured in manufacturer K and 0 whenbackorder is necessary
zj: Binary variable equals 1 when processing center j ishold, otherwise 0.
Objective Function
Minimize the transportation cost, occurred
backorder cost or inventory cost by the JIT delivery
according to waiting for the next arrival of goods

amount to the returning center or backordering
necessary parts for manufacturing.
In this paper, a reusable reverse logistics model
considering the decision of backordering or waiting forthe next arrival of goods on the base of the reusable
recovery is built. And, the optimization method of thereusable recovery to minimize the transportation cost
and the volume of the backorder or next arrival of goods
occurred by the just in time delivery of the final
delivery stage between the manufacturer and theprocessing center is described. In addition, this method
can be also applied to the remanufacturing recovery andthe recycling recovery.

II. BUILDING OF REUSABLE REVERSELOGISTICS MODEL AND OPTIMIZATIONCONSIDERING TRANSPORTATION,INVENTORY, AND BACKORDER COSTS
The reverse logistics model in newly building
reusable recovery is considered the Just in Timedelivery cost from the processing center to manufacturer
included decision whether waiting for arrival of the endof-life product or backordering necessary parts for
manufacturing when the end-of-life product gathering
goods to the processing center through the returning
center is less than the demand of manufacturer.
Processing center
Manufacturer

Reusedproducts

InventoryNo
Disposedproducts

DisposalReusedDemand > productsYesbackorder

Returned
products

Returning
centerInventory

next arrival

Purchased products
Supplier

Fig. 1. Reusable reverse logistics model considering
backorder or next arrival
Figure 1 describes the model combined thedecision factors of the backordering or wait for the next
arrival of goods based on the total inventory of the
manufacturer in case of the end-of-life productgathering goods to the processing center through thereturning center is less than the amount of demand.
Indicesi: returning center (i=1, 2, …, I ； I ： number ofreturning centers)
j: processing center (j=1, 2, …, J； J ： number ofprocessing centers)
t: time period (t=1, 2,…, T)
Parameters
ai: capacity of the returning center i©ISAROB 2011f  1 f1   2 f 2
T  J
I
J
J

1
2
 1    c op
j z j   cij xij (t )   c jK x jK (t ) t 1  j 1
i 1 j 1
j 1

T 
J
J

4
H
+ 2  cSK
xKB (t )  cKH 2 z KH (t )   c3jD x jD (t )   c H1j y j (t )   mint 1 j 1
j 1α1, α2：weight

512

(1)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

returning center and the processing center and thetransportation cost from the returning center to theprocessing center are shown in the Figure 2.

Subject to
The recoveredamount of the end-of-life product:J x (t )  r (t ),
j 1

ij

i

i, t

(2)

Capacities of the processing center and manufacturer:I x (t )  y (t  1)  b z ,
i 1
J

ij

H
j

j

j

(3)

j , t x (t )  y (t  1)  x (t )  e ,HKnode ID l ：t(4)
The amount of the backorder when selecting thebackorder:Jd K (t )  zKH (t  1)   x jK (t )  xKB (t ),
t
(5)
j 1
The inventories of the processing center andmanufacturer:I(6)y Hj (t  1)   xij (t )  ( x jK (t )  x jD (t ))  y Hj (t ), j, t
j 1

jK

KB

K

1

2

3

5

6

8

2

3

4

5

1

2

3

priority v (l) ： 3

6

5

2

4

7

8

1 Chromosome 2

Processing centers

Returning centers

1

40

Processing centers
1

2
3
30

3

4

110

70

Chromosome 12

3

18 12 16 
2 16 10 18
3 13 14 20 
4 19 15 13
5 12 18 10 1

Cost =

Returning centers

2

5

H
K

50

Fig. 2. Example of Representation for Chromosome
Decoding: In case of the first chromosome, the lowest
cost of returning center for processing center with thehighest priority is the returning center 5. The shipments(50) are decided by the maximum deliveries (50, 110).
Moreover, the returning center 4 with the second lowest
transportation cost is selected for the processing center
3. The Shipments (60) are decided by the maximum
deliveries (70, 60). Since the capacity 110 in theprocessing center 3 was satisfied, the priority is reset to0, and the returning center 1 with the second highest
priority is selected.
2. Genetic operators
The WMX crossover determines a cutting point
randomly and generates an offspring exchanging theright parts of the chromosomes from the cutting point.
The exchanged right parts are arranged by eachascending orders. Next, the numbers of genes that
become a pair to each other are checked and changed by
the relationship. The Swap mutation [4] selects
randomly the pairs of the gene exchanged in eight genes
and exchanges the selected genes.
3. Optimization by priority-based Genetic Algorithm
Priority-based genetic representation adopted for thechromosome representation is used to show the node of
the gene position and the value is used to show thepriority of the node.
4. Optimization by hybrid Genetic Algorithm
Besides the function of priGA, hGA improves thesearching ability of GA through adjusting the parameter
appropriately in each generation using FLC [4] andmaking a suitable situation by the optimal solution
search.

III. OPTIMIZATION OF REUSABLE
REVERSE LOGISTICS USING GENETIC
ALGORITHM
1. Priority based genetic representation
Encoding: The chromosome with length which is
totalized by returning center (I) and processing center(J) is generated. The value of each gene represents thepriority and an initial value allocated by the priority
starts from total of the gene, (I+J). Then the priority 1
less than the selected gene randomly is allocated till
every gene has the priority value. For example, twochromosomes with the total of returning centers, I=5
and total of processing centers, J=3, the capacities of the©ISAROB 2011

5

3

60

(7)
The restriction not to be able to execute waiting ofarrival of goods and relapse note at the same time in theprocessing center:
zKH (t )  uK wK (t ),  t(8)(9)
xKB (t )  uK (1  wK (t )),  t
Non-negativity of the decision variables:xij (t ), x jK (t ), x jD (t ),xKB (t )  0, i, j , t(10)
Decision variable of the holding determination:(11)z j  0,1 j
The reverse logistics is formulated as a mixed integer
programming problem, and is one of the NP-hardproblems. The mixed integer programming problem
with comparatively little integer variable is possible tobe solved in practicable time using traditional
optimization software. But it becomes impossible to be
applied for the large-scaled problem in this research
since the calculating time or using memory increase
geometrically. Therefore, the sub-optimal solution is
calculated using GA as the solution of this problem.
K

4

2

60

J

H
K

3

4

80

 x (t )  z (t  1)  d (t )  z (t ),  t
jK

2

1

1

i 1

j 1

1

priority v (l) ： 7
node ID l ： 1

513

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

IV. SIMUTAION AND RESULTS

cost of the supplier and the disposal cost. Neither
processing centers 2 and 5 are selected nor is holding
cost reduced.
2. Case of Bottle reusable reverse logistics of
distilling and Sale Company
The optimization problem of the bottle reusable
recovery case with a distilling and sale company inBusan, Korea by the real data was simulated. It set to 20of the population size, 0.7 of the initial WMX crossover
rate, 0.3 of the initial mutation rate, and 5000 of the
maximum generation as a simulation condition of the
proposed method. The minimum costs of the priGA andhGA are 3,975,048 and 3,963,330 respectively. Figure 5
shows the sub-optimal delivery routes and the selected
situation of the returning center. In this sub-optimaldelivery routes, the five recovery centers (dotted circle
in the Figure 5 are removed because of these highholding cost. The total of variables in this case study is22435, and it is impossible that the traditional
optimization software LINGO calculates the solution inpracticable time. However, the priGA and hGA could
calculate the solution in the practicable time, 6.42[sec]and 4.45[sec], respectively.

1. Numerical example
This paper considers the ten returning centers andthe six processing centers, the manufacturer, thedisposal and one each supplier. In the Figures 3 and 4,
the sub-optimal delivery route and the amount of
backorder in period t1 and t2 are shown respectively.
Returning center i

1
550(520)

Processing center j

460

30

2

1

470
2200

110

1829

470

300(110)30

Manufacturer K

3

100

2

150(100)
50
80

549

3

4

150

250(80)1

5550

Disposal D

350

650(150)
230

4

6

450

160

450(230)7570250(160)

120

460

8

6

170

150(120)460

9190 250(170)

10
j

1

2

1

0

2

0

3

0

4

0

0

5

0

0

6

0

0

7

160

0

8

120

9
10

i

220(190)

3

4

5

6

0

30

30

0

460

0

110

0

0

0

0

100

0

0

0

80

0

0

0

0

150

0

0

230

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

170

0

0

190

0

0

0

0

0

371

Supplier S

Amount of backorder in t=1: 371

Fig. 3 Sub-optimal delivery routes and amount of
backorder of the example at t=1
Returning center i

1
Processing center j

40

550(400)

360
2510

1

470

1819

2

200

470

300(200)

Manufacturer K

3

100

2

150(100)

50

4

80

549

250(80)3
1

Disposal D

5

140

550
340

650(140)
10

4

310

450

170

5

7

70

110 250(170)6150(110)

460

1

1

2

3

4

5

6

0

0

40

0

0

360

100

20

9

2

0

0

200

0

0

0

3

0

0

0

100

0

0

220(180)

0

0

0

80

0

0

5

0

0

0

140

0

0

6

10

0

310

0

0

0

7

170

0

0

0

0

8

110

0

0

0

0

0

9

0

0

0

20

0

100

10

180

0

0

0

0

0

REFERENCES

180 250(120)10

4Fig. 5 Sub-optimal delivery routes and selected or
unselected recovery centers

8
460

j

i

6
450(320)[1] Jayaraman V., Patterson R. A. and Rolland E.
(2003), The design of reverse distribution networks:
Models and solution procedures, European Journal ofOperational Research, 150:128-149[2] Tang O., Grubbstrom R. W. and Zanoni S. (2007),
Planned lead time determination in a make-to-orderremanufacturing system, Int. J. Production Economics,
108:426-435[3] Lee J. E., Gen M. and Rhee K. G. 5 (2008), A Multistage Reverse Logistics Networks Problem by using
Hybrid Priority-based Genetic Algorithm, IEEJ Trans.
on EIS, 128( 3): 450-455[4] Gen M., Cheng R. and Lin L. (2008), Network
Models and Optimization: Multiobjective Genetic
Algorithm Approach, Springer, London

Supplier S

0Amount of backorder in t=2: 0
Amount of inventory in t=2: 1819

Fig. 4 Sub-optimal delivery routes of the example at
t=2
This minimum costs 1,290,220 includes thetransportation cost of from the returning center toprocessing center, the holding cost of the processing
center, the transportation cost from the processing
center to the manufacturer, the inventory cost of theprocessing center and the manufacturer, the backorder©ISAROB 2011

514

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Advanced Pipe Inspection Robot using Rotating Probeand Image Processing
Ryuta Oyabu1, Kentarou Nishijima1,Zhicheng Wang1,Harutoshi Ogai1,Bishakh Bhattacharya2
1

Graduate school of Information, Production and Systems, Waseda University, Japan(Tel: +81-93-672-5147; Fax: +81-93-672-5147)(E-mail: ogai@waseda.jp)
2
Department of Mechanical engineering, Indian Institute of Technology Kanpur, India

Abstract: Recently many drain pipes used for transportation of water and gas at the plants have become old. These
pipes have many defects caused by corrosion and cracking and they cause serious accidents because of leakage, fireand blasts. Therefore, to forestall these accidents, we believe it is important to do drain pipe inspections andmaintenance using drain pipe inspection robots.
'Rotating probe' and 'Image processing' are used for the method of inspecting pipe in this research. Then, thedevelopment of the Pipe Inspection Robot measures the irregularity and the form of the defect in pipe. Therefore, thisrobot using the rotating probe of this touch sensor is profitable.
Keywords:

Drain pipes, Pipe inspection robot, Rotating probe, Image processing

I. INTRODUCTION

element for inspecting the inside of pipes with a touch
sensor system. It explains the composition and theoperation of the probe, and the outcome of anexperiment in the made pipe model is shown.

Recently many drain pipes used for transportation ofwater and gas at the plants have become old. These
pipes have many defects caused by corrosion andcracking and they cause serious accidents because ofleakage, fire and blasts. Therefore, to forestall these
accidents, we believe it is important to do drain pipe
inspections and maintenance using drain pipe inspection
robots. A lot of pipe inspection robot that uses image
processing has been developed up to now. However,whether it is a defect or a pattern might not be
understood only from the image processing. Then, thepresence of the defect is confirmed with a touch sensor.
In this research, the composition of the robot is
described, and the image processing, the explanation of
the rotating probe, and the outcome of an experimentare described. Subsequently, I describe the future plans.
First of all, the video shooting in pipe is done as a flowof the system of the picture processing. Next, theconfirmation and the correction of a center point are
done. Finally the developed figure of the video image ismade by development of the image and correcting theseam. The recording cameras of the image processing
are 300,000 pixels CCD.
Next, the probe (sensor) that enabled the defectdetection was developed. The performance experimentof the sensor is installed in the rotational mechanism of'Mogurinko 250' by 'Ishikawa iron works Ltd'. In
separate research, the Indian Institutes of Techonology
Kanpur has researched a rotating probe using piezo©ISAROB 2011

II. ROBOT STRUCTURE
In this research, the robot of 'Mogurinko 250' by'Ishikawa iron works Ltd' was used. This robot has CCD
camera in the front and the rotating probe was installedin the back. Fig.1 shows the appearance diagram.

Fig.1 The appearance diagram of robot
This robot can be controlled by wireless radio
communication in the inside pipe and can also transmit
image information of the inside of the pipe in real time.

515

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Specifications of this inspection robot:
・

3.2 Zonal Image Making
When a center point is found, circular image isdeveloped to the zonal image. The acquired image istaken by perspective. Therefore, the correction is put,
and a circular image is converted to a zonal image.

Size: length 370mm, width 180mm, height
160mm.
Moving speed: 13.7m/min,Driving mode: double motor,
Electric Power: rechargeable batteries 7.2V.
Wireless frequency: apply to 2.4/5 GHz andData transmission by 100 base-T Ethernet.
USB Camera(300 thousand pixel)

・
・
・
・
・

III. IMAGE PROCESSING METHOD
The image processing to be a center point correction ofpipe and develop of the image, it is very difficult torecognize the defect based on video in pipe. Then, theimage is converted from circular image to zonal image
and the inside of pipe is vertically seen. It becomes easy
to recognize the defect by processing it like that.
Fig.4 Zonal image making

3.1 Center Point Correction
The obtained video doesn't necessarily have the centerof the pipe at the center of the image. Accordingly, thecenter of the pipe is found and it is necessary to process
the image centering on the point.
Center point correction method is ' Houghtransformation method ' and ' Least squares method '[1].
Fig.2 is the second order polynomial approximation.

3.3 Development Image Making
When an arbitrary zonal image is made, a zonal image
is continuously made. And, the image of development inpipe is made by joining it.

Fig.5 Development Image

Fig.2 Least-squares method

Fig.3 Hough transformation method
Fig.6 Image processing result©ISAROB 2011

516

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

IV. ROTATING PROBE METHOD

V. Performance experiment of Probe

4.1 Structure of Rotating Probe
The probe consists of spring steel and piezo film is
positioned at the base of the probe. The steel strip canbe used as cantilever. To control the cable of the rotatingprobe, a steel strip was used, as shown in Fig.7

This experiment used a resting robot with a rotatingprobe in a clean vinyl chloride pipe with a 25cm
diameter, as shown in Fig.9. The defects were made ofslices of eraser.

Fig.9 Experiment Conditions
Fig.7 Rotating probe
5.1 Data Capture Method
The voltage change happens from bending of the probe.
The voltage change is taken into the microcomputer.
And, it displays it in CPU of the robot by using serial
communications. The sampling period at this time is0.01 seconds. The microcomputer used H83664tiny
microcomputer. Fig.10 shows the appearance diagramof microcomputer.

4.2 Operating Principles of Rotation Probe
When the rotating probe touched the defected area, thepiezo film could detect the curve and change of thestress. This stress change of the piezo film can bemeasured as voltage change. Movement of rotatingprobe is shown Fig.8.
First, four probes rotate and the probes approach thedefect of the inside pipe. (Fig.8-①)Next, the probes
start to touch the defect and its detected defect. (Fig.8②.③) Afterward the probe moves away from the defect.
(Fig.8-④)

Fig.10 The appearance diagram of microcomputer.
5.2 Experiment Method
The experiment doesn't drive the robot, and operate
only the rotation probe. And probe touched to the defectof experimental object. What voltage change happens ismeasured as a result.
Table.1 shows the experimental condition.

Fig.8 Movement of rotating probe©ISAROB 2011

517

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table.3 Experimental Results (Form)

Table.1 Experimental condition.

VI. AUTOMATIC DEFECT RECOGNITIONMETHOD
Automatic defect recognition method was made basedon these outcomes of experiments. Fig.13 shows the

5.3 Experimental Environment
Figure of each experimental condition is enumerated
as follows.
 Experiment 4

algorithm of automatic defect recognition method.
Fig.13 Pipe inspection algorithmVII. CONCLUSION

Fig.11 Defect form: Semicircular and Equilateraltriangle
Experiment 5

In this paper, a rotating probe in vinyl chloride pipe
was tested, and a new inspection robot system for drainpipe was developed. Moreover, the image wasprocessed, a circular image was developing, and it waspossible to make it to a plane image. In the future, these
two researches are combined, and the defect diagnostic
system with good accuracy will be made. And, it
actually experiments in pipe.

ACKNOWLEDGMENTS
We would like to thank ISHIKAWA IRON WORKS,
who provides the inspection robot for this research.

Fig.12 Defect form: Concave type

REFERENCES[1] Zhicheng Wang, Harutoshi Ogai, Shigeyuki
Takahiro, “Real-time Generation of Developed
View for Sewer Pipe Based on Web Camera
Video”, The 15th International Symposium on
Artificial Life and Robotics 2010, CD-ROM,
papers,OS3-4,2010,Beppu,Japan.
[2]Kentarou Nishijima, Yixiang Sun, Rupesh
Kumar Srivastava, Harutoshi Ogai and Bishakh
Bhattacharya, “Advanced Pipe Inspection Robotusing Rotating Probe”, The 15th International
Symposium on Artificial Life and Robotics 2010,
CD-ROM, papers,OS3-2,2010,Beppu,Japan.

5.4 Experimental Results
The Table.2 and Table.3 shows the outcome of anexperiment.
Table.2 Experimental Results (Size)©ISAROB 2011

518

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Application of the Genetic Algorithm on Face Recognition
Fengzhi DAI1, Liqiang SHANG2, Naoki KUSHIDA3, Masanori SUGISAKA4
1.

College of Electronic Information and Automation, Tianjin University of Science and Technology, daifz@tust.edu.cn
2.
Vestas Wind Technology (China) Co., Ltd, shliq@vestas.com
3.
Dept. Electronic Mechanical Engineering, Oshima .ational College of Maritime Technology, kushida@oshima-k.ac.jp
4.
.ippon Bunri University, ALife Robotics Co., Ltd, ms@alife-robotics.co.jp

Abstract: Computer vision and recognition is playing an increasing important role in the modern intelligent control.
Object detection is the first and the most important step in object recognition. Traditionally, special objects can bedetected and recognized by the template matching method, but the recognition speed has always been a problem. Also
for recognition by the neural network, training the data is always time consumption. In this paper, the genetic
algorithm-based face recognition system is proposed. The genetic algorithm (GA) has been considered as a robust andglobal searching method. Here, the chromosomes generated by the GA contain information (parameters) of the image,
and we use the genetic operators to obtain the best match between the original image and the face of interest. The
parameters are the coordinate (x, y) of the center of the face in the original image, the rate of scale and the angle Ѳ ofrotation. Finally, the experimental results and some other considerations are also given.
Keywords: Face detection and recognition, Genetic algorithm, Image processing, Template matching method.

to obtain the best match to the template [7]: searchingfor the best match is the goal of this paper.
This thought emerged from the features of GA, andthe need to recognize the faces of special people easilyand quickly by an intelligent system. The single concept
and feature of image processing and the GA will not be
introduced here, because there is already extensive
literature on these subjects.
In this paper, the encoding and decoding method of
the GA and the experimental setting, the experiment andthe analysis, and conclusion are addressed sequentially.

I. ITRODUCTIO
Lots of papers and applications are presented on theweb, in conference proceedings or journals about the
intelligent control. Among them, image processing andrecognition occupy a very large percentage [1-3]. The
higher the degree of intelligence is, the more importantis the image detection and recognition technology.
For an intelligent control system, it is necessary toacquire information about the external world
automatically by sensors, in order to recognize its
position and the surrounding situation. A camera is oneof the most important sensors for computer vision: theintelligent system endeavors to find out what is in animage taken by the camera: traffic signs, obstacles or
guidelines, etc.
The reliability and time-response of the object
detection and recognition have a major influence on theperformance and usability of the whole object
recognition system [4]. The template matching method
is a practicable and reasonable method for object
detection [5], but the recognition speed has always beena problem.
In addition, in order to search for the object ofinterest in an image, lots of data need to be processed.
The genetic algorithm (GA) has been considered to be arobust and global searching method (although it issometimes said that GA can not be used for finding the
global optimization [6]). Here, the chromosomes
generated by GA contain information about the image
data, and the genetic and evolution operations are used

©ISAROB 2011

II. THEORY AD EXPERIMET
If we want to find a special person in an image, wefirst have to detect people in the image, and thenrecognize which one is the person of interest(sometimes these two steps will be executed
simultaneously).
1. Genetic encoding
Since the chromosomes generated by the GA contain
information about the image data, the first step is to
encode the image data into a binary string to the GA [8].
Some important parameters of GA are given in Table
1, and the search field and region are given in Table 2.
Table 2 shows that there are 4 image parameters: thecenter of a face (x, y) in the original image, the rate ofscale of the face, and the rotating angle Ѳ, are encoded
into the elements of gene (the meanings of which will
be introduced below). Since one parameter uses 8 bits

519

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

division by 255 ensures that the resulting fitness is
between 0 and 1.
Based on Eq.1, in the program, the fitness iscalculated by the following four steps, and Fig.2 gives
the figural example.

(1 byte), and there are 4 image parameters, thus one
chromosome contains 4 bytes shown in Fig.1.

Ѳ

Scale
by rateFig.1. One chromosome contains 4 bytes
Table 1. Setting for the GA parameters
SourceThe original and the template images
Maximum = 300 (the stopping
Generations
criterion)
Population
200
Size
Pr of the best individuals will be
Reproduction selected to survive. The remained (1Pr) will be treated by the genetic(selection)
operators (crossover and mutation)
Offspring are produced from parents
Crossoverby exchanging their genes at thecrossover point, the ratio is Pc
Producespontaneous
randomchanges in various chromosomes. Th
Mutatione general random change method isused at the rate of Pm

x

Table 2. Setting for the experimentImage parameters (field)(0, max_x)
8 bitsy

8 bits(0, max_y)

rate

8 bits(1.0, 3.0)

Ѳ

8 bits(-35o, 35o)A

Template image

Translate by (x, y)Original image
Fig.2. Figural example of calculation by Eq.1

(1) The coordinate (i, j) in the template is scaled by thevalue of parameter rate, then we get the result f(x, y,
rate, Ѳ) = f(0, 0, rate, 0) = (i × rate, j × rate);(2) The result of step (1) is rotated by the value ofparameter Ѳ, thus f(x, y, rate, Ѳ) = f(0, 0, rate, Ѳ);(3) The result of step (2) is translated from thecoordinate (x,y) in the original image, then the graylevel of the pixel f(x, y, rate, Ѳ) is gotten;(4) All the differences between the gray level of thecoordinate (i, j) in the template and that of thecalculated f(x, y, rate, Ѳ) in the original image aresummarized by Eq.1 to calculate the fitness.
If the value of the fitness is lager than the preset
threshold, the search process is over and the result isgiven, otherwise the loop will be continued.

GA parameters
Pr

0.6

Pc

0.5

Pm

0.01

2. Genetic algorithm
The fitness is defined as [9]temp_y temp_x

3. Decoding
In order to obtain the true value of the image
parameters from the chromosome, decoding is needed.
In Eq.2, after decoding from datum of GA, the value ofeach parameter is standardized. A figural example isgiven in Fig.3.
MAX − MIN
(2)
Value = MIN +× (datum from GA )∑ ∑| f ( x, y, rate,θ ) − temp(i, j ) | (1)fitness = 1.0 −

j =0

i =0

(temp_x) × (temp_y) × 255

In Eq.1, temp(i, j) is the gray level of the coordinate(i, j) in the template image, the width and height ofwhich are temp_x and temp_y. f(x, y, rate, Ѳ) gives thegray level in the original image, the coordinate of which
are calculated by translation from (x, y), and bychanging the scale and the rotation angle Ѳ from thetemplate. Since the images have 256 gray levels, in Eq.1,©ISAROB 2011

Rotateby Ѳ

255

4. Experiments
The experiment is done by first loading the original
and the template images. GA is used to find whether or
not there is the object (face) of a template in the original520

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011is called chromosome, and represents a solution to the
problem. By stochastic search techniques based on themechanism of nature selection and natural genetics,
genetic operations (crossover and mutation) andevolution operation (selecting or rejecting) are used tosearch the best solution [10].
In this paper, the chromosomes generated by the GA
contain information about the image, and we use thegenetic operators to obtain the best match between theoriginal image and the template. The parameters are thecoordinate (x, y) of the center of the object in theoriginal image, the rate of scale, and the rotation angleѲ.
In fact, translation, scale and rotation are three main
invariant moments in the field of pattern recognition.
However, for face recognition, the facial features are
difficult to be extracted and calculated by the general
pattern recognition theory and method. Even these threemain invariant moments will not be invariant because
the facial expression is changed in different images.
Thus the recognition only gives the best matching
result within an upper predetermined threshold. The
result in the paper shows that the recognition is satisfied.
By using the GA-based recognition method, thesettings of the search field (in this paper, (x, y, rate, Ѳ)is selected), the determination of the genetic operations,and the selection and the optimization of the fitness
function all have a strong effect on the level ofrecognition of the resultant image.
Based on the results of experiments described here,
further work will emphasize (i) optimizing the fields ofchromosomes, and (ii) improving the fitness function byadding some terms to it. This work is important andnecessary in order to improve the GA-based facerecognition system.

Fig.3. Figural example for decoding
image. If the answer is YES, then in the original image
the result gives the coordinate of the center of the face,
the rate of scale, and the rotation angle from thetemplate.
Fig.4 and Fig.5 are the template and the original
images for the experiment. The values are the width ×
height in pixels of the image. In Fig.5, two original
images are presented, the sizes of which are the same
320×240.

30×35
Fig.4. Template for matching (temp_x × temp_y)
The maximum number of generation is limited to300, and the threshold of the matching rate is set to 0.85.
Thus if within 300 generations the matching rate canreach 0.85, it is said that the template is found in theoriginal image (the template matched the original image
by the threshold). The result of the GA-based facerecognition is given in Fig.6 and Table 3.

Acknowledgment
This work is sponsored in part by the Scientific
Research Foundation of Tianjin University of Scienceand Technology (China) for the Introduced Talented
Scholars, No. 20100408.

Table 3. Results of searching by GA
Result for Fig.5a Result for Fig.5b
Fitness
0.858
0.875
Generation
28
18(x, y)(71, 108)(51, 107)
Rate
2.00391.9333
Angle [deg]
12.59
-9.76

REFERECES[1] K. Deguchi, I. Takahashi (1999), Image-based
simultaneous control of robot and target object motion
by direct-image-interpretation, Proc. the 1999 IEEE/RSJ
International Conf. of Intelligent Robot and System, vol.
1, 375-380[2] H. Takimoto, Tasue Mitsukura, M. Fukumi, etal(2002), A design of face detection system based on
the feature extraction method, Proc. of 12th Symp. on

III. COCLUSIOIn this paper, the GA-based image (face) recognition
method is tested.
The GA starts with an initial set of random solutions
called the population. Each individual in the population©ISAROB 2011521

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fuzzy, Artificial Intelligence, Neural Networks andComputational Intelligence, 409-410[3] F. Dai, T. Kodani, Y. Fujihara (2006), Research on
Face Recognition System by Genetic Algorithm, Proc.
of the 11th Int. Symp. on Artificial Life and Robotics,
CD version, 63-66[4] M. Sugisaka, X. Fan (2002), Development of a facerecognition system for the liferobot, Proc. of the 7th Int.
Symp. on Artificial Life and Robotics, vol. 2, 538-541[5] K. R. Castleman (1998), Digital image processing,Original edition published by Prentice Hall, Inc., a
Simon & Schuster Company, Press of Tsinghua
University[6] H. Iba (1994), Foundation of genetic algorithm –solution of mystic GA, Omu Press (in Japanese)[7] T. Agui, T. Nagao (2000), Introduction to image
processing using programming language C, Shoko-dopress (in Japanese)[8] M. Gen, R. Cheng (1997), Genetic algorithms andengineering design, A wiley-Interscience Publication[9] F. Dai, T. Kodani, Y. Fujihara (2007), Research on aFace Recognition System by the Genetic Algorithm,
Journal of Artificial Life and Robotics, Issue: Volume
11, Number 1 / January, 2007, pp. 67-70[10] F. Dai, T. Adachi, Y. Fujihara, et al (2007),
Fundamental Research on Face Recognition by Genetic
Algorithm, Proc. of the 12th International Symp. on
Artificial Life and Robotics 2007, CD version, pp. 639642

(a)

(b)

Fig.5. The original images (size: max_x, max_y = 320×240)

(a)

(b)

Fig.6. Results of the GA-based recognition©ISAROB 2011522

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Study on the Disturbance Rejection of Virtual Slope Walking
by Stepper-2D Robot
Mingguo Zhao1,2, HaoDong1 and Naiyao Zhang1
1State Key Laboratory of Robotics and System (HIT)
Department of Automation, Tsinghua University, Beijing 100084, China( mgzhao@tsinghua.edu.cn, donghao00@mails.tsinghua.edu.cn, zlh@tsinghua.edu.cn )2

Abstract: Virtual Slope Walking is a new realization of powered walking based on Passive Dynamic Walking, which is
not only effective in generating fast walking, but also achieving advantages on disturbance rejection. Under the openloop condition without external sensing device，the step-handling walking with maximum step height of 10% leg
length is realized on a planar bipedal robot Stepper-2D. This paper theoretically studies the disturbance rejection of
Virtual Slope Walking by introducing the ground step perturbation. We theoretically proved that the step perturbation
can be transformed to the disturbance of initial system state and successful step handling walking comes from thesystem’s cyclic stability. The necessary and sufficient condition of recovering from the step perturbation is obtained andconfirmed by the experiment on Stepper-2D.
Keywords: Bipedal robot; Virtual Slope Walking; Disturbance Rejection.
walking comes from the fixed point’s stability. Then theI. INTRODUCTION
necessary and sufficient condition of recovering fromthe step perturbation is presented based on the analysis
McGeer [1] demonstrated that a Passive Dynamicof the relationship between the system state and theWalker can walk down a shadow slope with no control
minimum initial state in the transition phase, providing
and actuation in the early of 1990. Then the concept of
the possibility to quantitatively analyze the maximal
Passive Dynamic Walking has been used as a starting
relative step height.
point for designing powered walkers to walk on level
The remainder of this paper is organized as follows.
ground. Wisse[3], Hobbelen[4] and Collins[5].
InSectionⅡ, the model of Virtual Slope Walking isdemonstrated several realizations of powered walking
presented. In Section Ⅲ, the ground step disturbance
based on kinematic energy complement. Asano [6],Honjo [7] and Harata [8] introduced the parametrichandling is illustrated, and the performance of
excitation for potential energy restoration. In our
disturbance rejection is analyzed in Section Ⅳ. Section
previous work [9] [10], we proposed Virtual Slope
Ⅴ presents the experimental results and Section Ⅵ the
Walking by introducing the leg length modulation andconclusion and future work.
achieved a relative speed of 4.48leg/s on a planar
II. Model of Virtual Slope Walking
bipedal robot Stepper-2D.
Disturbance rejection, defined as the ability to deal
with unexpected disturbances [11], is considered as one
1 Model Descriptionof the fundamental performances for bipedal walking.
A cartoon of the Virtual Slope Walking model is
There exists several ways to measure the disturbanceshown in Fig. 1. We assumed that the model has two
rejection for a PDW based walker theoretically, such as
telescopic massless legs and a point mass body at the
Floquet multipliers, Basin of Attraction and the Gait
hip. The stance leg is actuated for extending from rs to re
Sensitivity Norm. But the most commonly usedfollowing a smooth leg length trajectory r(t), the swing
experimental measure is the ground step perturbation
leg is actuated for shortening from re to rs in one step.
that a walker can handle without falling. Wisse[12]The length shorten ratio is then defined as =rs/re. Since
realized powered walking under a step height of 2% leg
the swing leg is assumed massless, it can be swung
length disturbance. Pratt [13] realized powered walking
arbitrarily quickly to the position with constant inter-leg
under a step height of 9% leg length. Geng [14]angle φ0 before heel strike. The impact of the swing leg
achieved powered walking under a step height of 4%
with the ground is assumed to be fully inelastic (no slip,
leg length by on-line machine learning and PDW based
no bounce) and instantaneous, which implies that there
mechanism designed. We[15][16] have realized aexists discontinues change in the velocity of the center
powered walking under a step height of 10% leg lengthof mass and unchanged system configuration.
on Stepper-2D based on Virtual Slope Walking.
In this paper, we theoretically study the
disturbance rejection of Virtual Slope Walking andpresent its stabilizing mechanism under. Based on theasymptotic expression of stride function and the fixed
point, we theoretically proved that ground step
perturbation can be transformed to the disturbance ofinitial system state and the successful step handlingFig. 1 Model of Virtual Slope Walking©ISAROB 2011523

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

We define a walking step starts when the newstance leg (lighter line) has just made contact with theground in the upper left picture, namely instant I. The
stance leg swings to the position at θ=θII in the upper
right picture for the beginning of extension, namely
instant II. And the stance leg extension ends at θ=θIII in
the bottom left picture, namely instant III. The swingleg (heavier line) is shortened and swings to the position
with constant inter-leg angle φ0 just before heel strike inthe bottom middle picture, and hits the ground in thebottom right picture, namely instant IV. Then, the stanceleg and swing leg exchange subsequently, and thewalking cycle repeats continuously.

3 Stride Function and Fixed Point
The general procedure for the study of this modelis based on interpreting a step as a Poincaré map, or, as
McGeer termed it, a ‘stride function’ [1]. Our Poincaré
section is at the start of a step, namely instant I in Fig. 1.
Given the state of the system at instant I, the Poincaré
map f determines the state just after the next heelstrike.
Note that in the geometric collision condition Eq. (4),the stance leg angle θI is constant with inter-leg angle φ0  cos  0(6)
 I   arctansin  0So the heels trike transition reduces this problem in 2D
state space {θI, ωI} to a one dimensional map f, only
consisting of angular velocity ωI. So, while the system
has only one independent initial condition, we need to
specify ωI at the start of walking step n to fully
determine the subsequent motion at steps n+1, n+2,…
so that ωI(n+1) can be obtained from ωI(n) by the
Poincaré mapping. We have proved that under theEquivalent Definition, The Trajectory Leg Extension(TLE) can be equivalently transformed to the
Instantaneous Leg Extension (ILE) [15] in Virtual Slope
Walking. Consequently, defining a new variable q=ω2I
as the system state, the stride function f can beanalytically obtained under the Instantaneous Leg
Extension (ILE) as follows

2 Governing Equations
The governing equations of the system consist of
nonlinear differential equations for the swing phase andalgebraic equations for the transitions of heel strike.
(1) Swing phase from I to II: Using Lagrangian
Equations, the second-order differential equation of
motion is given below for the swing phase of the stanceleg with the constant length rs under the dimensionlesstime variable   g re t
1(1)
sin  ( )For the simplicity, we will refer to dimensionless time τas the time variable, henceforward.
(2) Stance leg extension From II to III: The stanceleg acts as an inverted pendulum with variable lengthr(τ). Using the Lagrangian Equations, the equation of
motion can be written as ( ) f ( q )   2 cos 2  0 q


re
2 
 
 ( )  r ( ) sin  ( )  r ( ) r ( )  ( )
(2)

 

 r ( )  r ( )  2 ( )  r cos  ( )  r F
e
e

mg
where F is the force that the leg exert on the center ofmass during the stance leg extension.
(3) Swing phase from III to IV: Similar to theequation in Eq. (1), the equation of motion for the
stance leg with the constant length re can be written as

 2 cos 2  0 [cos  II* (

1

1





  )  cos  I (2(7)
  )]

where θ*II is the equivalent extension angle. Since thewalking with TLE and its equivalent ILE produces theequivalent cyclic walking motion, ILE can be used as atheoretical tool for the analysis of Virtual Slope Walking
without the dependence on numerical simulation.
The fixed point of the stride function is defined asf(qf) = qf. From Eq. (7), the fixed point can then be
obtained as follows
2 cos 2  0 [cos  II* (1   3 )  cos  I (    3 )] (8)qf  2 (1  cos 2  0  2 )

(3)
 ( )  sin  ( )(4) Heelstrike transition from IV to I of thesubsequent step: The heelstrike from step n to thesubsequent step n+1 occurs when the geometriccollision condition
 I ( n  1)   ( 0   IV ( n ))

(4)


cos

I ( n  1)  cos  IV ( n )is met, where the ‘I’ and ‘IV’ subscripts denote theinstant I and IV respectively, φ0 is the constant of theinter-leg angle at heels trike. Eq. (4) also reflects achange of names for the two legs. The swing leg
becomes the stance leg, and vice versa.
From the conservation of angular momentum about
the swing foot contact point at heel strike, we obtain thefollowing transition equation©ISAROB 2011cos  0(5)
 IV ( n )Eq. (1)-(5) construct the dynamic equations of thishybrid system.

 I ( n  1) 

III. Ground Step Disturbance Handling
1. Transition Walking
After entering into the periodic state in Virtual
Slope Walking, once the system is perturbed by a single
step, there will be a transition phase in the subsequent
one or two walking steps. And after that, the system
state will approach the fixed point asymptotically thesame as the condition of initial state’s disturbance in
Virtual Slope Walking. We will illustrate such transition
walking by introducing the single step-up and step-updown perturbations in the following section.

524

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

2. Single Step-Up Perturbation
We assume that the system is in the periodic state
in step n-2, and a single step-up perturbation occurs at
the end of step n-1, just at heel strike. The subsequenttransition walking step n is shown in Fig. 2. In thetransition phase, there exist the perturbations not only
on the system state q, but also on the stance leg angle θIand θIV, which is constant in normal walking, resulting in
the variation of the stride function. After the transition
phase ends, only disturbance on the system state q exists.
 If

 IVf

 If
 IVf

 In  IVf

 IVn
n 1

n2

3. Single Step-Up-Down Perturbation
We assume that the system is in the periodic state
in step n-2, and a single step-up-down perturbation
occurs at the end of step n-1, just at heel strike. The
subsequent transition walking steps n and n+1 areshown in Fig. 3. The same as the step-up perturbation,in the transition phase, there exist the perturbations not
only on the system state q, but also on the stance leg
angle θI and θIV, which is constant in normal walking,resulting in the variation of the stride function. After thetransition phase ends, only disturbance on the system
state q exists.

If

 IVf

h

n

n 1

Fig. 2 Transition Walking of the Single Step-Up Perturbation

n2

1





 ) (
2

n

n 1

n2

The system state at the start of step n can be consideredas the output of the stride function with perturbed θIV ofstep n-1. So q(n) can be represented asq(n)   2 cos 2 0 q f(14)
1
1
2 cos 2 0 [cos  II* ( 2   )  ( 2 cos  IVn   cos  If )]Both perturbations on θI and θIV exist in transition
walking step n (Fig. 5), so q(n+1) can be considered as
the output of the stride function with perturbed θI of stepn and θIV of step n+1
q(n  1)   2 cos 2 0 q(n)(15)
112 cos 2 0 [cos  II* ( 2   )  ( 2 cos  IVn1   cos  In )]
There exists the perturbation on θI in transition walking
step n+1 (Fig. 3), so q(n+2) can be considered as theoutput of the stride function with perturbed θI of stepn+1
q(n  2)   2 cos 2 0 q(n  1)
(16)
1
1
2 cos 2 0 [cos  II* ( 2   )  ( 2 cos  IVf   cos  In 1 )]θI and θIV returns to the constant value from step n+2,
then q(n+3) can be represented asq(n  3)   2 cos 2 0 q (n  2)
(17)
1
1
2 cos 2 0 [cos  II* ( 2   )  cos  If (   )]
The transition phase ends after step n+1, and the system
state starts approaching the fixed point asymptoticallywith the initial state of q(n+2).
It can be concluded from Eq. (14)-(16) that the single
step-up-down perturbation introduces the same
disturbance as that of the single step-up perturbation. It(11)cos  IVf   cos  In )]2

θI and θIV returns to the constant value from step n+1,
then q(n+2) can be represented asq(n  2)   2 cos 2 0 q(n  1)
(12)
1
1
2 cos 2 0 [cos  II* ( 2   )  cos  If (   )]
The transition phase ends after step n, and the system
state starts approaching the fixed point asymptoticallywith the initial state of q(n+1).
It can be concluded from Eq. (10)&(11) that thesingle step-up perturbation introduces the disturbanceon the system state q of step n and transfers such
disturbance by the stride function in the subsequenttransition phase. It is indicated from Eq. (12) that after
the transition phase ends, the step perturbation can betotally transformed to the disturbance of initial system
state. So, once the continuous walking condition of theinitial state is satisfied, the system state will definitely
approach the fixed point in the following walking steps.

©ISAROB 2011

h

 IVf

,

n
n
n 1
n 1
cos  IV   cos  I  hr cos  IV   cos  I  hr

The system state at the start of step n can be consideredas the output of the stride function with perturbed θIV ofstep n-1. Let θfI and θfIV be the constant stance leg angle
which is corresponded to the fixed point, then q(n) canbe represented asq(n)   2 cos 2 0 q f(10)
1
1
2 cos 2 0 [cos  II* ( 2   )  ( 2 cos  IVn   cos  If )]
There exists the perturbation on θI in transition walking
step n (Fig. 2), so q(n+1) can be considered as theoutput of the stride function with perturbed θI of step n
1

n 1

 If

The perturbation on the stance leg angle θI and θIV
resulted from the step height disturbance in thetransition walking step n and n+1 can be obtained asfollows
 IVn   In  0 IVn 1   In 1  0
(13)


n
n
cos  IV   cos  I  hr

q (n  1)   2 cos 2 0 q (n) In1
 IVf

 In  IVn1nIV

Fig. 3 Transition Walking of the Single Step-Up-Down
Perturbation

Let h be the step disturbance height, and hr=h/re be
the relative value. Then, the perturbation on the stanceleg angle θI and θIV resulted from the step height
disturbance in the transition walking step n can beobtained as follows
 IVn   In  0(9)

2 cos 2 0 [cos  II* (

If

525

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011is indicated from Eq. (17) that after the transition phaseends, the step perturbation can be totally transformed tothe disturbance of initial system state. The only
difference is that the transition phase of single step-updown perturbation includes one more step than that ofsingle step-up perturbation. So, once the continuous
walking condition of the initial state is satisfied, thesystem state will also definitely approach the fixed pointin the following walking steps.
So we can draw the conclusion from the above resultsthat the step height perturbation can be totallytransformed to the disturbance of initial system state,
and the disturbance rejection problem in Virtual Slope
Walking can be transformed to the stabilizing problemof the fixed point if the continuous walking condition issatisfied.

lways restricted by the physical parameters of the real ro
bot.
0.35

0=55o

0.2

0=60

o

0.05
0

0.7

0.75

0.8

0.85

0.9

0.95

Fig. 4 Trajectory of maximum relative step height hrmax
versus length shorten ratio β

2.2 Effect of the Equivalent Extension Angle θ*II
The maximum relative step height hrmax is shown
as a function of θ*II in Fig. 5 with two values of β and φ0
respectively. Fig. 5 shows a second order relationshipbetween hrmax and θ*II. As θ*II approaching zero fromboth side, hrmax increases and reaches a maximum valueat θ*II =0o. The vertical projection of leg length
extension increases as θ*II approaching zero, and morepotential energy is complemented. As a consequence,q(n) and hrmax increase while qz(n) stays constant. The
vertical projection of leg length extension reaches its
maximum at θ*II =0o.


3
2
2
*
1  4q f  1 (1  )(1  cos 0 )cosII   3 cos f , *  0
I
II
2
2
2
 cos 0

0.25

0.25

0.2

0.2

0.15

0.15

0.1

0.1

0.05

0.05

=0.7 0=55o

=0.7 0=60o

=0.8 0=55o

=0.8 0=60onand θI satisfy
 IVn   In  0
(19)

n
n
max
cos
cos
h






r
IV
I
It can be concluded from Eq. (18)&(19) that the
maximum relative step height hrmax is determined by themodel parameters length shorten ratio β, equivalentextension angle θ*II and inter-leg angle φ0. We will
illustrate the influence of model parameters on the
disturbance rejection in the following section.

0

-10

0

0

-10 10 0

1020

20 3030

40

50

Fig. 5 Trajectory of maximum relative step height hrmax versus
equivalent extension angle θ*II

It can be concluded from this graph that extending
the stance leg more close to mid-stance will result ina greater hrmax and a larger disturbance rejection in
Virtual Slope Walking. We can extend this conclusion
to the Trajectory Leg Extension (TLE) that theextension phase being close to mid-stance also produces
larger disturbance rejection.
2.3 Effect of the Inter-leg Angle φ0
The maximum relative step height hrmax is shown as afunction of φ0 in Fig. 6 with four values of β. As shownin Fig. 6, hrmax decreases with an increase in φ0. The
dissipation energy Er increases as φ0 increases. As aconsequence, the system kinematic energy decreases,and q(n) decreases. On the other side, an increase in φ0causes an increase in qz. Consequently, hrmax decreases
from combined action with the effect of changing in
q(n) and qz(n).

2. Influence of Model Parameters
2.1 Effect of the Length Shorten Ratio β
The maximum relative step height hrmax is shown as
a function of β in Fig.4 with four values of φ0. It is
indicated from Fig.4 that hrmax decreases with anincrease in β. An increase in β causes a net decrease in
the extended leg length, resulting in a decrease in thecomplementary energy Ec and a decrease in the systemkinematic energy which is represented by the system
state q. On the other side, an increase in β causes anincrease in qz. Consequently, hrmax decreases fromcombined action with the effect of changing in q(n) andqz(n).
The main conclusion from this graph is that a decrea
se in β leads to a greater hrmax and a larger disturban
ce rejection in Virtual Slope Walking. However, β is a

©ISAROB 2011

0.25

0.1

1. Maximum Relative Step Height
There exists the maximum relative step height hrmax
when q(n)=qz(n) holds as hr increases, which describes
the performance of disturbance rejection of Virtual
Slope Walking. Therefore, hrmax can be obtained asfollows
hrmax   tan2 0 cosIn1 4 f
*
3
3
f
*
(18)
2  q  cosII (1  )   cosI  cos2  ,II  00where hr0=50o0.15

Ⅳ Analysis of Disturbance Rejection of
Virtual Slope Walkingmax0=45o0.3

526

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 20110.35

confirmed in the following experiment results.

=0.7
=0.75
=0.8
=0.820.3
0.25

Ⅴ. Experiment
1. Planar Bipedal Robot Stepper-2D
We use the planar bipedal robot Stepper-2D as atest bed of disturbance rejection in Virtual Slope
Walking under single step perturbation. As shown in Fig.
8, Stepper-2D is mounted on a boom to constrain the
body motion in the sagittal plane. The boom has threeorthogonal DOF and the length is six times more thanthe height of the robot, so its effect on the robot sagittal
movement can be ignored. Stepper-2D’s leg length is
250mm and hip mass is 390g.

0.2
0.15
0.1
0.05
45

48

51

54

57

60

Fig. 6 Trajectory of maximal relative step height hrmax versus
inter-leg angle φ0

So we can conclude from this graph that a smaller
φ0 results in greater hrmax and a larger disturbancerejection in Virtual Slope Walking.
2.4 Adjoint Relationship between the Walking Speed andDisturbance Rejection
The walking speed described by the Froude Number
Fr is also determined by the model parameters β, θ*II,and φ0 [16]. Therefore, as the model parameters change,
there exists an adjoint relationship between the walking
speed and disturbance rejection. The maximum relative
step height hrmax is shown as a function of Fr in Fig. 7.

Fig. 8 Planar Bipedal Robot Stepper-2D with Point Foot.

The leg with the point foot is actuated in the hip andknee joint by digital servo motors. The telescopic leg
motion is realized by bending and unbending the knee
joint. And the swing leg motion is achieved by hip
motor actuation [15]. All digital servo motors arecontrolled by a computer through serial bus.

0.35
0.3
0.25

2. Experimental Results
Stepper-2D successfully recovers from a maximum
single step perturbation of 25mm in height, with amaximum relative step height hrmax of 10% leg length.
The hip and knee joints data from the motor sensors in
the real walking experiment. Fig.9&10 presents theimage sequences of the walking experiment under asingle step-up and step-up-down perturbation of
Stepper-2D respectively. The robot reaches the periodic
state after several steps. And when the step height isgreater than 25mm, it falls backward.
All the videos about the walking experiments
including the single step-up and step-up-down could befound on our website
http://v.youku.com/v_show/id_XMjA3ODM5OTcy.htm
l.

hr

max

0.2
0.15
0.1
0.05
0
0.1

0.2

0.3

0.4

0.5

Fig. 7 Trajectory of maximal relative step height hrmax versus
walking speed Fr

It is indicated from Fig. 7 that hrmax increases with anincrease in Fr. Since the effect of model parameters on
hrmax is the same as that on Fr, this conclusion exists
distinctly. Such conclusion suggests that achieving fast
walking speed always accompanying with large
disturbance rejection.
From the above analyze results, it can beconcluded that the performance of disturbance rejection
in Virtual Slope Walking can be determined by themodel parameters β, θ*II, and φ0, which will be

Fig. 9 Image sequence extracted from video of a single step-up perturbation experiment (hrmax=25mm)

Fig. 10 Image sequence extracted from video of a single step-up-down perturbation experiment (hrmax=25mm)©ISAROB 2011

527

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

The compararison of maximum relative step height ofStepper-2D with other typical dynamic walkers isshown in table 1, suggesting that Stepper-2D achieves
an improvement on the disturbance rejection of theprevious research.

[4] D.G.E. Hobbelen, M. Wisse. Ankle Actuation for
Limit Cycle Walkers. International Journal of
Robotics Research, 27(6):709-735, 2008.
[5] Collins, S.H, Ruina, A. A Bipedal Walking Robot
with Efficient and Human-Like Gait. In Proc. IEEE
International Conference on Robotics andAutomation, 18-22 April, pp. 1983- 1988, 2005.
[6] F. Asano, Zhi-Wei Luo. Energy-Efficient and HighSpeed Dynamic Biped Locomotion Based on
Principle of Parametric Excitation, IEEE Trans.
Robot., vol. 24, no. 6, pp. 1289–1301, 2008.
[7] Toyoyuki Honjo, Zhi-Wei Luo, Akinori Nagano.
Parametric excitation of a biped robot as an inverted
pendulum," Proceedings of IEEE/RSJ 2008
International Conference on Intelligent Robots andSystems, pp. 3408-3413, 2008.
[8] Yuji Harata, Fumihiko Asano, Kouichi Taji and Yoji
Uno. Parametric excitation walking for four-linked
bipedal robot, Proceedings of the 9th International
IFAC Symposium on Robot Control (SYROCO), pp.
589-594, 2009.
[9] Mingguo Zhao, Ji Zhang, Hao Dong, Yu Liu, Liguo
Li, and Xuemin Su. Humanoid Robot Gait
Generation Based on Limit Cycle Stability, In theproceedings of the RoboCup Symposium, China,
JUL 15-18, 2008.
[10] Hao Dong, Mingguo Zhao, Ji Zhang, Naiyao
Zhang. Hardware design and gait generation ofhumanoid soccer robot Stepper-3D. Robotics andAutonomous Systems, 57:828-838, 2009.
[11] D. G. E. Hobbelen, M. Wisse. A DisturbanceRejection Measure for Limit Cycle Walkers: The
Gait Sensitivity Norm. IEEE TRANSACTIONS ON
ROBOTICS, VOL. 23, NO. 6, pp.1213-1224, 2007.
[12] M. Wisse. J. van Frankenhuyzen. Design andconstruction of mike; a 2d autonomous bipedbased on passive dynamic walking. Proc. Con
f. Adaptive Motion of Animals and Machines.
Kyoto, Japan: AMAM, 2003.
[13] Pratt J. Exploiting Inherent Robustness and Natural
Dynamics in the Control of Bipedal Walking Robots.
PhD Thesis, Massachusetts Institute of Technology,
2000.
[14] Geng T, Porr B, Wörgötter F. Fast biped walking
with a sensor driven neuronal controller and realtime online learning. International Journal of
Robotics Research, vol. 25 no. 3: 243 -259, 2006.
[15] Hao Dong, Mingguo Zhao, Naiyao Zhang. High
Speed and Energy Efficient Biped Locomotion
Based on Virtual Slope Walking. Autonomous
Robots, 2009.
[16] Mingguo Zhao, Hao Dong, and Naiyao Zhang.
The Instantaneous Leg Extension Model ofVirtual Slope Walking, In Proceedings of theIEEE/RSJ International Conference on Intelligent
Robots and Systems, 2009.

Table 1 Compararison of maximum relative step height
Type of dynamic walker
Stepper-2D
Flaminge [17]Runbot [18]Mike [15]Maximum
Relative Step Height
10%
9%
4%
2%

VI Conclusion and Future WorkIn this paper, we analytically study the disturbancerejection of Virtual Slope Walking by introducing the
ground step perturbation. We theoretically prove thatground step perturbation can be transformed to the
disturbance of initial system state and the successful
step handling walking comes from the system’s cyclicstability. We then obtain the necessary and sufficientcondition of recovering from the step perturbation by
analyzing the relationship between the system state andthe minimum initial state in the transition phase. Finally,
we illustrate the effect of leg length shorten ratio β,
equivalent extension angle θ*II and inter-leg angle φ0 on
the maximum relative step height hr, demonstrating that
achieving fast walking speed always accompanying
with large disturbance rejection in Virtual Slope
Walking. The step handling walking experiment of
Stepper-2D verifies the theoretical analysis results andpresents an improvement on the disturbance rejection
compared with the other current results.
Starting from the step handling walking under the
open-loop condition without external sensing device inthis paper, we will introduce the sensing data of the step
perturbation and study the sensor-based powered
walking from the kinematic energy complement
viewpoint, aiming at obtaining larger disturbancerejection for Virtual Slope Walking in the future work.

Acknowledgement
This work was supported partially by the National
Nature Science Foundation of China (No. 60875065)
and Supported by State Key Laboratory of Robotics andSystem (HIT) (No. SKLRS2010-ZD-01).

REFERENCES[1] McGeer, T., 1990a, Passive dynamic walking,International Journal of Robotics Research, 9(2), pp.
62–82.
[2] S. Collins, A. Ruina, R. Tedrake, et al. Efficient
Bipedal Passive-Dynamic Walkers. Science,
307(5712):1082-1085, 2005.
[3] M. Wisse. Three additions to passive dynamic
walking; actuation, an upper body, and 3d stability,
in Proc., Int. Conf. on Humanoid Robots. Los
Angeles, USA, 2004.

©ISAROB 2011

528

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A New Method for Mobile Robots to Avoid Collision with
Moving Obstacles
Yi Zhu1, Tao Zhang1, Jingyan Song1, Xiaqin Li1, Masatoshi Nakamura2
1

Department of Automation, Tsinghua University, Beijing 100084, China(Email: taozhang@tsinghua.edu.cn)
2
Research Institute of Systems Control, Saga University, Saga 840-0047, Japan
Abstract: A new method for mobile robots to avoid collision with moving obstacles is proposed in this paper. It adopts
the concept of safe sectors in the vector field histogram (VFH) method but simplifies its description. Moreover, the new
method takes the threat of moving obstacles into account when selecting motion direction and a new speed control lawthat considers more factors is designed. Hence it can better avoid moving obstacles than the VFH method. Simulation
results indicate that the new method also shows many advantages over the dynamic potential field (DPF) method which
is a representative approach for avoiding moving obstacles. Experiments have further verified its applicability.
Keywords: mobile robots; obstacle avoidance; moving obstacles
In this paper, we propose a new obstacle avoidance
method for mobile robots to deal with moving obstacles.
The new method adopts the concept of safe sectors inthe VFH method [1] which is a motion planning method
mainly for static environments. But the new method
simplifies its description to lower the computational andspatial complexity and takes the threat of moving
obstacles into account when choosing motion directions.
Hence it can better handle moving obstacles than the
VFH method. Another improvement of the new method
is its speed control law that takes more factors into
account than the VFH method. The new method also
shows advantages over the DPF method in many
aspects, which has been discussed in this paper. The
remainder of this paper is arranged as follows: the VFH
method is briefly reviewed in Section 2 and the new
method is presented in Section 3. Some simulations andexperiments are presented in Section 4.

I. INTRODUCTION
Obstacle is one of the key issues in many fields.
Many methods [1]-[9] have been proposed to indicate
the influence of obstacles and solve this problem overthe past twenty years. For example, In [2][3], theyfound recess shape to play an important role in theperformance of aerostatic bearings, and found vortex
flows in the recess by numerical experiments
which cause instabilities and vibrations of the bearing.
The elimination of obstacle influence is one ofimportant reasons in this research. This paper mainly
studies on the obstacle avoidance of mobile robots.
Although there are many methods on this research, mostprevious works focus on static obstacles and only a few
works [6]-[9] address the problem of dealing withmoving obstacles.
To deal with moving obstacles, one concept ispreviously planning a safe path which takes moving
obstacles into account to guide the robot [6]. The
drawback of this concept is that it assumes the
trajectories of moving obstacles are known in advance,
which is unrealistic in many scenarios. Another concept
is dynamically planning the motion in every control
cycle by sensory information [7]-[9]. This concept ismore practicable since it can adapt the changing motionof moving obstacles. One representative method basedon this concept is the DPF method [7], [8]. In thismethod, the target generates an attractive force and thethreat of all the moving obstacles is represented by arepulsive force. The robot always moves in the directionof their resultant force.

©ISAROB 2011

II. THE VFH METHOD
As an efficient obstacle avoidance approach, theVFH method [1] can generate smooth trajectory without
oscillations and guide the robot to go through narrowcorridors. The VFH method divides all the directions
around the robot into some safe sectors that the obstacle
density (a value that is proportional to the negative ofthe distance from the robot to obstacles) in any directionof these sectors is no less than a threshold. The middle
directions of such sectors are candidates for motion andthe one that has the minimal bias to the target direction
is selected as the final motion direction. Nevertheless,the VFH method only takes the distances of obstacles

529

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Note that two safe sectors can overlap in part. Only the
middle directions of the safe sectors can be selected as
the motion direction of the robot. In the example ofFig.1, there is only one safe sector whose middle
direction is θ (All the angles in this paper refer to the
local coordinates where the original angle equals to thehead direction of the robot and anticlockwise direction
is positive).
Step 2 (Calculate the corresponding maximum speed ofevery candidate direction) To keep safe, every candidate
direction θ has a corresponding maximum speed thatis calculated byinto account and ignored their velocities. Therefore it’s
not suitable to be applied in environments containing
moving obstacles especially when they move fast.

III. THE PROPOSED METHOD
The new method proposed in this paper adopts thebasic concept of safe sectors in the VFH method but
improves it in three main aspects. Firstly, the new
method directly compares the obstacle distance of one
direction with a threshold to judge whether the
direction is safe without figuring out a density value
based on a grid map that needs updating in every
control cycle as the VFH method does since the latteris unnecessary but computational and spatial expensive.
Secondly, we design a new speed control law thatconsiders more factors especially the obstacle speed.
The third also the key improvement is that in the new
method, we take the threat of moving obstacles into
account when selecting motion directions. Hence the
new method can better deal with moving obstacles.
The process for selecting the direction and speed of therobot in the new method is presented below. It can beillustrated by the example shown in Fig.1.
v′

0≤ n≤ N⎧ 2a (d n − d S )⎪⎪ cos θ n − θnvmax = ⎨2v0 ⋅ d n⎪⎪⎩ d 2 ⋅ cos θ − θSnv (θ , v )vO
Safe Sector

Threshold

π ⎞

⎟

2 ⎠

dn > dS(1)

dn ≤ dSvO⎧⎪
Tht (θ ) = ⎨ 2a ( DS (θ , v) − r − rO )⎪+∞⎩Fig.1. An example that θ 0 = 30 and N = 3oStep 1 (Find all the safe sectors and take their middle
directions as candidates for the final motion direction)
The new method divides all the directions around therobot into a series of sector units whose width is θ 0o( θ 0 = 5 in our experiments). Any sector that consists of
N ( N = 24 in our experiments) continuous units
whose minimal obstacle distances are all larger than d S( d S = 0.4m in our experiments) is considered as a safe
sector. Additionally, there is a special sector whose
middle direction is the target direction θ T and width isNθ 0 / 2 . If the minimal obstacle distance in this sector
is larger than d S , it is also considered as a safe sector.

©ISAROB 2011

{ }

⎛
⎝

where d n and θ n are the obstacle distance and the
direction of the nth sector unit, a is the averagenacceleration of the robot, v0 is a constant, vmax
represents the maximum speed limited by the obstacle
distance of the nth sector unit based on the requirementthat the obstacle distance must be large enough for the2
process of brake. The item cos (L) in (1) is used toslow down the speed when the bias between θ and thecurrent direction of the robot is large, which can shorten
the path length generated by turning.
Step 3 (Evaluate the threat from moving obstacles forevery candidate direction) We define the threat valueTht (θ ) from a moving obstacle for a candidate
direction θ asMoving Obstacle

Robot

2

n

DS

−v O

{ }

vmax = min vmax ⋅ cos ⎜ min θ ,⎧⎪DS > r + rO
DS ≤ r + rO

(2)

⎫⎪
⎬
0 ≤ v ≤ vmax⎪⎩ 2a ( DS (θ , v) − r − rO ) ⎪⎭where r and rO are the radius of the robot and the
obstacle, vO is the obstacle speed, DS is the distancefrom the obstacle to the straight line that passes throughthe robot’s center and parallels the vector v ′ which is
the relative velocity between the robot and the obstacle
if specific θ and its corresponding v are selected asshown in Fig.1. Note that v is the speed that generates
the lowest threat value in the speed boundary if specificv = arg min ⎨

530

vO

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011θ is chosen. If there are multiple moving obstacles, thefinal threat value is the maximum Tht (θ ) generated byvO ( vO = 0.1m/s ) into account, which is the concept of

the VFH method. The result is that the robot hits the
obstacle at 60T (T is the length of the control cycle).
This collision can be avoided if we increase d S tokeep enough distance to the obstacle. However, it will
be not safe again if the obstacle increases its speed and alarge d S will make it difficult to go through narrowcorridors. As a comparison, navigated by the new
method proposed in this paper, the robot smoothly
bypasses the same obstacle as shown in Fig.2(b).
Moreover, the new method is adaptive when the
obstacle increases its speed as shown in Fig.2(c)( vO = 0.4m/s ). The simulation results of Fig.2 indicate
the importance of taking the obstacle speed into accountfor obstacles avoidance, which is just the advantage ofthe method proposed in this paper over the VFH method.

them.
Step 4 (Select the final motion direction and speed fromall the candidates) The final direction is selected byθ = arg min {α1 ⋅ θ − θ T + α 2 ⋅ Tht (θ )}(3)where α and α 2 are constants. For soft-landing, thefinal speed is calculated by min {v , 0.5 ⋅ DT } where v
is the corresponding speed calculated in Step 3, DT isthe distance between the robot and the target.
The new method described above shows many
advantages over the DPF method for avoiding moving
obstacles. In the DPF method, all the effects of moving
obstacles are abstracted as a repulsive force. Such a
description is simple for implementation. However, as
pointed out in [1], it loses detailed information about the
obstacle distribution and can lead to a series ofproblems, e.g., oscillations in the presence of obstacles,difficulties in going through narrow corridors [10]. The
VFH method has well solved these problems byintroducing safe sectors to describe the distribution of
obstacles [1]. The new method proposed in this paper
has inherited the concept of safe sectors in the VFH
method and therefore it can also avoid the above
problems in the DPF method. Furthermore, some works[7] about the DPF method hasn’t paid much attention to
the speed control law as the method proposed in thispaper. Simple linear functions are usually adopted,
which will affect their performances. The advantage of
the DPF method is that some related works address the
problem of how to pursuit a moving target [7], [8],which hasn’t been taken into account in this paper.

(a) The method ignoring the obstacle speed(b) The proposed method ( vO = 0.1m/s )IV. SIMULATIONS AND EXPERIMENTS
To show the performance of the proposed method,
several simulation results are presented below.
For testing the performance of the new method
proposed in this paper, we design a scenario that the
robot meets an obstacle moving toward it as shown in
Fig.2 (In all the simulations in this paper, the start pointsof the robot and the obstacle are respectively at (0,8)
and (0,0); the robot’s target is at (0,-2); the dash circle
represents the locations of the robot and the obstacle atthe time labeled aside). Fig.2(a) shows the result if therobot moves in the direction that has the minimal bias tothe target direction from all the middle directions of safe
sectors ( d S = 0.4m ) without taking the obstacle speed©ISAROB 2011(c) The proposed method ( vO = 0.4m/s )Fig.2. Results of simulation 1
The simulations in Fig.3 have compared the new
method with the DPF method proposed in [7]. Fig.3(a)is the result of the work in [7] in the same scenario of
Fig.2(b). Compared with Fig.2(b), there are oscillations

531

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

in the trajectory of Fig.3(a) due to the shake of thepotential force shown in Fig.3(b). Such shakes occur
frequently when the robot suddenly meets an obstacleand it is an inherent drawback of the DPF method due toits oversimplified description of the obstacle effect. It
can be also observed that the trajectory in Fig.3(a) ismuch longer than the result in Fig.2(b). The proposed
method also shows advantages over the work in [7] inmany aspects of the speed control. As an example, inthe scenario of Fig.3(c) and Fig.3(d) (the robot moves
from (0,8) to (0,-2) but its initial direction is opposite to
the target), the method proposed in this paper generates
shorter path than the work in [7] when turning due to2
the item cos (L) in (1).

The proposed method has also been implemented onreal Pioneer3-AT robots as shown in Fig.4. Experimentresults have further verified its validity and applicability.

V. CONCLUSIONS
A new method for mobile robots to avoid collision
with moving obstacles is proposed in this paper. The
new method adopts the concept of safe sectors in the
VFH method but simplifies its description. Moreover, ittakes the threat of moving obstacles into account whenselecting motion direction and a new speed control lawthat considers more factors is designed. Hence it canbetter deal with moving obstacles than the VFH method.
Simulation results show that the new method also
performs better than the DPF method in many aspects.

REFERENCES[1] Borenstein J, Koren Y (1991), The vector fieldhistogram – fast obstacle avoidance for mobile robots.
IEEE Journal of Robotics and Automation, 7(3): 278288.
[2] Chen X D, He X M (2006), The effect of the recess
shape on performance analysis of the gas-lubricatedbearing in optical lithography. Tribology International,
39(11): 1336-1341.
[3] He X M, Chen X D (2007), The dynamic analysisof the gas lubricated stage in optical lithography.
International Journal of Advanced Manufacturing
Technology, 32(9-10): 978-984.
[4] Beaufrere B, Zeghloul S (1995), A mobile robot
navigation method using a fuzzy logic approach.
Robotica 13: 437-448.
[5] Minguez J, Montano L (2004), Nearness diagram(ND) navigation: collision avoidance in troublesome
scenarios. IEEE Trans Robotic Autom 20(1): 45-59.
[6] Conn R A, Kam M (1998), Robot motion planningon N-dimensional star worlds among moving obstacles.
IEEE Trans. Robotic. Autom 14(2):320–325.
[7] Ge S S, Cui Y J (2002), Dynamic motion planningfor mobile robots using potential field method.
Autonomous Robots 13: 207–222.
[8] Huang L (2009), Velocity planning for a mobile
robot to track a moving target - a potential fieldapproach. Robotics and Autonomous Systems 57: 55-63.
[9] Martinez-Gomez L, Fraichard T (2009), Collision
avoidance in dynamic environments: an ICS-based
solution and its comparative evaluation. IEEE
International Conference on Robotics and Automation:
100-105.
[10]Koren Y, Borenstein J (1991), Potential field
methods and their inherent limitations for mobile robot
navigation. IEEE International Conference on Robotics
and Automation 2: 1398–1402.

(a) The DPF method(b) The force components in X-axis and Y-axis(c) The proposed method (d) The DPF method
Fig.3. Results of Simulation 2

Fig.4. A scenario of experiments on real robots©ISAROB 2011532

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Some Thought for the Mckibben Muscle Robots
Huailin Zhao1
1

Xiaoqing Jia2

Masanori Sugisaka3

School of Electrical & Electronic Engineering, Shanghai Institute of Technology, China
2

College of Logistics Engineering, Shanghai Maritime University, China
3

Mec. & Ele. Course, Nippon Bunri University, Japan

Abstract: This paper introduces the background of the research.

will easily generate soft touching and relative safe

It summarizes the applications related with robots actuated byoperation. And human will feel the robot more like an

Mckibben muscles. Based on the robot joint structure, theanimal than a machine.

robots are classified into two types. The paper indicates that the

The world should be welfare. The development of thedifferent modeling methods and the control algorithms arerobot capable of assisting elders, handicappeds, andneeded to the two types of the robots.

patients is one of the main directions. Developing the

Keywords: artificial suit, Mckibben muscle, Modelingdevices capable of helping getting well for patients is oneof the main tasks of the modern technology. It’s expectant

1.

for the McKibben muscle to play an important role in these

Introduction
How to simulate the human behave and develop humanoid

fields. In some places such as Japan, America, and Europe,devices has been a hot spot and a difficulty in the robot field forthe artificial assisted suit made of McKibben muscles has

times. As it is thin and light and soft and similar to thecome out. The suit is called as “dressable robot”. Besides,

biological muscle, the Mckibben muscle has the special

the bipedal robot actuated by McKibben muscles has been

advantage for developing humanoid robots when it is used as

developed too. These artificial assisted suit or bipedal

the actuator to robots. Right just due to its above characteristics,

robot can only realize the main actions. They can’t

the requirement for its connection to robots is not too high. The

perceive the intention of human actions.

connection to robots can simulate the way by which theUsually,the

McKibbenmuscleconsistsofanbiological bones and muscles connect with each other in the

expandable rubber tube and a fiber weave mask. The outeranimal joint. Based on this kind of connection structure and thefiber mask expands when the rubber tube stimulated by theappropriate control algorithm, the robot can imitate animals

pressured gas, pulls its two ends and shortens themuch better.

McKibben muscle, just like the biological muscle contracts.

To the robot, in order to complete special tasks, it has to

The weave mesh changes its length and diameter byinteract with the outer environment. A very important problem

changing the incline angle. The pulling force F is linear tois the security, especially when interacting with human such as

the gas pressure P and nonlinear to its length L, which can

medical nursing. This means that the robot should keep stablebe calculated by formula (1) in which both b and n are theand compliant at the same time when it is carrying out tasks

structure parameters. The maximal contraction ratio isaccurately. The traditional robot, consisting of motors or fluid

usually less than 30%. Fig.1 shows its structure and theactuators and stiff linkers, is heavy and joint-stiff. So it is not

simple operation. It can output enough force and keep

suited to carry out these kinds of tasks, except the very

definite compliantness, so the robot actuated by McKibben

complicated control algorithms are designed. But themuscles is suited to carry out the “environment friendly”

McKibben muscle is inherently compliant – just like biologicaltasks such as nursing handicappeds and snatching at fragile

muscles. The robot actuated by it will act easily like animals. It

goods and so on.

©ISAROB 2011533

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

F=Pb2(3L2/b2-1)/(4πn2)(1)

Fig.1 The Mckibben muscle
Our project aims at an artificial suit actuated by pieces ofFig.2(b)Mckibben muscles. The artificial suit can put on the humanupper body including the two arms. Not only it can act likehuman arms, but also it can perceive automatically the action
intention of the human arms and output the intended actions.
But how to configure the Mckibben muscles in the artificial
suit and how to model and design the control algorithm become
the key problems.

2.

The applications
There have been many applications about the robots

related with our problems. Fig.2 shows the photos of the related

Fig.2(c)robots actuated by Mckibben muscles. Fig.2(a) shows a robot
which’s arms are actuated by Mckibben muscles[1]. Fig.2(b)shows a bipedal robot designed by Shadow Robot Company
which’s legs are actuated by Mckibben muscles[2]. Fig.2(c)shows a bipedal robot designed by Oita University which’s legs
are actuated by Mckibben muscles too[2]. But the connection
structure is different from that in Fig.2(b). Fig.2(d), Fig.2(e)and Fig.2(f) show the artificial suit actuated by Mckibben
muscles which are designed by Tokyo University of Science[4].

Fig.2(d)Fig.2(g) shows an assisted device which can help the patient’s
ankle.

Fig.2(e)Fig.2(a)©ISAROB 2011534

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig.3(b)Fig.2(f)Fig.3(c)Fig.2(g)Fig.2 The applications

3.

The two types of robot joints
To all the different applications, the structure of the robot

joint can be classified into two types on whole. One is the“regular” structure with which two Mckibben muscles parallel
to each other in the same size connect with each other by apulley, such as Fig.2(a) and Fig.2(b). The other one is theFig.3(d)“irregular” structure, such as Fig.2(c), Fig.2(d), Fig.2(e),Fig.2(f) and Fig.2(g). Fig.3(a), Fig.3(b) and Fig.3(c) show thatthe robot joint with “regular” structure can be analyzed as
one-input and one-output system. Fig.3(d), which is theconfiguration of Mckibben muscles in the robot shown in
Fig.2(c), can be used to represent the robots with the“irregular” joint structure. Fig.3(e) shows the relationshipbetween the three inputs and the one output. With three

Fig.3(e)Pressure inputs, it can’t be simplified as one-input andone-output system.

Fig.3 The “regular” structure and the “irregular” structure
robot

4.

How to model and control the robot
To analyze the robot, the model of it should beestablished. And to control it, the control algorithm should

Fig.3(a)©ISAROB 2011535

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

be designed. To the “regular” structure robot joint shown inhuman’s. One of the main reasons is that the contraction

Fig.3(a), the pressures of the two McKibben muscles vary

ratio of the Mcibben muscle is less than that of the humanrelatively instead of independently. At the initial, the two

muscle’s[6].

pressures of the two Mckibben muscles are P10 and P20
respectively. When the joint bends, P1 increases by ∆P and P2

5.

Conclusion:

decreases by ∆P at the same time. Contrariwise when the joint

Based on the characteristics of the Mckibben muscles

stretches, P1 decreases by ∆P and P2 increases by ∆P at theand the different applications, it is possible to develop an

same time. The change of ∆P leads to the change of both P1

artificial suit which can realize the main action of theand P2 in the opposite direction. In this way the joint angle canhuman arm. And most of the related applications configure

be regulated. Therefore the joint system can be considered as a

the Mckibben muscles in “irregular” structure. But how tosingle input (∆P) and single output (θ) system. At this rate, themodel and control it are the key problems. The achievablesystem is simplified and the control algorithm for it may beangles will be less than the human’s. They may be 70-90%

simpler.

of the angles which the human arm joints can realize.

But to the “irregular” structure robot, the structure
parameters are not “regular”. Different from the “regular”

References

structure joint system, there are more than two Mckibben[1] Kazuhiko Kawamura, R.Alan Peters, D.Mitchellmuscles to actuate one robot joint, such as shown in Fig.3(d)Wilkes, W.Anthony Alford, Tamara E.Rogers. ISAC:

and Fig.3(e). The lengths of them are different and they are not

Foundations in Human-Humanoid Interaction，IEEE

parallel to each other. And there is coupling between theIntelligent Systems，Jul.2000，p38-45.

neighbor robot joints. So how to model the robot system will be[2] http://www.shadow.org.uk, 2010.11

a great challenge.

[3] Masanori Sugisaka, Kouta Imamura, Kouji Tokuda,
Naoki Masuda, A new artificial life body: biologically

As to the application shown in Fig.2(g), the robot joint

inspired dynamic bipedal humanoid robots, Artificial

structure can be either “regular” or “irregular”.

life and robotics, Vol.8(1), 2004, p1-4

With the above application examples, the operation to the
Mckibben muscles is mainly based on the operation[4]

Hiroshi

Kobayashi,

Takamitsu

Aida,

Takuyaexperiences. Few literatures explain their modeling and theHashimoto, Muscle suit development and factory

related control algorithms.

application, Journal of Automation Technology,

Anyway, we are going to configure the Mckibben muscles in[5] R. Nakamura, H. Saito, Fundamental kinesiology, 5th“irregular” structure. And our project aims at the following:1.

Modeling of the artificial suit physically. The key isthe configuration of Mckibbe muscles;

2.
3.

edition, Tokyo, Japan: Ishiyaku Publishers, 2000
[6] Ching－Ping Chou, Study of human motion control

Designing the neural network based intelligent control

with a physiology based robotic arm and spinal level

algorithm;

neural controller:doctor dissertation, Washton Univ.,

Automatic sensing the behaving intention andMay.1996.

complete the action rapidly.
4.

Vol.3(6), 2009，p209-213

Realizing the main actions of the human arm.

Based on the literature and the above applications, the elbow
joint can bend larger than 100 degrees. The shoulder angle canget to about 90 degree, and so on. Comparing with human[5].
The robot arm joints’ rotation angles may be 70-90% of the©ISAROB 2011536

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Vehicle 3D Estimation Based on Time Series Images andPrior Knowledge
Haoyin Zhou, Tao Zhang, Changshui Zhang, Peng He
Department of Automation, Tsinghua University, Beijing 100084, China(Tel: +86-10-62797710, zhouhaoyin@gmail.com)

Abstract: Vehicle 3D estimation is important in intelligent transportation systems. To simplify system structure andimprove system accuracy, an algorithm based on one-camera system and prior knowledge is presented. The
experimental result shows that the algorithm can achieve satisfied accuracy.
Keywords: prior knowledge, 2D image, 3D estimation, shadow removal

I. INTRODUCTION
In intelligent transportation systems, it is usually to
use images captured by video cameras to identification
information of vehicles. The most usually method to getvehicle boundary box is to use 2 or more video cameras
for 3D estimation. This kind of system considers vehicles
as moving objects with no prior knowledge. Hence theyusually have low accuracy for 3D estimation. In themethod proposed in this paper, we want to take fully useof prior knowledge and estimate 3D information ofvehicles based on single video camera.
We get some helpful prior knowledge, including thephysical coordinates of 3 or more feature points in theimage, physical coordinates of the video camera, thedriving direction of vehicles, the range of length andwidth of a certain kind of vehicle, and the position of light.
The mapping between image coordinates andphysical coordinates can be acquired from the physicalcoordinates of feature points. So vehicle area that is got
by motion detection can be projected to physicalcoordinate. When the vehicle is moving along the street,the length direction of the vehicle can be confirmed.
Intuitively, the length of the vehicle can be got bymeasuring the move area along the driving direction.
However, because of the height and shadow, thevehicle area got by motion detection is not precise. To
remove the impact of height information, we can do themeasurement in different frames of the same video. In
this process, some height information can be got.
Shadow removal is a great problem in CV. If we get
the prior information of the position of the light and theaverage length of this type of vehicle, it is easy to know
the approximate location of the shadow. And a suitable©ISAROB 2011

537

location of the shadow segmentation can be found by
calculate variance of a slide window along the length
direction. It is the same to estimate the width of a vehicle
as the length estimation. In the end, the result of 3D
estimation is given.

II. METHOD
1. Motion Detection
Motion Detection is to detect the moving areas in each
video frame from the background. When a car is runningon the road and a static camera captured the entire scene
saved as a piece of video, what we do is to find themoving car area in every video frame. The called“moving area” means that there are one or more moving
objects in these labeled moving areas. Effective andefficient motion detection is the basement of tracking ofmoving objects [1].
Gaussian Background Model has been widely used forrobustly modeling complicated backgrounds, especiallythose with small repetitive movements (such as leaves,bushes, rotating fan, ocean waves, rain). According toGaussian Background Model, the distribution of eachpixel’s lightness of a background image meets Gaussian
distribution, which means for an image B [2]:I B ( x , y ) ~ N (u , σ ) =
2

1
2πσ

−

e

( x − μ )22σ

2(σ > 0)(1)

Thus, each pixel has two parameters which are2average u and variance σ . Meanwhile, thebackground image is changing slowly with time. The
parameters of each pixel in the background image should
be updated:u (t + 1, x , y ) = α ⋅ u (t , x, y ) + (1 − α ) ⋅ I t +1 ( x, y ) (2)The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Where u (t , x, y ) is point ( x, y ) ’s average in the timet , I t +1 ( x, y ) is the pixel of point ( x, y ) in t + 1 frame,2and α is the updating speed. The changing of σ is solittle to be ignored [3].
According to the model above, we use the first frame
without moving to build the background model andupdate the model with the latest input frame. The moving
areas can be got by the current frame subtracting
background model. The pixels with high difference
belong to moving areas. Then we filtrate the noise in theimage by morphologic processing and Gaussian filter. For
visual effect, we label background pixel black and object
pixel white, producing a binary image. The result showsthat the Gaussian background model achieves better
performance. Fig. 1(a) is the Gaussian background model.
Fig. 1(b) is some frame with a moving object. Fig. 1(c) is
got by Fig. 1(b) subtracting background model. Fig. 1(d)is the moving object detected after dilate and erode.

(a)f ( x 4 − x0)( x0 − x 4)( x − x0 −x 4 − x0) + ( y 4 − x0) + (h)2)

x 4 − x0) + ( y 4 − x0) + (h)22fhx 4 − x0) + ( y 4 − x0) + ( h)
2

2

2

2

(3)

)

)=0

Fig 2 Schematic diagram of the system geometry
The 3 straight line equations that separately get passthe 3 edge points of the view field and camera opticalcenter point are similar, take the line that get pass (x1, y1,0) and (x0, y0, z0) for example:⎧ x = ( x1 − x 0)t + x 0(b)⎪⎨ y = ( y1 − y 0)t + y 0⎪ z = ( z1 − z 0)t + z 0
⎩

(c)
(d)
Fig.1 (a) Gaussian background, (b) Frame with vehicle, (c)Subtract result, (d) Dilate & erode
2. Convert to physical coordinate
Image coordinate can be converted to physicalcoordinate with some necessary prior knowledge. The
system geometry is shown below. Here we take the leftbottom point as the origin, and direction of vehicle as yaxis. The z-axis is perpendicular to the ground.
At first, we measure the physical coordinate of thefollowing points: 2 edge points of the view field, which
are named (x1, y1, 0) and (x2, y2, 0); point which is in thecenter of the image (x4, y4, 0); camera optical center
points (x0, y0, z0).
If the focus length of the camera is f, the camera
imaging plane equation is:©ISAROB 20112f ( y 4 − y 0)+( y 0 − y 4)( y − y 0 −+(h)( z − h +2With the plane equation and 3 straight line equation,
the physical coordinate of the camera imaging plane edge
point (X1, Y1, Z1), (X2, Y2, Z2), (X3, Y3, Z3) are got.
If pixel size of the image is 780*560, the physicalcoordinate of (i, 0) and (0, j), which is image coordinate,in the camera imaging plane is:i
U1 =( X 2 − X 1) + X 1
780
V1 =

i

780

W1 =
U2 =
V2 =
W2 =

538

(4)

(Y 2 − Y 1) + Y 1

i

780
j
560
j

560
j

560

(5)

( Z 2 − Z 1) + Z 1( X 2 − X 1) + X 1(Y 2 − Y 1) + Y 1( Z 2 − Z 1) + Z 1(6)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Physical coordinate and image coordinate are both describing the position of the same point. When (i, j) = (i, 0)+ (0, j) in image coordinate, the physical coordinate of (i,j) is
U = U1 + U 2
V = V1 + V 2(7)

W = W1 + W 2
The equation of the straight line that get s
pass the camera optical center and (U, V, W) is:⎧ x = (U − x 0)t + x 0⎪⎨ y = (Y − y 0)t + y 0⎪ z = (W − z 0)t + z 0⎩(8)

Fig 3 Virtual grid

The intersection of this line and z=0 plane is:
− z0
+ x0
x = (U − x 0)
W − z0y = (v − y 0)− z0
W − z0

+ y0

(9)

z=0
Here we find the mapping between image pixels toreal world point. So the virtual grid can be drawn in images. Every grid is 10cm*10cm in physical world; thesides of grids are separately along the vehicle length andwidth direction.

Fig 4 Histogram calculate area

3. Shadow Cutting
We use slide window and histogram for shadow
cutting. At first, convert the image from RGB space to
HSV space, which is more suitable for dividing shadow
and vehicle body. Secondly, we create narrow and long
window along length direction. And calculate histogramof H component of HSV. Thirdly, compare the differenceof histogram of adjacent window, as shown in figure 4,
and find some local maximum line. The algorithm ofcomparison is as follows [4]:d ( H1 , H 2 ) =∑ ( H ( I ) ⋅ H ( I ))∑ (H (I ) ) ⋅ ∑ (H (I ) )I'I

H k (I ) =
'

1

'

'

1

2

2

'

I

2

Fig 5 Shadow cutting line
4. Get Real Size
After shadow cutting, we can get the 4 edge point of
the vehicle, from which the look length and look width
can be got.
The look length of the vehicle can be got in every
image, as shown in Fig.7:

2

(10)

Hk (I ) − 1
N⋅∑ H (J )jkWith time-series images, we can find local maximum
lines in every image. We can find some stable lines afterconverted to physical coordinate and fitting,
Here the prior knowledge of statistical average vehicle
width can be used, and choose the nearest line as shadow
cutting line, Fig. 8 shows the result of shadow cutting.

(a)Fig.6 (a)Edge points, (b)Edge points©ISAROB 2011

539

(b)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Mini-Car
Truck
Bus

19.00
35.00
24.50

20.19
35.49
25.53

-1.19
-0.49
-1.03

Table 3. 3D estimation result：hight
Vehicle typeCar
SUV
Mini-Car
Truck
Bus

Vehicle information extraction is being widely
studied. This paper comes up with an algorithm forvehicle 3D estimation with prior knowledge. With timeseries images, 3D information can be extracted from the
2d information in every image.
The shortcoming of the algorithm is that it cannot
accurate estimate vehicle information when it changes
moving direction.

REFERENCES

To get a least squares solution of the over determined
equations, we can calculate the solution of equation:
A Ax = A b(12)Because of Yi ≠ Yj (i ≠ j ) , the rank of matrix A is 2.
So we can get
T

(13)

T

−1

x = ( A A) A b
T

(14)

T

III. RESULT
We test our algorithm in 5 different kinds of different
vehicles, of which the result is shown in table 1, 2 and 3.
Table 1. 3D estimation result：length
Vehicle
type
Car
SUV
Mini-Car
Truck
Bus

Real Length

Estimate Length

48.00
46.00
36.00
60.00
72.00

50.31
45.35
34.47
58.23
69.78

Length
Error
-2.31
0.65
1.53
1.77
2.22

Table 2. 3D estimation result：width
Vehicle typeCar
SUV

©ISAROB 2011

Real Width
20.00
21.00

Estimate Width
22.03
21.40

Height Error
4.2
1.04
4.36
1.94
1.22

IV. CONCLUSION⎪⎪⎪ L + Y 2 h = L 2
(11)
H
⎨
⎪ Y3
⎪ L + h = L3
H⎪⎪⎩......
⎡1 Y 1/ H ⎤⎡ L1 ⎤⎢1 Y 2 / H ⎥⎢ L2⎥
L⎥ , X = ⎡ ⎤ ,b = ⎢ ⎥Define A = ⎢⎢h ⎥⎢1 Y 3 / H ⎥⎢ L3 ⎥
⎣ ⎦
⎢
⎥
⎢ ⎥
⎣...
⎦
⎣... ⎦

rank ( A A) = rank ( A) = 2
Here is the lease square solution:

Estimate Height
9.80
16.46
9.64
31.06
26.78

In table 1, 2 and 3, most estimate error is less than 2
cm, except the height of car and mini-car. That is because
the real top point does not influence the look length andlook width of the vehicle in images.

Fig 7 System geometry
With time-series images, a lot of look length can be
got by measurement⎧ L + Y 1 h = L1
⎪ H

TReal Height
14.00
17.50
14.00
33.00
28.00

Width Error
-2.03
-0.4

540

[1] A. J. Lipton, H. Fujiyoshi, and R. S. Patil, “Moving
target classification and tracking from real-time video,” inProc. IEEE Workshop Applications of Computer Vision,
1998, pp. 8–14.
[2] C. Stauffer andW. Grimson, “Adaptive backgroundmixture models for real-time tracking,” in Proc. IEEE
Conf. Computer Vision and Pattern Recognition, vol. 2,
1999, pp. 246–252.
[3] H. Z. Sun, T. Feng, and T. N. Tan, “Robust extractionof moving objects from image sequences,” in Proc. Asian
Conf. Computer Vision, Taiwan, R.O.C., 2000, pp. 961–
964.
[4] M. Kiger, “A shadow handler in a video-based realtime traffic monitoring system,” in Proc. IEEE Workshop
Applications of Computer Vision, CA, 1992, pp.11-18.

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Automatic Drawing of Correct Topographical Distribution of EEG
Rhythms Based on Unified Suitable Reference Selection
Bei Wang1, Xingyu Wang1, Akio Ikeda2, Takashi Nagamine3, Hiroshi Shibasaki4, Masatoshi Nakamura5
1

Key Laboratory of Advanced Control and Optimization for Chemical Processes, College of Information Science andTechnology, East China University of Science and Technology, Shanghai 200237, China
2
Department of Neurology, Kyoto University, Kyoto 606-8501, Japan3
Department of System Neuroscience, Sapporo Medical University, Hokkaido 060-8556, Japan
4
Takeda General Hospital, Kyoto 601-1495, Japan
5
Research Institute of Systems Control Institute for Advanced Research and Education,
Saga University, Saga 840-0047, Japan(Tel: 86-21-64253581; Fax: 86-21-64253386)(beiwang@ecust.edu.cn)

Abstract: Electroencephalography (EEG) interpretation is important for brain diseases inspection. In this study, an
automatic technique was developed to realize the automatic drawing of correct topographical distribution of EEGrhythms, which would be an assistant tool for EEG interpretation. Unified suitable reference electrode was selected
automatically to construct the common referential derivation. Topographies were drawn according to the amplitudesof EEG rhythms calculated among the scalp of head. The final result of topographical distribution was helpful tohighlight the EEG rhythms of interest for automatic EEG interpretation. The developed technique has application
significance for real clinics.
Keywords: Topographical distribution, EEG rhythm, Reference selection, Automatic EEG interpretation

purpose was to obtain the correct interpretation result
for clinical practice. The reference was selected basedon an iterative method, to construct the common
referential derivation. The obtained EEG waveforms
and periodograms under common referential derivation
were analyzed. The amplitudes of EEG rhythms were
calculated among the scalp of head based on thecommon referential derivation. Finally, the obtained
topographical distributions of EEG rhythms were
evaluated comparing with the visual inspection.

I. INTRODUCTION
The electroencephalogram (EEG) waveforms are
generally described by kinds of rhythms according to
the frequency, amplitude and shape. The interpretationon the distribution of EEG rhythms had clinicalsignificance for brain diseases inspection, when thenormal properties of EEG rhythms were becoming
abnormal.
An automatic EEG interpretation system had been
applied for real clinics. The relative EEG potential
between the recorded electrode position and thereference were analyzed to judge the grade of normality
or abnormality of awake EEG records [1] [2]. The EEG
waveform under different reference potentials, which
was called derivations, could bring different results.
When the reference was contaminated by artifacts, thederived EEG waveform would be unsuitable for
automatic interpretation. Therefore, the selection ofreference was important to obtain correct EEGinterpretation result.
In this study, an automatic technique was developed
to realize the automatic drawing of correct
topographical distribution of EEG rhythms based on
unified suitable reference electrode. The ultimate©ISAROB 2011

II. METHOD
1. Data acquisition and visual inspection
The EEG data of one patient suffered by brain
disease was analyzed. The data was recorded at Kyoto
University, Japan. According to the International 10-20
System [3], totally 19 channels of EEG waveforms wererecorded including Fp1, F3, C3, P3, O1, Fp2, F4, C4,
P4, O2, F7, T3, T5, F8, T4, T6, FZ, CZ and PZ which
covered the scalp of head. Another 2 channels recorded
at left and right ear-lobes of A1 and A2. The recording
was done with the time constant of 0.3 s, the high cut
filter of 120 Hz and a sensitivity of 0.5 cm/50μV. The
sampling rate was 200 Hz for all the channels. The long541

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

EEG record was divided into consecutive segments of
5- second long each for analysis.
The recorded data was inspected by a qualified
clinician, especially the EEG rhythms distribution. The
EEG waveform of test subject was moderately abnormal.
There were ‘continuous irregular slow wave more on
the right posteriorly and can be a localized slow wave inthe right midtemporal (T4) to central (C4) region’.

B. Distribution of amplitude of EEG rhythm
Fast Fourier Transformation (FFT) was taken for the
EEG waveform under the constructed referential
derivation. According to the sampling rate, the data
length of each 5-second segment was 1000 points. By
taking 1024-point FFT, the frequency resolution ofpeirodogram was 0.2 Hz. The amplitude of EEG rhythm
was calculated by(1)
A(f)  4 S(f) ,where S was the amount of power within the frequency
band of EEG rhythm.
C. EEG rhythm separation
The obtained amplitude value for totally 16 channels
indicated the topographical distribution of EEG rhythm
among the scalp of head. A band pass filter was utilized
to extract the EEG rhythm component from the EEG
waveform to evaluate the consistency between theautomatic result and visual inspection.

2. Automatic reference selection
An iterative method had been developed to find out
the unified suitable reference for EEG interpretation.
The flowchart of automatic reference selection was
illustrated in Fig. 1.

III. RESULTS
1. Unified suitable reference selection
The EEG data of one 5-second segment was
analyzed. Based on the original EEG data from EEG
recorder, bipolar derivation was constructed. The crossspectrum of slow wave 0.5-8 Hz was analyzed for bothhorizontal and vertical directions of bipolar EEG.
The result of cross-spectrum analysis for phasereversal detection was illustrated in Fig. 2. The left side
was cross-spectrum, coherence and phase for horizontal
bipolar EEG. The right side was the cross-spectrum,Fig.1. Flowchart of automatic reference selection
There were four main steps. Firstly, the crossspectrum of bipolar EEG was adopted to detect thefocus of EEG rhythm as step A. Under the bipolar
derivation, the reference was the neighbor electrode.
The cross-spectrum showed the relation of two bipolar
EEGs. Parameters of coherence and phase were
calculated to analyze phase reversal. The detected phasereversal area corresponded to the focus of EEG rhythm.
In step B, C and D, the distribution of amplitude of EEGrhythm was analyzed to detect the distributed area
among the scalp of head. Finally, a unified suitablereference was selected excluding all the distributed
electrodes. The detail algorithm was described in [4].

8
[µV]
6

4

2

2
0

2

4

6

8 [Hz]

Coherence (horizontal)

1

0

0.8

0.6

0.6

0.4

0.4

0.2

0.2
0

2

4

6

0

8 [Hz]

0

Cross spectrum (vertical)

2

0

2

Phase (horizontal)

6

4

6

8 [Hz]

100

0

0

-100

-100
2

4

6

8 [Hz]

0

2

4

6

Fig.2. Cross-spectrums of bipolar EEG

542

8 [Hz]

Phase (vertical)

100

0

4

Coherence (vertical)

1

0.8

0

©ISAROB 2011

8
[µV]
6

4

0

3. Topographical distribution of EEG rhythms
A. Referential derivation construction
The 16 channels of EEG waveforms, Fp1, F3, C3,
P3, O1, Fp2, F4, C4, P4, O2, F7, T3, T5, F8, T4, and T6
covering left and right hemisphere, were obtained bytaking the relative potential of each electrode with theselected unified reference.

Cross spectrum (horizontal)

8 [Hz]

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

100μV

50

1s

50

40

Fp1-Ave
F3-Ave

Fp1

30

20

20

10
0

C3-Ave50

P3-AveO1-AveFp2-Ave

O2-AveF7-Ave

0

0

10

20

50

40

F3

50

40

F4

40

30

30

20

20

20

20

10

10

10

0

10

20

0

0

10

20

50

40

P4-Ave20

30

50

C4-Ave1030

0F4-Ave

F7

Fp2

10
0

50

40

40

30

T3

0

10
0

10

20

50

40

C3

0

40

C4

30

30

20

20

20

20

10

10

10

T3-Ave50

T5-Ave

40

10

20

0

0

10

20

50
T5

0

P3

10

20

0

40

P4

30

30

F8-Ave

20

20

20

20

T4-Ave10

10

10

T6-Ave

10

20

0

10

20

50

Fig.3. EEG waveforms under referential derivation

0

O1

20

T6

10
0

10

20

40

30

30

20

20

10
0

10

0

0

10

20

50

40

coherence and phase for vertical bipolar EEG. For thetest subject, T4 was detected automatically as the focus.
By using the iterative method, F8, C4, P4, O2 and T6
were detected as the distributed area.
Finally, the automatically selected reference was theaverage potential of all electrodes excluding T4, F8, C4,
P4, O2 and T6. The EEG waveform under the new
constructed referential derivation was shown in Fig. 3.

O2

10
0

10

20

0

0

10

20

Fig.4. Periodograms.

F729

2. Periodograms analysis
The periodogram of the EEG waveform shown in
Fig. 2 was obtained by taking 1024-point FFT. The
periodograms were illustrated in Fig. 4. For eachchannel, the power of frequency band from 0.5 to 25 Hz
was displayed. The frequency band from 0.5 to 25 Hz
covered the main activities of EEG rhythms. The slow
wave from 0.5 to 8 Hz was marked by grey color. The
scale of each peirodogram was the same and the amountof slow wave can be observed among the scalp of head.
The empty circles indicated the automatically detectedpeaks within the slow wave frequency band. The
detected peaks would be utilized for EEG rhythm
separation.

T3
21
T5
40

FP1
32

FP2
33

F3
31

F4
38

C3
25

C4
54

P3
39

P4
68

O1
35

F8
54
T4
71
T6
82

O2
54

Fig.5. Topographical distribution of EEG rhythm ofslow wave among the 16 EEG channels.
observed at the dotted area of focus T4 and distributed
F8, C4, P4, O2 and T6 in Fig. 5.
Furthermore, the detected peaks in periodograms
showed that the distributed slow wave frequency was
lower than 4 Hz. A band pass filter of FFT-IFFT was
adopted to separate the frequency activity from 0.5 Hz
to 4 Hz out of the EEG waveforms. The separated
results were illustrated in Fig. 6. It can be observed that

3. Topographical distribution of EEG rhythms
According to the peridogram, the amplitude of slow
wave from 0.5 to 8Hz was calculated for each channel.
The amplitude values were shown as the topographicaldistribution of EEG rhythm of slow wave. Comparing
with other channels, the lager amplitude values can be©ISAROB 2011

0

0

40

30

0

T4

50

30

0

20

10
0

50

40

10

40

30

0

0

50

30

0

F8

543

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

T4, F8, C4, P4, O2 and T6 had obvious slow wave. The
separated results were consistent with the visualinspection.

100μV
1s

Fp1-Ave
F3-AveC3-AveP3-AveIV. DISCUSSION

O1-Ave1. Unified suitable reference selection
The visual inspection on EEG interpretation is a
critical skill for neurologists. Automatic EEGinterpretation had been developed which can bringsubjective inspection result as an assistant tool forclinical practice. The reference selection problem wasimportant for both visual inspection and automaticinterpretation. When the reference was unproper orcontaminated by artifacts, the derived EEG waveform
would affect the interpretation result. In this study, weinvestigated on the reference selection for automaticinterpretation. Focus was detected by cross-spectrum of
bipolar EEG and distributed area was detected byconsidering the distribution of EEG rhythm among thescalp. The obtained unified suitable reference canhighlight the EEG rhythm of interest and bring correctinterpretation result.

Fp2-Ave
F4-AveC4-AveP4-AveO2-AveF7-AveT3-AveT5-AveF8-AveT4-AveT6-AveFig.6. Separated EEG rhythm from 0.5 to 4 Hz.

ACKNOWLEDGMENTS
The authors are grateful to Dr. Takenao Sugi,
Department of Electrical and Electronic Engineering,
Saga University, for the technique help on EEG data.
This study is supported by Nation Nature Science
Foundation of China 60674089, Shanghai Leading
Academic Discipline Project B504, and Fundamental
Research Funds for the Central Universities
WH0914028.

2. Topographical distribution of EEG rhythms
According to the selected unified suitable reference,
an automatic drawing method for topographicaldistribution of EEG rhythm was developed. The
amplitude of EEG rhythm was calculated for eachchannel. Band pass filter was adopted to separate EEG
rhythm according to the peaks in periodogram. The
topographical distribution can be observed based on theamplitude values. The separated waveform can reflect
the frequency property of the distributed EEG rhythm.
The obtained result highlighted the EEG rhythm ofinterest and was fit to the visual inspection. For the test
subject, the peaks in the periodogram were rather closed.
According to the brain diseases, there may be several
peak groups in other case. Further proper separation of
EEG rhythms would be developed as the future works.

REFERENCES[1] Nakamura M, Shibasaki H, Imajoh K, Nishida S,
Neshige R, Ikeda A(1992), Automatic EEGinterpretation: a new computer-assisted system for the
automatic integrative interpretation of awake
background EEG. Electroenceph. Clin. Neurophysiol.
82: 423-431[2] Nakamura M, Sugi S, Ikeda A, Kakigi R, Shibasaki
H (1996) Clinical application of automatic integrativeinterpretation of awake background EEG: quantitativeinterpretation, report making, and detection of artifacts
and reduced vigilance level. Electroenceph. Clin.
Neurophysiol. 98: 103-112[3] Jasper HH (1958), Ten-twenty electrode system of
the international federation. Electroenceph Clin
Neurophysiol. 10:371-375[4] Wang B, Wang X, Ikeda A, Nagamine T, Shibasaki
H, Sugi T, Nakamura M (2010), Automatic Reference
Selectionfor
Quantitative
EEG
ComponentInterpretation: Cross Spectrum Analysis Based on
Bipolar EEG, Proceedings of International Conferenceon Artificial Intelligence and Computational
Intelligence, 79-86, Oct. 23-24 2010, Sanya, China

V. CONCLUSION
The automatic drawing of topographical distributionof EEG rhythm was investigated. The main method wasbased on a unified reference selection technique. The
obtained automatic result was consistent with visualinspection. The developed technique had clinicalapplication for EEG interpretation.

©ISAROB 2011544

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Research on Surface Crack Detection based on Laser Scanning and Image
Processing Techniques
Guangming Cai1 Jiwu Wang1 Mingcheng E1 Wenliang Guo1
Department of Mechanical and Electrical Engineering, Beijing Jiaotong University, Beijing, China, 100044 1
Sugisaka Masanori2
Nippon Bunri University, Oita, Japan2(jwwang@bjtu.edu.cn)
Abstract: The traditional Surface crack detection method based on artificial vision, due to low detection efficiency and highlabor strength etc., can hardly fit the demand of the actual testing operation. In order to meet this demand, in this paper we
propose a new method based on laser scanning and image processing techniques for surface crack detection. The analysis ofexperimental results has shown the feasibility, accuracy and effectiveness of this method, and detection results were quitesatisfied.
Keywords: crack; detection; laser scanning; image processingI.

laser scanning and image processing techniques for

INTRODUCTION

surface crack detection. The main steps of this method are
as follows. Firstly, a red laser emitted a laser beam which

For engineering structures, products, parts, etc., theformulated a straight laser line in the detected surface.

crack resulting from various reasons will affect their

Secondly, we used a CCD camera to capture target images

regular service, durability and security. In many cases, thefrom the detected surface. Thirdly, we applied the image

occurrence of crack is inevitable, and the propagation ofprocessing techniques to judge whether the red laser line

crack is usually ongoing. In order to guarantee the safety

in the detection surface is continuous, to identify whetherof life and property, the crack detection is very necessary

cracks are exist in the images. And the results of[1] [2].

experimental have shown that this method can meet the

The commonly used methods for surface crack

demand of surface crack detection.

detection include manual detection, mechanical detection,
optical fiber detection, laser detection, machine vision

II. THE OVIEW

detection and so on. Among them, the manual detection
has many shortcomings, such as low detection efficiency,

The core idea of this paper is to use machine visionhigh labor strength, poor quality and low detection
factors.

instead of artificial vision for automatic detection of

Mechanical detection is contact detection, and its

surface crack. Machine vision for crack detection depends

detection efficiency is low. Both optical fiber detection

on image processing technology [4]. Considering both theand laser detection are high precision methods, but theirdisadvantages of image processing technologies and thecost is also too high. Although machine vision detection

advantages of laser technology, we use a combination of

method based on image processing techniques is objective,

them to detect surface crack.

reliability

dependedonmany

subjective

The sketch map of the detection method is as

non-contact, high accuracy and convenient, it also can notfollows.

meet the demand. That is because images contain many
interferences except crack, and there are no common andappropriate methods to eliminate these effects [3].
In this paper we propose a new method based on©ISAROB 2011545

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig.3The case with crack and interference

Therefore, we can do the following analysis.

Fig. 1 The sketch map of the detection method

As shown in Fig.1, the 5 is a red laser. It emitted a

As shown in Fig.2, a red laser beam was emitted to

red laser beam and formulated a straight red laser line in

the detected surface and most of it will be reflected back.

the detected surface (1). And, we used a CCD camera (4)And in the captured image, we can see a continuous red

to capture the image from the captured region (2). Then,laser line. As shown in Fig.3, there is a case when surface

we saw the CCD camera (4) and the red laser (5) as acontains crack and interference. In the crack region, thewhole thing, and let them move together along with the

red-laser beam will through the crack and not be reflected

direction of movement. Therefore, we can capture theback basically. In the interference region, there were still

entire image information of the detected surface. In thesome of the red laser will be reflected back. Then in theend, we applied the image processing techniques to judge

captured image, we can see a red laser line which iswhether the red laser line in the detection surface isdiscontinuous and uneven thickness.

continuous, to identify whether crack is exist in the2.

images.

Experiment
To verify the above analysis, we made the following

experimental test.

III. THE BASIC PRINCIPLE AND

Red laser line

EXPERIMENTAL TEST

Adhesive
Material

Cement1.

StainsThe basic principle of the crack detection method
In addition to crack, the interferences, such as stains,adhesive materials, texture etc., were also contained in thedetected surface. Before identifying crack from theinterferences, we need to know their differences. And theobvious difference is as follows. When there is a crack in(a)the surface, the material in the crack region is no longer(b)close and a gap will appear in the surface. On the contrary,
the interferences usually do not have this feature [6].

TextureInk stains

(c)

(d)

Fig.4. Testing images: containing inferences. (a) Cement stains, (b)Fig.2 The case without crack and interferences

Adhesive materials, (c) Texture, (d) Ink stains.

©ISAROB 2011

546

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Red laser line

Crack

Fig.5. Testing image: containing crack

For the testing images that contain inferences, which

Fig.7. Flow chart of crack detection method.

is shown in Fig.4, the red laser line is continuous. As

A.

Original Image: Use a CCD camera to capture

B.

Gray Scale: Make the color image into theshown in Fig.5, the red laser line disconnected obviously

images.

in the crack region. According to the above results, we cansee that the method for surface crack detection is feasible.

gray-scale image, so is suitable for image processing.
C.

Analysis Region: According to the least square
method, we can get the analysis region (the red laser

IV. THE IMAGE PROCESSING SYSTEM

line region).
D.

In this paper, the surface crack detection method is

Threshold: Apply Dual Threshold Method. And therequired threshold could be got by calculating thebased on image processing technology. And the image

histogram.

processing system is a core part of the crack detectionE.

method. Therefore, image processing system is directly

Judge the continuity: According to the distribution ofgray, we can select the analysis region. Then, in therelated to the accuracy of the results of image processing.

selected region, we can judge the continuity by1.

analysis the ratio of the target gray value.

Hardware programF.

As shown in Fig.6, we designed a hardware program

Crack Measurement: Combined with the analysis
results of multiple images, we can measure the crack.

for surface crack detection [5].
G.

Data Out: Save data to Access database.

V. THE PROBLEMSFor this crack detection method, we found thefollowing problems by the experimental tests:
A.

Cannotidentify

themicro-cracks:

For

the

micro-crack (its width less than 0.2mm), theintermittent feature of the red laser line in the crack
region is unobvious, which is shown in Fig.8.
However, by increasing the intensity of laser and theFig.6 the hardware program

magnification of the camera, we can solve this2.

Flow chart of crack detection

problem.

The image processing flow chart of crack detection
is shown as follows [7] [8].

©ISAROB 2011

547

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

VI. CONCLUSION
In this paper, we propose a new method based onlaser scanning and image processing techniques forsurface crack detection. For relying solely on image
processing techniques, the crack detection method wehave proposed can make up its shortage. The role of laser
scanning is equivalent to the function of image filtering. It
can effectively filter the interferences except crack form

Fig.8. Testing image: containing a micro-crack.

B.

the image. Through experimental tests, we verified the

The impact of a large contrast ratio: When the color

feasibility of this method. However, any method has itsof interferences is much difference from that ofshortcomings, this method also exist shortcomings. For

background, this method can not rule out the impact

example, it is difficult to detect the micro-crack (its widthof interferences, which is shown in Fig.9.

less than 0.2mm); this method has improved the control
requirements of movement and detection angle for camera
and red laser. The purpose of this paper is to explore newideas for workers who engaged in the work of crack
detection.

REFERENCE[1] Ikhlas Abdel-Qader, Osama Abudayyeh, M.ASCE,
Michael E. Kelly. “Analysis of Edge-Detection
Techniques for Crack Identification in Bridges” [J].
Journal of Computing in Civil Engineering, pp.
255-256, October 2003.
[2] Y. Fujita, Y. Mitani, Y. Hamamoto, “A Method for
Crack Detection on a Concrete Structure”, ICPR ’06,
Hong Kong, pp.901-904, August 2006.
[3] Sun Zhaoyun, Chu Yanli, Fan Yao, Dang Le,“Pavement Crack Image Processing System Researchbased on VC++,” Computer Application and
Software, vol. 26, no. 8, pp. 82-85, August 2009.
[4] Je-Keun Oh, An-Yong Lee, Se Min Oh, Youngjin Choi,
Byung-Ju Yi and Hai Won Yang. “Design and Controlof Bridge Inspection Robot System” [J]. IEEE
Computer Society: 3634-3635.
[5] Liu Xiaorui, Xie Xiongyao “Rapid Crack Inspection of
Tunnel Surface Based on Image Processing,” Chinese
Journal of Underground Space and Engineering, vol.
5, no. 2, pp. 1624-1628, December2009.
[6] Fu Hua, Chi Jihui, “Research on Non-destructive
Testing of Mold Micro Crack Based on Laser
Ultrasonic Surface Wave,” Optoelectronic Technology,
vol. 30, no. 1, pp. 29-32, March 2010.
[7] Jeong Ho Lee, Jong Min Lee, Hyung Jin Kim andYoung Shik Moon. “Machine Vision System for
Automatic Inspection of Bridges” [J]. 2008 Congress
on Image and Signal Processing: 363-366.
[8] Zou Yiqun, Hou Guicang, Yang Feng. “A Surface
Crack Inspection Algorithm based on Digital Image
Processing Techniques” [J]. Control & Automation,
pp. 98-99, April 2004.

Fig.9. Testing image: the color of interferences is much difference fromthat of background.

C.

The impact of shooting angle of the camera: Where
the shooting angle of the camera is not perpendicular
to the detection surface, this method can not rule out
the impact of interferences, which is shown in
Fig.10.

(a)(b)
Fig.10. Testing images: containing ink stains. From left to right:
the angle between the shooting direction of camera and detection。

。

surface is (a)90 , (b)45 .

©ISAROB 2011

548

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Intelligent Speech Recognition Filtering
Young Im Cho
Dept. of Computer Science, Univ. of Suwon, San2-1, Wau-ri, Bongdam-eup, Hwaseong, Gyeonggi-do(Tel : 82-31-229-8214; Fax : 82-31-229-8281)(ycho@suwon.ac.kr)
Abstract: In Emergency situation, speech recognition speed is very important. Therefore, in this paper, we propose a fast
filtering algorithm. Firstly, FIR filter selectively passes through the frequency range of speech, and secondly, the Wiener
filter filters out the extraneous noises. Because of that, the processing time is reduced.

Keywords:

FIR filter, Speech recognition, Combination filtering algorithm

Depending on the presence of feedback processes, thedigital filter is divided into IIR(Infinite Impulse
Response) and FIR(Finite Impulse Response) filters.
The latter is known to be less error-prone. For noise
elimination in the subsequent processes, the Wiener andKalman filters[3] are widely used. In emergency
situations that require accurate interpretation of a rather
brief voice data, the Wiener filter is usually preferred.
The general model-based Wiener filtering process canbe expressed as follows:

I. INTRODUCTION
One of the key factors in the speech recognition isnoise[1]. The real situation is quite different from thecontrolled environment of the speech laboratory.
However, the surrounding noises are a particularly
difficult problem in the real speech recognition. The
difference in the controlled environment and the realenvironment in speech recognition comes into play inthree distinct processes: signal process, feature spaceprocess, and model process. Of these three processes,
the difference is most evident in the signal process [2].
Here, the noise in the speech data after the signal
process is filtered by a novel digital filtering system. A
FIR filter is first used to separate the speech region andthe noise region, and then a Wiener filter is used to
improve the overall speech recognition.

Where, s(t ) is the speech to be recognized, s(t ) is
the speech data containing noise, n(t ) is the noise, andg (t ) is the Wiener filter.

II. Combined Filtering

In Eq. (1), s(t ) is being sought. In it, an estimate ofn(t ) is derived from s(t ) , and then the approximate^s(t )  g (t ) * ( s(t )  n(t ))

(1)

^

^

^

value of

Generally, the speech recognition system isconfigured by six stages. In stage 1, voice data are
inputted by converting the audio signals into theelectrical digital signals. In stage 2, the voice signals are
separated from the surrounding noises. In stage 3, usefultraits in speech recognition are extracted by using aspeech recognition model. In stage 4, a standard speech
pattern database is formed by speech recognition
training. In stage 5, new voice data are compared to thestandard speech pattern database, and the closest matchis searched. In the final stage of 6, the matched result is
put to use through the user interface.
In the preprocessing (noise elimination) stage of 2,
analog audio signals from a CCTV or a sensor are
digitized and then fed to the digital filter. The digital
filter, which is widely used and proven, selects the
passband and filter out the stopband.

obtained by using n(t ) . In order to^achieve a better approximation of s(t ) , the GMM as
expressed below in Eq.(2) is used. It expresses
mathematically the general characteristic of speech data.
K

P( s )  p(k ) N (s; 
k



;

k)

(2)

Based on Eq. (2), the model-based Wiener filter isdesigned per following steps: In the inputted current
frame, the noise region is determined by a statisticallybased VAD. In the noise region found, the noise modelis renewed to the previous value. In the preprocess-WF
block, a temporally noise-free clean speech is estimatedusing the decision-directed Wiener filter. Using theestimated values from the previous step, the Gaussian
post probabilities of the GMM are calculated. In the
final WF using the MMSE method, the probabilities areused to estimate the noise-free clean speech. The
estimated noise-free speech and the noise model areused to design the final Wiener filter. The current frame-----------------------------------This paper is supported by Gyeonggi-do Regional
Research Center in Korea (suwon GRRC 2009-B3)©ISAROB 2011s (t ) is549

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011is processed using the Wiener filter designed, and thenoise-free clean speech is obtained. Then the above five
steps are repeated for the next frame.
For emergency detection, we propose a fast
recognition filtering method. The basic concept is toselectively use the audio signal being transmitted fromthe CCTV's. That is, from the transmitted signal, only
the audio energy spectrum that is relevant to the speech
is to be selected, digitized, and saved for further
analysis. A high-performance FIR Wiener filer can beused to digitally filter out the unwanted portion of theaudio signal, prior to actual speech recognition.
As human speech generally falls within 300-3400khz,
the FIR filter [4,5] can separate the incoming audio data
into passband(the speech region), stopband, andthreshold-band. This will greatly reduce the time andimprove the performance of a speech recognition
system.
The basic mathematical concept of the FIR filter canbe expressed as follows:.

effectively separating the speech signal and the noise is
implemented.
Afterwards, the noise signals are extracted bysubtracting the output speech data from the incoming
speech data. Then the extracted noise data and theincoming speech data are used in Eq. (1) to design an
improved noise filter.
In terms of mathematical expression, the general
Wiener filter consists of multiplications andsummations of current and past data and filtering
coefficients. Thus, it can be designed using the device
transfer functions and the mathematical expressions.
Within the scope of this research, the physical states,
such as operational stability and sensitivity and the safe
transmission of data, are assumed to be steady, and themain priority is placed on minimizing the number ofdevices and increasing the speed of the filter operation.
Finally, the noise elimination Wiener filter isexpressed as follows:
So(w)  H (w)S (w)

(6)

N 1

y[ x]   h[k ]x[n  k ]
k 0

(3)

In Eq. (6), s(w) is the noise-containing speech signal,s0 ( w)is the noise-free speech signal, and H (w) isestimation function of the Wiener filter. An effective

In Eq.(3), is the speech information input, is theoutput speech information after filtering, is the finite
impulse response characteristic, and N is the filtering
step number. As the input information and coefficientsare multiplied and summed, the time required for noise
filtering is quite long if Eq. (3) is implemented as is.
However, the multiplication steps in Eq. (3) can beeliminated if a bit-serial algorithm [6] is applied. The
result is expressed in Eq. (4) below:
N 1y[ x] way to determine H (w) is a major focus of this research.
Accordingly, a mathematical expression for H (w) is
proposed as follows:H ( w)  k 0hj[k ]·2 j ) x[n  k ]j 0speech signal, and Pd (w) is the audio spectrum of thenoise signal. An error is introduced in estimating theaudio spectrum of the original speech signal during thefiltering process. To reduce the error, a coefficient isintroduced as below:(4)Here, h i , N , and M represent the coefficient h 's
j th bit, tab number, and coefficient bit number,respectively. The bit-serial algorithm multiplies
multiplicand to the multiplier while shifting LSB to
MSB and then adds the result to the previous sum. To
reduce the total multiplication cycles, the odd and evenpart of the Eq. (4) can be separated and the result can be
written as follows:
N 1

M
1
2

k 0

j 0

y[ x]  

 (h [k ]·2
2j

2j



Ps ( w)
H ( w)  
 Ps ( w)    Pd ( w) 

(8)Here, parameters α, β and squaring the averages ofthe signals are used to reduce the error.
The Wiener filtering processes the noise-containing
speech information effectively, but it takes time, so thatspeech recognition is delayed. To minimize the timedelay, the concept expressed in Eq. (8) is applied during
the statistically-based VAD process[7] of stage. The
resulting process model is expressed in Eq. (9) below.
In this model, the speech data and the noise data areconsidered to be asymmetric. By applying asymmetricwindow to these two data in designing the Wiener filter, h 2 j[k ]·2 2 j 1 ) x[n  k ](5)

Eq. (4) requires a total of multiplication cycles, while
Eq. (5) requires a total of multiplication cycles, a factorof 2 increase in the speed.
In this way, utilizing the benefits of the FIR filter, aWiener filtering that minimizes the noise error by©ISAROB 2011

(7)

Here, Ps (w) is the audio spectrum of the original

M 1

(

Ps ( w)
Ps ( w)  Pd ( w)

550

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

the time required for noise filtering can be significantly
reduced.

 2n ,0  n  n0
0.54  0.46 cos  p1 
H ( n)    2 n  n0  , n0  n  Ncos 
p2
 

IV. Results
To test the speech recognition system developed thus
far in this work, speaking word database developed by
SITEC is used. The database is recorded in 16kHz/16
bit, and contained the voices of 500 individuals. For
comparison to the database, voice data from a
microphone or CCTV in 16kHz/16 bit format areused[9].
As discussed in the introduction, it would be unwise
to use all the collected voice information in speech
recognition, as it will consume too much time and may
result in inaccurate analysis. By first extracting theaudio frequency region useful for speech recognition byusing the FIR filter proposed, the overall processing
time can be greatly reduced.
The noise that escapes the initial filtering will then be
eliminated by the Wiener filter. By comparing two
filtering, it is found that the FIR Wiener filter visibly
eliminates the background noise. Also, the effect can be
audibly felt by listening to the before and after sounds.
By comparing the noise-free speeches processed
through the MATLAB-constructed Wiener filter, to theexisting database of key words, a very accurate speech
recognition effect was realized.
By filtering out the unnecessary portions of the
speech information, such as non-audible frequency
regions, environmental noise, and transmission noisefrom the CCTV's, the level of speech recognition isfound to be greatly increased. Also, the word models arefound to be very useful in the success of speech
recognition and in the reduction of the processing time.
That is, the word model that considers the relationships
between the words being searched had much more
successes.
Using the database complied with noise-free sounddata, two-pass bigram and bigram+trigram searches
under ECHOS were found to indicate that the wordcorrelated model was much superior in terms of therecognition success rate and the search time than thesimple model that does not consider the relationships
between words.
As a result, in case of using model for each words,
the speech recognition rate is much faster than withoutmodel. In model case, it is about 88.9% in bigram and
90.0% in bigram and trigram combination model ofspeech recognition rate(%) respectively. However, in
without model case, it is about 77.2% in bigram and
80.1% in bigram and trigram combination model ofspeech recognition rate(%) respectively. Also, therecognition time(sec/sentence) is much faster than
without model. It is about 21.0% in bigram and 22.1%
in bigram and trigram combination model of speech
recognition rate(%) respectively. However, in withoutmodel case, it is about 5.4% in bigram and 6.3% inbigram and trigram combination model of speech
recognition rate(%) respectively.

(9)

In Eq. (9), P1 , P2 , respectively, represents the periodof the left and right portions of the asymmetric windowfunction, n0 is the location of the maximum value, andN is the total length of the window function.
Based on the noise-free speech signal obtained thus
far, a speech recognition database is complied and usedin analysis of emergency situations. In this effort, thephonemic recognition of individual words is initiallychosen as the key element of speech recognition, andthe database is complied accordingly.

Fig.1. The overview of proposed speech recognition
systemFig. 1 illustrates the overview of the speech
recognition system founded on the concepts describedin Fig. 1. MATLAB[8] is used for the proposed FIR
Wiener filter, and HTK and ECHOS are used for thesubsequent processes. The finished speech recognition
system basically uses sound models to search key words,
and the flat lexicon and lexical tree are used in thisword-based speech recognition system. The lexical tree
is efficient in the usage of the memory, but is somewhat
slow in applying the probability values of the language
models and is also somewhat complex in implementing
word models. Thus, a duplicate tree algorithm is used.
That is, a parallel structure is used in the lexical tree forsingle-phoneme words.
The speech recognition result obtained by these
serious of processes is then sent to the user interface ofthe system.

©ISAROB 2011551

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

It is interesting to note that in HMM the success rate
for the standard bigram (left to right) search method islower by 8% than the trigram search method thatsearches in the reverse direction and also considers therelationships between different phonemes. Nonetheless,the search time was longer for the latter method.
Lastly, a significant processing time reduction and afast situation response were realized by selectively
processing the audible voice region of the audio signals
transmitted from the CCTV.

frequency region is not yet completely understood,
further research in this front is currently underway.

REFERENCES[1] H. Kruegle, “ CCTV Surveillance ” , Analog andDigital Video Practices and Technology, Elsevier,pp.227-239, 2007.
[2] C.-H.Lee, "On Stochastic Feature and Model
Compensation Approaches to Robust Speech
Recognition", Speech Communication, pp.29-47.
1998.
[3] J.K. Kim, "Min/Max Estimation and Base
Estimation for Kalman Filter", Natural Science
Research (Korean), vol. 5, pp.21-30, 1995.
[4] T.K. Ryu, K.H, Park, D.S. Hong, C.O. Kang,"Channel Estimation by Sero-Forcing Method intheFrequency
Region,"Kor.
J.
Telecommunications, vol.31, no.1, pp.38-47, 2006.
[5] Y.S. Park, J.H. Jang, "Echo Filtering by Soft
DecisionintheFrequency
Region",Telecommunications Review (Korean), vol.19, no.5,
pp.837-844, 2009.
[6] Robert E.Morley, Jr. Gray E. Christensen, Thomas
J. Sullivan, Orly Kamin, " The Design of a BitSerial Coprocessor to Perform Mulitiplication andDivison on a Massively Parallel Architecture", in
Proc IEEE, The 2nd Syposium on the Frontiers ofMassively Parallel Computation, Farifax, U.S.A,
pp.419-422, 1998
[7] J.H. Jang, D.K. Kim, N.S. Kim, "A New Statistical
Method for Speech Recognition Systems",
Telecommunications Review (Korean), vol.15, no.1,
pp. 201-209, 2005.
[8] K.S. Kim, "MATLAB Signal and Image
Processing", Ajin Publishing, Korea, pp.213-250,
2007.
[9] Y.I. Cho and S.S. Jang, "Intelligent Speech
Recognition System for CCTV Surveillance", Kor.
J. Intelligent Systems, vol.19, no.3, pp.415-420,
2009.

Fig.2 Simulation Result ‘Hello’

V. Conclusion
Unlike the controlled environment where a speech
recognition system can easily filter out the extraneous
noises, it is rather difficult in the real environment
where a sensor, such as a CCTV, collects abundantnoises from various human, mechanical, and natural
sources. The success of speech recognition in the realenvironment thus depends critically on how well thesenoises are filtered. Just as important, the processing
time for noise filtering needs to be reduced, as time is
the most critical element in emergency situations. Thus,
effective noise filtering combined with fast processing
time is considered to be the essence of speech
recognition. Towards these goals, an improved speech
recognition system is proposed in this work. The system
has the FIR and Wiener filters as the key elements andeffectively filters out the extraneous noises andproduces clean noise-free speech data in a reasonabletime.
One of the problems cited during the work is that theextraneous noise that is present in the audible band of
300-3400khz can still pose some problems even withthe proposed FIR filter. As the noise filtering in this©ISAROB 2011

552

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Multi Robotic System and the Development of Cooperative
Navigation Behaviors for Humanitarian Demining
Maki K. Habib
The American University in Cairo
New Cairo, Egypt
maki@ieee.org

Abstract: Multiple robotic systems can accomplish tasks that no single robot achieve, since ultimately a single robot, nomatter how capable, is spatially and physically limited. However, achieving cooperative robotics is quite challenging.
Many issues must be addressed in order to develop a working cooperative team, such as action selection, coherence,
conflict resolution, resources management, coordination, cooperation and communication. In this paper Pemex-BE
robot is used to represent the individual robot that makes up a team for multi robotic system dedicated for humanitarian
demining. The multi Pemex-BE robots for mine clearance represents an attempt to reduce the gap between the researchlevel and the actual needs on the ground. The technical features and navigation system with obstacle avoidance alongwith the scenario of multi robotic system is presented.
Keywords: Robotics, Multi robotic systems, Navigation, Sensors, Antipersonnel mines, Humanitarian demining
psychology), and life sciences (theoretical biology,
animal ethology).
Achieving cooperative robotics, however, is quite
challenging. Many issues must be addressed in order todevelop a working cooperative team, such as action
selection, coherence, conflict resolution, resourcesmanagement, coordination, and communication.
Furthermore, these cooperative teams often work in
dynamic and unpredictable environments, requiring therobot team members to respond robustly, reliably, andadaptively to: unexpected environmental changes,
failures in the inter-robot communication, modificationsin the robot team that may occur due to mechanicalfailure, learning of new skills, the addition or removalof robots from the team by human intervention, or fullrobot failure etc.
Small, lightweight, and inexpensive robots tend tohave better mobility but it might be unavoidably slow
[5-7, 9, 10]. Smaller in size and light in weight alsomeans reducing certain capabilities. With the use of alarge number of such robots, the good mobility cancompensate for the low speed while it is necessary todevelop efficient group behaviors to compensate for the
reduced capabilities. When designing the multi robotic
systems for demining, it is important to decide the typeof movement strategy the robots adopt when scanningthe minefield, the standard set of behaviors that allindividual robot should have and the set of specialized
behaviors that are assigned to specific individual robots
and the way robots are going to communicate
information.
Random collective behaviors with improved
algorithms have been proposed to look for mines onbeaches [8]. This technique cannot fulfill humanitarian
demining requirements, as there is a need to assure thatevery square inch of the terrain is explored reliably and

I. INTRODUCTION
Research on using population of robots forachieving a given task efficiently has mostly been
inspired by animal behaviors. There has been increased
research interest in systems composed of multiple
autonomous mobile robots exhibiting cooperative
behavior. The study of multiple-robot systems naturally
extends research on single-robot systems. Multiplerobot systems can accomplish tasks that no single robot
can accomplish, since ultimately a single robot, nomatter how capable, is spatially limited. Achieving
cooperative robotics is desirable for a number ofreasons [1-4].
1. Many robotic applications are inherently distributedin space, time, or functionality, thus requiring a
distributed solution. In addition, tasks may be
inherently too complex or impossible for a single
robot to accomplish it,
2. It is quite possible that many applications could be
solved much more quickly if the mission could be
divided across a number of robots operating inparallel by duplicating capabilities across membersof the robot team.
3. Building and using several simple robots can beeasier, cheaper, flexible, and has the potential ofincreasing the robustness and reliability of theautomated solution through redundancy. It would bemuch cheaper and more practical in many
applications to build a number of less capable robots
that can work together at a mission, rather thantrying to build one robot which can perform theentire mission with adequate reliability, and
4. The constructive and synthetic approach inherent incooperative mobile robotics can possibly yieldinsights into fundamental problems in the social
sciences (organization theory, economics, cognitive©ISAROB 2011

553

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011as safely and as fast in a minimum amount of time andcost

detection sensors, mine sensor, and radio
communication. Some of the robots might have extra
sensing and physical capabilities depending on theassigned task requirements.

II. MULTI ROBOTIC SYSTEM SCENARIO
FOR HUMANITARIAN DEMINING
The outline of multi robotic system scenario forhumanitarian demining using Pemex-BE robot is statedas follows:a. The higher level of control is represented by amobile monitoring station. The monitoring station
receives the task description either from humanoperator through a multimodal graphical interface, orby using a topographical map, or an aerial picturewith precise coordinates. The operator divides thearea of the assigned minefield into sectors andallocates a robot for each sector. Then, themonitoring station informs the task and the starting
global /reference location of each sector to therelevant robots through radio module. Periodical
polling tracks task execution, clarifications, andreported difficulties that might be raised by any of
the robots or by the searching mission. The
monitoring station helps to resolve any problems by
individual robots that are in deadlock situations or
robots that require additional resources. In this casethe monitoring station and human operator can
instruct and extend help to the relevant robot directly
to fulfill its needs in finding safe path, resolve thedeadlock, emergency help and guide, etc. If this is
not possible, the monitoring station reports that tohuman operator. Human operator interacts with themonitoring station to instruct a specialized robot for
emergency to help the robot in question and resolve
the deadlock, or give charging service, etc.
b. Individual robots. Each robot (see Figure 1)
initializes itself and performs a self-check to emulate
his readiness to achieve a set of possible tasks andinforms the monitoring station about its availability
and readiness to execute these tasks. Each robot in
the multi robot team receives the assigned task fromthat monitoring station and reports back on its
activity and performance update: scanning for mines,
marking mines, and communicating with otherrobots. In addition, each robot tries to get out of adetected deadlock situation by means of its own
available knowledge and resources before reportingit to the monitoring station. Mines are marked andpossibly exploded using a small charge placed by the
robot and triggered after the robot has backed at asafe distance. Each robot has its behavior navigation
system with set of standard behaviors available forall robots, and set of specialized behaviors that arededicated for special tasks and needs (See Figure 2).
The principle requirement is that each of thedemining robot in the team should operate in aremote control mode or, at least, semi-autonomously.
All robots are assumed to have the following
capabilities: position encoders, GPS, obstacle©ISAROB 2011

Fig.1. Pemex-BE in different environments

Fig.2. Pemex-BE in different environments

III. IMPLEMENTATION OF THE ADOPTED
MULTI ROBOTICS SYSTEM SCANRIO
Pemex-BE robot was used to represent theindividual robot that makes up the team for multirobotics system. The mobile monitoring station isconsidered as a car of suitable size, which can adjust its
location as needed to support the demining process andits requirements. In this case, the mobile monitoring
station represents the higher level of control. Within thisarchitecture, human-operator stations inside the carinteracting with the monitoring station with thepossibility to communicate with other land-based suboperators through radio. The operator describes the task
to the monitoring station. The monitoring station

554

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

interacts with the assigned operator and takes care oftask distribution and allocation for individual robots
according to the size of the minefield and the available
number of robots. The station supervises the execution
and control of tasks, and communicates with robots as
individuals, groups or as a whole to follow-up task
execution, mapping, and problem solving updates. The
skill required by human-operator should be kept tominimum. Figure 3 shows an example for dividing aminefield into three sectors, the mobile monitoring
station and some high level behaviors. Efficient
distributed robotics architecture must allow robots to beefficiently added to the system or removed from it.
Once, a Pemex-BE robot starts to operate; it is in astandby mode while reporting to the monitoring station
its readiness in performing a set of tasks according to
the associated physical functional capabilities.

robot and the monitoring station from the other side.
Each robot maintains its own version of the map basedon its searching activity and shares it with themonitoring station. For the purpose of this work, theoperator at the monitoring station is responsible decide
whether the minefield is completely cleared or there is
still remaining issues or regions within the assigned
sectors that need to further follow-up (see Figure 4).
Hence, the operator must have a comprehensive view ofthe demining process performed by the participating
robots that constitute the demining team. Marking a
detected mine within an assigned land on a digital map
enables immediate and easy interactive interpretation byhuman operator or the monitoring station. In addition, it
helps to decide whether further exploration for that landis needed or not.
During the actual operation of mine searching, anassigned working area for each individual robot can be
redefined depending on the circumstances facing therobot and the searching progress, e.g., help can berequested when one robot is discovering more mines ormore obstacles within its own assigned area and needssome help from other robots that already finishedsearching their sectors. Coordination of movements isrequired if one robot is stuck into a hole or at a corner orlost its detection or mechanism capabilities; in suchsituation, another robot (either specialized emergency
robot for such mission or any of the other individual
robots) should pull it back and such a decision is madein consultation through the monitoring station andpossibly through human operator too.
The main functional capabilities of the monitoring
station can be summarized as:
1. It receives a demining task description throughhuman-friendly graphical user interface using, forexample aerial pictures, topological maps, etc.
2. It divides the located mined area into sectors andallocates a robot to each sector. Human has thecapability to interfere and do or adjust this job
directly.
3. It communicates with the available robots by radiotransmission through periodical polling or per
demand as needed. This aims to announce a task,
track task execution, solve deadlock, manage
collaborative action, mapping searched area, replanning, etc.
4. Tracks all marked mines.
5. It maintains a global and updated digital map of theminefield.
6. It confirms and announces the completion of ademining task.
7. Adding new robots to the team or removing
damaged members.

Fig.3. Terrain to be cleared with three robots

Fig.4. Terrain scanned after certain time by the work of
three robots
Each of the dedicated robots for mine searchreceives the global reference location of an assigned
sector along with its width and depth. A digital map of
the minefield and the searched area within each of theassigned sectors have to be maintained and continuously
updated with the support of a seamless interactive
interface between the human operator and themonitoring station from one side and between each

©ISAROB 2011

IV. COMMUNICATION MANAGEMENT
In order to enable parallel behaviors and efficient
task achievement, follow up progress and update, and to555

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

extend support to any robot in need, a communication
management module at each robot and at the mobile
monitoring station is considered necessary under thework of this paper for message and informationexchanges. Group communication among all robots(inter robots communication) including the monitoring
station requires protocols that can operate without
central control and handle dynamic topology changes
due to the mobility of the robots and the station itself.
Multicast is the most important group communication
primitive and it is critical in applications where close
collaboration of teams is needed. A radio based
communication module has been developed and its
communication framework consists of communication
protocol to establish physical communication link andmessage protocol to exchange messages corresponding
to necessary information. The main functions of themessage protocol are negotiation (task assignment,cooperation, etc.), inquiry (look for information, etc.),offer (information, help, etc.), announce andsynchronize, deadlock solve, etc.
It is important to remember that even the mostsophisticated radio systems are subject to interference,both loss of bandwidth due to RF noise and competition
for bandwidth. In addition, when dealing with tasks inreal-time, much information has a severely limited
useful lifetime which influences how reliability ofcommunication must be addressed. The communication
management will be helpful to add new robots to thesystem and removing damaged ones. When a robot isunreachable through communication by others forvarying periods of time, it is considered temporarily to
be damaged and the system requires dynamically to dochange in the makeup of a robot team.

Information Systems Analysis and Synthesis(ISAS ’98), vol. 3: 89-94.
[2] Uny Cao Y; Fukunaga A S; Kahng A B (1997).
Cooperative Mobile Robotics: Antecedents andDirections, Autonomous Robots, 1997 Kluwer
Academic Vol. 4: 1–23.
[3] Arkin R C; Balch T (1998). Cooperative multiagent
robotic systems. In David Kortenkamp, R. Peter
Bonasso, and Robin Murphy (Edrs), AAAI Press.
[4] Mataric M J (1990). Navigating with a rat brain: A
neurobiologically-inspired model for robot spatial
representation. In Proceedings of the FirstInternational Conference on Simulation of Adaptive
Behavior: From Animals to Animats (SAB’1990).
MIT Press: 169 – 175.
[5] Gage D W (1995). Many-robot MCM Search
Systems,
Autonomous Vehicles
in MineCountermeasures Symposium, Monterey, CA: 9.56–
9.64.
[6] Loh J; Hengm J; Seet G; Sim S K (1998).
“Behavior-based search using small autonomous
mobile robot vehicles, Second International
Conference on Knowledge-Based Intelligent
Electronic Systems, Adelaide, Australia, Vol. 3:
294-301.
[7] Roumeliotis S I,; Pirjanian P; Mataric M J (2000).
Ant-inspired navigation in unknown environments,
Autonomous Agents 2000, Barcelona, Spain: 25-26.
[8] Greiner, H.;
Shectman, A.;
Chik-yung
Won; Elsley, R.; Beith, P.; (1996). Autonomous
Legged Underwater Vehicles for Near Land
Warfare,Autonomous
Vehiclesin
Mine
Countermeasures Symposium,(AUV’96), Monterey:
41 – 48.
[9] Habib M K (2002). Mine Clearance Techniques andTechnologies for Effective Humanitarian Demining,
International Journal of Mine Action, Vol.6, No.1,
2002.
[10] Habib M K (2001). Mine Detection and Sensing
Technologies: New Development Potentials in theContext of Humanitarian Demining”, 2001 IEEE
International Conference of Industrial Electronics,
Control and Instrumentation “IECON’2001”: 16121621.

V. CONCLUSIONS
The multi Pemex-BE robots for mine clearance
structure presented in this paper as an attempt to reduce
the gap between the research level and the actual needson the ground. This requires proper understanding of theexact problems at the minefields sites and concludes a
design of a low cost, flexible, and light weight mobile
robot while considering local resources and constraints.
There is still a need for further analysis to formulate
the efficient navigation system for Pemex and its
motion. A better sensory combination and fusion
techniques are still needed. Better mobility is of highdemand. Further analysis is required to develop
dynamic
behaviors
supporting
coordination,communication and cooperation capabilities.

REFERENCES[1] Parker E (1998). Distributed Control of Multi-Robot
Teams: Cooperative Baton Passing Task,
Proceedings of the 4th International Conference on©ISAROB 2011

556

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Development of Flexible Surgical Manipulator
for Natural Orifice Transluminal Endoscopic Surgery
Jungwook Suh, Hoseok Song, Kiyoung Kim, and Jungju Lee*
*Dept. of Mechanical Engineering, KAIST, 335 Gwahangno, Yuseong-gu, Daejeon, Republic of Korea(Tel : 82-42-350-3033; Fax : 82-42-350-3210)(leejungju@kaist.ac.kr)

Abstract: Natural Orifice Transluminal Endoscopic Surgery is an advanced and experimental surgical techniqueperformed via natural orifices (mouth, anus, vagina, urethra, etc.). Therefore, unlike other surgery methods such as
laparoscopic surgery or single port access surgery, NOTES can avoid remaining external incisions or scars. Surgicalmanipulator for NOTES should be flexible in order to be inserted into channels of overtube. This paper presents adevelopment of flexible robotic manipulator for NOTES, including design and kinematic analysis. Developed thinmanipulator has 4 DOF motion and only 5.0 mm in diameter. And it uses multi-revolute joints that have gradual curvein order to enlarge the workspace and minimize the diameter of manipulator.
Keywords: NOTES (Natural Orifice Transluminal Endoscopic Surgery), MIS, surgical robot

surgery remains no visible scar after a while, external
incision on the patient’s skin is still needed. On theother hand, NOTES has significantly different
procedure to laparoscopic surgery. The biggest
advantage of NOTES is that it makes no incision on
patient’s skin. Instead, it is performed via naturalorifices such as mouth, anus, vagina or urethra, andinternal incision on the wall of organ is necessary toapproach the surgical site. Patient barely feels pain on
the internal incision and can leave the hospital muchearlier. Besides, lower anesthesia requirements andavoidance of the potential complications are also big
merits of NOTES [1]. Therefore, NOTES can beconsidered an ideal abdominal surgery from theviewpoint of MIS. However there is an obstacle to carryout this brand-new technique that is absence of proper
instruments for NOTES. Unlike SPA surgery, NOTES
cannot utilize traditional laparoscopic instruments,
because flexible surgical tools should be inserted via
bent human orifices [2]. Therefore various surgicalinstruments for NOTES should be developed so thatNOTES can be one of general surgical operations for
abdominal surgery. Furthermore, robotic system for
NOTES is expected to change the status of NOTES inabdominal surgery as in the case of laparoscopic surgery.

I. INTRODUCTION
Laparoscopic surgery, a kind of MIS which stands
for Minimally Invasive Surgery, means a surgical
technique performed in the abdomen through somesmall incisions. These days it is common surgicalmethod in the case of simple abdominal surgery such as
cholecystectomy because of advantages like short
hospital stay and small scars. However it usually takes
longer time to operate laparoscopic surgery than
traditional open surgery. Development of laparoscopic
instruments helped to make the procedure short andsimple, and finally the introduction of medical robot
caused a shift of power in the medical surgery. Da
Vinci®, developed by Intuitive Surgical Inc., is therepresentative medical robot system for the laparoscopicor thoracic surgery. This kind of surgical robot canoperate surgery much more dexterously than traditional
manual surgery.
The smaller the incisions for surgery are, the bigger
the merits of MIS are. Traditional open cholecystectomyneeded an incision bigger than 20 cm, while recent
laparoscopic operation makes three to five small
incisions of only about 10mm. Furthermore, surgeons
are trying to minimize the incision for surgery as ever.
Therefore they invented two kinds of new experimental
surgical methods for abdominal surgery in recent years.
One is SPA(Single Port Access) surgery, and the other is
NOTES(Natural Orifice Transluminal Endoscopic
Surgery). The SPA surgery is performed using single
entry port, typically the patient’s navel. Though SPA©ISAROB 2011

II. DESIGN OF ROBOT ARM
More than two surgical instruments are needed toperform surgical procedures, and they should beexchangeable. Therefore, for the sake of NOTES, two

557

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Further, flexibility and force are important
requirements for robot arm for NOTES. Thin andflexible robot looks nearly impossible to apply heavyradial force on the tip. Therefore, flexible shaft
composed of lots of short rigid links are used in order toapply axial and torsional forces effectively. 2 DOF
active joint also can be composed of similar links. There
are several types of small sized joint mechanism thatcan be applied to NOTES instruments [4]. Multispherical joints design is frequently applied for many
researches because of the advantage that it’s relatively
easy to product small sized joints [5][6]. However thiskind of manipulators has a severe problem that they areso weak to external forces that the assembled joints areeasily bent. In order to prevent the S-shaped bending by
applied external forces, French CNRS fixed lots ofactuating wires on each links and actuated them withdifferent speed using pulleys [6]. 5 mm sized Endowrist,®
laparoscopic instrument for da Vinci system, hassimilar but advanced mechanism using universal joint at
proximal driving part. It helps the system minimize thesize of driving part. Notched nylon rod or superelastic
NiTi tube can be used for bending joint similarly, andthey are good to serve a working channel for gradualbending [7]. Nevertheless, they cannot beat mutispherical joints because of elasticity and small
applicable forces.
Although there is no commercial robotic NOTES
system now, lots of researchers are trying to develop
NOTES system. Purdue University develops a NOTES
robot based on the laparoscopic surgery robot named
Laprotek, while Columbia Univ. and Intuitive Surgical,
Inc. are developing SPA surgery robot [7][8]. These
three systems have elbow-out function in common,
because the scope and two arms should be located insmall sized overtube head. Therefore the distance
between two robot arms is so narrow that it’s difficult tosecure enough workspace and field of view without
elbow-out joints. They are essential for NOTES robot,
but wire-driven toggles are sufficient for them. As
shown in Fig.4, surgical robot arm consists of tooltip,
long flexible shaft and driving part. Three DC motors in
the driving part drives the 2 DOF bending joints and theforceps using stainless steel wire ropes. And Fig.5
shows the actual built robot arm combined with theovertube. The overtube serves three working channels
for a camera and two arms.

instruments should be inserted along an overtube withan endoscope as shown in Fig.1. At this moment, there
is no commercial NOTES robot system, but some
NOTES prototypes, such as EndoSAMURAI (Olympus
Corp., Tokyo, Japan) and DDES (Boston Scientific,
Natick, MA), are considerable for development [3].

Fig.1. Overview of NOTES robot
The target surgery of this research is transgastriccholecystectomy. The diameter of overtube is limited to
25 mm due to the size of patient’s esophagus. Of course,
it’s obvious that small diameter is good for smoothinsertion. In this research, the diameters of overtube androbot arms are determined to 20 mm and 5 mm each.
Different from the conventional flexible endoscopic
instruments, 2 additional DOF (degrees of freedom)
joint can make surgical procedures easy. Thus eachrobot arm has 4 DOF including translation and axial
rotation except grip motion.

Fig.2. Flexible shaft composed of short links

Fig.3. 2 DOF multi-spherical joints driven by 8 wires©ISAROB 2011

558

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig.4. Designed NOTES robot arm

Fig.6. Kinematics of 4 DOF manipulator

d3 =

2 LT

q3

sin

q3
2

(1)

LT is the length of bending section when unfolded,and θ4 is the angle between x2 axis and the projectedline of bending section on the x2 y2 plane. As a result,the bending section has 2 DOF motion and it can be
expressed as a transformation matrix with two variables,θ3 and θ4, as follows.

Fig.5. Built robot arm combined with the overtube

III. KINEMATIC ANALYSIS
2

2'

2''

2'''

2''''

4

2'

2''

2'''

2''''

4

é
2
2
ê c3 c 4 + s 4 (c3 - 1) c 4 s 4êê(c - 1) c s
c3 s 4 2 + c 4 2
4 4
=ê 3
ê
ê
- s3 c 4- s3 s 4êêë00

s3 c 4s3 s 4c3
0

LT (1 - c3 ) c 4 ùúq3úLT (1 - c3 ) s 4 úú (2)q3
LT s3 ú
ú
q3 ú
1 úû

By the benefit of assumption on the bending section,
the overall kinematics also can be expressed with quite
simple transformation matrix. (eq.4) and (eq.5) indicate
the rotation matrix and the position vector in (eq.3).

Link
αi-1
ai-1
di
θi
1
0
0
0
θ1
2
0
a1
d2
0
2’
0
0
0
θ4-90˚
2’’
-θ3/2
0
0
0
2’’’
0
0
d3
0
2’’’’
-θ3/2
0
0
0
4
0
0
0
90˚-θ4500d40Table.1. Denavit-Hartenberg parameters©ISAROB 20112

T »T T T T T

2 DOF bending section is composed of 4 unit
revolute joints. If the number of segmental revolutejoints is large enough, this kind of 2 DOF joint can beconsidered as a long flexible shaft even though it
consists of rigid links. Furthermore, it makes a circular
arc shaped bending under the assumption of zerogravity and no friction. This bending section is difficult
to express using traditional D-H (Denavit-Hartenberg)
parameters. It causes some duplicated symbols like θ3and θ4 as shown in Table.1. There are four variables (θ1,d2, θ3, and θ4) because this manipulator has 4 DOF. And
here d3 is a function of θ3 as follows.

é 0
0
1
2
4
q
q
q
»
=
(
,
d
,
,
)
1
2
3
4
T5
T 1 T 2 T 4T 5 ê R0 5
ëP ùú (3)éc14 c3c4 + s14 s4 c14 c3 s4 - s14 c4
ê
R 5 = ê s14c3c4 - c14 s4 s14c3 s4 + c14c4
êë- s3c4- s3 s4

c14 s3 ùs14 s3 úú (4)
c3 úû

0

0

559

0

5

1 û

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011éùLTê c14 ((1 - c3 ) + s3 d 4 ) ú
q3
ê
ú
LT
0
ê
ú
P 5 = ê s14 ((1 - c3 ) q3 + s3d4 ) úêú
s3 LT úêd 2 + c3 d 4 +
ê
q3 úû
ë

grip. It’s designed to apply high axial and torsional
forces and to have small radius curvature for smoothinsertion into bent channel. Even though the result of
the repeating experiment was not so good, it showed thepossibility of NOTES. In the near future, NOTES is
expected to be established as one of the standard
procedures for abdominal surgery.

(5)

In spite of the rough assumption, this result does notshow a significant error. Besides, the bending amount ismeaningless in the NOTES arm because of long andflexible shaft, while the bending direction is quite
accurate.

This work was supported by the Korea Science andEngineering Foundation (KOSEF) grant funded by the
Korea government (MEST) (No. 2009-0063570)

IV. EXPERIMENTREFERENCES

A simple experiment was performed to confirm thelinearity between commanded angle and actual
measured angle in the bending section. Angles wereobserved for several cycles of simple bending motion inorder to check the repeatability. This experiment was
performed with 8 segmented bending section and
400mm long flexible stem in S-shaped experimental
working channel whose size is only 70 mm in radius ofcurvature. The two values of commanded and actual
angles would be same on the idealized assumption.
Unfortunately, however, the plot shows severe
hysteresis. It’s impossible to eliminate this phenomenon
completely. Nevertheless, it can be improved byminimizing friction and tolerance and slope correction
technique [7].

[1] T. H. Baron, “Natural Orifice Transluminal
Endoscopic Surgery”, British Journal of Surgery, Vol.
94, No.1, pp. 1 – 2, 2007
[2] L. L. Swanstrom, M. Whiteford, and Y. Khajanchee,“Developing Essential Tools to Enable Transgastric
Surgery”, Surgical Endoscopy, Vol. 22, pp. 600 - 604,
2008.
[3] Sohail N Shaikh and Christopher C Thompson,“Natural orifice translumenal surgery: Flexible platform
review”, World Journal of Gastrointestinal Surgery,Vol.2, No. 6, pp. 210 – 216, 2010
[4] A. Faraz and S. Payandeh, “Synthesis andWorkspace Study of Endoscopic Extenders with
Flexible Stem”, Journal of Mechanical Design, Vol. 119,
pp. 412 – 414, September, 1997
[5] Peter Berkelman and Ji Ma, “A Compact Modular
Teleoperated Robotic System for Laparoscopic
Surgery”, The International Journal of Robotics
Research, Vol. 28, No. 9, pp. 1198 – 1215, September,
2009[6] Frederick Van Meer, Alain Giraud, Daniel Esteve,and Xavier Dollat, “A Disposable Plastic Compact
Wrist for Smart Minimally Invasive Surgical Tools”,
Proceedings of the 2005 IEEE/RSJ International
Conference on Intelligent Robots and Systems, Alberta,
pp. 919–924, 2005.
[7] Daniel J. Abbott, Chris Becke, Richard I. Rothstein,and William J. Peine, “Design of an Endoluminal Notes
Robotic System”, Proceedings of the 2007 IEEE/RSJ
International Conference on Intelligent Robots andSystems, pp. 410 - 416, San Diego, CA, USA, Oct 29 –Nov 2, 2007.
[8] Jienan Ding, Kai Xu, Roger Goldman, Peter Allen,
Dennis Fowler, and Nabil Simaan, “Design, Simulationand Evaluation of Kinematic Alternatives for Insertable
Robotic Effectors Platforms in Single Port Access
Surgery”, Proceedings of
The 2010 IEEE
International conference of Robotics and Automation,
pp. 1053 – 1058, Anchorage, Alaska, USA, May 3-8,
2010.

ACKNOWLEDGEMENT

60

Actual Angle (deg)

40

20

0

-20

-40

-60
-200

-150

-100

-50
0
50
Commanded Angle (deg)

100

150

200

Fig.7. Hysteresis loop for repeated bending motion

V. CONCLUSION
Long, thin and flexible surgical robot suitable for
NOTES procedure was developed. Although thediameter is only 5 mm, it serves 4 DOF motion except©ISAROB 2011560

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Swing-up and LQR stabilization of rotary inverted pendulum
Minho Park, Yeoun-Jae Kim and Ju-Jang Lee
Department of Electrical Engineering
Korea Advanced Institute of Science and Technology (KAIST)
Daejeon, Republic of Korea(Tel : 82-42-350-5432; Fax : 82-42-350-5432)(cyrano@kaist.ac.kr)

Abstract: In this paper, we considered swing-up and LQR stabilization of rotary inverted pendulum. A DC motor
rotates a rigid arm. At the end of the rigid arm, joint is attached and a pendulum is suspended. Two encoders check thedegree of the rigid arm and pendulum every 0.5ms. This paper proposes a modified bang-bang control which swings up
a pendulum fast and safe. In order to solve the stabilization problem, this paper used linear quadratic regulator. Whenthe user gives a large disturbance to the pendulum and when the pendulum loses its position, the pendulum quickly
recovers to upright position. Experimental results showed that the proposed bang-bang controller and LQR controller
can stabilize a rotary inverted pendulum system within 3.0s for any starting point. The system also showed robustness
from large disturbance.
Keywords: Inverted pendulum, Swing-up, Linear Quadratic Regulator

II. ROTARY INVERTED PENDULUM

I. INTRODUCTION
Inverted pendulum has been a great test bed for
decades to deal with the control problems. Inverted
pendulum has nonlinear characteristics and are notdifficult to analyze systems near the equilibrium point.
For these reasons, a lot of researchers used this tool toverify their ideas.
Furuta proposed rotary inverted pendulum which
has a direct-drive motor as its actuator source and apendulum attached to the rotating shaft of the motor [1].
Yoshida proposed an energy-based swing-up control
and Å ström showed that swinging up a pendulum byenergy control is very effective and if acceleration of thepivot is sufficiently large, the pendulum can reach
upright position in one swing [2], [3].
In recent years, a lot of control theories have been
applied to control inverted pendulum such as fuzzycontrol [4], adaptive PID control [5], iterative impulsive
control [6], neural network control, sliding mode control
and other various control methods.
Most papers concentrated on how to swing-up the
pendulum and how to stabilize it to upright position. In
this paper, we will also deal with situations after thependulum was stabilized. The user applied large
disturbance to a stabilized pendulum which pushed thependulum to an unstable position. Once the pendulum
lost its position, the controller returns it back to stable
position quickly. We adjusted the parameters of LQR
controller by simulations and experiments.
This paper is organized as follows. In section II,
design and kinematics of rotary inverted pendulum by[1] will be introduced. Section III, IV discusses swingup strategy and LQR controller. Controllers are
switched properly by control laws. Experimental results
are shown section V. Conclusions are in the last section.

©ISAROB 2011

Fig. 1. Sketch of the rotary inverted pendulum
The above figure shows a sketch of the rotary
inverted pendulum [1]. To use the Euler-Lagrange
dynamic equation,d LL( )  ( )  T,
dt 


d L
L
( )( )  0
dt 


(1)

we need to get the kinetic and potential energy.
Assuming that mass of the rotating arm is M and themass of pendulum is m. The kinetic energy of thissystem is

K net  K arm  K pen
1
1I  Mr 2 2
2
6m L
K pen {(r ) 2  (l ) 2  2r l cos  }dl
2 L 01
1
1
 m(r ) 2  L2 m 2  Lmr cos 2
62

K arm 561

(2)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

the motor in the opposite direction when  is extremely
high. Overall, this controller is same as the bang-bang
controller when  is in the normal range.
If you change the working range of swing-up
controller,  should be reconsidered.

A rotating arm’s potential energy is zero, so the net
potential energy of system is
L(3)
Pnet  Ppen  mg cos 
2
Putting (2) and (3) to Euler-Lagrange equation (1),
we can get two equations of motion.

IV. LQR STABILIZATION

L  K net  Pnet

When the swing-up controller brings the pendulum
to the range between -30 degrees and 30 degrees, the
LQR controller will operate. We can linearize (4) near
the equilibrium point (   0 ). For convenience, we set
the auxiliary variables as shown in the equations below.

11
T  ( Mr 2  mr 2 )  mrL( cos    sin  ) (4)
3
2
1
1
1
0  mL2  mrL cos   mgL sin 
3
2
2

1
1
1
1
a1  Mr 2  mr 2 , a2  mrL, a3  mL2 , a4  mgL
3
2
3
2
(6)
a a a T
a a a T  2 2 4 3 ,   1 24 2a2  a1a3
a2  a1a3

The table below shows a specification of SRV02
with ROTPEN.
Table 1. Specification of inverted pendulum
Parameter Definition
Value
Units
r
Rotating arm length
20cm
LPendulum length
35cmmPendulum mass
0.128
kg
M
Rotating arm mass
0.278
kg

By (6), we make the state-space representation:  x1 ,   x2 ,   x3 ,   x4
0
 x1   x  0
 2   x3  0  
 x4 0III. SWING-UP STRATEGY
Swing-up strategy is giving 90% of maximum
torque to the opposite direction of pendulum speed
when the pendulum heads near the ground. It is a very
simple strategy and it works if motor’s torque is enough.
However, if a pendulum being pushed by a user lostits position, it could revolve very fast. If a pendulum’s
speed is high, then swing-up control laws gives
acceleration to the pendulum. When acceleration andspeed are increased, LQR controller couldn’t hold it
anymore. This can makes the pendulum rotates all thetime and it may cause malfunction of a motor.
To prevent this problem, we need to modify thecontrol laws. We introduce the weighting factor
 , which decides the working range of swing-up
controller. Because encoders give  and  every
0.005sec, we can get pendulum’s speed easily. 
decreases when the pendulum’s speed is high and viceversa. A motor operates if  located between
210    150 degrees, and  changes according
to the speed of the pendulum. Output torque is 90% of
maximum torque multiplied by . By trial and error,
if  is faster than 900(deg/ sec) , the pendulum
revolves. Therefore we determined  as follows.

 1
  3

900

for 600   (deg/ sec)

00aa
0  2 1 4a2  a1a3

0
0




 x
a
 

0  1   2 3
 a2  a1a3  (7) x2
T 
1   x3  0  x4  
a2
0
 2


 a2  a1a3 After digitizing (7) with 2 kHz, the system changesx(k  1)  x(k )  u(k )(8)The system (7) is unstable but controllable.
Therefore by using linear quadratic regulator, we canmake it stable. LQR minimize the object function

1 n 1 T [ x (k )Q1 x(k )  uT (k )Q2u(k )]2 k 01
 xT ( N )Q0 x( N )
2

J

(9)

and its control law is

u (k )   K  x(k )K   (Q2  T S )1 S 

(10)

where
(5)

S  T [ S  S R 1T S ]  Q1

(11)
R  Q2  T S 
We want to settle  and  as fast as possible. Q2
is just 1 and Q1 is

Modified control law slowly shuts down the motor
when the pendulum’s speed is too high. It also operates©ISAROB 2011

0

0
a2 a4
2
a2  a1a3

tofor   600(deg/ sec)



1

562

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 201110
0
Q1  
000 0 0
1 0 0 0 50 0 
0 0 1

moved more than 100 degrees but the controller wasable to recover their position quickly. This shows thatboth controllers work correctly.

(12)

2. Large disturbanceThis time, the user will apply a large disturbancewhich can make the pendulum fall down to the bottom.
We will show that the controller is still able to move itback to upright position.

which emphasize  and  . After calculate (10) and(11), we can finally get K  .

V. EXPERIMENTAL RESULTS
Controllers discussed in section 3 and 4 are used forthese experiments. We used SRV02 with ROTPEN.
1. Swing-up and small disturbanceAt the start, the pendulum is located at -180 degree.
After the pendulum is stabilized, the user push it to theleft and to the right. This changes alpha by almost 20
degrees but not enough to make the pendulum fall down.

Fig. 3(a). Angle of a pendulum

Fig. 2(a). Angle of a pendulum

Fig. 3(b). Angle of a rotating arm
Fig. 3(a) and Fig. 3(b) shows the second experimentresult. In the figures, 0 and 360 degrees represent sameangle. Between 2 and 3 seconds, the pendulum fell
down but recovered to the upright position right away.
Between 4 and 5 seconds, user applied disturbance tothe opposite direction and it still operated.

VI. CONCLUSION
In this paper, the modified bang-bang controller and
LQR controller worked successfully. The pendulum
showed convergence and achieved robustness from thelarge disturbance. Swing-up time was less than 3.0s andthe whole recovery process worked immediately.

Fig. 2(b). Angle of rotating arm
Fig. 2(a) and Fig. 2(b) show the first experimentresult. A pendulum swung to upright position in 2
seconds. Between 10 ~ 14 seconds, the user applied
disturbance to the left and right. The pendulum head
was moved about 20 degrees and the rotating arm was©ISAROB 2011563

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

REFERENCES[1] K. Furuta, M. Yamakita and S. Kobayashi (1991),
Swing Up control of Inverted Pendulum. IECON’91, pp.
2193-2198.
[2] K. Yoshida (1999), Swing-up control of an inverted
pendulum by energy based methods. Proceedings of theAmerican control conference, pp. 4045-4047.
[3] K. J. Å ström, K. Furuta (2000), Swinging up apendulum by energy control. Automatica, vol. 36, no. 2,
pp. 287-295.
[4] J. Krishen and V.M. Becerra (2006), Efficient fuzzycontrol of a rotary inverted pendulum based on LQR
mapping. IEEE International Symposium on Intelligent
Control, pp. 2701-2706.
[5] T. C. Kuo, Y. J. Huang and B. W. Hong (2009),
Adaptive PID with Sliding Mode Control for the Rotary
Inverted Pendulum System. Advanced Intelligent
Mechatronics, pp. 1804-1809.
[6] Z. Wang, Y. Chen and N. Fang (2004), MinimumTime Swing-up of A Rotary Inverted Pendulum by
Iterative Impulsive Control. Proceedings of the 2004
American Control Conference, vol. 2, pp. 1335-1340.
[7] N. Muskinja and B. Tovornik (2006), Swinging up
and stabilization of a real inverted pendulum. IEEE
Transactions on Industrial Electronics, vol. 53, no. 2, pp.
631-639.

©ISAROB 2011564

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Intelligent Information Retrieval System
Young Im Cho
Dept. of Computer Science, Univ. of Suwon, San2-1, Wau-ri, Bongdam-eup, Hwaseong, Gyeonggi-do(Tel : 82-31-229-8214; Fax : 82-31-229-8281)(ycho@suwon.ac.kr)
Abstract: In this paper, we propose a multi agent based information retrieval system in digital library. We propose a newalgorithm for each multi agent module in detail. Each module is for personalized information retrieval. In the future, we haveto apply to a real situation for proving.
Keywords: Information retrieval, Multi agent system, Agent platform

to many researchers. It is that their search results include
undesired results because of no information of users‟
profile. Thus, users have to establish the directories inusers‟ personal computer for developing personalized
digital library. In order to solve the problem, thepersonalized digital library systems have been developed.
Bollacker et al. [8] introduced a personalized CiteSeer
digital library system which is an automatic paper searchsystem. The system can track and recommend the similar
papers whose topics are very relevant, using the contentbased relatedness measures [10]. Torres et al. [9]improved the performance of the automatic paper searchsystem by developing the hybrid recommender which isthe combination of user-based collaborative filtering
method [11] and content-based filtering method [12].
The personalization is done by analyzing the topic ofpapers. In this case, the papers whose topic is not
relevant to the queried keywords request by a user but
contents are relevant to them are not recommend. Inorder to solve the problem, we consider the abstracts inthe papers for providing a personalized paper search list
according to the user‟s behavior on the papers and therelevance among keywords in the abstracts.
Also, another problem the multi agent-based digital
library systems have is that users themselves should visit
all possible search servers one by one. To overcome theproblem, we propose a new platform of multi agent
digital library system which is mobile search system.
Users do not need to visit all possible search servers withthe same query by using the new platform. Our system
automatically visits to all possible search servers whenuser requests a query. In this case, the scheduling ofvisiting the search servers is needed. Also, thenegotiation for the results searched from the servers
needs to be made for complicated situations such as the
duplicated search results from multiple servers. Also, ourpersonalized paper search algorithm builds user‟s
individual relevance network from analyzing theappearance frequencies of keywords in the searchedpapers. The relevance network is personalized byproviding weights to the appearance frequencies of

I. Introduction
Recent developments of the internet and network
technologies evoke the technical change of the dataprocessing from conventional centralized and local
processing system to the distributed processing system.
Many studies have been actively carried out in a
distributed processing environment by using agent
systems for efficient network management. Multi agent
systems promote the efficiency in solving problems bycooperating among agents. Also, each agent
independently manages its own tasks by dividing a whole
work into smaller units and assigning them to each agent.
There are many application areas in the real world, using
the multi agent systems. One of these areas is the digital
library system. This is a library developed to replace theconventional library, in order to serve information fromdatabases on the web to users according to thedevelopment of computers and the related fields.
University of Michigan Digital Library (UMDL) [1, 2]
is one of the most famous agent-based digital librarysystems. Agents in UMDL autonomously use theirresources through negotiation among them. Also, the
UMDL is very flexible for updating the library system
when a new agent needs to be added. As another famous
agent-based digital library, Green Stone Digital Library(GSDL) [3] was developed by New Zealand Digital
Library project team. The GSDL provides a novel
method for organizing information using the open source
digital library [4, 5] and making it useful over theinternet. McNab et al. [6] improved the open source
digital library system by proposing a flexible protocol forcommunicating between an interface server and a searchengine. Their system is suitable for distributed
computing environment. Also, Witten et al. [7] improved
the functionalities of finding, collecting, and organizing
information in the GSDL for the distributed environment.
Even though most conventional digital library systems
provide the convenient life to users for searching andmanaging information, a hot issue has been paid attention©ISAROB 2011565

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

keywords according to users‟ behaviors on the searched
list, such as “downloading,” “opening,” and “no-action.”
Also, we enable interaction among multi agents bydeveloping an artificial negotiation algorithm.
In the experimental section, we demonstrate ourmethod using 100 faculties‟ search information
employed in the University of Suwon. Also, theperformance of our method is compared with that of theconventional paper search system by surveying thesatisfaction of users for both systems. In addition, we
analyze the searching speed of our system to show boththe advantage and disadvantage of our mobile-based
searching system.

Once a user requests a query with keywords, Agent
Manger (AM) in DMMAF receives the query through the
Matchmaker Agent and prepares the user‟s profile
because the search results are obtained by the user‟s
profile. Scheduling Agent determines the order of theagents‟ visit to the digital library server. The documents
are collected from an agent‟s visit to web servers(DBMS) according to the scheduled order. Digital
Library Server (sub host) accepts the agents by defined
port, agents approach the DBMS of web servers. During
the agent visiting, Negotiation Agent customizes thecollection at each web server by eliminating duplicate
documents with its previously visited web servers via a
network. Each visit is done in company with the Mobile
Multi Agent (MMA). The attributes of MMA includes
the plan code, agent ID, host URL, and resources (user‟s
profile). The plan code is the agent‟s task generated by
Scheduling Agent. The agent ID is the unique number of
each agent, which is generated by Matchmaker Agent.
The host URL is the address to return. The resource isthe user‟s profile. Personalization Agent provides theuser a personalized search list from the negotiateddocuments according to the user‟s profile. Then,
Information Collection Agent shows the final searched
results to user through the user interface (UI).
DMMAF is a proposed agent framework for
distributed environment in this paper. DMMAF has fivecomponents such as Agent Manager, Plan Editor, ANS,
MMA , Transfer Protocol.
Agent Manager receives a user‟s query, creates agents,
and controls the agents‟ operations. Matchmaker Agent
communicates between Matchmaker Agent and UI. Also,
Matchmaker Agent creates agents according to theschedule generated by Scheduling Agent. Scheduling
Agent searches the optimal path to visit servers andinforms it to the plan editor in DECAF. The plan editor
creates the plan codes. Information Collection Agent
sends the final searched results to the user through UI.
Negotiation Agent removes the duplicated results
through KQML communication language among MMAs.
Agent Name Server creates the MMA and saves themulti agents‟ record about creation and deletion
information of each MMA. The transfer protocol of the
MMA is to encrypt and decrypt the agent byte code, plancode, user profile, and resources. Byte codes which aresent through network are doing the parsing process.
MMA visits the host having user keyword and user
profile. MMA negotiates with other agents using
negotiation algorithm and search the DB.
The proposed negotiation algorithm in Negotiation
Agent is explained in detail as seen in Fig. 2.

II. Proposed Multi Agent-based Platform for
Information Retrieval
A digital library serves a lot of information on-line.
The advantages of digital libraries over conventional
libraries are user friendly, on-site service, andaccessibility. However, in case of not having
standardized platform, the search of heterogeneous
information from digital libraries may be hard, or
impossible in some cases. To solve the problem, wedevelop a mobile multi agent-based digital library systemusing DECAF. The „mobile‟ means that agents
autonomously move a server to a server without user‟sinterference. The mobile multi agent-based digital librarysystem can eliminate unnecessary and duplicate
information stored in Internet or DB. Also, existing
digital libraries do not have or learn about the user‟s
information. It causes unnecessary or useless information
for the user to appear in the searched results. To solve
the problem, we propose a mobile multi agent-based
personalized digital library system (MAPS). MAPS
provide the personalized search list from users‟ usage
history. The overall structure is in Fig.1.

/* improved negotiation algorithm of MMA */Switch(one of 5 relations) { /* relation_name(sender, receiver) */case 1: Add_R(Ai_Mi, Aj_Mj)/* (i≠j), Ai=agent, Mi=method */while(Aj_Mj is finished)
no-operation (Ai-Mi); break;Fig.1 Overall Structure©ISAROB 2011566

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

3
case 2: Compensate_R(Ai_Mi, Aj_Mj)
Ai_Mi and Aj_Mj are operated in real situation
by call(inference) function;
Inference() is from user profiles or user information repository;
break;
case 3: Replace_R(Ai_Mi, Aj_Mj)
Aj_Mj=Ai_Mi;
break;
case 4: Contradict_R(Ai_Mi, Aj_Mj)
Aj_Mj=Aj_Mj;
break;
case 5: Activate_R(Ai_Mi, Aj_Mj)
if wait(Aj_Mj) then awake(Aj_Mj) and restart;
break;
default:}

Fig.3 The flowchat of the negotiation process between
two MMAs

Fig.2 Negotiation method among MMA
Fig.4 The negotiation mechanism of informationexchange between two MMAs by KQML protocol

Agent_Task Group assigns Task_1, Task_2, andTask_3 to Agent 1, Agent 2, and Agent 3 respectively.
The tasks are automatically decomposed into methods,
and the agents execute the assigned tasks by the methods.
There are five types of relationships among methods.
Add_R is the add relationship, i.e., adding the result of amethod to that of other methods. Activate_R is therelationship making the running method keep running
continuously. Compensate_R is the relationship thatcompensates the results of methods if needed. Replace_R
replaces the results of receiving methods with those of
sending methods. Contradict_R ignores the results ofreceiving methods. Also, there are lots of relationships
between methods and tasks and between methods andresources, such as Enable, Facilitate, Produce, Consume,
and Limits. In the negotiation algorithm, if the agents inthe same level take different actions, then max operation
operates to produce the output of the agents. Otherwise,
min operation is operated.
Fig. 3 is the flow chart of the negotiation process
between two MMAs. Fig. 4 is the schematic of thenegotiation mechanism of information exchange between
two MMAs by KQML protocol.

The information about MMA moving is transferred inbyte stream type surrounded by the defined tag of the
KQML protocol. The byte stream is parsed according to
the defined tag and the clone byte of the byte stream istransferred to the next web server. The detailed algorithmof the moving MMA is the followings.
receive( Socket s ) {InputStream inputByteStream = new InputStream();inputByteStream = new InputStream( s.getByteStream() );while( ( tempByte = inputByteStream.read()) != End ) {Parsing()
if( nextHost == null ）｛try{
ByteCode[index++] = (byte)tempByte ;} catch( ArrayIndexOutOfBoundsException ) {System.out.println("Termination of all host visiting");
Notify to Agent Manager about Job termination;} // end of try} else if ( Exist sending socket to next host? ){nextHostSocket.write( (byte)tempByte );}} // end of while} // end of receive()

The algorithm of parsing process is the followings.

©ISAROB 2011

567

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

parsing( InputStream in ) {while( ( tempByte = in.read() ) != End ){if( Start character of Tag? ){inputStream mark();
Read byte until the Tag length;
Identification the Tag type;if( fail the defined TagSearching ?){Reset according to marked Stream index;
Process according to current Tag type;} else {Branch or activate according to the current Tag and theidentified next Tag;}} ese if( Ending character of Tag?){if( nTag inTag = false;
else Process according to current Tag type;}} // end of while} // end of parsing()

00.
[8] K.D. Bollacker, S. Lawrence, C.L. Giles, “A System
for Automatic Personalized Tracking of Scientific Lit
erature on the Web”, Proceedings of ACM Conferenc
e on Digital Libraries, pp. 105-113, 1999.
[9] R. Torres, S.M. McNee, M. Abel, J. A. Konstan, andJ. Riedl, “Enhancing Digital Libraries with TechLens”, ACM/IEEE-CS Joining Conference on Digital Lib
raries, pp. 228-236, 2004.
[10] K. Bollacker, S. Lawrence, and C.L. Giles,“ An Aut
onomous Web Agent for Automatic Retrieval and Ide
ntification of Interesting Publications”, Proceedings of The Second International Conference on Autonomo
us Agents, pp. 116-123, 1998.
[11] W.P. Lee and T.H. Yang, “Personalizing Information Appliances: A Multi-agent Framework for TV Prog
ram Recommendations”, Expert Systems with Applications, vol. 25, no. 3, pp. 331-341, 2003.
[12] T. Kamba, K. Bharat, and M.C. Albers, “An Interact
ive Personalized Newspaper on the Web”, Proceedin
gs of International World Wide Web Conference, pp.
159-170, 1995.
[13] J.R. Graham and K.S. Decker, “Towards Distributed, Environment Centered Agent Framework”, Appear
ing in Intelligent Agents IV, Agent Theories, Archite
ctures, and Languages.
[14] Lesser, Victor; Horling, Bryan; Klassner, Frank;
Raja, Anita; Wagner, Thomas; and Zhang, Shelley.
“BIG: An Agent for Resource-Bounded Information
Gathering and Decision Making”, Artificial
Intelligence Journal, Special Issue on Internet
Information Agents, vol. 118, no. 1-2, Elsevier
Science, pp. 197-244. May 2000.
[15] John Graham, Victoria Windley, Daniel McHugh,Foster McGeary, David Cleaver, and Keith Decker,“Tools for Developing and Monitoring Agents inDistributed Multi Agent Systems”, Workshop on
Agents in Industry at the Fourth International
Conference on Autonomous Agents, Barcelona,
Spain, June, 2000.
[16] Baeza-Yates, Ribeiro-Neto, Modern Information
Retrieval, Addison-Wesley, 1992

III. Conclusion
In this paper, we proposed a platform of a mobilebased personalized paper search system. By using oursystem, users do not need to visit all possible search
servers with the same query by using the new platform.
Also, our system provides users‟ personalized search listby building user‟s individual relevance network fromanalyzing the appearance frequencies of keywords in the
searched papers.
We have to apply this algorithm to the real system inthe future for proving of this algorithm.

REFERENCES[1] W. P. Birmingham, An Agent-based Architecture for
Digital Libraries, D-Lib, July 1995.
[2] W.P. Birmingham, E.H. Durfee, T. Mullen, “The Distributed Agent Architecture of the University of Michi
gan Digital Library”, AAAI Spring Symposium on In
formation Gathering form Heterogeneou, Distributed
Environments, AAAI Press.
[3] http://www.greenstone.org/cgi-bin/library.
[4] C. Logoze and D. Fielding, “Defining Collections inDistributed Digital Libraries”, D-Lib Magazine, vol.
4 no. 11, 1998, http://www.dlib.org/dlib/november98/
lagoze/11lagoze.html.
[5] C.M. Bowman, P.B. Danzing, U. Manber, and M.F. S
chwartz, “Scalable Internet Resouce Discovery: Research Problems and Approaches”, Communications of
the ACM, vol. 37, no. 8, pp. 98-107, 1994.
[6] R.J. McNab, H. Witten, and S.J. Boddie, “A distribut
ed Digital Library Acchitecture Incoperatiing different index styles”, Proceedings of IEEE Advances in Dig
ital Libraries, pp. 36-45, 1998.
[7] H. Witten, R.J. McNab, S.J. Boddie, and D. Bainrige,
Greenstone, “A Compreshensive Open-Source Digit
al Library Software System”, Proceedings of Fifth A
CM Conference on Digital Libraries, pp. 113-121 20©ISAROB 2011

568

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A Study on Real-time Face Verification and Tracking with
Segmented Common Vector
Dong-kyu Ryu, Min-ho Park and Ju-Jang Lee
Department of Robotics Program
Korea Advanced Institute of Science and Technology (KAIST)
Daejeon, Republic of Korea(Tel : 82-42-350-8032; Fax : 82-42-350-5432)(E-mail: ghostdz@naver.com)

Abstract: Recently, There has been much interest in automatically face recognition and tracking in many areas such asIntelligent Robotics, Military, Intelligent Transport System (ITS) and Smart device applications. However so far there
has not been simultaneously face tracking and verification algorithms, so we propose the method in simultaneously
verifying and tracking face which is segmented common vector method and It is theoretically based on discriminative
common vector method and Fisher’s LDA. The algorithm trains segmented and shift face image to obtain new facevector. The Gram-Schmidt orthogonalization is employed to calculate the orthogonal projection matrix. Our goal can be
defined as the identification of individuals and real-time face tracking from video images simultaneously, so we use
segmented common vector for two kind of different task at the same time.
Keywords:

Discriminative Common Vector, Fisher’s LDA, Gram-Schmidt orthogonalization.

After proposed eigen face, several research have been
more performed. The Linear Discriminant Analysis(LDA) method is proposed to overcome the limitationsof the eigen face method. This method can findprojection directions that on one hand maximize thedistance between samples of different face and on theother hand minimize the distance between the sample of
the alike face [3]. But, in face recognition task, we can’t
directly calculate projection directions, because
dimension of the image samples is very larger than thenumber of samples which is known as the “small
sample size problem”
To solve this problem, Chen proposed the Null spacemethod based on the modified Fisher’s LDA [4]. The
Null space method is based on the modified Fisher’sLinear Discriminant criterion,|WT S W |(1)
J MFLD (Wopt )  arg max T B| W STW |W
For this method, all image samples are first projected
onto the null space of SW and then result in a new
within –class scatter that is a zero matrix.
In the special case where W T SWW  0 and W T SWW  0 ,for all w  R d \{0} , and then a better criterion is givenas

I. INTRODUCTION
In computer-vision research area, there has been muchdemand in verification of human face in video andphoto images due to intelligent robotics, military, smart
phone and surveillance system. The problem of face
verification can be defined as finding the identificationof test faces from sample face images which waspreviously learned, however there are many complex
problem in this task. For example, face data has
numerous changeable factors such as 3-D pose, hairstyle, make up, and facial expression. In addition tothese can be varied due to lighting, background, andscale changes. Thus there are many researches solving
face verification problem as follow.
For face recognition, Harr-like feature and AdaBoost
algorithm of Viola and Jones is very famous in facerecognition. It is very fast and has high-rate accuracy[1]. In face identification tasks, the verifying face has
many hard problems, so it is very important to extractcommon factors from various data sets for face
identification.
To do this, eigen face of Turk and pentland isintroduced, and this method uses Principal component
analysis (PCA) which is used to find the best set ofprojection directions in the sample space. From here,
the projection directions are called the eigen faces
which is vector set having common factors. [2]©ISAROB 2011

J (Wopt )  arg max | W T S BW | arg max | W T STW | (2)|W T SW W | 0

|W T SW W | 0

The Discriminative Common Vector Method was
proposed to extract the common properties of classes inthe training set by eliminating the differences of the

569

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

people becomes each classes, thus the training set iscomposed of C classes.
To solve real-time tracking problem, new face sample
set be generated by using shift-image method.
If we have P samples in each all classes, firstly weshould shift face image to 1-3 pixels in order of left,right, top and bottom. This is called shifted sample set,so we can get N  5  P samples. And then we takesegmented samples apart this N samples.

samples in each class, it gives more efficient algorithm
to find Null space of within-class scatter matrix [5]. In
the special case where W T SWW  0 and, W T SWW  0 for
all w  R d \{0} , and then a better criterion is given as

J (Wopt )  arg max | W T S BW | arg max | W T STW | (2)|W T SW W | 0

|W T SW W | 0

To find the optimal projection vectors w in the nullspace of SW , we have to project the samples onto thenull space of SW and then obtain the projection
vectors by performing PCA.

C

- Definition of sample vectors xN , S

xmi , k be a d -dimensional column vector andLet

C be denoted by i-th number of classes, S be denotedby k-th segment set in i-th classes and N is index of mth sample from k-th segment set in i-th class.

II. Segmented Common Vector
1. Definition of Segmented Common Vector
We propose the simultaneously face verifying andtracking method which is segmented common vector
method. It is theoretically based on discriminative
common vector method. However this method presents
that several weak classifiers can more easily solve hard
problem than one strong classifier such as face tracking
problem. So that we have to make several feature vector
from each one sample to create various sub classifiers,
before the one sample was transferred to only one facefeature vector.

The segmented face vector sets is proposed as subclass sets which made by segmenting face images from
C classes as below.

sub class : xki  span { x1,i k , x2,i k ,..., xNi , k } (3)i  th Class : xi  x1i , x2i , ... , xSi
So we can have N x S samples from existing one
class and C x S sub-classes which has N samples,
then the number of total samples become
M  N  S  C in training set. Suppose that d  M  C .
In this case, SW , S B , and ST are newly defined as

2. Algorithm to calculate Segmented Common Vector
A. Span new image sample vector
In this section, we account for process in making
segmented common vector in detail.
To make segmented common vectors, in advance wehave to make segmented image to span new face vector.
The size of segmented image is usually 60~90% thanoriginal face size. The size of segmented image is saved
as cell size value and it used for face tracking. We
should take apart scrap images to be overlapped,because the center of face has many information such aseyes, eyebrows, nose and mouse. The segmented feature
extract manner is presented as below.

SW 

C

S

N

 ( xim, ki 1 k 1 m 1

SB 

C

 ki )( xmi , k  ki )T ,(4)S

 (   )(   ) ,i
ki 1 k 1

i
k

T

(5)

and

ST 

C

S

N

 ( xi 1 k 1 m 1im, k  )( xmi , k   )T  SW  S B (6)B. Span difference space
To span difference spaces B , we choose any one of theface vectors from the k-th segment sets in i-th class.
Difference space is made by subtraction of vector.

bmi , k  xmi 1, k  x1,i k ,(7)( i  1, ..., C , k  1, .., S , m  1, ..., N  1)im, k

The b is difference vector from k-th segment in i-thiclass, so difference space is defined as Bk .
The complete difference subspace is represented as
below
1
1
1
2
C
B  B1  B2  ...  BS  B1  ...  BS1
 span{b1,1,..., bN1 1,1 , b1,1 2 , b2,1 2 , ..., bNC 1, S }Fig.1 The Manner of Segmenting face image

In this paper, the number of segmented face feature has
range from 4 to 9 features
Suppose that we have several people images, each

©ISAROB 2011(8)

To obtain orthonormal basis vectors 1 ,....,  M C , thedifference vectors is orthonormalized by using theGram-Schmidt-orthogonalization procedure.

570

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Its Algorithm is explained as below.

III. EXPERIMENTAL RESULTS
To classify the identification of face, each 10 samplesof 7 people is learned by using OPENCV library. The
samples is fragmented by 4 pieces and changed as
segmented common vectors. The mean shift tracking
algorithm [6] is used to track position of face, however
segmented common vector is employed to calculate
target candidates instead of color histograms which isusually adopted in face tracking. The experiments havebeen processed with a 3GHz Pentium PC and 2GB ofmemory under windows. Our implementation of tracker
is made from visual studio 2008 environment.
The goal of experiments is to show how well our
method tracks the optimal position of face. The tracked
face has been changed by position and 3-D pose of face.
The face sequence of 1200 frames is shown as below.

C. Algorithm of learning Segmented Common vectori

Step 1: Find the linearly independent vectors bm , k that span
the difference subspaceB  span{b1,1 ,..., bN 1,1 , b1, 2 , b2, 2 , ..., bN 1, S }
1

1

1

1

C

Step2: Apply the Gram-Schmidt orthogonalization procedure
to obtain an orthonomal basis 1 , ...,  M  ( C  S ) for B
Step 3: Choose any sample from each class and project it onto
B to obtain common vectors by using

U  [ 1

 M  ( C  S ) ],....

P  UUT

xk , com  Pxm , k  xm , k  Pxm , k  xm , k  UU xm , k ,Step 4: Find the difference vectors that span Bcom
i

i

i

i

bk , com  xk 1, com  x1, com ,
i

i

i

T

i

i  1, ..., C , k  1, ..., S

1

Step 5: Apply the Gram-Schmidt orthogonalization to obtainan orthonormal basis 1 , ... , ( C  S ) 1 for Bcom
Step 6:

The projection matrix W  [1 , ... , ( C  S ) 1 ]be used to obtain feature vectors in m, k  W xm , k , m  1,..., N , i  1,..., Ci

Ti

We call the feature vectors  m , k segmented common vectors
i

300

600

from k-th segmented in i-th class and they are used forclassification of segmented face images.

4. Classifier of Segmented Common Vector
For classification, various classification methods canbe used. However, in this special case which is real-timetracking, more fast method is needed to reduce
calculation time. In this paper, Euclidean distancemethod is employed for fast classification.
After calculating projection matrix W , the segmentediicommon vectors  m , k is spanned from samples xm , k .
Also mean of segmented common vector is calculated
as below.
mean k i

1 N i
1 N T i m , k N  W xm , k ,N m 1
m 1

1200
900
Fig.3 The experimental result

The experiment result shows that this method wellfollow the face of human, and face classification is done
as class 1 from 7 classes. The tracker is very tough at
various pose and has high precision of position face.
The frame rate generally shows speed of algorithm, and,in this experimental, the frame rate of video was 14.5
per seconds. The tracker is very robust at illumination,
since proposed method is not based on color feature, so
it is also strong point of proposed method. The tracker
can efficiently and successfully handle non-rigid andfast moving objects under different background.

(9)( i  1, ..., C , k  1, ..., S , m  1, ..., N )Each sub classes of segmented face sets have mean ofsegmented common vector which is a classifier of thismethod.

©ISAROB 2011571

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

IV. CONCLUSION
From experiments, the segmented common vector
tracker successfully estimated position of face incomplex background and can verify the identification
face from several faces which was learned before.
However, the boundary of estimation should be needed
for avoiding measurement error about position and size.
Since proposed method is not based on color feature,
the tracker is very robust at illumination and the tracker
can efficiently and successfully handle non-rigid andfast moving objects under similar background. Our
implementation of segmented common vector algorithm
has proven to be robust to changes in shape, occlusionand color in experiment. However there is limitation oflearning peoples, becaused  M
Cshould beguaranteed in Null-space method.

REFERENCES[1] PAUL VIOLA and MICHAEL J. JONES (2004)“Robust Real-Time Face Detection”, International
Journal of Computer Vision 57(2), 137-154[2] M. Turk and A.P. Pentland, (1991) “Eigenfaces for
Recognition”, J. Cognitive Neuroscience, vol. 3, no. 1,
pp. 71-86[3] P.N Belhumeur, J.P. Hespanha, and D.J. Kriegman,(1997), “Eigenfaces vs. Fisherfaces: Recognition Using
Class Specific Linear Projection”,IEEE Trans. PatternAnalysis and Machine Intelligence, vol. 19, no. 7, pp.
711-720[4] L.-F. Chen, H.-Y.M. Liao, M.-T. Ko, J.-C. Lin, and
G.-J. Yu,(2000) “A New LDA-Based Face Recognition
System Which Can Solve the SmallSample Size
Problem,” Pattern Recognition, vol. 33, pp. 1713-1726[5] Hakan Cevikalp, Marian Neamtu, Mitch Wilkes andAtalay Barkana (2005) “ Discriminative Common
Vectors for Face Recognition”. IEEE TPAMI vol.27[6] Dorin Comaniciu Visvanathan Ramesh and peter
Meer (2000)”Real-Time Tracking of
Non-Rigid
objects using Mean shift”, IEEE CVPR

©ISAROB 2011

572

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Social Modification using Implementation of Partial Agency
toward Objects
Hirotaka Osawa* and Seiji Yamada***JST PRESTO, Honcho 4-1-8, Kawaguchi, Saitama, Japan(Tel : 81-48-226-5601; Fax : 81-48-226-5651)(Email osawa@nii.ac.jp)**National Institute of Informatics, Hitotsubashi 2-1-2, Chiyoda-ku, Tokyo, Japan(Tel : 81-3-4212-2000)(Email seiji@nii.ac.jp)
Abstract: This paper discusses what kind of partial agency is implementable for objects to bring more suitableagencies toward human agent interaction. Human beings have an ability to inform fellows about our intentio
n, internal states and requirements using verbal talk, gestures, attitudes, timings and other representations. The
se representations can keep our belief that we are sufficient agents mutually. The robots and virtual agent also mimic these representations, reproduce as if they have an agency, and interact with them. However, their a
gencies are sometimes too excessive compared to its task. This mismatch leads high cognitive load toward us
ers and brakes interaction consequently. This defect prevents to apply human agent interaction method towardapplication field. The authors consider that our agency is constructed by multiple features and dividable. If these features are selectable, we can choose more proper design for virtual agents, robots, machineries, daily ho
me appliances according to their traits. The authors categorized these agencies in several group and discusses
about what elements achieves these features. The paper also shows what method can extract these features fro
m human being.
Keywords: Anthropomorphization, Human Agent Interaction, Human Robot Interaction, Human Interface

Another policy places social buffer between users
and system and convert several input/output using thisbuffer (shown in Fig. 1-B). The buffer is called agent[1]. The agent uses several metaphors like internal states,
emotions, requirements to inform users. This method is
supported by several cognitive science results, theI. INTRODUCTION
Today's human computer interaction field is widely
developed by several different background researchers.
Various robotics technologies like actuators and sensors
gives the researches on informatics field the accessmethod toward real-world. On the other hand, rapid
development of computer gives mechanics engineers
more sophisticated control method. Ubiquitous, humanrobot interaction, interfaces is occupied several
researchers. Challenges are tried in several fields.
These studies which related in human computerinteraction are conducted by two different policies. One
policy is called extension of human's ability. Figure 1-A
shows the brief image of this policy. Technologies are
accepted as extended third arm and extended machinesinterfaces. For example, glass type augmented reality(AR) technology gives users more visible information.
If someone gets the glass type AR technology, s/he
simply gets upgraded eyes. Tele-operation is anotherkind of extension about our own bodies. If we canmanipulate actuators in another location according to
our will, these actuators behave as third arms or legs forus.

©ISAROB 2011

Fig.1. Difference between Extension andAgent Technologies in human computerinteraction

573

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

background of artificial intelligence and enthusiasm
toward lifelike agent. In this paper, we used humanagent interaction as covering word of this ideology. We
call human agent interaction as HAI. We also calls HRI(human robot interaction) as a HAI in the real world.
HAI method has advantages to inform several
complex states toward users, especially for implying
intentions and internal states by metaphors. For example,
several states of video recorder, computers are possible
to represent as its emotional states. If the object behaves
with sad emotion, it suggests some wrong treatment isongoing. On the other hand, if the object represents
happy emotion, it multiply implies that the transaction isin order. The human-like social stance can bring
restrictions toward determination of meanings. Ishida et
al showed how conversation system improves quality byusing several roles in the conversation.
Our study focuses to extend the application field of
human-agent interaction toward more broad area using
separation of agency. In this paper, we defines agency as
any kind of representations that evocate user's
intentional stance described in Dennett's work [2]. If weselect and add agency separately according to required
task of the system, system's ability will be expand.
Figure 2 and 3 show how our approach extends humanagent interaction. Figure 2 shows normal human agent
interaction method. An agent is placed between thesystem and a user. It wraps system's input and output
using social channels like attitude, emotion, gestures,and joint attentions. In our study, these channels are
divided and applied partially according to the task themachineries required. Figure 3 shows the examples ofpartial agency method. If a machine runs under alocation based task like a vacuum shown in Fig.3-A, it
uses location related social channels. An attitude canemphasize which direction is informed. This works as a
restriction of the representation. Joint attention has moresophisticated role in human-human interaction, but it
also works to show a clue of positioned place [3]. On
the other hand, emotions and gestures work more in thesituation that a machine does not have an actual
movement but have complex states like a printing
machine shown in Fig.3-B. These social channels caninform several states to a user using metaphors.
This paper is organized as follows. Section 2 refers
what kind of studies are conducted psychological fieldand HAI. Section 3 discusses what kind of problems
disturbs the usage of agent toward application field.

©ISAROB 2011

Fig.2. Previous Agent System

Fig.3. Implementation of Partial Agency
Section 4 shows the detail notion of partial agencies
based on section 2 and 3. It also discusses how to
extract partial agency from interactions by several
implementations. Section 5 concludes our method.

II. STUDIES ABOUT USEFULNESS OF
AGENT AND AGENCIES
Psychological studies and human agent studies havebeen revealed social features like emotions, gestures
and attitudes that useful to improve machineries'
interactions. We think that these features are importantelements to construct agency. In this section, we
examine how we have used elements of ourselves.
Emotions are one of the most important parts toevoke agency toward users. Human beings and otheranimals express emotions. One of the earliest studies
about emotion is conducted by Darwin [4]. Hecompared several animals' expressions and discusses its
influences. Paul Ekman categorized human faces with 6
types, happy, sad, angry, fear, disgust and surprise [5].
Robert Levenson organized another model forevaluation [6]. Even if the emotions do not be told justfrom abstracted appearances, these abstracted models
are proposed and commonly used in virtual agent andcommunication robot. The utilization of emotions is

574

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

proposed several times. Rosalind Picard proposed
effective computing that uses emotional cues as input
and output of the system [7].
Gesture is another good example that is used torealize agencies. It is commonly used to support ourconversations [8]. We use some gestures unconsciously,
and other gestures intentionally. Mental space studies in
psychological field describe how these gestures transfer
information from one to another. When we hear spatial
information like a feature location, we map the
instructed position into our mental space. Psychologists
analyzed this situation using blended mental space. The
concept of blended mental space is proposed by
Fauconnier [9]. Liddell suggests that mental space isalso applicable to analyzing gestures [10]. The blended
mental space helps to analyze the meaning of users’conversations using the virtual space of each user. For
example, if you used the right hand to point a dial onwall and left hand for turning gesture, it is not
instructing users to turn a dial in front of them. It
instructs the user to blend these two gestures into thesame mental space to turn one dial one way.
These human like features are selected from humanactivities. They have enough ability to make agency
toward the object. Several human robot interaction and
HAI studies uses these result for the improvement ofinteraction [1].

2.

Excessive anthropomorphization and agency
The researchers have a trend to hypothesize thatanthropomorphism sometimes attracts people. However
over-anthropomorphism sometimes distracts people
from the interaction. This problem is based on threereasons.
First reason is mismatch of the agency. If the agent
expresses the interaction that implies complex agency,
the user estimates much more agency from the behavior.
Adaptation gap study also noted this problem in theinteraction [12].
Second problem is the mismatch of relationshipbetween task and agents. Even if an agent acts is
attractive and enough to express the appropriate agency,
the mismatch of appearance still harm an interaction.
For example, bug type robots are not appropriate forcooking task because it implies dirty image toward
users. Robot designer Sonoyama illustrated several
mismatch of the design of robots in his book [13].
Lastproblemisdisgusttowardanthropomorphization itself. Overwhelming feeling ofsocial connection leads too much cognitive load toward
users. Epley et al showed their study that people
requires anthropomorphic attitude when they are
disconnected by social connections [14]. This result
indicates that if one is satisfied for social connection,
additional anthropomorphic representation becomesunwanted exaggeration.

III. PROBLEMS BETWEEN HUMAN
AGENT INTERACTION AND APPLICATION

IV. HOW TO EXTRACT AGENCIES

Above section shows what kind of basic knowledge
about human beings support HAI technologies.
However, agent based interaction is sometimes not usedin the real application instead of their usefulness. In thissection, we discusses about several defects of an agent
and HAI studies.

In Section 3, we discussed what kind of
shortcomings interfere the smooth applying of agency
toward interface. To solve above problems, we simply
propose the separated evaluation of agency using
separated anthropomorphic devices. Each feature for an
agency is constructed by each device. Extraction of theagency is achieved by subtraction of the interface fromthe previous agent equipped full of agencies.
Figure 4 shows what is achieved by our method.
Figure 4-A describes the kind of features could be
extracted by separation of agency. Each feature has been
discussed in previous studies referred on Section 2.
Previous HAI studies already researched what kind of
agency is useful [1]. However, we can evaluate eachfeature simultaneously and separately if we divide eachagency. As a result, we can evaluate more precise
arrangements of agencies according to each user's
tendency and attributes (like Fig. 4-B). Our previous
study suggested that female users like anthropomorphic

1.

Compatibility problems
Agents' various modalities prevent to organize
standard knowledge from comparing each study. Rene
Descartes noted the importance "to divide each of the
difficulties under examination" in his work [11]. This
policy is basement of Science. However, HAI and HRI
sometimes fails to order this requirement because theresults of these studies are strongly related to theagent/robot themselves and difficult to discuss
separately. For example, a result based on agent A may
not be possible to be applied to another study using
agent B, because of their different attributes.

©ISAROB 2011575

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

representation more than male participants [15]. If auser does not want too much anthropomorphization, we
can decrease features and can adjust most appropriate
agency according to the user. Figure 5 shows some
implementation according to our method. Figure 5-A
shows partial embodiment. Eyes and arms are attached
to a vacuum because a vacuum requires precise
positioning toward users during cleaning. On the other
hand, eyes and arms are removed from Fig. 5-B. Insteadof rich appearances, 2 axis motors are implemented torepresent to show an attitude because a microwave ovendoes not requires precise positioning.

V. CONCLUSION
We considered that our agency is constructed bymultiple features and dividable. If these features are
selectable, we can choose more proper design for virtual
agents, robots, machineries, daily home appliances
according to their traits. We categorized agencies
referring previous studies and discussed about what
elements are required. We also proposed what method
can extract these features from human being using
several devices.

Fig.4. (A) Separated anthropomorphic elements.
(B) Appropriate anthropomorphizationaccording to each user.

ACKNOWLEDGEMENT
This work was supported by JST PRESTO program.

REFERENCES[1] Yamada S, Kakusho K, Ono T, Ogawa K, Nakano Y,
Komatsu T, Imai M, Osawa H, Komatsu K, Terada K,
Kojima H, Nakagawa K, Kiyokawa S, Ueda K,
Takeuchi Y (2008), Designing space between humanand robot (in Japanese)[2] Dennett DC (1987), The Intentional Stance, The
MIT Press[3] Tomasello M (1995), Joint attention and social
cognition. In: C. Moore, & P. Dunham, editors. Joint
attention: its origins and role in development, pp. 103–130
[4] Darwin C (1872), The Expression of the Emotionsin Man and Animals, 1872
[5] Ekman P (1999), Handbook of Cognition andEmotion, John Wiley & Sons, Ltd.
[6] Levenson RW (1999), The intrapersonal functionsof emotion. Cognition and Emotion, 13, 481-504.
[7] Picard R (1997), Affective Computing, MIT Press[8] Morris D (1995), Bodytalk: The Meaning of Human
Gestures, Crown[9] Fauconnier G (1995), Mental Spaces, MIT Press[10] Liddell S (1996), Spatial representations in
discourse: comparing spoken and signed language,
Lingua 98: pp. 145-167[11] Descartes R (1637), Discourse on the Method,
http://www.gutenberg.org/ebooks/59©ISAROB 2011

Fig. 5. Partial embodiment andpartial attitude[12] Komatsu T, Utsunomiya A, Suzuki K, Ueda K,
Hiraki K, Oka N (2005), Experiments toward a mutual
adaptive speech interface that adopts the cognitive
features humans use for communication and induces
and exploits users' adaptation, International Journal of
Human-Computer Interaction, vol.18 (3), 243-268.
[13] Sonoyama T (2007), Introduction for Robot Design(in Japanese)[14] Epley N, Waytz A, Cacioppo JT (2007), On Seeing
Human: A Three-Factor Theory of Anthropomorphism,
Psychological Review, 2007, Vol. 114, No. 4, 864-886[15] Osawa H, Mukai J, Imai M (2007), Display Robot"- Interaction between Humans and Anthropomorphized
Objects, 16th International Symposium on Robot andHuman Interactive Communication, Vol.16, pp. 451456, Jeju, Korea, 2007.8

576

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

©ISAROB 2011

577

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

©ISAROB 2011

578

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

©ISAROB 2011

579

The Sixteenth International Symposium on Artificial Life and Robotics 2011(AROB 16th ’11),
B-Con Plaza, Beppu, Oita, Japan, January 27-29, 2011

©ISAROB 2011580

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Behaviors for Getting Conscious Responses
Toshihiro Osumi*, Masato Noda*, Hirotaka Osawa**, Yuki Kuwayama*,
Kazuhiko Shinozawa*** and Michita Imai*****Graduate School of Science and Technology, Keio University, Yokohama, Kanagawa, 223-8522, Japan(tosihiro@ayu.ics.keio.ac.jp)**PRESTO, Japan Science and Technology Agency, Chiyoda-ku, Tokyo, 102-0075, Japan***ATR Intelligent Robotics and Communication Laboratories, Hikaridai, Kyoto, 619-0288, Japan****Faculty of Science and Technology, Keio University, Yokohama, Kanagawa, 223-8522, Japan
Abstract: This paper investigates robot’s behaviors to get voluntary conscious responses from users. Our final goal is toconstruct an asynchronous human-to-human communication mediated by a potable robot. The robot has a behavior
system which makes the robot behave in response to the acquired conscious response. The system also enables the user
to give the robot feedback at anytime and anywhere while the robot presents a script which an author prepared. We
investigate how well the behavior system encourages the users to give the robot feedback and how much the users
consider inputting the response to be meaningful. The results show that there are definite relation between response
behavior and the acquisition of the conscious responses.
Keywords: Conscious response, Computer-Mediated Communication, Input and Interaction technologies

I. INTRODUCTION
In this paper, we investigate robot’s behaviors to get
a voluntary conscious response from users. Our final
goal is to construct an asynchronous human-to-humancommunication mediated by a portable robot (Figure 1).
The robot has an advantage over the other interface
when giving user information. It can tailor the contentsof the information to him/her based on his/her
environment by using emotional expressions andgestures [2, 3]. In particular, the emotional and gesturalexpressions provide an author with the memorable wayof presenting his/her view and emotion experienced at a
certain place. The embodied expressions of a pointing
gesture and a gaze movement also provide him/her withthe exact way of presenting information related to acertain place or an object. The author-to-usercommunication via the robot increases the author's
motivation for creating and revising the content of theinteraction. In addition, the interactive presentation by
the robot can prompt a user to prepare his/her responseto the presented information. To substantialize the robot
mediated communication, we must develop an
individually-owned robot, an environment that author
can create contents without an expert knowledge and abehavior system that generates robot's behaviors forpresenting information and encouraging the user to givehis/her response to the presentation. In this paper, we
focus on the acquisition of a conscious response andinvestigate the effect of the robot's behaviors when
getting the users responses.

©ISAROB 2011

Fig.1. Communication via the portable robot

Fig.2. Motivations for updating blogs
The style of the author-to-users communication isthe common on the web. The style is that an author
creates his/her original content and a PC user provides acomment after reading it. For example, blogs and social
networking services have the function which is able tobe posted comments about the entry as the feedbacks
from users. The feedbacks are strongly linked tomotivation to use the service. In Japan, more than halfof users said that getting to know people who share theirinterests and including entries that visitors enjoyed weregreat motivations for them to update their blogs [5] (seeFigure 2). This result suggests the significance of581

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

feedback such as a comment, a reply and a ReTweet.
Nico Nico Douga [6], a characteristic video-sharing
website in Japan, has caught on with many people. In
contrast to other video-sharing sites such as Youtube,
Nico Nico Douga allows comments on specific timecode and can superimpose the comments directly on thevideo when showing the commented video frames. The
feedbacks on the web are posted voluntarily andconsciously from users and are not the one that a system
obtains by analyzing the user's behavior of browsing oraccess logs.
However, many communication robots acquire
feedback from a user by using sensors (e.g. cameras,
touch sensors, laser range finder and other devices) or
the user’s choices in a scenario. The feedback is theunconscious responses of the user's behaviors. It isdifferent from the communication services on the web
acquire conscious feedback which is provided
voluntarily from the user. But feedbacks by using
sensors and the users' choices are insufficient for the use
as the author-to-user communications, because theunconscious feedbacks do not reflect the user's
impression of the presented content. To communicate
with other people via a robot, the robot requires thefunction that can acquire voluntary conscious feedbacks
from users, like the services on the web.
The design which gives users a motivation for
inputting conscious feedback is necessary to acquire
conscious feedbacks efficiently. Kuno et al. [1] showedthat museum visitors’ nodded and mutually gazed more
frequently when a robot turned its head at significant
points than when a robot did not. This result suggeststhat felicitous behavior at a relevant point establishes
joint attention and arouses interest in the robot.
Therefore we developed a behavior system which
generates a behavior for encouraging the user to feed
conscious response anytime and anywhere while therobot presents the content which an author provided.
The behavior is generated in response to acquired
conscious responses. The reactive behavior induces
natural interaction which includes the conscious
responses. We installed the system into the mobile
phone based robot called “TenoriAvatar”. We also
conducted two experiments to observe the effects of the
behaviors on the frequency and range of the conscious
response and on how meaningful users thought their
response was. We believe that the acquisition of the
conscious response from users is necessary in theauthor-to-user communication via a robot or an agent.

©ISAROB 2011

The study provides a fundamental knowledge to acquire
conscious responses.

II. EXPERIMENT
1. Apparatus
In our experiments, we used a phone based portablerobot called “TenoriAvatar” (Figure 1), a modified
version of “BlogRobot” [4]. TenoriAvatar is HTC
Corporation's mobile phone HTC P3600, upon which a
robotic head and robotic arms are installed.
In the experiments, we used simple linear scenarios
which were composed only of sentences for utterances
and tags for gestures and images. Thus, the scenarios
had no branches. When a scenario is selected,
TenoriAvatar loads the scenario and utters sentences inturn. If TenoriAvatar loads a gesture tag in a sentence,
TenoriAvatar does the gesture associated with theloaded tag right there. The gestures come in twelve
types including happy, unhappy, bye-bye, wow, andwalking. In addition, when TenoriAvatar is not doing a
gesture, it blinks at fixed intervals to show the system isrunning.
2. Behavior system
The user feeds conscious response by pushing a feed
button set up on the display of the mobile phone. Only
two types of conscious response can be fed: positive andnegative. The arrangement of feed buttons is shown in
Figure 3. The lower left button is the positive feed
button, and the lower right is the negative feed button.
When the user sympathizes with the content or finds acontent funny, s/he pushes the positive feed button.
When the user does not sympathize with the content orfinds the content anemic s/he pushes the negative feed
button.

Fig.3. Arrangement of feed buttons
When the user pushes a feed button, TenoriAvatar
plays the sound and stops the playback and thenperforms a response behavior for the fed conscious
response. We believe that a break in the playback gives
the user the impression that the robot responded to the582The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

fed conscious response. The response behavior is arandom combination selected from a gesture list and anutterance list that are prepared based on type of
conscious response. The list consists of suitableutterances and gestures for the type of feed button.
Thus, a response behavior that is suitable for the feed
button that the user pushed is executed. After the
response behavior, TenoriAvatar utters a sentence
selected from the utterance list for restarting the
playback scenario. When restarting, the scenario isplayed back from beginning of the line of theinterrupted sentence. To prevent the same sentence frombeing uttered over and over, if a user already fed
response at the sentence, the response behavior is not
executed when s/he feeds conscious response again.
Therefore, the response behavior is executed only once
for each sentence. If conscious response was fed
between the end of the utterance of one sentence and thestart of the utterance of the next sentence, the response
is contained within the sentence which was uttered atthat time. In this case, response behavior is executed,
but TenoriAvatar will not play back the uttered sentence
and will start playing back from the next sentence.

An experimenter handed TenoriAvatar with orwithout the response behavior to a participant, and theparticipant played back the scenario about our boothand fed conscious response. An experimenter
randomized participants to receive TenoriAvatar with orwithout it, and to the extent possible, kept the number ofexperiments with or without the response behavior thesame.
B. Evaluation of Motivation
The experiment on conscious response input
motivation was conducted at our university laboratory.
Fourteen Japanese engineering students (20 - 29 years
old) participated. Participants played back the two
scenarios with and without the response behavior. We
investigated the effect of the response behavior on the
motivation to provide conscious response input from thequestionnaires that participants filled out after theexperiment.
Participants played back two scenarios that
explained a college cafeteria and a local character inJapan with TenoriAvatar. One scenario was with the
response behavior, and the other was with only sound.
To avoid the order effect, the order of scenarios with the
response behavior and with only sound was changed forevery participant. The order of the scenarios remained
the same. After the playbacks, the experimenter had theparticipants answer questionnaires about theirimpressions of the first and second playbacks.
Evaluation items were about how meaningful they felt
their input conscious response was, pleasantness of theprocess, and whether they would use continuously inseven levels.

3. Evaluation of Response Behaviors
Conscious response function attempts to help theuser to feel meaningful in input conscious response. We
conducted two experiments to observe the effects on thefrequency and range of conscious response inputs andon how meaningful users thought their response was.
The playback time for each scenario was about two
minutes, and each scenario had about ten gesture tags.
Each participant held TenoriAvatar in one hand at about
50-cm distance from them. Before the experiment, theparticipants received a briefing from an experimenter onTenoriAvatar and on the response input method.
Participants did not receive an explanation about the
response behavior, regardless of the function of the
response behavior.
A. Evaluation of Frequency
The experiment on conscious response input
frequency was conducted at our university festival.
Fifty-eight Japanese people (10 - 59 years old) who
came to view our booth participated. Participants playedback the scenario with TenoriAvatar with or without
response behavior and fed conscious response. The
content of the scenario was about expressing asentiment about the robot, the device, and TenoriAvatar,
which were displayed in our booth.

©ISAROB 2011

III. RESULTS
Figure 4 plots the mean number of conscious
responses in the experiment on the response input
frequency. In the response behavior condition, the mean
number of positive feedback responses was 9.19,
negative feedback was 2.90, and the sum total was
12.09. In only the sound condition, the mean number of
positive feedback responses was 7.50, negative
feedback was 1.04 and the sum total was 8.54. There
were no significant differences in mean number of
positive feedback responses (p=.44) or in the total
conscious feedback (p=.18). There was a significant
difference in the mean number of negative feedback
responses (p=.02).

583

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

VI. DISCUSSION AND CONCLUSION
This paper reported our investigation of the effect of
response behaviors on the acquisition of the conscious
responses. The results of the first experiment indicatedthat the existence of the response behavior enhanced theacquisition of negative feedback. These data indicate thepossibility of increasing the number of fed conscious
responses. Without response behavior, many users
pressed the negative feedback button only once. With
response behavior, the negative feedback button waspressed more than two times. The result indicates thatthe input of negative feedback is not done casually likepositive feedback is, and response behavior makes it
easier to input negative feedback. From the results ofthe second experiment, the existence of response
behavior had an effect on significance and enjoyment.
We believe that the result indicate that frequency ofacquiring conscious responses motivates the scenario
author.
The results indicate that there are definite relation
between response behavior and the acquisition of the
conscious response. In obvious, the methods using only
positive and negative feedback have limitations. We
must therefore think of more appropriate ways to obtain
conscious responses. If the robot can obtain the richcontent of the conscious response, the robot can behave
in a very adaptive way.

Fig.4. Mean number of input responses with SE
Figure 5 shows the mean number of sentences inwhich positive or negative feedback was fed. In the
response behavior condition, the mean number ofsentences was 7.38. In only the sound condition, themean number of sentences was 5.29. There was adifference in the mean number of sentences in which
feedback was fed (p=.07).

Fig.5. Mean number of sentences with SE
Figure 6 shows the questionnaire results from theexperiment on conscious response input motivation. In
the response behavior condition, the average mark of
meaningfulness in conscious response input was 5.53,
pleasantness was 6.00, and will use continuously was
5.13. In only the sound condition, the average mark of
meaningful in conscious response input was 3.20,
pleasantness was 4.98 and will use continuously was
4.27. There was a significant difference inmeaningfulness (p=.01) in conscious response input andpleasantness (p=.02). There was no significant
difference in will use continuously (p=.14).

REFERENCES[1] Kuno, Y., Sadazuka, K., Kawashima, M., Yamazaki,
K., Yamazaki, A., and Kuzuoka, H. (2007). Museum
guide robot based on sociological interaction analysis.
In Proceedings of CHI ’07, 1191-1194.
[2] Satake, S., Kanda, T., Glas, D., Imai, M., Ishiguro,
H., and Hagita, N. (2009). How to Approach Humans? Strategies for Social Robots to Initate Interaction. In In
Proceedings of HRI ‘09, 109-116.
[3] Lee, M. K., Kiesler, S., and Forlizzi, J. (2010).
Receptionist or information kiosk: how do people talkwith a robot? In Proceedings of CSCW ’10, 31-40.
[4] Osumi, T., Fujimoto, K., Kuwayama, Y., Noda, M.,
Osawa, H., Imai, M., and Shinozawa, K. (2009).
Blogrobot: Mobile terminal for blog browse using
physical representation. In International Conference on
Social Robotics, Vol44, 96-101.
[5] Investigation research of realities of blogs
Institute for information and communications policy
http://www.soumu.go.jp/iicp/chousakenkyu/data/researc
h/survey/telecom/2009/2009-02.pdf (in Japanese)[6] Nico Nico Douga. http://www.nicovideo.jp

Fig.6. Questionnaire results with SD©ISAROB 2011584

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Cross-modal effects between gestures and words in human robot
interaction
Takamasa Iio1,2, Masahiro Shiomi1, Kazuhiko Shinozawa1, Takaaki Akimoto1,
Katsunori Shimohara2 and Norihiro Hagita1
1

Advanced Telecommunications Research Institute International, Kyoto, JAPAN
2
Doshisha University, Kyoto, JAPAN(Tel : 81-774-95-1405; Fax : 81-774-95-1408)(iio@atr.jp)

Abstract: This paper reports a new finding of a phenomenon that person's gestures or words are implicitly modified byrobot's gestures or words. Previous researches focused on an implicit effect of robot's gestures on person's gestures oran implicit effect of robot's words on person's words, but they did not focused on an implicit effect of robot's gestureson person's words or an implicit effect of robot's words on person's gestures. We supposed that there was such an effectas to arise between different modalities, and we defined it as a cross-modal effect. In order to verify hypotheses about
the cross-modal effect, an experiment was conducted, in which a pair of a pointing gesture and a deictic word was
focused on. This result showed that participants used a pointing gesture more often when a robot used a deictic word,
and participants used a deictic word more often when the robot used a pointing gesture. Therefore, person’s pointing
gesture was implicitly modified by robot’s deictic word, and also person’s deictic word was implicitly modifie
d by robot’s pointing gesture. The cross-modal effect is expected to be applied to robot's dialog design to elicit
comprehensible behavior from a person.
Keywords: Entrainment, Cross-modal effects, Multi-modal interaction, Human-robot interaction.

person. Therefore, we should understand how to modify
person’s behavior by robot's behavior.
The effect between each similar modality like theentrainment has been researched in human robot
interaction. However, we have to consider not only theeffect but also the effect between each different
modality because all modalities are not independent
respectively but some modalities, especially gestures
and words are dependent on each other [4, 5]. If there is
the effect, the robot could effectively elicit a certain
gesture or a word from the person by using the effectwith the entrainment.
We defined the effect between different modalities
as cross-modal effect. Based on the entrainment
between similar modalities and the mutual dependenceof gestures and words, we supposed that there was theI. INTRODUCTION
Social robots that support people in daily life are
expected to communicate with them in humanlike
manners such as body movements or voices. That is
because even people who do not use computers well
could smoothly converse with the robots as if they
converse with other people. For achieving naturalcommunications, it is important to research humanrobot interaction [1]. One of purposes of the research isto understand how people communicate with the robots.
When a person communicates with a robot, aninteresting phenomenon called entrainment often arises.
This is the phenomenon that the person’s gestures aresynchronized with the robot’s gestures or the person’s
words are synchronized with the robot’s words asshown in Fig.1. For example, Ono et al. reported that
person’s gestures became similar with robot’s gesturesin a route direction conversation [2]. Iio et al. showedthat when a person and a robot repeatedly referred to thesame objects, the person tend to use the same words as
the robot used [3]. By using the entrainment, a robot
could implicitly elicit a certain behavior from a person.
Such elicitation would make it possible for the robot toimprove its recognition capability because the robot
could elicit comprehensible gestures or words from the©ISAROB 2011

[Ogawa & Watanabe 2001][Ono et al. 2001]

GesturesGesturalentrainment

GesturesResearch objective

Cross-modal Effect
Words

Lexicalentrainment

Words[Iio et al. 2009]

Fig.1. Entrainment and cross-modal effect.

585

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

following cross-modal effect: We assume that a gestureand a word are used together at the same time. If a robot
often uses the gesture, a person would often use thesimilar gesture too; this phenomenon is gestural
entrainment. Since the gesture tends to be used with theword, the person is also likely to use the word often.
This means a cross-modal effect of the robot's gestureon the person's word. Based on the above logic, we canconsider a cross-modal effect of the robot's word on the
person's gestures.
In this paper, we supposed that there was the effectbetween different modalities. We defined it as crossmodal effect. The hypotheses about the cross-modal
effect were made and verified through a laboratory
experiment. Finally, a process of the cross-modal effectwas discussed.

found in many households; moreover, books involve thevarious referential expressions, such as title, color,category, author and location.
3. Hypotheses
We made the following hypotheses about a crossmodal effect during the object reference conversation.
H1: When the robot uses a deictic word, the rate of hisor her pointing gesture is high.
H2: When the robot uses a pointing gesture, the rate of
his or her deictic word is high.
4. Conditions
We controlled robot’s pointing gesture and robot’s
deictic word used to confirm a book. The experiment
had four conditions listed in Table 1. The detail of eachcondition was as follows.
PD: The robot turned its face on a book and pointed at
the book, saying “Sore desuka?” (In English, That one?)
PnD: The robot only turned its face on a book but kept
its arms stationary in the side of its body, saying “Soredesuka?”PDn: The robot turned its face on a book and pointed at
the book, saying the book title.
PnDn: The robot only turned its face on a book but kept
its arms stationary in the side of its body, saying thebook title.

II. EXPERIMENT
1. Target gesture and word
This experiment aimed at a pair of a pointing gestureand a deictic word because they are coupled with eachother. The deictic word cannot identify an object byitself; therefore they are likely to be used with apointing gesture.
2. Experimental conversation
In order to introduce a pointing gesture and a deictic
word, we employed an object reference conversation in
the experiment. The conversation was as follows; therobot asked a participant to choose an object and theparticipant chose an object. Then, the robot confirmed
the object. If the confirmation was correct, theparticipant indicated another object; otherwise theparticipant indicated the same object again.
We adopted books as the objects because books are

5. Measurement
We measured the pointing gesture rate and thedeictic word rate to verify our hypotheses.
The pointing gesture rate: We counted how manytimes a participant did references with a pointing
gesture and verified the rate is changed by the lexical
factor.
The deictic words rate: We counted how many times
participants did references with a deictic word andverified the rate is changed by the gestural factor.

Table 1. The experimental conditions.
6. Experimental environment
Fig. 2 depicts our experiment. A participant wasseated in front of the robot. The robot was Robovie-Rver.2, which is a humanoid robot developed by theIntelligent Robotics and Communication Labs, ATR.
There were five different books between the participant
and the robot.

Gestural factor
Pointing

No Pointing

LexicalDeicticPD

PnD

factorNo deictic

PDn

PnDn

7. Procedure
A participant was first given a brief description of
the purpose and the procedure of the experiment. We
told the participant that we were developing a robot for

Fig.2. The pictures of our experiment.

©ISAROB 2011586

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

The result says there was gestural entrainment of robot’s
pointing gesture on participant’s pointing gesture.
Main effect of the lexical factor: The pointing
gesture rate of PD and PnD was higher than that of the
PDn and PnDn. The difference was significant (x2(1) =12.719, p < 0.01). Therefore, when the robot used a
deictic word, participants tended to use a pointing
gesture. The result says there was the cross-modal effectof robot’s deictic word on participant’s pointing gesture.
That is to say, the result supports our hypothesis 1.
Interaction between the gestural factors and thelexical factors: There was no significant interaction
between these factors.

recognizing an object and would like their help in
evaluating the design. Then, the participant was
assigned randomly to the four conditions. A participantreferred five books three times, that is to say, theparticipant did 15 references. Therefore, we totallyobtained 90 references in each condition.
The robot was controlled remotely by an operator,that is to say, our experiment employed the Wizard of
Oz method. That was because the difficulty ofrecognizing participant’s pointing gesture andparticipant’s voice automatically. Considering robot’s
recognition capability expected in the future, theoperator rejected participant’s references which did notpass the following rules.
- Reference by bibliographical information.
- Reference by attributions able to identify each book.
- Reference by pointing a finger at a book.

2. Deictic word rate
The deictic word rate of each condition is shown in
Fig.4. The analysis found out there was significant
interaction between the gestural factor and the lexical
factor (x2(1) = 7.209, p < 0.01). We describe the detailof the interaction.
Simple main effect of the gestural factor: The
deictic word rate of PDn was significantly higher thanthat of PnDn (x2(1) = 12.216, p < 0.01), but the deictic
word rate of PD was similar with that of PDn. Therefore,
when the robot used a pointing gesture without a deictic
word, participants tended to use a deictic terms. The
result says there was partially the cross-modal effect ofrobot’s deictic word on participant’s pointing gesture.
That is to say, the result partially supports our
hypothesis 2.
Simple main effect of the lexical factor: The
deictic word rate of PnD was significantly higher thanthat of PnDn (x2(1) = 14.417, p < 0.01), but the deictic
word rate of PD was similar with that of PnD. Therefore,
when the robot used a deictic word without a pointing
gesture, participants tended to use a deictic terms. The
result says there was partially lexical entrainment ofrobot’s deictic word on participant’s deictic word.

8. Participants
There were 24 participants in the experiment. They
were native-Japanese-speaking university students from
Kansai area. Since they were assigned randomly to thefour conditions, each condition has six participants.

III RESULTS
In order to analyze main effects and interaction of
the gestural factor and the lexical factor, we did theanalysis of variance using chi-square distribution basedon the arcsine transformation method. This analysis cantest the difference between proportions of unpaired two
factors.

1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0

PD

PDn
Deictic word rate

Pointing gesture rate

1. Pointing gesture rate
The pointing gesture rate of each condition is shownin Fig.3. The analysis is as follows.
Main effect of the gestural factor: The pointing
gesture rate of PD and PnD was higher than that of PDn
and PnDn. The difference was significant (x2(1) = 11.017,
p < 0.01). Therefore, when the robot used a pointing
gesture, participants tended to use a pointing gesture.

Robot’s gesture factors
Pointing

PnD
PnDn

No Pointing

PD

PnD

Robot’s word factors
Deictic
No Deictic

PDn
PnDn
Pointing
No Pointing
Robot’s gesture factors

Deictic
No Deictic
Robot’s word factors

Fig.4. Deictic word rate of each condition.

Fig.3. Pointing gesture rate of each condition.

©ISAROB 2011

1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0

587

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

IV. DISCUSSION

a gesture and a word are coupled, it might be possible
for the cross-modal entrainment to arise in the other pair.

1. The process of the cross-modal effectAt first, we investigate whether a pointing gestureand a deictic word was used with each other. We
grouped person's references into (A) a reference withboth a pointing gesture and a deictic word, (B) areference with a pointing gesture and without a deictic
word and (C) a reference with a deictic word andwithout a pointing gesture. These rates were A = 0.625,
B = 0.346 and C = 0.029, and they were significantly
different (x2(1) = 149.621, p < 0.01). According to a
multiple comparison, the rate of A was higher than thatof B or C. Therefore, a pointing gesture and a deictic
word are more likely to be used together.
Next, we consider the process of the cross-modal
effect of robot’s deictic word on participant’s pointing
gesture. According to the results in the section 4.2, there
was partially lexical entrainment of robot’s deictic wordon participant’s deictic word. Considering that apointing gesture and a deictic word tended to be used
together, we can say that, if only partially, the crossmodal entrainment arose from the mutual dependence ofthe pointing gesture and the deictic word and lexical
entrainment.
Finally, we consider another process of the crossmodal effect of robot’s pointing gesture on participant’s
deictic word. According to the results in the section 4.1,
there was gestural entrainment of robot’s pointing
gesture on participant’s pointing gesture. Like the above
consideration, considering that a pointing gesture and a
deictic word tended to be used together, we can say thatthe cross-modal entrainment arose from the mutual
dependence of the pointing gesture and the deictic wordand gestural entrainment.

V. CONCLUSION
This paper defined an effect of one robot's modality
on another person's modality as cross-modal effect. We
supposed that if a gesture and a word are more likely to
be used together, the cross-modal effect would arise
through gestural entrainment or lexical entrainment, andwe investigated that through a laboratory experiment.
The experiment focus on a pointing gesture and a
deictic word during a conversation that a person and arobot refer to objects and investigated how the rate ofperson's pointing gestures and that of person's deictic
words were changed by the combination of robot's
pointing gestures and robot's deictic word. The
experimental results were as follows; (1) a person used
pointing gestures more often when the robot used
deictic words; (2) a person used deictic words more
often when the robot used pointing gestures.
In the cross-modal effect, a robot's modality elicits
a similar person's modality by entrainment and theperson's modality riggers another coupled modality.
Therefore, when we design robot's behavior for
maneuvering person's behavior, we should take accountof not only the entrainment but also the cross-modal
effect. We believe that this knowledge is useful for
designing new conversational strategies for a robot.

REFERENCES[1] Ishigro H, Ono T, Imai T, Maeda T, Kanda T and
Nakatsu, R (2001), Robovie: An interactive humanoid
robot, International Journal Industrial Robotics,
28(6):498-503.
[2] Ono T, Imai M and Ishiguro H (2001), A Model ofEmbodied Communications with Gestures between
Humans and Robots, Proc. 23rd Annual Meeting of the
Cognitive Science Society 732-737.
[3] Iio T, Shiomi M, Shinozawa K, Miyashita T,
Akimoto T and Hagita N (2009), Lexical Entrainment in
Human-Robot Interaction: - Can Robots Entrain Human
Vocabulary? -", IEEE/RSJ International Conference onIntelligent Robots and Systems, 3727-3734.
[4] Kita, S (1998), Expressing turns at an invisible
location in route direction: the interplay of speech
and body movement, Sign-space Raum zeichen, 160-172.
[5] Kita, S (2000), How representational gestures he
lp speaking, Language and Gestures: Window into
Thought and Action, 162-185.Casti JL (1995), Bellcurves and monkey languages. Complexity 1:12-15

2. Application of the cross-modal effectWe can consider to applying the cross-modal effectto a design of robot’s behavior. For example, a robot
could raise the possibility that a person use a pointing
gesture by the gestural entrainment and the cross-modal
effect. This helps the robot to recognize a referred-to
object correctly because there are various objects in thereal environment and sometimes the robot cannot
identify each object only with words.
Although the experiment aimed at a pointing gestureand a deictic word, the results suggest that the crossmodal effect would arise from the mutual dependence ofa gesture and a word and the entrainment. Therefore, if©ISAROB 2011

588

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Adaptive co-construction of state and action spaces in reinforcement learning
Masato Nagayoshia , Hajime Muraob , and Hisashi Tamakicab

Niigata College of Nursing, 240, Shinnan, Joetsu 943-0147, Japan
nagayosi@niigata-cn.ac.jp

Faculty of Cross-Cultural Studies, Kobe Univ. 1-2-1, Tsurukabuto, Nada-ku, Kobe 657-8501, Japanmurao@i.cla.kobe-u.ac.jpcGraduate School of Engineering, Kobe University, Rokko-dai, Nada-ku, Kobe 657-8501, Japan
tamaki@al.cs.kobe-u.ac.jp

Abstract

lem of designing state and action spaces involves thefollowing two requirements: (i) to keep the characteristics (or structure) of an original search space as muchas possible in order to seek strategies that lie close
to the optimal, and (ii) to reduce the search space asmuch as possible in order to expedite the learning process. These requirements are, in general, in conﬂict.
Until now, we have proposed an adaptive state
space construction method which is called “state spaceﬁlter[2]” and an adaptive action space construction
method which is called “switching learning system[3]”, after the other space has been ﬁxed. In this paper, we reconstitute these two construction methods
as one method by treating the former method and thelatter method as the combined method for mimicking
infants’ perceptual and motor developments respectively. The proposed method is to construct state andaction spaces adaptively by introducing and referringto the “entropy” as indexes of both necessity for division of the state space in the state and suﬃciency forthe number of learning opportunities in the state. Further, a computational experiment was conducted byusing a so-called “path planning problem” with continuous state and action spaces.

Reinforcement Learning (RL) attracts much attention as a technique of realizing computational intelligence such as adaptive and autonomous decentralized
systems. In general, however, it is not easy to put RL
into practical use. This diﬃculty includes a problem ofdesigning suitable state and action spaces of an agent.
Until now, we have proposed an adaptive state
space construction method which is called “state
space ﬁlter” and an adaptive action space construction method which is called “switching RL”, after theother space has been ﬁxed. In this paper, we reconstitute these two construction methods as one methodby treating the former method and the latter method
as the combined method for mimicking infants’ perceptual development in which perceptual diﬀerentiation progresses as infants become older and more experienced, and infants’ motor development in whichgross motor skills develop before ﬁne motor skills respectively. Then the proposed method is based on
introducing and referring to the “entropy”. Further,a computational experiment was conducted by using
a so-called “path planning problem” with continuous
state and action spaces. As a result, the validity ofthe proposed method has been conﬁrmed.

2
1

Introduction

2.1

Engineers and researchers are paying more attention to reinforcement learning (RL)[1] as a key technique of realizing autonomous systems. In general,
however, it is not easy to put RL into practical use.
Such issues as satisfying the requirement of learning
speed, resolving the perceptual aliasing problem, anddesigning reasonable state and action spaces of anagent, etc. must be resolved. Our approach mainly
deals with the problem of designing state and action
spaces. By designing suitable state and action spaces
adaptively, it can be expected that the other two problems will be resolved simultaneously. Here, the prob-©ISAROB 2011

Typical RL Methods
Q-learning

Q-learning works by calculating the Quality of astate-action combination, namely Q-value, that gives
the expected utility of performing a given action in agiven state. By performing an action a ∈ AQ , where
AQ ⊂ A is the set of available actions in Q-learning
and A is the action space of the agent, the agent canmove from state to state. Each state provides theagent a reward r.
The Q-value is updated according to the following
formula, when the agent is provided the reward :Q(s(t-1), a(t-1)) ← Q(s(t-1), a(t-1))

589

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

+αQ {r(t-1) + γ max Q(s(t), b) − Q(s(t-1), a(t-1))}(1)b∈AQ

SwitchingmoduleDiscrete action spacelearning module N

where Q(s(t-1), a(t-1)) is the Q-value for the state andthe action at the time step t-1, αQ ∈ [0, 1] is the learning rate of Q-learning, γ ∈ [0, 1] is the discount factor.
The agent selects an action according to thestochastic policy, π(a|s), which based on the Q-value.
π(a|s) speciﬁes probabilities for taking each action ain each state s. Boltzmann selection, which is one of
the typical action-selection methods, is used in thisresearch. Therefore, the policy π(a|s) is calculated asfollows:
exp(Q(s, a)/τ )π(a|s) = ∑(2)
exp(Q(s, b)/τ )b∈A
where τ is a positive parameter.

2.2

Agent
X
State space filter

Discrete action spacelearning module 1Sx

s=f(x)sa=g(s)

Aa

Switching learning systemx

Systema

Figure 1: Proposed developing RL model.
error could be 0 or a negative value. So, it is necessary
for the setting of the standard error to be creative to
specify the range, etc.

Actor-Critic

Actor-Critic methods have a separate memory
structure to explicitly represent the policy independent of the value function. The policy structure iscalled “Actor”, which selects actions, and the estimated value function is called “Critic”, which criticizes the actions made by the Actor. The Critic is astate-value function. After each action selection, theCritic evaluates the new state to determine whether
things have gone better or worse than expected. That
evaluation is TD-error:
δ(t-1) = r(t-1) + γV (s(t)) − V (s(t-1))
(3)

3
3.1

Developing RL
Outline of a Computational ModelIn this section, we propose developing RL model
to mimic processes of infants’ perceptual and motor
developments. The proposed model is constructed by“state space ﬁlter[2]” to mimic a process of perceptual
development in which perceptual diﬀerentiation progresses as infants become older and more experienced
and “switching learning system[3]” to mimic a process
of motor development in which gross motor skills develop before ﬁne motor skills, as shown in Fig. 1.
This model mimics the process of perceptual development by diﬀerentiating the state space graduallyfrom the undiﬀerentiated state space. In parallel, thismodel mimics the process of motor development byswitching discrete action space learning modules (hereafter called “DA module”) from more coarse-grained
DA module to more ﬁne-grained DA module, and ﬁnally switching to a continuous action space learning
module (hereafter called “CA module”).

where V (s) is the state Value.
Then, V (s(t-1)) is updated according to Eq. (4)in the Critic, based on this δ(t-1). In parallel, it is
updated for the stochastic policy, π(a|s), in the Actor.
V (s(t-1)) ← V (s(t-1)) + αC δ(t-1)(4)
where αC ∈ [0, 1] is the learning rate of the Critic.
It is typical for the normal distribution to be used,shown in Eq. (5), as the stochastic policy in the Actor,
when Actor-Critic is applied to a continuous action
space.In this case, both the mean the mean µ(s) andthe standard error of the mean σ(s) about the normal
distribution are calculated using TD-error δ(t-1) in the
Actor, as Eq. (6),(7).
1−(a − µ(s))2√ exp(
π(a|s) =)(5)
2σ(s)2σ(s) 2πµ(s(t-1)) ← µ(s(t-1)) + αµ δ(t-1)(a(t-1) − µ(s(t-1)))(6)
σ(s(t-1)) ← σ(s(t-1))+ασ δ(t-1)((a(t-1) − µ(s(t-1)))2 − σ(s(t-1))2 )(7)

3.2State and Action Spaces Construction
Method

3.2.1 Basic Idea
A variety of methods to acquire the state space ﬁlter
and to switch learning module can be considered. In
this paper, we propose a method based on introducing and referring to the “entropy”, which is deﬁned
on action selection probability distributions in a state,
and the number of learning opportunities in the state.
It is expected that the proposed method (i) is able tolearn in parallel the state space ﬁlter and the switching learning system, (ii) does not required speciﬁc RL
methods for the learning module.

where αµ ∈ [0, 1], ασ ∈ [0, 1] are the learning rate of
the mean and the standard error of the mean respectively. Here, if Eq. (7) is used directly, the standard©ISAROB 2011

Continuous action spacelearning module

590

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

(0,0)

The entropy of action selection probability distributions using Boltzmann selection in a state, HD (s),is deﬁned by∑
HD (s) = −(1/ log |AD |)π(a|s) log π(a|s) (8)a∈AD
where AD is the action space and |AD | is the numberof available actions of the DA module.
The state space ﬁlter is adjusted and the learning
module is switched by treating this entropy H(s) as
an index of necessity of division for an inner state s
and the action space. In parallel, the learning moduleis switched by treating this entropy H(s) as an index
of suﬃciency for the number of learning opportunities
in the state.
If the entropy does not get smaller despite being thelearning module learned a suﬃcient number of opportunities in the inner state, then the state space ﬁlter isadjusted by dividing the inner state and the learning
module is switched to more ﬁne-grained one. In contrast, if the entropy get small regardless of the number of learning opportunities, the learning module isswitched to the CA module due to the number of learning opportunities being suﬃcient.
In this paper, Q-learning and Actor-Critic are applied to the DA module and the CA module respectively. The learning module is switched in the order of
Q-learning with an action space divided evenly inton, 2n, · · · , 2(N −1) n, ﬁnally ending with Actor-Critic,
where N is a number of DA modules.

start

450
goal
450 (500,500)

Figure 2: Path planning problem.
portunities being suﬃcient. In the procedure to switch
controllers, the result of Q-learning is succeeded by
Actor-Critic. The following procedure is conducted :i) the state value of the Critic,V (s), is initialized by∑
V (s) =
π(a|s) · Q(s, a)(9)a∈AQii) the normal probability distribution used by the Actor is calculated byµ(s) = arg max Q(s, a),
(10)
a∈AQ
σ(s) = |AQ (arg max Q(s, a))|/6
(11)
a∈AQ
where |AQ (i)| is a range of the action space which
represents action i of Q-learning.
If L(s) > θL and H(s) > θH , then the learning
module is switched to more ﬁne-grained DA module,
and ﬁnally ending with to the CA module. Simultaneously, the state space ﬁlter is adjusted.
Q-values of actions newly added ai at this time
are set according to the following formula : Q(s, i) =maxj∈i−1,i+1 Q(s, j) where action i − 1 and i + 1 areadjacent to action i. This formula is set in consideration of a more eﬃcient search as well as the idea ofthe optimistic initial values.

3.2.2 Adjustment of State Space Filter
If L(s) > θL and H(s) > θH , where L(s) is thenumber of learning opportunities in s, θL is a threshold value of the number of learning opportunities, θH
is a threshold value of the entropy, and θL is set at
a suﬃciently big number, then the state space ﬁlter is adjusted by dividing a range of the input state
mapped to the inner state s into 2 parts for each dimension, and mapping each part to a diﬀerent innerstate respectively. Simultaneously, the learning module is switched. Through this operation, a size of theinner state space after divided increases by (2M − 1),
where M is a number of dimension. Also note thatthe values of the new 2M inner states are the value ofthe inner state before divided.
In addition, after the learning module is switchedto the CA module, if L(s) > θL , then the state spaceﬁlter is adjusted by dividing the inner state to be more
ﬁne-grained.

Computational Example

4.1

Path Planning Problem

The proposed method is applied to a so-called “pathplanning problem” where an agent is navigated froma start point to a goal area in a continuous space
as shown in Fig. 2. Here, the agent has a circularshape (diameter = 50[mm]), and the continuous space
is 500[mm] × 500[mm] bounded by the external wallwith internal walls as shown in black. The agent canobserve the center position of the agent: (xA , yA ) as
the input, and move 25[mm] in a direction, i.e., decide
the direction: θA as the output.
The positive reinforcement signal rt = 10 (reward)
is given to the agent only when the center of the agent
arrives at the goal area and the reinforcement signal
rt = 0 at any other steps. The period from when theagent is located at the start point to when the agent
is given a reward, labeled as 1 episode, is repeated.

3.2.3 Switching of Learning Module
If H(s) > θH , then the learning module is switchedto the CA module due to the number of learning op-©ISAROB 2011

4

591

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

4.2

Comparison to Adaptive Methods

10000

average steps

We have conﬁrmed that a combined method of thestate space ﬁlter and the switching learning system(hereafter called method “FS”) demonstrates better
performance than three Q-learning methods with theaction space divided evenly into 4, 8 and 16 ,and two
Actor-Critic methods with the state space divided
evenly into 10 × 10 and 40 × 40 in this task.
In this section, method FS is compared with two
methods using the switching learning system with thestate space divided evenly into 10 × 10 and 40 × 40
spaces (hereafter called method “S10” and “S40” respectively), Actor-Critic using the state space ﬁlter
and Q-learning with the action space divided evenly
into 4 spaces using the state space ﬁlter (hereafter
called method “FAC” and “FQ4” respectively). Here,the initial state space ﬁlter is designed that divides thestate space evenly into 10 × 10 spaces.
Then, the entropy of a continuous action space in astate for method FAS, HC (s), is deﬁned by∫ ∞HC (s) = −π(a|s) log π(a|s)da.
(12)

S40
100

FS

FQ4

FAC

S10

10
0

500 1000 1500 2000 2500 3000 3500 4000 4500 5000
episodesFigure 3: Required steps.
lations with various methods as described in Fig. 3.
Learning speed and obtained control rule : It can be
conﬁrmed from Fig. 3 that, 1) method FS have worse
performances than method FQ4, FAC and S10 , butbetter performances than method S40 with regard to
the learning speed, 2) method FS has good performance as well as method FAC and S40 with regard to
the obtained control rule,
Therefore, we have conﬁrmed that method FAC andmethod FS, in that order, demonstrate better performance than any other method on the path planningproblem with the continuous state and action spaces.

−∞

By substituting
the Eq. (5) into this formula, HC (s) =√log( 2πeσ). In method FAS, if HC (s) < θHC , then thestate space ﬁlter is adjusted.
All initial values and the range of σ(x) are set at π
and [0.001, 2π] respectively, all initial means are set torandomize within a range of [−π, π] for Actor-Critic.
Then, all initial state values and Q-values are set at
5.0 as the optimistic initial values[1] for Actor-Critic
and Q-learning respectively. Here, the initial values
and the maximum limit of σ(x) are set so that ±1σ
and the maximum limit become the size of the action
space: 2π. Further, the adjustment of the state spaceﬁlter is assumed until the third attempt in all innerstates because it is impossible to evaluate suﬃciency
for division of the state space.
Computer experiments have been done with parameters as shown in Table 1. Here, θHD was set referring
to about 0.312 : the maximal value of the entropy
when the highest selection probability for one action
is 0.9, θHC was set referring to about 0.335 : the entropy when the standard error σ is π/6, θL was set inconsideration of the enough big number.
The number of average steps required to accomplish
the task was observed during learning over 20 simu-

5Conclusion

In order to design the suitable state and action
spaces adaptively, we propose, in this paper, the developing RL model, and state and action spaces construction method referring to the “entropy”. Then,through the computational experiment, we have conﬁrmed that the combined method of the state spaceﬁlter and Actor-Critic, and the combined method of
the state space ﬁlter and the switching the learning
system, in that order, demonstrate better performance
than any other method on the path planning problem
with the continuous state and action space.
Our future projects include to apply more complicated problems and real world problems, etc.
Acknowledgment
This work was supported in part by a Grant-in-Aid for Young
Scientists (B), (No. 21700258), from MEXT, Japan.

References[1] R.S. Sutton and A.G. Barto: Reinforcement Learning, A
Bradford Book, MIT Press (1998).
[2] M. Nagayoshi, H. Murao, and H. Tamaki: A State Space
Filter for Reinforcement Learning, Proc. AROB 11th’06,
615-618(GS1-3) (2006).
[3] M. Nagayoshi, H. Murao and H. Tamaki: A Reinforcement Learning with Switching Controllers for Continuous
Action Space, Proc. the 15th International Symposium on
Artiﬁcial Life and Robotics 2010, 236-239 (2010).

Table 1: Parameters for experiments
Parameter
Value Parameter ValueαQ , αC , αµ , ασ
0.1
γ
0.9
θ HD , θ HC
0.3
τ
0.1
1000
θL

©ISAROB 2011

1000

592

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Autonomous acquisition of cooperative behavior based on a theory of mind
using parallel Genetic Network Programming
Kenichi Minoya1, Takaya Arita2 and Takashi Omori3
1, 2

Graduate School of Information Science, Nagoya University
Furo-cho, Chikusa-ku, Nagoya 464-8601, JAPAN3
College of Engineering, Tamagawa University
6-1-1 Tamagawa Gakuen, Machida, Tokyo 194-8610, JAPANE-mail: 1kenichiminoya@alife.cs.is.nagoya-u.ac.jp, 2arita@nagoya-u.jp, 3omori@lab.tamagawa.ac.jp

Abstract: Understanding of others as having intentional states such as beliefs and desires is called Theory of Mind
(ToM). To clarify the mechanism of the autonomous acquisition of cooperative behavior based on the ToM, weconstructed a functional model of the brain based on the Functional Parts Combination (FPC) model. This model
consists of a set of functional parts and activation signals specifying selective activated patterns, and activated modules
can be executed in parallel based on the flow of control tokens. The module network and activation signals can beacquired by the evolutionary computation techniques used in the Genetic Network Programming and Genetic
Algorithm, respectively. We use a hunter task as a task to be solved by the agents, and encode inherent activation
signals into the genome as a first step. The result of computer simulation shows an emergence of the pattern of thefunctional parts for processing ToM through evolution characterized by punctuated equilibrium.
Keywords: Functional Model of the Brain, Theory of Mind, Cooperation, Evolution, Genetic Network Programming.
[4] [5]. However, functions of ToM in these studies areprocedurally defined a priori by the designers.
We focus on the emergence of a ToM without
defining it a priori by modeling the brain at thefunctional level. Next section explains a functionalmodel of the brain and Section 3 illustrates a task andcomponents of the brain. Section 4 shows theexperiments and Section 5 summarizes the paper.

I. INTRODUCTION
Humans are extremely social animals. One aspect ofsocial cognition sets us apart from other primates:
Theory of Mind (ToM). It enables us to understandothers as having intentional states such as beliefs anddesires [1]. The evolutionary origins of it can be traced
back in extant non-human primates; ToM probably
emerged as an adaptive response to increasingly
complex primate social interaction [2]. The aim of ourstudy is to investigate how cooperative behaviors basedon ToM emerge through evolutionary processes bymodeling the brain at the functional level.
A Genetic Network Programming (GNP) is one of
the evolutionary computation techniques which canautonomously generate behavior sequences by evolution.
Eto et al. (2006) realized functional localization of GNP
by switching the nodes depending on the situation [3].
However, their model cannot realize parallel processing
because nodes activate sequentially from a start node as
well as the conventional GNP. Considering the fact thatthe multiple areas in the brain activate simultaneously,
we assumed that nodes can be executed in parallel.
A limited number of attempts have so far been madeat the constructive approach to ToM characterized by
the use of computational models for simulating its
evolution. Among them, there are only a few studies
which investigate the underlying mechanism of
evolutionary acquisition of the recursion level in a ToM©ISAROB 2011

II. FUNCTIONAL MODEL OF THE BRAIN
As the functional model of the brain, we adopted the
Functional Parts Combination (FPC) model [6] in order
to control the topology of the modules. The FPC modelis based on the neuroscientific fact that each cerebral
cortical area has a different role and is selectively
activated depending on the task [7]. This model consistsof a set of functional parts and activation signals
specifying selective activated patterns. Fig. 1 shows afunctional model of the brain based on the FPC model.
There are modules Mi in the brain, which constitute a
module network. A set of modules in the network areactivated by a set of activation signals. A set of
activation signals A is represented as a vector of binary
values 0 and 1: A = ( a0, …, ai, …, ak-1 ), where k is thenumber of modules, and ai is an activation signal formodule Mi. The activation signals are searcheddepending on the tasks. In the module network, parallel
computation is controlled based on the simple parallel
control flow paradigm [8] as follows. All data are

593

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

action-selection process as if they were in the same
situation [9]. Action-selection process is represented bya probability of action a under the condition state s andgoal G; P(a|s,G) [10]. We defined following strategies
based on a Dennett's intentional stance [11]: (1) Agentat level 0 takes action based on own goal independentlyof the intention of others; (2) Agent at level 1 estimatesthe intention of others by assuming that others would be
at level 0, and takes action based on it; (3) Agent at level2 estimates the intention of others by assuming that
others would be at level 1, and takes action based on it.
In order to realize a smooth cooperative behavior, we
adopted the mixed strategy [10] which dynamicallychanges above three strategies. The module network andfunctions of each module we adopted in the experiments
are described in Box 1.

Figure 1. Functional model of the brain.
transferred indirectly between modules via updatable
memory cells. The execution starts from the sensory
input. In a case that all input links receive a control
token the activated modules begin its computation while
the non activated modules do not execute it. Then bothactivated and non activated modules output tokens fromall output links. However, the modules whose input
links are not connected do not output tokens, regardlessof whether or not the modules are activated. Besides,
the memory cells are initialized before each sensory
input. In a case that the non-written data is attempted toload, an error occurs, the process is suspended andtokens are output from all output links.

IV. EXPERIMENTS
1. Experimental Setup
We conducted simulations in which the activation
signals of agents were evolved by using a genetic
algorithm. A chromosome was represented by binary
encoding, which represents the activation signals A =(a0, …, ai, …, ak-1). We first created N individuals
whose activation signals were randomly generated, andevery pair of agents solved the hunter task E times in around robin manner. Then, time steps to solve the task
were averaged over those games, and agents were
evaluated as: Fitness = 100/step. The offspring in thenext generation were selected by the linear ranking
selection method by Baker [12]. Then, cross over was
performed on the parents to form a new offspring(single point crossover) with a crossover probability Pc,
and each activation signal of all offspring was mutatedwith a mutation probability Pm. We conducted
evolutionary experiments 13 times using the parameters
shown in the Table 1.
Table 1. Experimental setup.

III. MODULE NETWORK AND TASKThe module network can be acquired by the GNP,
however; this paper focused on the emergence ofactivation signals for forming ToM sub-networks toachieve cooperative behavior in a hunter task as a firststep. The discussion on the emergence of modules
network is outside the scope of this paper and weassumed that they had been acquired.
1. Hunter Task
There are two hunter and two prey agents in a 20×20
a two-dimensional grid folded to a torus. Each huntermoves one cell per step to the left, right, up or down, or
stays in the current cell according to its own strategy,
while each prey moves one cell per step stochastically(right; 40 %, up; 20 %, or stop; 40 %).
When starting the task, all 4 agents are randomly
located in the grid, and each hunter selects the closer
prey as an initial target. Each episode ends when eachhunter captures the different prey or the number of time
steps exceeds the upper limit stepmax.

length of the history:T
temperature parameter:βα1α2threshold:θ
upper limit:stepmaxbi (i = 0, 1, 2)

wi (i = 0, 1, 2)
episode: E
population size: N
generation
crossover probability:Pc
mutation probability:Pm
Baker parameter

4
10
20
4000
0.001
0.001
2

2. Results
Fig. 2 shows the transition of the ratio of theactivation signals in the population (black lines) and thefitness (the gray line) on a certain trial. The bar above
Fig. 2 represents the acquired activated patterns. We see

2. The Function of Each Module and Its Networks
We assumed that humans estimate the intention orgoal of others by simulating it based on their own©ISAROB 2011

5
1
0.4
0.6
32
500
5

594

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

agents were (0, 1, 1, 0, 0, 0, 1, 0, 0, 1, 1, 1, 0, 1, 1, 0, 1).
that the fitness in the early stage remained very low.
This means that agents acquired the network for level 0This is because agents randomly selected their actions,ToM. The fitness remained stable in the subsequentand thus they could not solve the task within the upper
generations, and then it increased at around 1100th
limit (500 steps). The fitness slightly increased at
generation with the activation of M5 and M7. By then,around 450th generation with the activation of M9. At
agents had acquired the following activation signals:
that time, the activation signals of the major portion ofBox 1. The module network and function of each module.
M3: clarity comparison
A hunter judges which more precise is: the clarity of theestimated other's goal (L1(Go,t|mo)) or that of the estimated own
𝛼 𝑖 𝑒𝑥𝑝 (𝛽 𝐿𝑖 )goal (L2(Gs,t|ms)): 𝑝𝑖 =(𝑖 = 1,2), where 𝛼𝑖
𝛼 𝑒𝑥𝑝 (𝛽 𝐿 )j=1,2Figure 1. The module network used for the experiments.
M10: state recognition
Own and other's state (ss(t) and so(t)) and action (as(t) andao(t)) at the time t are recognized. The state is defined as
relative coordinates between the hunter and two preys.
M12: working memory
Own and other's state (ss(t – 1) and so(t – 1)) and action (as(t –1) and ao(t – 1)) at the time t – 1 are recognized.
M0, M1: likelihood estimationa(t – 1), s(t – 1) and 𝐺 are substituted to own action-selection
process 𝑃(𝑎|𝑠, 𝐺) in order to calculate the likelihood thatgoal would be 𝐺 as follows: l(G, t) = P(a(t – 1)| s(t – 1), G).
Likelihood l(G, t) is calculated for all possible goals, and isstored in a likelihood history to make estimation of intention
stable: m(G, t) = {l(G, t), …, l(G, t – T + 1)}, where T is thelength of the history. Then, cumulative log likelihood L(G, t|m) is calculated for all possible goals: L(G, t | m) = Σl∈m(G, t)
log l. In particular, conviction degree C which represents thereliability that the estimated other's goal would be G iscalculated in M0: C = L(G1st , t| m) – L(G2nd , t| m), where G1st =argmaxG L(G, t| m) and G2nd = argmaxG≠G1st L(G, t| m).
M2, M4: intention estimation
Others' goal Go or own goal Gs is estimated by an actionselection function based on soft-max reinforcement learning
in
M2
and
M4,
respectively:𝑔 𝐺, 𝑡|𝑚 =𝑒𝑥𝑝 (𝛽𝐿 (𝐺,𝑡|𝑚)), where 𝛽 is a parameter called the𝑒𝑥𝑝 (𝛽𝐿 (𝐺 ′ ,𝑡|𝑚 ))
𝐺′

𝑗

𝑗

is the weight to the Li (𝛼1 + 𝛼2 = 1).
M8: conviction and clarity judgment
In a case that conviction degree C (calculated in M0) is less thanthreshold θ or there is no necessity to change own goal (i.e.
the cooperated rule is already satisfied) the bias of M15 is set tob0. If this is not the case, the bias of M6 is set to b1 when theclarity of the estimated other's goal (L(Go,t|mo)) is higher thanthat of the estimated own goal (L(Gs,t|ms)), otherwise; the biasof M7 is set to b2.
M16: intention formation
A hunter judges whether the others' goal 𝐺𝑜 and own goal 𝐺𝑠
satisfy a cooperated condition. For this task, 𝐺𝑜 ≠ 𝐺𝑠 (i.e.
others' goal differs from own goal) is simply a condition ofcooperation.
M5, M6, M7: intention formation
In M6, own goal Gs1 is formed to satisfy the cooperated
condition, and the weight of the connection between M6 and M9
is set to w1. In M7, own goal Gs2 is also formed to satisfy thecooperated condition, and the weight of the connection between
M7 and M9 is set to w2. Other's goal is also formed to satisfy thecooperated condition in M5.
M15: intention
Own goal 𝐺𝑠 t − 1 selected in M9 (intention selection) at thetime t − 1 is stored in the own goal Gs0, and the weight ofthe connection between M15 and M9 is set to w0.
M9: intention selection
Own

goal

Gsi

is

selected:𝐺𝑠𝑖 =𝑒𝑥𝑝 (𝛽ℎ 𝑖 )𝑗 =0,1,2 𝑒𝑥𝑝 (𝛽 ℎ 𝑗 )𝑖=0,1,2,where hi = bi + wi (i = 0,1,2).
𝑀11 : action selection
Action 𝑎 is selected by an action-selection function:
exp (𝛽𝑄 (𝑎 |𝑠,𝐺))
𝑃 𝑎|𝑠, 𝐺 =,where Q represents an evaluation𝑎 ′ exp (𝛽𝑄 (𝑎 |𝑠,𝐺))value which is acquired by reinforcement learning.
𝑀14 : action-selection function
It is the one based on soft-max reinforcement learning.
𝑀13 : Q-Table
It is an evaluation value 𝑄 which is acquired by reinforcement
learning [18]. Before we conducted experiments in Section 4,each agent had acquired a different Q-Table on its own by thesoft-max reinforcement learning in the setting where there werea hunter and a prey (temperature parameter β=1).

temperature.
1) ss(t), so(t), as(t), ao(t), ss(t – 1), so(t – 1), as(t – 1), ao(t – 1) and as(t + 1) (action output) are set randomly per step, and then thosevalues are updated if related modules are activated. Also, cumulative log likelihood L(G, t | m) and conviction C are set to －10000 per step, and then those values are updated if related modules are activated.
2) Sub networks including modules {M9, M10, M11, M13 and M14}, {M9, M10, M11, M13, M14, M12, M16, M0, M2 and M6} and {M9,
M10, M11, M13, M14, M12, M16, M1, M4, M5 and M7} correspond to ToM 0, 1 and 2, respectively.

©ISAROB 2011595

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011acquired networks
level 0 ToM

V. CONCLUSION

2
level 1v.

In this paper, we constructed a functional model ofthe brain using a functional parts combination (FPC)
model to clarify the mechanism of the autonomous
acquisition of cooperative behavior based on the ToM.
The result of computer simulation shows an emergenceof the pattern of the functional parts for processing ToM
through evolution characterized by punctuatedequilibrium as: (1) level 0 ToM; (2) level 1 ToM; (3)level 2 ToM. The next step would be to investigate theacquisition of not only the activation signals but also theconnections between modules. We believe that the
proposed method would contribute to clarify the originof ToM. It might be also interesting to discuss thefeasibility of the acquisition of ToM in humanoid robots.

2

Figure 2. The transition of the ratio of the activation signals
in the population and the fitness on a certain trial.

REFERENCES[1] Premack, D. and Woodruff, G., Does the chimpanzee(0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1), in otherwords, all modules other than M0, M4, and M8 wereactivated. This means that agents could not estimate
other's and own goal correctly as M0 and M4 were notactivated, and changed own goal randomly.
Subsequently, the activated pattern for level 1 ToM wasemerged with the activation of M0 under the condition
that the other prerequisites for level 1 ToM (M10, M11,
M13, M14, M2, M6, and M12) had been acquired. Next,
there was a remarkable increase in the fitness in parallelwith the activation of M8. By then, all modules otherthan M4 were activated. This means that agents couldchange strategies between level 0 and level 1 ToM
based on the conviction degree C representing thereliability that the estimated other's goal would be G.
After that, the activated pattern for level 2 ToM wasemerged with the activation of M4 under the condition
that the other prerequisites for level 2 ToM (M10, M11,
M13, M14, M1, M5, M7, and M12) had been acquired. Thismeans that agents changed strategies between level 0,level 1, and level 2 ToM based on the conviction degree
C and the comparison of the clarity of the estimatedother's goal (L(Go,t|mo)) and that of the own goal
(L(Gs,t|ms)).
What it comes down to is that the activated patternof the functional parts for processing ToM tended to
evolve in incremental steps as: (1) an emergence of theactivated pattern for level 0 ToM; (2) an emergence ofthat for level 1 ToM; (3) an emergence of that for level
2 ToM. Looking at other trials, the same tendency could
be found.

©ISAROB 2011

have a theory of mind? The Behavioral and Brain
Sciences, 1, 515–523, 1978.
[2] Brothers, L., The social brain: A project for integrating
primate behavior and neurophysiology in a new domain.
Concepts Neuroscience, 1, 27–51, 1990.
[3] Eto S., Hatakeyama H., Mabu S., Hirasawa K., Hu J.,
Realizing Functional Localization Using Genetic
Network Programming with Important Index, Journal ofAdvanced Computaional Intelligence and Intelligent
Informatics, 10 (4), 555-566, 2006.
[4] Takano, M. and Arita, T., Asymmetry between Even and
Odd Levels of Recursion in a Theory of Mind, Proc. of
the 10th International Conference on the Simulation and
Synthesis of Living Systems (ALIFE X), 405-411, 2006.
[5] Noble, J., Hebbron, T., Horst, J., Mills, R., Powers, S.
and Watson, R., Selection pressures for a Theory-ofMind faculty in artificial agents. Proc. of the 12th
International Conference on the Simulation and Synthesisof Living Systems (ALIFE XII), 2010.
[6] Omori, T. and Ogawa, A., Two hypothesis for realizationof symbolic processing in brain. Proc. of the 9thInternational Conference on Neural Information
Processing, 2001.
[7] Liu, M. J., Fenwick, P. B. C., Lumsden, J., Lever, C.,
Stephan, K.-M. and Ioannides, A. A., Averaged andsingle-trial analysis of cortical activation sequences inmovement preparation, initiation, and inhibition. Human
Brain Mapping, 4, 254–264, 1996.
[8] Trealeven, P. C., Hopkins, R. P., and Rautenbach, P. W.,
Combining data flow and cotrol flow computing,Computer Journal, 25 (2), 207-217, 1982.
[9] Gallese, V., & Goldman, A., Mirror neurons and thesimulation theory of mind-reading. Trends in Cognitive
Sciences, 2, 493–501, 1998.
[10] Nagata, Y., Ishikawa, S., Omori, T., and Morikawa, K.,
Computational model of cooperative behavior based on
dynamical selection of intention based action decision
strategy, Cognitive studies，17(2), 280-286, 2010.
[11] Dennett, D., The Intentional Stance, MIT Press,
Cambridge, 1987.
[12] Baker, J. E., Reducing bias and inefficiency in theselection algorithm, Proc. of the Second International
Conference on Genetic Algorithms, 14 - 21, 1987.

596

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A comparison of learning performance in two-dimensional Qlearning by the difference of Q-values alignment
Kathy Thi Aung, Takayasu Fuchida(Department of System Information Science, Graduate School of Science and Engineering, Kagoshima University,
Kagoshima, Japan)(Tel: 81-99-285-8453; Fax: 81-99-285-8464)(kathythiaung@gmail.com, fuchida@ibe.kagoshima-u.ac.jp )Abstract: Q-learning is a kind of reinforcement learning where the agent solves the given task based on rewards
received from the environment. In this paper, we compared the performance of the Q-learning based on the reward
examined by the difference of Q-values alignment in two-dimensional (2D) state space under various conditions suchas angle of VQE rotation which is arranged like a lattice and angle of the agent’s action rotation to correctly evaluate
the optimal Q-values for state and action pairs, in order to deal with continuous-valued inputs. We apply the proposed
method with an agent that learns to reach the reward area successfully during the reward-based learning process. A
reward is given to the learning agent if the agent reaches the reward area during a process of trial.
Keywords: Q-learning, Q-value, VQE
strategy strongly depends on the situation of oursimulation methods and the simulation results are
explained.

1 Introduction
Learning algorithms based on evaluative feedback
signal is generally referred to as RL algorithms.[1,4] In
a RL paradigm, [4] a system called agent senses theenvironment and produces control actions. The
environment responds to these control actions. Based onthese responses a reward function will evaluate thecontrol actions. The agent tries to optimize the control
policy to maximize the total expected reward over afinite time-span. Learning may occur using theprediction error of expected rewards.
Reinforcement learning (RL) methods are a
powerful and useful way to control agents such as anautonomous robot. [1] Q-learning [2] is the most widelyused in RL method which deals with only discretevalued inputs (states) and outputs (actions) to represent
Q-function (action value function) that evaluatesstate/action pairs.
The aim of this work is to significantly improve thelearning performance of Q-learning between the state
space and the action space. Therefore, we implement
and investigate here in order to clearly show theeffectiveness of proposed learning method undervarious problems.
In this paper, we first briefly explain our agent
model in a single-agent environment. Secondly, wedefine VQE with radius to decide the position. Next we
describe each of the simulation methods, and then theperformance of each strategy is examined by computer
simulations on competitive situations of several
strategies. We also show that the performance of each

©ISAROB 2011

2. Q-Learning
In Q-learning, [2] the expected value of each actionin each state is stored. In the other way, the Q-value isthe expected value of each action in a certain state,
which is the discounted sum of the rewards agent
received for state and action pair. We can estimate andupdate the Q-value, which is denoted by Qst , at 
according to the following equation by taking the one
with the maximum Q-value (highest expected value) forthe current state.
Q(st , at )  Q(st , at )   (r   max Q(st 1, a)  Q(st , at ))a

Eq. (1)

When the agent is in state t, the agent observes thestate st and executes the action at . The agent obtains the
reward rt , and senses a new state st 1 by selecting anaction at in state s t . In this equation,  is thelearning rate and  is the discount rate, both are
between 0 and 1.
In the standard Q-learning implementation, Q-values
are stored in a table, is known as Q-table. It looks like asquare lattice in two dimensions and one cell is required
per combination of state and action. This
implementation is not amenable to continuous state andaction problems. [3] As the number of state and action
variables increase, the size of the table used to store Qvalues grows exponentially, is called Curse ofdimensionality. On the other hand, as the number of597

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

between the direction of agent’ action and the reward
area. The state space is constructed with these two input
values. In this model, the agent is represented by atriangular arrowhead which indicates the direction inwhich the agent is moving. The rectangular area
represents as the reward area.

dimensions increases, the state space also increases
exponentially and the learning speed decreases
dramatically.
2.1. Behavioral Decision
The agent selects the next action which has thehighest Q-value. An action which has a large Q-value isconsidered to be a good way to achieve the reward.
However, selecting the highest Q-value continually
decreases the opportunity to find a better way. Therefore,
the agent sometimes selects the next action at random.
This random selection is useful for exploring the state
space and finding a new better way which has not beenfound yet.

3.1. Experimental Environment
An agent moves in a closed two-dimensional state
space. Figure 2 shows an image of the agent that learns
to reach the reward area. A reward is given to thelearning agent if the agent reaches to the reward area, in
the other states the reward is always zero.

2.2. Voronoi Q-value Element (VQE)
VQE is a point that has Q-value but we don’t know
where VQEs should be placed in the states space at theinitial time. If we got many rewards, we would be able
to evaluate the positions of VQEs for the optimal policyof state-action pair.
By using the VQE, we can place the Q-value
arbitrarily in the state space and reduce the waste ofspace. Therefore, as the degree of freedom to select theposition of VQE is so numerous, we have to decidecarefully where the VQEs are placed in the state space.

Fig.2. Agent Problem
In Figure 2, since the number of reward area is one,the agent observes two parameters. These twoparameters construct the two-dimensional state space.
The dimension of the state space goes up by twodimensions when one reward area is increased.
The possible actions of the agent are: 1, straightahead; 2, turn left; 3, turn right. If the agent reaches the
reward area, the position of the agent is randomlychanged in this state space.
3.2. Experimental Parameters

3. Experimental ModelIn these experiments, we tested with one agent and
one reward area. The working environment of the agent
is shown in Figure1, in where, it is intended to learn
efficient action of an agent which is to reach the reward
area in the state space.
The agent’s action of maximum Q-value is selected
while observing in a certain condition and the agent is arandom action with a fixed probability. The agent
observes the distance (r) and the angle (  ) toward thereward area.

width and height of space
size of reward area
learning rate 
discount rate 
random action rate
initial Q-value-100,100
5
0.1
0.9
0.3

travel distance of agent
Number of execution times
One trial
One episode2.0~5.0
10 trials
20 episodes
100000 times

0  Qst , at   0.01

4. Simulation Methods and Results
We carried out three types of simulation experiment.
4.1. Square Lattice with random noise
The lattice below is a general 2-dimensional latticeon partition number 10, shown in figure 4.1.1. The ideaof the lattice was proposed by VQEs and designates a 2-

Fig.1. Working environment of an agent
There are two input variables: 1, the distance
between an agent and the reward area; 2, an angle©ISAROB 2011

598

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

dimension of 100 VQEs in lattice by symbol 10*10 gridenvironment. If we conduct this general 2-dimensionallattice with random noise (i.e., 0, 0.01, 0.02, 0.03, 0.04,
0.05 randomly arranged VQEs), we get the lattice likefigure 4.1.2.

Fig.4.1.1 Original 2-D lattice

As in the experimental results shown in figure 4.2.1
for the 10*10 grid environment, the number of rewards
clearly was slow, and a similar result also occur in a
20*20 grid environment because the reward propagation
is delayed.
As the number of reward count decreases, weconsidered the following possible casual points.
 When the state changes, a state certainly transit to adifferent state if the action is different.
 But if lattice VQE rotates, also take in different action,
grows the probability that the state of result go into thesame state.
 Q-value decreases if the action is acting in the same
area by the Q-learning method as shown in above
equation (1).
 Since the angle of lattice VQE rotation for learning isenormous, we implemented the next strategy.

Fig.4.1.2. Random Lattice

4.3. Rotation of Lattice VQE and agent’s action
Here, we propose a new simulation experiment
method in which the reward area is denoted by closed
circle and the agent is denoted by open circle. The newstrategy of learning for our target problem is defined as
figure 4.3.1. Note that the agent takes actions with four
directions: go up, go down, go left and go right.

Fig.4.1.3. From regular to random VQE
As we can see from this above figure, the number ofrewards is slowing down, thus its performance by thisstrategy depends on the arrangement of VQEs.
4.2. Lattice VQE rotation by Degrees
In this subsection, we rotated the above original 2dimensional lattice by the angle of clockwise rotation on
partition number 10 and 20. It turns for each degree ofrotation by five degrees of angle intervals in the ranges
from 0 to 90 degrees.

Fig.4.3.1. Structure of a new agent
In this case, we rotated the location of reward area
and initial/default position of agent also. In each degreeof our rotation, the action selection is simultaneously
performed by the agent.
In our computer simulations, we used two elements
rotations: act-angle and VQE-angle of rotations whichmeasure between 0 degrees and 90 degrees by fivedegrees of angle intervals. (Figure 4.3.2)

Fig.4.2.1. Reward count in a 10*10 grid environment

VQE_angle0

act_angle0

act_angle45

Fig.4.3.2. Rotation of VQE and agent’s action

Fig.4.2.2. Reward count in a 20*20 grid environment©ISAROB 2011

VQE_angle30

599

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig.4.3.6. From regular to random VQE
Fig.4.3.3. Reward count in a 10*10 grid environment

5. Conclusion and Future WorkAs a result, the learning performance was decreaseby turning the angle of VQE only. When we turn act
angle and VQE angle, it improves in performance. The
difference between the number of actions 3 and 4 isaccording to the following reason.
The agent’s action space and the state space is matchor not. In the case of the agent’s action spacecorresponds to the state space, the performance
improves while shifting in the angle of rotation
especially VQE angle and act angle is shifting by 45
degrees is greatly improved. Therefore, it is match ofagent’s action space and the state space that gives good
performance.
In fact, it is different by expression of state input. Itmeans when the number of action is 4, the agent
observes by its position but in the case of action number
3, it observes the distance and the angle of agent relative
to the reward area.
In the current research, we proposed learning
method of Q-learning based on Voronoi Q-value
element. In this paper, we examined the learning
performance of various strategies in a different situationfor our experimental environment. Although at thecurrent stage, we only performed experiments in two
dimensions, as future topics of research, we plan to ahigh-dimensional problem by generating an Ndimensional state space.

Fig.4.3.4. Reward count in a 20*20 grid environment

(a)

(b)

Fig.4.3.5. Angle of rotation for (a) 0,90,180,270,360
degrees and (b) 45,135 degrees
As Fig. 4.3.3 and Fig.4.3.4 reveals, when the turning
angle of VQE are 0 degrees and 90 degrees, the results
are entirely same at all, in fact there’s no difference
between them. In addition, Fig.4.3.5 also has totallysame at the turning angle of 0, 90, 180, 270, 360
degrees and 45, 135 degrees. In this situation, we cansee that the number of rewards increases due to theagent’s optimum actions are four in this strategy.
Since Fig.4.3.6 generates the number of rewards thatare put VQE randomly with noise on a number of(0,1,2,3,4,5) is relatively compared with in the Fig.4.1.3
above.

©ISAROB 2011

References[1] R.S. Sutton and A.G. Barto, “Reinforcement
Learning: An Introduction”, MIT Press, 1998.
[2] C.J.C.H. Watkins and P. Dayan, “Q-learning”,
Machine learning, Vol8, pp.279-292, 1992.
[3] Christopher J.C.H. Watkins. “Learning from
Delayed Rewards”, PhD thesis, University ofCambridge, 1989.
[4] K.Kiguchi, T.Nanayakkara, K.Watanabe, T.Fukuda,“Multi-Dimensional
Reinforcement
Learning
Using a Vector Q-Net - Application to Mobile
Robots”, Internal Journal of Control, Automationand Systems, vol.1, no.1, pp.142-148, 2003.

600

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Robot path planning in unknown environment based on ant colonyalgorithm
Cu Xuan Tien, Young Sik Hong
Department of Computer Science, Dongguk University
26 Pil-dong, 3- ga, Jung gu, Seoul, South Korea, 100 - 715{tienbkct8@gmail.com, hongys@dgu.edu}

Abstract: In this paper, we present Ant Colony Optimization Algorithm for path planning to a target for mobile robot
in unknown environment. The proposed algorithm allows a mobile robot to navigate through static obstacles, and findthe path in order to reach the target without collision. This algorithm provides the robot the possibility to move from theinitial position to the final position (target). The proposed path finding strategy is designed in a grid-map form of an
unknown environment with static unknown obstacles. The robot moves within the unknown environment by sensing
and avoiding the obstacles coming across its way towards the target.
This work provides overviews of swarm intelligence and Ant Colony Optimization applications applied to many
fields of our life particularly we will focus in mobile robot routing. Various mechanisms are presented in this paper todemonstrate the useful and wide combination of Ant Colony Optimization. Finally, enhancement to the mechanism issuggested and it is evaluated in MATLAB environment.
Keywords: Ant Colony Optimization (ACO), mobile robot, path planning
The paper is organized as follow: in the second
section the ant colony algorithm is introduced and thenext section is discussed about the path planningalgorithm realization. In the last section, some result,experiment and conclusion are provided.

I. INTRODUCTION
Today, the robotics become favorite topics of manystudies and its applications are applied in various
problem of our life. There are many type of robot andthey perform many functions. In the conquest of space
exploration where they can be used to collectenvironment data such as rock, soil samples… In
medical field, some specialized robots support
physicians diagnose and help people with disabilities.
Scientists in the world have created special type ofrobots replace human activities in the particularly
dangerous position, for example in case of incidents at
nuclear power factory or hygiene robot working in highbuilding…
One of most important tasks in the control of mobile
robot is path planning. In environment with obstacles,
the role of path planning is to finding the appropriate
collision – free path for mobile robot to move from startlocation to the destination point. There are manymethods have been studied and applied in grid-based
path planning such as A * (A star) algorithm [1], roadmap (Voronoi diagram and visibility graphs) [2],artificial potential field [3]…
In robotics field, working environment is divided into
three kinds: the totally known, partly known, and total
unknown. This paper will focus to total unknown
environment, each robot has forage to find the way thetarget by co-operate with each other.
Ant colony algorithm was invented in 1992 by Marco
Dorigo in his doctoral thesis. Since then this algorithm
is used in many studies and achieve high performance.
Multi ant system has been successfully applied inTravelling Salesman problem (TSP) [4], the Quadratic
Assignment problem [5], Vehicle Routing problem [6],etc…

©ISAROB 2011

II. ANT COLONY ALGORITHM
Ant colony algorithm, based on behavior of real ants, isa nature approach to establish optimization path from ne
st to food source, it is a stochastic search algorithm andgood effect has been obtained in solving combination op
timization, function optimization, system identification,
robot path planning, data mining, and network routing algorithm.

Figure 1: Ant colony foraging foodAt the time of beginning, all ants move at random
direction to find the way to the destination. Biologist
have found by many studies and experiment that ants
leave some chemical substance on the route that they
have passed, which call “pheromone”. Other ants cansmell this pheromone, and their path-taking decisions
are depended on the amount of pheromone. Whensubsequent ants sense this trail of pheromone, they will
follow the trail with a probability that is proportional tothe concentration of pheromone – the higher theconcentration, the larger the probability that subsequentants will follow the trail. The pheromone evaporates at a

601

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

given rate which means that the strength of pheromone
encountered by another ant is a function of the original
pheromone strength and the time since the trail was laid.
Besides pheromones are deposited on the foraging path,ants also rely on heuristic information as well. The
heuristic information is defined as the inverse of
distance between the current point and the next point
which ant can be go as show in below equation:
1=

attractive for the ant robot. This is the parameter will be
used to calculate the cost function in probability
transition in next section.
From the matrix in figure 2 the environment for robot
path planning will be constructed as figure 3 with the
top – left are the starting cell and the bottom – right isthe destination cell

Therefore, the shorter the distance between two
locations is, the better their heuristic information andvice-versa. Following with the intensity of pheromone
trails and the heuristic information, the probability ofthe path which will be chosen by ant can be calculated.
This probability is known as the transition probability:( )] [ ( )]∈∑ ∈ [ ( )] ][ ( )]0∉Where t is the time index,
is the heuristic
information,is the pheromone trail which has been
evaporated when the ant move from point i to point j,, are the parameter of pheromone function andheuristic function, Z is the set of all moveable pointfrom current point.
( )=Figure 3: Robot environment
2. Coordinate and mapping
Each cell is assigned by a number from 1 to 400 as
figure below which is defined by marking sequentiallyfrom left to right, top to bottom.

III. IMPLEMENTATION OF ANT COLONY
ALGORITHM IN ROBOT PATH PLANNING
1. Environment initialization
We will use 20x20 grid model to present theenvironment of mobile robot. Assume that the space inwhich a robot moves is a limited region with several
obstacles of different size in two – dimensional plane.
Assume that the edge of each cell is 1 unit that presents
the radius size of scope in which a robot could movefreely. Obstacles occupy one or more grids are createdby a matrix which each value is distributed from 1 to 5.
The value 1 is indicated a cell of obstacles, and thevalues from 2 to 5 are the free cell.

Figure 4: Environment mapping
The mapping relationship between ( , ) and node
number i can be defined by formula:( , ) − 0.5
=
=

+ 0.5

If
= −0.5 then
= − 0.5. Where N is the( , ) is function to calculate
size of map (N=20),
the remainder value when integral number i divide by N,ceil(x) is function which return the largest integral valueof x.

Figure 2: Robot environment matrix

3. Movement Rule
Moveable area for ant robot in each step is chosen

As we can see in the figure 2 the zone near thedestination point have high value which is the more

©ISAROB 2011−

602

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

from maximum 8 cells around the current cell without
obstacles. For example, assume that the current positionof robot is cell number 168 then the set of next step are
148, 149, 167, 169, 187, and 188

IV. EXPERIMENT AND RESULTThe algorithm has been simulated by MATLAB
language by Intel Pentium IV with Core2Dual 1.8 GHZ,
1Gigabytes
memoryrunninginWindowXp
environment. This experiment we use a map with 20x20
grid and all operations were done by 10 ant robots. All
parameters have been using for simulation = 0.5,
= 5, = 0.5,
= 0.5, = 0.1, number iteration
is 50 and initialization value of pheromone is 1. The
figure below is the result when ant robot using basic
algorithm, the length of minimal path is 31.5563 (28cells

Figure 5: Moveable area
Each ant robot will chose the next cell to moveaccordance the visibility of the cell and pheromone
intensity. In the initialization state, all ants have no
information about environment so they forage to reachthe destination by random step.
After that ant robot will go to the next point
given by:=[ ( )] ][ ( )] [( )] ,
,
>

≤

and J

, ∈

Where q is the random number uniformly distributed
between [0, 1],∈ [0,1] is the threshold value set in( ) is the value of cell ,the initialization,
denotes the relative significance of cost value and J is
calculated by:[ ( )] ][ ( )] [
( )]
( )=
∑ ∈ [ ( )] ][ ( )] [( )]Z is the set of moveable area which is discussed above
andis the transition probability to point J. The
heuristic information will be determined as equation
below to improve the efficiency and getting better
solution:1=( − ) +( − )Here i is the next node and e is the destination node.

Figure 6: Basic algorithm result
And the result with improved ant colony algorithm withminimal length is 29.779 (25 cells)

4. Pheromone Updating Rule
- Localized pheromone update
When the ant has passed the cell , the pheromone will
be updated by:( + 1) = (1 − ) ( ) +- Global update:
After all ants reached the target, the pheromone will be
refreshed and update by equation below:( + 1) = (1 − ) ( ) + ∆ ( )∆ ( + 1) = (1 − )∆ ( ) + /min ( )Where 0 < < 1 is the erased rate of pheromone andis the initial value of pheromone
Figure 7: Improved algorithm result©ISAROB 2011603

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

V. CONCLUSION
In this paper, ant colony optimization is presented
detail to provide an efficient method for solving robot
path planning problem. Ant colony optimization is ameta-heuristic in which a colony of artificial ants
cooperates in finding good solutions to difficult discrete
optimization problems. Cooperation is a key designcomponent of ACO algorithms: The choice is toallocate the computational resources to a set of
relatively simple agents (artificial ants) that
communicate indirectly, that is, by indirectcommunication mediated by the environment.
The application of ACO is particularly interesting
for: (1) NP – hard problem, which cannot be efficiently
solved by most traditional algorithms; (2) Dynamic
shortest path problem in which some properties of the
problem’s graph representation change over time
concurrently with the optimization process; and (3)problems in which the computational architecture isspatially distributed. The key elements are cost value
and improved heuristic function, the robot path planningscheme has been simulated using MATLAB language tosolve efficiently the Ant Colony Algorithm using twodimension grid model with static obstacles in order tomake the robot’s environment.

REFERENCES[1] (2010) Junfeng Yao, Chao Lin, Xiaobiao Xie, Andy
JuAn Wang, Chih-Cheng Hung, "Path planning for
Virtual Human Motion Using Improved A* Algorithm"
2010 Seventh International Conference on Information
Technology, DOI 10.1109/ITNG.2010.53[2] (2007), Priyadarshi Bhattacharya and Marina L.
Gavrilova,"Voronoi diagram in optimal path planning",
4th International Symposium on Voronoi Diagrams in
Science and Engineering (ISVD 2007)[3] (1991), J. Borenstein,Y. Koren,"The vector fieldhistogram -fast obstacle avoidance for mobile robots",
IEEE Journal of Robotics and Automation Vol 7, No 3,
June 1991, pp. 278-288.
[4] (1997), Marco Dorigo, Luca Maria Gambardella,"Ant colonies for the traveling salesman problem",publication in BioSystems.
[5] (2006), Malek Mouhoub and Zhijie Wang," Ant
Colony with Stochastic Local Search for the Quadratic
Assignment Problem", Proceedings of the 18th IEEE
International Conference on Tools with Artificial
Intelligence (ICTAI'06)[6] (2010), Hua Bin, Ding Huai-long."Vehicle Routing
Problem of Logistics Based on Dynamic Ant Colony
Algorithm", 2010 Second International Workshop on
Education Technology and Computer Science©ISAROB 2011604

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Improved Artificial Bee Colony Algorithm for Large-Scale Optimization Problems
Ryuta Gocho†, Akihide Utani‡, and Hisao Yamamoto‡†Graduate School of Engineering, Tokyo City University‡Faculty of Knowledge Engineering, Tokyo City University
1-28-1, Tamazutsumi, Setagaya, Tokyo 158-8557, Japan{autani; yamahisa}@tcu.ac.jp
Abstract: This paper proposes a new optimization algorithm based on Artificial Bee Colony (ABC) algorithm that has the good performance on large-scale optimization problems. We evaluate the proposed algorithm through numerical experiments on well-known benchmark functions, such as Rosenbrock function, Rastrigin function, Schwefel function, Ackley function and Griewank function, and discuss its development potential. In numerical experiments, the performance of the proposed algorithm is compared with those of the existing optimization algorithms.
Keywords: Wireless Sensor Networks, Particle Swarm Optimization, Query Dissemination, Long-Term Operation.
many difficult optimization problems. The ideas that underlie Particle Swarm Optimization algorithm are inspired not by the evolutionary mechanisms encountered innatural selection, but rather by the social behavior of flocking organisms, such as swarms of birds and fish schools. Particle Swarm Optimization algorithm is a population-based algorithm that exploits a population of individuals to probe promising regions of the search space.
The algorithm is simple and allows unconditional application to various optimization problems. However, it has been confirmed that the performance of Particle Swarm Optimization algorithm on large-scale optimization
problems is not always satisfactory.
This paper proposes a new optimization algorithm based on Artificial Bee Colony (ABC) algorithm that has the good performance on large-scale optimization problems. We evaluate the proposed algorithm through numerical experiments on well-known benchmark functions, such as Rosenbrock function, Rastrigin function, Schwefel function, Ackley function, and Griewank function, and discuss its development potential. In numericalexperiments, the performance of the proposed algorithm
is compared with those of the existing optimization algorithms. The rest of this paper is organized as follows.
In Section II, the proposed algorithm is described. In Section III, experimental results are reported. Finally, thepaper closes with conclusions and ideas for further study in Section IV.

I. INTRODUCTION
The minimization of multimodal functions with many local and global minima is a problem that frequently
arises in diverse scientific fields and numerous engineering design problems. This problem is NP-hard in the sense of its computational complexity even in simple cases. As techniques of computing a global minimum of the objective function, many meta-heuristics, which aresearch algorithms for optimization based on heuristic knowledge, have been proposed. Some well-known representative meta-heuristics are Simulated Annealing and
Tabu Search, which are the traditional optimization algorithms, Genetic Algorithms and Immune Algorithms,
which are classified as evolutionary computation techniques, and Ant Colony Optimization algorithms and Particle Swarm Optimization algorithms, which belong to
the category of swarm intelligence algorithms.
In meta-heuristics, Genetic Algorithms and Immune
Algorithms, classified as evolutionary computation techniques, are generally techniques for combination optimization problems. In Genetic Algorithms and Immune
Algorithms, the variables of continuous type are frequently translated into those of discrete (genetic) type. If there is a dependency between variables, therefore, a promising solution may be destroyed during the solution
search process (genetic operation) and the solution search performance may deteriorate. To the contrary, Particle Swarm Optimization algorithm can directly handle the variables of continuous type. Even when there is a dependency between variables, therefore, an efficient andeffective solution search can be realized. Recently, Particle Swarm Optimization algorithm is intensively researched because it is superior to the other algorithms on©ISAROB 2011

II. PROPOSED ALGORITHM
In ABC algorithm, the colony of artificial bees contains three groups of bees: employed bees, onlookers andscouts. In the initial state of “artificial bee colony”, thecolony consists of the employed bees and the onlookers.

605

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

blem and faccuracy shows the exactness of convergence
to fbound.
2)Improvement of Step1 and Step3 in ABC algorithm
For improving the performance of solution search, inthe proposed algorithm, a more suitable solution point is determined by roulette or elite selection based on
the probability (Pik) of Eq.(2).
3)Improvement of Step4 in ABC algorithm
The search by scouts corresponds to the mutation of
Genetic Algorithms. In the proposed algorithm, the search by scouts is not executed to no effect.

At the initial stage of ABC algorithm, multiple solution
points are randomly set in multidimensional solution search space. For every solution point, there is only one employed bee. In other words, the number of employed
bees is equal to the number of solution points. The employed bee of an abandoned solution point becomes a scout. The search carried out by the artificial bees can besummarized as follows:
1. Each employed bee randomly searches a more suitable solution point within the neighborhood of the solution point in its memory.
2. Employed bees share their search information with onlookers and then onlookers select one of solution points by the following equations:1⎧⎫,f ( xik ) ≥ 0⎪k⎪
fit ik = ⎨ 1 + f ( xi )⎬⎪ 1 + abs ( f ( xik )) , f ( xik ) < 0⎪ (i = 1,…,SN)⎩⎭

Pi k = fit ik

III. EXPERIMENTAL RESULTS
Through numerical experiments on the following D
dimensional benchmark functions, the performance of the proposed algorithm is investigated to verify its effectiveness.
- Rosenbrock function

(1)

D −1

∑ fit

k
n

}

j =1

(2)
where f (x) is the objective function of variables (x).
The subscript i(i =1, … , SN) and the superscript k indicates the solution point’s index and the number ofsearch iterations, respectively. SN is the number of solution points. Onlookers select one of solution points by referring to the probability (Pik) of each solution
point based on the search information from employedbees.
3. Each onlooker randomly searches a more suitable solution point within the neighborhood of the solution point chosen by itself.
4. The employed bee of an abandoned solution point becomes a scout and starts to search a new solution point randomly.
The proposed algorithm is an advanced ABC algorithm. For effectively searching a global optimum solution,
ABC algorithm is improved as follows:1)Improvement of Eq.(1) to compute fitness (fitik)
For improving the adaptability to various engineering
problems, in the proposed algorithm, the fitness (fitik)of each solution point is computed as follows:
n =1

subj. to − 100 ≤ x j ≤ 100,x* = (1,...,1),

j = 1,..., D

f1 ( x* ) = 0- Rastrigin function
min. f 2 ( x) = ∑ { x 2j − 10 cos(2π x j ) + 10 }D

j =1

subj. to − 5.12 ≤ x j ≤ 5.12,
x* = (0,...,0),j = 1,..., Df 2 ( x*) = 0- Schwefel function
D

min. f 3 ( x ) = 418.98288727 D + ∑ − x j sin( | x j |)j =1

subj. to − 512 ≤ x j ≤ 512,

j = 1,..., D

x* = (420.968750,...,420.968750),

f 3 ( x*) = 0- Ackley function⎛
1 D 2 ⎞⎟
min. f4 ( x ) = 20 + e− 20 exp⎜ − 0.2∑ xj⎜
D j =1 ⎟⎠⎝⎛1 D⎞− exp⎜⎜ ∑ cos(2π x j ) ⎟⎟⎝ D j =1⎠
subj. to − 30 ≤ x j ≤ 30, j = 1,..., D
x* = (0,...,0),1⎧⎫, f ( xik ) − f bound ≥ f accuracy ⎪⎪⎪ f ( x k ) − f
⎪
k
i
bound
fiti = ⎨⎬1k⎪, f ( xi ) − f bound < f accuracy ⎪f accuracy⎪⎩⎪⎭(i = 1,…,SN)

f4 ( x*) = 0- Griewank function
min. f 5 ( x ) =(3)
where fbound represents the boundary value of f (x+) on
x+ acceptable as a solution for every engineering pro-

©ISAROB 2011{

min. f1 ( x) = ∑ 100( x j +1 − x 2j ) 2 + ( x j − 1)2

SN

⎛ x ⎞
1 D 2 Dx j − ∏ cos⎜ j ⎟ + 1∑⎜ j⎟4000 j =1
j =1⎝⎠

subj.to − 600 ≤ x j ≤ 600,
x* = (0,...,0 ),606

f 5 ( x* ) = 0

j = 1,..., DThe Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

x* of each benchmark function represents a global optimum solution. Fig.1 shows landscapes of multimodal functions (D = 1).
In experimental results reported, the proposed algorithm is evaluated through the comparison with the existing ones, which are Particle Swarm Optimization algorithm[1,2], Differential Evolution algorithm[3], and theoriginal ABC algorithm[4]. The results of numerical experiments are shown in Tables 1, 2, 3, and 4. Experimental results indicate that the proposed algorithm is a promising one for large-scale optimization problems.

4540
f2(x1)
35

系列 1

30
25
20
15
10
5
0
-5

-4

-3

-2

-1 0
1
(1) Rastrigin

2

3

4
x1

5

IV. CONCLUSIONSIn this paper, a new optimization algorithm based on
Artificial Bee Colony (ABC) algorithm has been proposed. The proposed algorithm has been evaluated through numerical experiments on well-known benchmark functions, such as Rosenbrock function, Rastrigin function, Schwefel function, Ackley function, and Griewank function. From experimental results, it has been confirmed that the proposed algorithm has the development potential as an efficient one for large-scale problem.

900

f3(x1)800
700

系列 1

600
500
400
300
200
100
0
-500

-300

-100
100
(2) Schwefel

300

x1

REFERENCES

500[1]Kennedy J and Eberhart RC (1995), Particle swarm
optimization. Proc. IEEE International Conference
on Neural Networks, pp.1942-1948[2]Kennedy J and Eberhart RC (2001), Swarm intelligence. Morgan Kaufmann Publishers, San Francisco.
[3]Storn R and Price K (1997), Differential evolution A simple and efficient heuristic for global optimization over continuous space. J. of Global Optimization,
vol.11, pp.341-359[4]Karaboga D and Basturk B (2007), A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm. J. Global Optimization, vol.39, pp.459-471.
--------------------------------------------------------------------

25

f4(x1)

系列 1

20
15
10
5
0
-30

-20

-10

0
10
(3) Ackley

20
x1

30

400
x1

600

100
f5(x1) 90
80

系列 1

70
60
50
40
30
20
10
0
-600

-400

-200
0
200
(4) GriewankFig.1. Landscapes of multimodal functions (D = 1)

©ISAROB 2011

607

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 1 Experimental results on PSO algorithm

Table 3 Experimental results on ABC algorithm

Problem

Dim.

Problem

Dim.

Rosenbrock50

Rosenbrock

Schwefel

4.63×1075

2.30×100 5.36×102 9.79×103

3.18×10

6100

9.96×101 9.07×102 9.82×103

150

9.76×102 2.85×103 1.40×104

150

9.86×104 1.48×106 5.91×106
250

4.51×10-4 3.53×100 1.25×101

75

1.58×10

3.26×10375

4.93×100 2.42×101 9.82×101

100

2.72×102 3.71×103 6.10×103

100

6.22×101 7.16×101 1.02×102

150

1.28×102 2.77×102 3.22×102

21.81×102.30×10

3

Rastrigin

3.14×10350

3

6.60×10

3

3.54×1050

5.39×103 9.78×103 1.25×104

50

8.98×102 1.30×103 1.97×103

4.52×1075

2.73×103 6.01×103 6.56×103

9.88×10

4100

6.02×103 9.31×103 1.00×104

150

9.60×103 1.62×104 3.20×104

9.20×10100

1.92×104150

3.35×104 4.07×105 5.65×105
050

4.74×10-4 1.71×10-3 2.24×10-2

75

2.54×106.02×10175

1.30×10-1 7.96×10-1 1.58×100

100

4.84×100 2.40×102 4.30×102

100

1.89×100 2.49×100 4.12×100

150

4.57×100 5.75×100 6.68×100

1.23×100

3.89×10004.05×101

8.90×102

2.13×10150

8.50×10-7 9.21×10-3 3.73×10-2

9.83×10175

4.26×10-4 4.57×10-2 1.28×10-1

100

4.01×10

1.51×102100

8.17×10-3 7.94×10-2 5.34×10-1

150

5.99×100 7.05×101 1.87×102

150

1.79×10-2 7.17×10-1 9.97×10-1

50

5.79×100

2.11×101

Ackley2.65×10150

1.82×10

6.92×104Schwefel

4

30

3.48×104

9.31×10

3

150

75

2.61×102

1.91×101

4.93×101

7.12×10

1

Griewank

Table 4 Experimental results on the proposed algorithm

Table 2 Experimental results on DE algorithm
Problem
Rosenbrock

Rastrigin

Dim.

AckleyGriewank©ISAROB 2011

Best

Ave.

Problem

Worst

Rosenbrock

Dim.

Best

Ave.
-2

8.73×10Worst
0

9.71×10150

1.39×10

7.09×102 1.38×103 1.92×103

75

7.01×10-1 2.57×101 1.14×102

100

1.08×104 1.73×104 2.79×104

100

5.75×100

5.34×101 2.10×102

150

8.45×105 1.97×106 4.12×106

150

9.98×101

1.32×102 4.39×103

50

7.62×10-250

9.01×101 1.32×102 9.02×102

75

4.55×100

Rastrigin

4.23×10150

2.08×10-10

8.10×10-1 2.99×100

08.00×100 1.57×101

75

1.53×101 8.77×101 1.81×102

75

2.80×10100

1.54×102 2.21×102 3.22×102

100

1.21×101

1.39×101 2.92×101

150

4.67×101

5.66×101 9.53×101

50

1.18×1022.03×102 1.18×103

75

1.43×103

2.39×103 3.06×103

3

3.03×103 5.26×103

150
SchwefelWorst

1.36×100 1.25×102 8.80×102

5.08×10
1.05×10

7.28×10

5

Ave.

50

100

2

1.79×10

5

Best

5

4.38×103 7.00×104 2.26×105
4150
Griewank

Worst

9.20×1075

AckleyAve.
375

Rastrigin

Best

2.98×102 5.59×102 7.38×102

50

1.14×103 2.24×103 4.62×103

75

6.27×103 9.87×103 1.11×104

Schwefel100

1.24×104 3.45×104 4.10×104

100

2.54×10150

5.72×104 6.70×104 7.54×104

150

6.78×103

9.02×103 9.91×103

50

9.89×10-450

3.30×10-95.70×10-5 2.21×10-3-5

2.44×10-4 1.55×10-3

2.44×10-3

Ackley3.56×10-375

2.66×10-1 8.52×10-1 7.25×10.075

1.85×10100

5.26×100 7.81×100 8.01×100

100

1.51×10-3 7.17×10-2 3.74×10-1

150

8.59×100 9.15×100 9.45×100

150

1.48×10-1 1.96×10-1 2.46×100

50

5.42×10-6 9.80×10-3 7.30×10-2

50

1.14×10-16 9.13×10-5 4.56×10-3

75

5.72×10-2 1.47×10-1 3.89×10-1

75

7.92×10-11 3.61×10-3 6.84×10-2

100

4.19×10-1 6.24×10-1 3.83×100

100

4.00×10-8 5.49×10-3 3.76×10-2

150

2.02×100 3.08×100 5.31×100

150

1.74×10-4 1.68×10-2 2.56×10-1

Griewank

608

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Simulation of self-reproduction phenomenon of cells in two-dimensional
hybrid-cellular automata modelTakeshi Ishida
Nippon Institute of Technology, 4-1, Gakuendai, Miyashiro, Minami-Saitama, Saitama, 345-8501 Japan(Tel : 81-480-33-7719; Fax : 81-480-33-7745)(Email ishida06@ecoinfo.jp)

Abstract: Understanding the generalized mechanism of self-reproduction is considered to be fundamental for
application in various fields such as mass-production of molecular machines of nanotechnology. We developed a model
for simulating cellular self-reproduction in two-dimensional cellular automaton. We demonstrated that the following 3
functions can be realized. (1) Formation of a border similar to cell membrane. (2) Self-replication is achieved while
maintaining a carrier containing information. (3) The division of the cell membrane is achieved while maintaining thetotal structure. Furthermore we constructed a hybrid cellular automaton model. To reduce the transition rules number,
we considered not only the state transition rules but also the concentration diffusion of the Gray Scott model, which
emerges self-reproduction phenomenon with certain parameters.
Keywords: self-reproduction, cellular automaton, cell division, Gray Scott modelIn our previous study[4], we developed a model for
simulating cellular self-reproduction in
twodimensional cellular automaton. We demonstrated thatthe following 3 functions can be realized by thetransition of two adjacent cells.
(1) Formation of a border similar to a cell membrane.
(2) Self-replication is achieved while maintaining acarrier containing information (information carrier).
(3) The division of the cell membrane is achieved while
maintaining the total structure of the cell.
This study demonstrated the self-reproducing abilityof a shape that was similar to that of real living cell.
Figure 1 showed the results of a cell-type selfreproducing two-dimensional cellular automaton. This
is not a study to clarify all the necessary and sufficientconditions of self-reproduction. It is considered that it is

I. INTRODUCTION
Understanding the generalized mechanism of selfreproduction is considered to be fundamental for
application in various fields such as mass-production ofmolecular machines of nanotechnology and artificial
synthetic of biology (synthetic biology). Furthermore, it
is considered that large, complex machine systems ofover a certain size are difficult to construct by the topdown approach. Therefore, these complex systems are
required to be constructed by the bottom-up approach,by applying the phenomenon of biological selforganization. Thus we have to elucidate not only thedetails of the real cellular reaction network but also thenecessary conditions for self-organized, self-replicating
cells.
Fifty years ago, von Neumann[1] initiated the studyof self-reproduction model from a mathematical view
point. This study theoretically proved the possibility of
constructing a self-reproducing machine by cell state
and transition rules of two-dimensional square cells. On
the other hand, Neumann’ self-reproducing machine
was large in size; therefore, it is difficult to implement
this machine perfectly in a computer system [2].
Thereafter, Langton[3] developed a simple machine
capable of self-reproduction abandoning thecompleteness of Neumann’s machine. Although theshape was very simple, the rules of transition arecomplicated and it could reproduce specific shapes.

©ISAROB 2011

Step 0

Step 9

Step 136

Step 232

Step 410

Step 650

Fig.1. Results of a cell-type self-reproducing twodimensional cellular automaton [4]609The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Cellular automaton model

Control the quantity offield with cell transition

Control the cell transition
with quantity of field

Reproducingof cell divisionphenomenon
by mutualinteraction of
simple rules
and field
equations

State distribution based on theGray Scott modelFig.2. Outline of hybrid cellular automaton model
possible to simulate self-replication in a real dynamic
chemical reaction environment by applying thetransition rules determined in this study.

transition rules of cell membrane formation and settled
the total states in all cells. Then, we applied thetransition rules for the division of information carriers,
following which we applied the transition rule ofmovement of the information carriers and formation of
the nuclear membrane.
To cause objective state transitions of the cellular
automata, these rules cause unexpected side effects. We
divided the transition into 4 phases and discovered thetransition rules because the discovery of wholetransition rules was difficult at a time.

II. OBJECTVEIn this study, we constructed a hybrid cellular
automaton model. Figure 2 shows the outline of hybrid
model. To reduce the transition rules number, weconsidered not only the state transition rules but also theconcentration diffusion of the field. We chose the Gray
Scott model, which emerges self-reproduction
phenomenon with certain parameters. In this hybrid
model, information carriers activate the trigger of theself-reproduction phenomenon of the Gray Scott model,
and cell membrane was formed by a part of the specificconcentration of Gray Scott model. Using this model,
we reduced the number of transition rules.

Each cell;・state; 0～19a・direction; 1～6

Fig.3 Triangular grid model in this study

III. RESARCH METHOD
2. Gray Scott model
Concerning this cellular automaton model, celltransitions patterns are similar to the physicalphenomena. Thus we considered the possibility of
substitute of transition rules to non-liner quantity model
like Gray Scott model. Equations of Gray Scott model is
as follow. Self-replication patterns are caused undercertain conditions (Du = 0.04，Dv = 0.02，F = 0.02，
k = 0.06, in this study).

1. Cellular automaton model
A two-dimensional triangular grid model was usedin this study (Fig.3). The cell automaton wasconstructed by transition rules such that the state of thenext step was decided by the state of the own cell andthat of 6 neighboring cells. Each cell has a state (0–19)
and direction (6 directions) as an attribute. In thetriangular grid, calculation starts from a certain initialcondition. The transition rules were divided into thefollowing 4 phases: 1) state transition concerning cell
membrane formation, 2) division of the information
carriers, 3) movement of the information carriers, and 4)
formation of the nuclear membrane surrounding theinformation carriers. In other words, first we applied©ISAROB 2011

¶V
= Da DV - U 2V + F (1 - V )¶t¶U= DU DU + U 2V - ( F + k )U
¶t

610

(1)
(2)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Cell membrane

Initial condition
Only information carriers

Information
carriers

Nuclearmembrane

CA modelGS model0 Step2 Step10 Step

20 Step

50 Step

66 Step

80 Step

97 Step

101 Step

CA model32 Step
GS modelCA model90 Step
GS modelFig.4 Results of hybrid cellular automaton model©ISAROB 2011

611

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table.1 Number of transition rules

CA modelHybrid model

Process

Number oftransition rules

Alternate physicalphenomenon

Number of transition
rules

Application of transition
rules of cell membrane
formation

34

Gray Scott Model

4

Application of transition
rules of information
carriers division

17

―

17

Application of transition
rules of information
carriers movement

12

―

12

Application of transition
rules of nuclear
membrane formation

13

―

5

We believe that transition rules of this model can beapplied to simulate self-replication phenomena in a real
dynamic chemical reaction environment by applying
transition rules determined in this study. We plan first toachieve cell-division simulation of a discrete particle
reaction. It is comparatively easy to replace statetransition rules with collision / reaction rules of discrete
particles.
We next plan to achieve cell-division simulation ofcontinuous chemical reaction simulation. We canconvert discrete particles rules to chemical equations.

3. Transition rules
Each cell was renewed by transition rules and thestate of the next step was decided by the state of the celland that of 6 neighboring sites. It is not found themethod to derive transition rules according touniformity law automatically at this time. Therefore, weconstructed step-by-step transition rules according to
the movement of the cell in the automaton.
In this hybrid model, firstly information carriers
activate the trigger of the Gray Scott model. States ofcell membrane appears under certain concentration ofGray Scott fields. The movement of nuclear membrane
was controlled by concentration of Gray Scott fields.

REFERENCES[ 1 ] J. von Neumann; Theory of Self-replicating
Automata, University of Illinois Press, 1966
[2] D. Mange, A. Stauffer, L. Peparodo, and G. G.
tempesti; A Macroscopic View of Self-Replication,
Proceeding of the IEEE, vol.92, No.12, Dec. 2004
[3] C. Langton ed, Artificial Life, 1/48, Addison-Wesley,
1989[4] Takeshi Ishida, Simulating self-reproduction of cells in a two-dimensional cellular automaton., Journal of Robotics and Mechatronics, Vol.22, No.5, pp.6
69-676, 2010

IV. RESULTS
Figure 4 shows the process of formation of cell
membrane, and the process of division of theinformation carriers with the cell membrane. Thus, wewere able to replicate the phenomenon of cell-likedivision.
Table1 shows the number of transition rules ofcellular automaton model and hybrid model. Using thishybrid model, we reduce the number of transition rules.

V. CONCLUSION
Our
model
producedaself-reproducing
phenomenon in a cell shape with few state transition
rules. Future directions are as follows:- Find other transition rule sets.
-Find a way to automatically derive transition
rules based on the uniformity law.
-Apply transition rules to particle collision
theoretically.

©ISAROB 2011

612

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A Physics Modeling of Butterfly’s Flight Control by GA and ANN
and Its Over-evolution Problem
R. Ooe, I. Suzuki, M. Yamamoto and M. Furukawa
Graduate School of Information and Science, Hokkaido University, Sapporo, Hokkaido, Japan(Tel : +81-11-706-6445; Fax : +81-11-706-6443)({ooe, ikuo, masahito, mack}@complex.eng.hokudai.ac.jp)

Abstract: We describe a simple physics model of a butterfly and an approach to its flight control by the genetic
algorithm (GA) and the artificial neural network (ANN). A physics model consists of two kinetic equations which are
led by a simplification of fluid force. A butterfly's flight is controlled by an ANN. The GA optimizes weights of the
ANN for the suitable flight. This approach resulted in the flight which is obtained by maximizing the prepared fitness.
However, the optimized ANN did not have a generality and a butterfly fell down in changing the initial height. A
transition of fitness throughout processes of evolution showed that too much optimization tends to break the generality.
We call this phenomenon "over-evolution". Changing conditions of experiments prevented the over-evolution.
Keywords: Physical simulation, Drag force, Butterfly, Real-coded genetic algorithm, Artificial neural networkfor various types of artificial life using physics engines[6]. To simplify the flight mechanism and to reduce a
computational amount, we modeled a butterfly’s flightinto two kinetic equations. In this paper, we describe themethod of a model and control of butterfly. A model is
described in Section Two and the control in Section
Three. We also refer to the problem about an optimized
controller. Section Four explains the details of thisproblem and a solution based on properties of actions.
Finally, our work is summarized in Section Five asconclusion.

I. INTRODUCTION
Flights by a flap of wings are difficult actions. The
motion of life existing on the ground is restricted to thetwo-dimensional space, that is, the ground. On the other
hand, life in the air, such as the bird and the butterfly, isable to move freely in the three-dimensional space
without falling down. It is very interesting to understandthese flying creatures deeply. However, there are few
researches on them as the artificial life because of therequirements of many computational resources.
We first overlook previous works on the threedimensional physical simulation. Terzopoulos et al [1].
modeled the artificial fish by springs and sensors.
Although it seemed to be a real fish, the calculation of
the drag force was dispensed with. Usami [2] simulated
the motion of a fish using the moving particle method.
However, it was confined to the two-dimensional
simulation. Wu et al [3]. proposed a model of bird forthe computer graphics. Their bird consisted of partsconnected by springs and it was taken account of the liftand drag forces. It is one of a few studies of the flying
creatures.
Sims [4] showed various shapes of artificial life andbehaviors, such as walking, jumping and swimming.
Both shapes and behaviors had been acquired by theevolutionary algorithm (EA). This work showed that theenvironment gave life various shapes and behaviors, so
it is very remarkable. Reil et al [5]. applied EAs to thecontrol problem of bipedal walking.
We had studied a modeling and simulating method©ISAROB 2011

II. MODELING
1. Drag force
The external force that acts on a flying butterfly isdrag force besides gravity. Simplified the drag force ∆D
for each surface of an object is∆D =1
ρ ∆A p C d v p 22(1)

where ρ is the density of air, ∆Ap is the area of a surface,
Cd is the drag coefficient and vp is the velocity relative
to air.
The drag force works at the center of gravity and it
is perpendicular to a surface. In applying (1) to an
object, the drag force is calculated for only the surface
facing to the direction of the velocity. If the surface isrelatively large, we divide it into sub-surfaces andcalculate for each sub-surface for accuracy.

613

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011x

Table 1. The parameters of the modelyW

Explanation
Length of wings
Width of wings
Angular velocity of wings
Flapping angle of wings
Representative area of a body
Mass of a butterfly
Tilt angle of a butterfly
Horizontal distance
Forward velocity
HeightUpward velocity

ω
L

dS

θ

dx
z

0

x

0
y
Lω

ϕ

vz

vz sinϕ0z

Fig.1. The shapes and parameters of the model

Variable
L
W
ω
θ
Sb
M
φ
z
vz
y
vy

v w' = L ω − v z sin ϕ
2. Modeling butterfly
Based on (1), two kinetic equations are formalized.
For simplification, we assume that a wing of butterfly is
an inverted triangle plane which is shown in Fig.1.
Parameters used in the model are listed in Table 1.

Accordingly, the drag force (4) is also re-expressed.
Dividing it into the forces in the direction of y and z axis
and considering the flapping angle θ lead to

M

Wx
dx
L

(2)

dF ( x ) =1
ρ C d ( x ω ) 2 dS
2

(3)

M

α
4

vw 2

Fby =

vw = L ω

dt 2

=

α
4

α
4

v w' 2 cos ϕ cos θ + Fby(7)
2

v w' sin ϕ cos θ + Fbz

11
ρ C d S b v y 2 , Fbz = ρ C d S b v z 2
2
2

(8)

The flight of a butterfly is controlled by an evolving
artificial neural network (EANN) [7]. θ is given by thecosine function of 10[Hz], while φ is controlled by an
EANN.

(4)

1. A controller by EANN
We use a three-layered feed-forward artificial neuralnetwork (ANN) as a controller. There are six neurons in
the input layer, six in the middle layer and one in the
output layer. The input signals are φ, ω, θ, vy, y and vz.
The output is a difference value of φ.
A real-coded genetic algorithm (RCGA) is used to
optimize weights of ANNs. The tournament selection ofsize = 2, the elite selection, the BLX-α crossover of α =0.45 and multiplying each weight by a random value inrange of [-2,2] for a mutation are used as operators of
the RCGA. The group size is 40, the crossover rate is
1.0 and the mutation rate is 0.01 for each weights.

(5)

B. Change of drag force by tilt angle
It is known that the flapping motion of a butterfly isexpressed by the cosine function. In this case, the total
amount of the drag force for one cycle of the flapping
equals zero, then a butterfly goes down by gravity.
Changing the tilt angle φ makes a butterfly fly up.
For simplification, we assume that the upwardvelocity vy is relatively small. When a butterfly is tilted
by φ and moving forward at speed of vz, vw is replacedby vw’. It is expressed by©ISAROB 2011d2z=

III. CONTROL

where α is a coefficient and vw is a velocity of the end of
a wing. They are expressed by

α = ρ Cd L W ,2where Fby and Fbz are the drag forces acting on a body.
They are expressed by(3) is integrated from x = 0 to x = L, then the totaldrag force F acting on two wings is given by

F=

d2 y
dt

A. Drag force acting on wings
Wings rotate up and down on the body. A minutearea dS(x) at a distance x from the body, and drag force
dF(x) which acts on dS(x) is expressed bydS ( x ) =

(6)

614

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

30

0.1

20

50000

Maximum
Averageφ [rad]0.05

100000y [m]Fitness H

150000

0
-0.05

0

-30

0

1000

20
z [m]

Generation

Fig.2. The fitness in evolution

0
-10
-20

-0.1

0

10

Fig.3. The track of a flight0

3
Elapsed time [s]Fig.4. The optimized output φ

Fig.5. The snapshot of a optimized flight

1. Generality for changes of initial heights
The ANN learned by evolution in Section Threeenables a butterfly to keep a height. Therefore we
supposed that this ANN works well for different initialheights. When a butterfly starts at 10[m] or -10[m],however, it falls down quickly and can not keep at 0[m].
This might happen unfortunately, so we tried sometimes. Through some experiments, we found out that
ANNs in a middle of evolution tend to be successful fordifferent initial heights. Typical transitions of the fitness
are shown in Fig.6. Fitness starting at different initialheights, that is to say, a generality, mostly increases inearly generations. However, as the fitness for 0[m] thatis an evolutionary guidance increases enough, agenerality tends to be lost. A generality for -10[m] iseasier to be lost than one for 10[m].
We call this reduction of generality “over-evolution”.
It resembles the over-learning, which appears in a
training of ANNs with sample data sets. An excessive
learning reduces a generality in both over-learnings andover-evolutions. However, an over-evolution does notalways happen, but happens by the probability.
To show a characteristic of over-evolutions
numerically, we have experimented on evolution threehundred times. Each evolution is continued to the two
thousandth generation. The average fitness is shown in
Fig.7. The average fitness for 0[m] gradually increases,
while the average fitness for 10/-10[m] decreases after acertain generation in a middle of evolution. The
probabilities that the fitness for 10/-10[m] is within arange from a half times to one and a half times as many

2. A result of evolution
We have experimented on a simple evolution of ourbutterfly model. The fitness value H is given by
t max

H=



v (t )∑  abs( y(zt )) + 0.1

(9)

t =0

where t is a time in the simulation and tmax is aterminated time of a simulation. H is very high when abutterfly goes forward rapidly and stays at height of
0[m]. In this experiment, we use tmax = 20.0[s], a time
step in Runge-Kutta method is 1/240 and the initialheight of a butterfly is 0[m].
Transitions of the fitness are shown in Fig.2. The
maximum fitness hardly increase in late generations,
therefore ANNs are evolved enough.
A track of the flight by the best ANN is shown in
Fig.3 and a transition of φ in Fig.4. A height of abutterfly changes little in early time and finally keeps at
0[m]. φ always changes periodically like θ. However, arange of φ changes along with a change of a height. A
snapshot of a flight is shown in Fig.5.

IV. OVER-EVOLUTION
It is known that an ANN has a generality. For
example, an ANN learned by sample data usually
recognizes other data in the pattern recognition.
Therefore we have examined a generality of the evolved
ANNs. Then we have discovered a problem.

©ISAROB 2011615

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

150000150000100000

0[m]

50000

10[m]
-10[m]

1

100000

0[m]
10[m]

50000

0

-10[m]

1000

0

10[m]
-10[m]

0

2000

2000
Generation

Generation

Generation

Fig.6. The typical evolution

Fig.7. The average fitness of 0[m]Fig.8. The probability

120000Evaluation : H

150000Evaluation : H

0.5

0

0
0

100000
0[m]
10[m]

50000

-10[m]
0

80000
0[m]
10[m]
-10[m]

40000
0

0

2000

0

Generation

2000
Generation

Fig.9. The average fitness of 10[m]Fig.10. The average fitness of -10[m]as the fitness for 0[m] are shown in Fig.8. As evolution
progresses, the probability decreases.

can happen in evolution of EANNs. The generality of
EANNs are rarely discussed. However, if a universal
method to achieve high generality is discovered,
EANNs become more useful. Our research can be a key
for leading to this great discovery. In the future, there
will be need to examine whether an over-evolution canappear in other model or not.

2. Variation of generality by aimed tasks
Based on the results above, we set a hypothesis; thegenerality depends on the behavior led by given
conditions. In our evolution, the best behavior is aflying forward as possible as fast at height of 0[m].
Compared with 0[m], a descending from 10[m] to 0[m]is additionally needed in starting at 10[m]. In the sameas 10[m], flying up is additionally needed in starting at 10[m]. The best behavior in starting at 10[m] or -10 m]
includes one in starting at 0[m], therefore we supposedthat a generality of the ANN learned by the evolution instarting at 10[m] or -10[m] is higher than one at 0[m].
The average fitness in three hundred evolution
started at 10[m] is shown in Fig.9 and one of -10[m] isshown in Fig.10. The ANN evolved by starting at 10[m]performs well in starting at 0[m] and 10[m] only, while
the ANN evolved by starting at -10[m] is successful inall three cases. Flying up is more difficult action than
flying down, therefore this result probably happened. In
this simulation, the flight starting at a low height is a“critical task”, which leads to higher generality.

REFERENCES[1] Terzopoulos D, Tu X, Grzeszczuk R (1994),
Artificial fishes: Autonomous locomotion, perception,
behavior, and learning in a simulated physical world.
Artificial Life IV, pp.327–351[2] Usami Y (2007), Re-examination of swimming
motion of virtually evolved creature based on fluid
dynamics. Advances in Artificial Life, pp.183-192[3] Wu J, Popovic Z (2003), Realistic modeling of bird
flight animations. Proceedings of ACM SIGGRAPH
2003, pp.888-895[4] Sims K (1994), Evolving virtual creatures.
Proceedings of the 21st annual conference on Computer
graphics and interactive techniques, pp.15-22[5] Reil T, Husbands P (2002), Evolution of central
pattern generators for bipedal walking in a real-time
physics
environment.
IEEE
Transactions
onevolutionary computation 6(2):159-168[6] Yamamoto M, Iwadate K, Ooe R, et al (2009),
Autonomous animated robots. International Journal of
CAD/CAM 9(1):85-91[7] Yao X (1999), Evolving artificial neural networks.
Proceedings of the IEEE 87(9):1423-1447

VI. CONCLUSION
We modeled a butterfly based on the physical law
and controlled it by EANNs. By many evolutionary
simulations, it is shown that an over-evolution problem©ISAROB 2011

Evaluation : H

Evaluation : H

Evaluation : H

200000616

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Emergence of Behavior Intelligence on Artificial Creature
in Different Virtual Fluid Environments
Keita Nakamura, Ikuo Suzuki, Masahito Yamamoto, and Masashi Furukawa
Graduate School of Information Science and Technology, Hokkaido University,
Kita 14, Nishi 9, Kita-ku, Sapporo, Hokkaido, 060-0814, Japan(Tel: +81-11-706-6445)({poco, ikuo, masahito, mack}@complex.eng.hokudai.ac.jp)

Abstract: This paper presents how the differences appear when the artificial creature, which has wings inright and left sides of the body, autonomously behaves in the air environment and in the water environment.

We construct approximate fluid environments with low computing costs to simulate the behavior acquisiti
on for artificial creature. Also, we propose a simulation method for the artificial creature in two environ
ments. As a result of simulation, we verify that it is possible for the creature to acquire behaviors in t
wo different virtual fluid environments. After evolution, the creature behaves effectively by using leverag
e fluid forces in each virtual environment.
Keywords: Artificial Life, Evolutionary Computations, Control Methods

analyzed. Instead, the implemented force adopted thesimple calculation methods for reducing the computing
time. On the other hand, in a field of numerical fluid
dynamics, many fluid simulations are based on a finite
element method and a particle method and so on.
S.Koshizuka et al.[7] suggested the moving particle
semi-implicit method. It makes it easy to create
animation on the water surface. Y.Usami[8] did asimulation of swimming motion on Anomalocaris
model in the virtual two-dimensional water environment
using the particle method. The finite element method
and the particle method give accurate results but
consume much computational time. Therefore, it isunsuitable for a real-time simulation to acquireappropriate behaviors in virtual environment. However,
we consider that the virtual environment needs to obey
the physical laws for the virtual creature to acquire amore natural policy of adaptive behaviors.
In this study, we construct approximate fluid
environments which enable us to do the behavior
acquisition simulation with low computing costs. Also,
we propose a simulation method for the artificial
creature in consideration of two different environments,
under-water and air. The artificial creature imitating bird
is modeled by connecting rigid bodies. This artificial
creature can behave by flapping their wings in two
environments, which are given by changing somephysics parameters such as density and drag coefficients.
To control wings and learn the behaviors, an artificial
neural network (ANN) is implemented with the creature.

I. INTRODUCTION
Many computer simulations have been performed forstudying acquiring behaviors, evolution, and learning
methodologies on a virtual artificial life creature in thefield of artificial life (Alife) and evolving robotics. X.Tu
et al.[1] realized a behavior of an artificial fish whose
controller learns swimming in the virtual water
environment. C.W.Reynolds[2] proposed a flock
simulation approach based on a distributed behavioral
model without setting the orbit of each bird. Thisapproach makes it easy to create flock animation.
K.Sims[3][4] showed that the virtual creature canacquire its morphology and behavior by an evolutionary
methodology based on the creature's competition. Many
studies for behavior acquisition are based on Sims'
studies. N.Chaumont et al.[5] applied Sims' model toevolution of virtual catapults. This creature could throw
its parts as far as possible. There are a lot of simulations
for artificial creature using physical calculating engines.
They enable these creatures to obey physics law easily.
I.Tanev et al.[6] simulated a side winding locomotion ofa “snake-like robot”. In these studies, the experimental
environment is set as an ideal environment in acomputer simulation space because they considered thatthe methodology of evolving learning behavior in an
ideal environment is more important than acquisition of
the similar behavior in a realistic environment.
Therefore, the influenced force from the fluid
environments to the artificial creature is not precisely©ISAROB 2011

617

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Left Wing
Buoyancy (FB)

Body

Plane view
Gravity (mg)

Fig.2. Drag model

Genetic algorithm (GA) is applied to ANN by its
evolution. As a result of simulation, we verify that it ispossible for this creature to acquire behaviors in two
different virtual fluid environments. After evolution, thiscreature behaves effectively by using leverage fluid
forces in each virtual environment. In addition, we
analyze the acquired behaviors by the examining a
relation between fluid environments and acquired
behaviors.

+θL

+θR

-θL

-θR

(b)
Fig.3. Artificial creature model
where ρ[kg/m3] is the density of the fluid, V[m3] is thevolume of the object, and g[m/s2] is the acceleration of
the gravity.

II. CONSTRUCTION OF VIRTUAL FLUIDENVIROMENTS

2. Modeling Drag
Based on fluid dynamics, we model the drag asuniformly distributed forces to the surface of the object
in the opposite direction of moving direction (Fig.2). In
the field of fluid dynamics, using the dynamic pressure
1of a flow 2 ρU2[kg/m・s2] derived analytically as thestrength of the drag in the fluid, FD[N] is given byequation (2),We assume that the buoyancy and drag act as theforce that virtual objects (rectangular parallelepipeds)
receive from the fluid effect. We construct a virtual fluid
environment by modeling two forces acting on theobject in the environment. These two forces compare to
the buoyancy and drag respectively. The simulation is
performed by calculating movement of the object which
obeys a physics law, resulting in an animation. We use
the "PhysX[11] (offered by the NVIDIA)" as a physicalcalculating engine. PhysX is applied to calculate a basic
physical operation, for example, gravity, frictional force,collision among virtual objects, and so on. In the virtualenvironment, the density of the air ρA is 1.205[kg/m3],
the density of the water ρw is 998.203[kg/m3] andacceleration of the gravity g is 9.8067[m/s2]. We
construct two fluid environments by changing theparameter of fluid density.

1

FD = 2 CDρSU2(2)

where CD is a scalar quantity called the drag coefficient,and S[m2] is the reference area of the object. The dragcoefficient depends on the shape of the object. In thisstudy, the drag coefficient of a rectangularparallelepiped CD is 1.50. The reference area of theobject is the projection area of the object to the plane
which is perpendicular to a flow.
Artificial creature can generate propulsion force bymoving its bodies because the modeled drag force isadded to its bodies when this creature moves its bodies.

1. Modeling Buoyancy
Based on Archimedes' principle, we model thebuoyancy as a force whose strength FB is equal to theweight of the fluid volume which an object occupied in
the fluid. This force acts on the center of mass in the
opposite direction of gravity (Fig.1). The strength of thebuoyancy in the fluid environment, FB[N] is given byequation (1),FB = ρVg(1)©ISAROB 2011○：Actuator

Front view(a)Drag (FD)
Fig.1. Buoyancy model

Right Wing

Moving Direction

III. EXPERIMENT FOR BEHAVIORACQUISITION
We examine how the differences appear when theartificial creature autonomously behaves in the air
environment and in the water environment. It is
assumed that the model must move upward as
efficiently as possible. Evolutionary computing (EC) is
adopted to obtain the adaptive behavior.

618

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 1. Dimensions of each part of model
Parts
Size [m]Density [kg/m3]
100.0 (in air)Body
4.0×4.0×0.50
998.2030 (in water)
100.0 (in air)Right Wing 6.0×3.0×0.250
998.2030 (in water)
100.0 (in air)Left Wing 6.0×3.0×0.250
998.2030 (in water)

ANN

Table 2. Input and output parameters of ANN
A flapping angle of a wing for a gradientof body in each time (θR, θL)
An angular velocity of a wing for a bodyin each time (ωR, ωL)
Input
An inner product of vertical upward vector
and cross direction vector of its body
An inner product of vertical upward vector
and forward direction vector of its body
A difference flapping angle of a wing for a
Outputgradient of body in each time (θR, θL)

GA

Step
Feval =

1. Artificial Creature ModelWe create the artificial creature by connecting rigid
bodies with actuators. The modeled artificial creature
imitates a bird, which can behave by controlling its
wings. After evaluation of the artificial creature by EC
in two environments, this creature behaves effectively
by using leverage fluid forces in each virtualenvironment. Figure 3 (a) shows an artificial creaturemodel. This model consists of three rectangularparallelepipeds. Table 1 shows the dimensions of eachpart of this model. The model has two actuators with
one degree of freedom (Fig.3 (b))

 Height(t)

(3)

t=0
where Step is the number of steps used for thesimulation at each generation, Height(t) is the height of
the artificial model at step t, We use a rank selection as areproduction operation based on the evaluated value andan elite preserving operation in the GA. We sort theindividuals in ascending order of their evaluated value
and preserve the best five individuals. The others are
modified by crossover and mutation operations. The
artificial creatures evolve, learn, and acquire thebehaviors in the two fluid environments.
4. Result and Discussion
We upload the movies to URL[12] that the artificial
model acquired behaviors in two fluid environment.
Figure 4 shows the angle between the body and bothwings in the air environment for one second after
learning. Figure 5 shows the angle for one second after
learning in the water environment.
From these results, in the air environment, bothwings of the model oscillate periodically. The speed toswing down the wings is faster than that to swing up thewings. Therefore, the model after leaning can move
upward by rather swinging down the wings than swing
up the wings to generate a drag difference (Fig.6 (a))
and needs to balance itself by generating the force larger
than gravity. On the other hand, in the water
environment, the angle between the body and bothwings propagate from the right wing to the left wing.
The angle between the body and left wing is larger thanthat between the body and the right wing. Therefore, the

2. Control Method for Artificial Creature Model
ANN is introduced to control artificial creature's
actuators for flapping autonomously based on its state.
ANN consists of the outputs of the three-layer feedforward ANN. Table 2 shows the input and output
parameters of the ANN. The number of the neurons in
the hidden layer is same number of ones in the input
layer. Synaptic weights of the ANN are initialized by arandom value at first. This creature enables itself tobehave by optimizing the ANN synaptic weights.
3. Experiments for Behavior Acquisition
We experiment to examine how the differences
appear when the artificial creature autonomously
behaves in the air environment and in the water
environment. The GA optimizes the synaptic weights ofan ANN. Table 3 shows experimental conditions. An
evaluated value for the GA as a fitness function is set sothat the creature flies as high as possible. This evaluatedvalue Feval is given by equation (3).

©ISAROB 2011

Table 3. Experimental condition
The number of the neuron
6in the input layer
The number of the neuron
6in the hidden layer
The number of the neuron
2
in the output layer
A range of an object[-50°, 50°]
flapping angle
Genotype
Weightij
Phenotype
Feval
Population30
1Step
1/120 [s]Simulation step
1200
Generation number
200
Crossover probability
0.10
Mutation probability
0.90

619

Angle between the bodies [degree]

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Left Wing
Right Wing(a) Air environment(b) Water environment
Fig.6. The model acquired behavior

Elapsed step

Angle between the bodies [degree]

Fig.4. Angle of body and both wings
in the air environment

body acquires behaviors to move freely in the air
environment and in the water environment bycontrolling flapping angles and feathering angles.
Additionally, we would like to explore "life-as-it-couldbe" by controlling the artificial creature which has
many wings.

REFERENCES[1] X.Tu and D.Terzopoulos (1994), Artificial fishes:
physics, locomotion, perception, behavior. In
Proceedings of the 21st annual conference on Computer
graphics and interactive techniques:43-50[2] C.W.Reynolds (1987), Flocks, herds and schools: A
distributed behavioral model. In Proceedings of the
14thannual conference on Computer graphics andinteractive techniques:25-34[3] K.Sims (1994), Evolving virtual creatures. In
Proceedings of the 21st annual conference on Computer
graphics and interactive techniques:15-22[4] K.Sims (1994), Evolving 3D morphology andbehavior by competition. In Proceedings of the FourthInternational Workshop on the Synthesis andSimulation of Living Systems:28-39[5] N.Chaumont, R.Egli, and C.Adami (2007), Evolving
Virtual Creatures and Catapults. Artificial Life
13(2):139-157[6] I.Tanev, T.Ray, and A.Buller (2005), Automatedevolutionary design, robustness, and adaptation of
sidewinding locomotion of a simulated snake-like robot.
IEEE Transactions On Robotics 21(4):632-645[7] S.Koshizuka, A.Nobe, and Y.Oka (1998), Numericalanalysis of breaking waves using the moving particle
semi-implicit method. International Journal for
Numerical Methods in Fluids 26(7):751-769[8] Y.Usami (2007), Re-examination of Swimming
Motion of Virtually Evolved Creature Based on Fluid
Dynamics. Lecture Notes in Computer Science
4648:183-192[9] NVIDIA PhysX: http://www.nvidia.com/object/ph
ysx_new.html[10] Result Movie: http://autonomous.complex.eng.ho
kudai.ac.jp/researches/physics-modeling/movies/nakam
ura/

Left Wing
Right Wing

Elapsed Step

Fig.5. Angle of body and both wings
in the water environment
model after leaning can move upward by oscillating
periodically like a sea snake (Fig.6 (b)). The right wing
plays a role of head, and the left wing plays a role of tail.

VI. CONCLUSION
In this paper, we modeled the received effect fromthe fluid environment by introducing two forcescomparing to buoyancy and drag and constructed two
fluid environments (in air environment and in water
environment) by changing the parameter of fluid density.
Secondly, we examine how the differences appear when
the artificial creature model autonomously behaves in
the air environment and in the water environment. From
the result, it is possible for the model to acquire
behaviors in two different virtual fluid environments byusing evolutionally computations (ANN and GA). Afterlearning, this creature behaves effectively by using
leverage fluid forces in each virtual environment. In a
future, we would like to experiment that the artificial
creature which has wings in right and left sides of the©ISAROB 2011620

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Behavior emergence of virtual creature living in complex environments
Kenji IWADATE, Ikuo SUZUKI, Masahito YAMAMOTO and Masashi FURUKAWA
Affiliation of authors, address(Tel : 81-11-706-6445; Fax : 81-11-706-6445)(Email address : kenji-i@complex.eng.hokudai.ac.jp)

Abstract: This study aims at establishing a new computer aided animation method using the agent-based and physics
modeling based animation. The specific problem we treat in this paper is to acquire an adaptive behavior of a virtual
creature placed in a complex environment and to create the animation of its behavior automatically. The virtual creatureis regarded as an autonomous agent who has sensors, actuators, and controllers. For controlling the virtual creature,
ANN and CPG are adopted as the controllers. Optimization algorithm is introduced for learning the controllers.
Numerical experiments proved that the virtual creature acquires effective motions (walking, swimming) to pursuit thedestination, and to avoid the obstacle and other creatures.
Keywords: Artificial Neural Network, Central Pattern Generator, Evolutionary Algorithm, Artificial Life
Evolution of the virtual creature is realized bylearning ANN and CPG. An optimization algorithm isintroduced into learning for evolving the virtual creatureplaced in complex environments. An evaluation
function to evaluate the behavior of the virtual creature
for the optimization algorithm is based on radical
characteristics of life such as energy conservation, target
pursuit and evasion from obstacle. This evaluation
function is applicable to evaluate virtual creature which
has any shapes.

I. INTRODUCTION
Computer-aided animation using the computer
graphics technology becomes more important in various
fields such as physics, engineering, entertainment, andmedical science. In entertainment field, physics
modeling based simulation for generating the realistic
object motion has attracted researchers attention, andmany works have been presented in this decade[1],[2].
On the other hand, the physics modeling based
animation for artificial-beings, which can autonomously
behave as living organisms in the earth, is still anundergoing research matter. A motion capture method ismostly adopted to create the animation for the artificialbeings. However, this method consumes lots of time andrequires an expert knowledge for creating the animation.
The agent-based approach is efficient solutions forovercoming problems that the motion capture methodhas. This study aims at establishing a new computer
aided animation method using the agent-based andphysics modeling based animation. The specificproblem we treat in this paper is to acquire an adaptive
behavior of a virtual creature placed in a complex
environment and to create the animation of its behavior
automatically. The complex environment means adynamically changed environment caused by fluid
influences, obstacles and interaction among agents.
For generating a control signal with rhythmic patternwhich is observed in living organisms, we implement a
controller of the virtual creature by a combination of acentral pattern generator (CPG) and an artificial neuralnetwork (ANN).

©ISAROB 2011

II. VIRTUAL ENVIRONMENTAll experiments described below are performed
under virtual environment which fundamental physicallaw is considered. We adopt a dynamics engine for
implementing the fundamental physical law with virtualenvironment. The adopted engine is PhysX, presented
by NVIDIA[4]. PhysX allows us to simulate physicaldynamics and phenomenon of a rigid object such as
gravity, action-reaction, friction, restitution, andcollision. Additionally, this environment is implemented
influenced forces caused by fluid.

II. VIRTUAL CREATURE
1. Salamander model
A salamander is modeled as a virtual creature. The
salamander can behave itself under the water and air
resistances namely a complex environment. Then, we
examine the salamander behavior by evolving it in thecomplex environment. The salamander consists ofseventeen rigid objects whose geometric and physicaldata are shown in Table1. The salamander has two
sensors and twelve actuators (see Fig.1).

621

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig.1. Salamander model
Table1. Data of components of the salamander model
Body Components
Value
Density
ρ=1000 [kg/㎥]
Restitution
0.2
Static friction
0.2
Dynamic friction
0.1
Limb Components
Value
Density
ρ=1000 [kg/㎥]
Restitution
0.2
Static friction
0.9
Dynamic friction
0.8

Fig.2. Controller Configuration diagram
Table2. Input signal of ANN
From each sensor
Value
Light strength
SL [-1, 1]
Inclination of sensor
SI [-1, 1]
Environmental pressure
SP [0, 1]
From each actuator
Value
Angle
cosθA [-1, 1]
Normalized angular velocityωA /ωAmax [-1, 1]
ANN is a well-known brain model. It consists of aset of neurons and set of synapses. Learning of ANN isperformed by adjusting a set of weights assigned tosynapses. A neuron model of ANN is defined by (4) and(5), where ui is an input of each neuron, vi is an output
of each neuron, wji is a synaptic weight, and T is atemperature coefficient. An output of ANN becomesparameters for modifying a CPG output.
(4)
u
w v

2. Sensors
Optical sensors can detect a light strength SL from alight source placed in a virtual space. SL is defined by(1), where θL is an angle between the light source andthe sensor direction, and Br is brightness of an lightsource. Inclination sensors detect a Inclination of
sensors SL according to (2), where θI is a inclined angle.
Pressure sensors detect a environmental pressure SP
according to (3). These values turn into inputs of thesalamander controller.
/2/ 2 : Br cos L
L
(1)
SL
:
0
otherwise
SI

SP

cos
P

i

threshold : 1: 0

otherwise1(5)
e u /T 1
CPG is a well-known neural model for generating arhythmic pattern which is observed in behavior of livingorganisms [4],[5]. A neuron model of CPG is defined by(6)-(8), where Tr and Ta are coefficients of time response,
aij is a synaptic weight between CPG neurons, gi is aparameter for modifying an amplitude of output signal,and bi is a parameter for modifying a frequency of theoutput signal. CPG parameters are optimized by an
optimization algorithm to acquire a periodical signal
whose frequency is under 60[Hz] (see Table3). The
number of CPG neurons is equals to the total number ofdegrees of freedom for actuators. Actuators are driven
according to output signals from CPG as control torques.
(6)
T x& xa y bz g(3)

3. Actuators
Actuators have a three degree of freedom in rotation,and these are driven according to driving torques whichis generated by the controller.
4. Controller
For generating a control signal with a rhythmic
pattern, a controller of the virtual creature is modeled by
a combination of a central pattern generator (CPG) andan artificial neural network (ANN). A structure of thecontroller is shown in Fig.5. ANN receives signals fromsensors and actuators (see Table2), and it calculate CPG
parameters. CPG converts ANN outputs into control
signals which have the rhythmic pattern.

©ISAROB 2011

j

vi

(2)

I

ji

j

r

i

i

ij

j

i

i

i

j

Ta z& i
yi

622

yi

(7)

max( 0, x i )

(8)

zi

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table3. CPG constants
Synaptic weight
aij =1.070772
Rise time constantTr =1.181958
Time lag of the adaptation Ta =0.153337
effectLight source
θLvt

3. OPTIMIZATION OF CONTROLLER
A salamander’s behavior is dominated by thecontroller. Therefore, adapting the salamander to an
environment depends on adjustment of weights assigned
to synapses in ANNs and CPGs. This adjustment is socalled learning. However, it is difficult to define
learning signals to train controller when the virtual
creature has complicated shapes and it virtually lives in
the complex environment. For letting the controllers
learn, we adopt Genetic Algorithm (GA) with real
number encoding. The GA optimizes synaptic weights
of ANNs and CPGs. A chromosome is represented by aset of weights. Table4 shows parameters for
optimization.

Fig.3. Evaluation values
Br = 1.0(0,0,100)(0,0,-100)
Br = -1.0(0,0,-100)
Br = 1.0(c) pattern C(d) pattern D
Fig.4. Settings of the Evaluation
We treat a learning problem what behavior asalamander can acquire when it moves towards a given
destination and evade from obstacle. It is expected thatwalking, running, or swimming emerges from theacquired behavior in the given environment.
In this problem, a light source is set in the field as apheromone (destination or obstacle). All light source
can change these brightness Br within [-1,1]. When thebrightness has positive value, we treat it as thedestination. Otherwise, we treat it as the obstacle.
A fitness function consists of two fundamental
behavior evaluations. The first evaluation is formulated
so as to maximize the cumulated propulsive componentof velocity V during the salamander moving (see Fig.3).
The second one is formulated so as to maximize thecumulated cosine value of the angle between the sensor
and the light source L during the salamander moving.
These two evaluations are expressed in (11)-(12),where NS is the number of sensors.

The optimization process consists of the following
steps.
I.
Simulate each virtual creature in a constructedenvironment according to ANNs and CPGs
generated by each chromosome.
II.
Evaluate behaviors of each creature by use of agiven fitness function.
III.
Reproduce new creatures.
IV.
Perform GA operations (selection, crossover,and mutation).
V.
Return to step II and repeat until thetermination condition is satisfied.
The mutation operation randomly selects achromosome (a set of weights) and extracts a gene (aweight) from it with the mutation rate PM and replace
the gene to a random number [-1.0, 1.0]. The crossover
operation selects a couple of chromosomes with thecrossover probability PC and selects one of the output
neuron and its neighborhood synapses (which areconnected to the selected neuron unit directly). Thenthese selected synapses of one chromosome areswapped to those of other chromosome.

Vstep N S
t

L

623

vt ,i

(11)

SL

(12)

i

step N St©ISAROB 2011

Br = -1.0(b) pattern B(a) pattern A

Table4. Parameters for Genetic Algorithm
Number of chromosome
200
Mutation rate
PM =0.05
Crossover probability
PC =0.1(0,0,100)i

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

These values are calculated under four initial
positions shown in Fig.4. Therefore, the fitness function(Eq.(13)) is expressed as a cumulated product of Eq.(11)and Eq.(12).
patternr r(13)f (V , L )V Lp

Fig.5. An acquired behavior on the groundpp

4. EVOLVING SALAMANDER BEHAVIOR
Experiments are performed in the following
conditions. Basic parameters of experiments are shownin Table 5.

Fig.6. An acquired behavior in the water

Table5. Parameters for optimization
Time resolution
1/60 [sec]Simulation time3000 [step] = 50 [sec]Each experiment is performed under the different
environment described bellow.

(a) First destination

1. Acquired behavior on the groundIn the first experiment, the salamander is placed on
the ground. Also, the friction and air influence are
implemented with the environment.
Fig.5 shows a snapshot of a motion that thesalamander moves on the ground after the optimization.
The salamander acquires effective motions to reach thegoal as fast as possible. The achieved motion looks likea “walk” behavior.

(d) Swarm behavior(c) Third destination
Fig.7. behavior emergence in complex environments

2. Acquired behavior in the water
In the second experiment, the salamander is placedin the water environment.
Fig.6 shows a snapshot of a motion that thesalamander moves in the water after learning. The
controller acquires control signals which cause rapid
propagation of vibrations for generating a thrust force
by harnessing the water resistance. The achieved motion
looks like a “swim” behavior.

and physics modeling-based animation method is
introduced. Numerical experiments proved that thevirtual creature acquires effective motions (walking,swimming) to pursuit the destination and to avoid the
obstacle and other creatures.

REFERENCES[1] J. J. Monaghan (1992), Smoothed Particle
Hydrodynamics. Annual Review of Astronomy andAstrophysics[2] S. Koshizuka, A. Nobe, Y. Oka (1998), Numerical
Analysis of Breaking Waves Using the Moving Particle
Semi-Implicit Method. Numerical Methods in Fluids
26(7):751-769[3] NVIDIA PhysX,

3. Behavior emergence in complex environments
Fig.7(a)-(c) show behavior emergence of the virtual
creature in the environment with an obstacle. Fig.7(d)shows emergence of a swarm behavior. Each Creatureacquires the tracking behavior toward to the destination,and the avoiding behavior from obstacles and othercreatures.

http://www.nvidia.com/object/nvidia_physx.html[4] K Matsuoka, “Sustained oscillations generated by mutually inhibiting neurons with adaptation”, Biol Cybern
52, pp. 367-376, 1

VI. CONCLUSION[5] L. N. Patel, A. Murray, J. Hallam (2005), Increased
Swimming Control with Evolved Lamprey CPG Controllers.
Neural Networks, IJCNN '05 4:2195-2200

We aim at establishing a new computer aided
animation method. For this purpose, the agent-based©ISAROB 2011(b) Second destination

624

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Three-dimensional Morphogenesis by Cell Division and Death inViscoelastic Amorphous Computing
Eisuke Arai, Fumiaki Tanaka and Masami Hagiya
Department of Computer Science, Graduate School of Information Science and Technology, University of Tokyo,
7-3-1 Hongo, Bunkyo-ku, Tokyo 113-8656, JAPAN.
(Tel : +81-3-5841-4093)(arai917@is.s.u-tokyo.ac.jp)
Abstract: Amorphous computing is a computational paradigm for realizing global computation as a result of local
communication among identical computational particles which are irregularly distributed over space. Each
computational particle has a small computing power and a small amount of memory, and executes an identical programwith no synchronization. In the original model of amorphous computing, particles did not have motility resulting fromphysical interactions and could not form any significant structures by their self-organization in three-dimensional space.
In this paper, we propose a new approach to construct desired structures of programmable particles in three-dimension
by extending the original model of amorphous computing. In addition to general operations in the original model, we
introduce operations for generation and destruction of particles. Furthermore, we allow particles to have slight motility
by viscoelasticity, because we assume physical interactions among particles. So far, we have built a simulator toobserve behaviors of moving particles in three-dimensional space and control their morphogenetic process. In thissimulator, the particles execute an identical program which includes the operations for generation and destruction. We
have investigated and optimized parameters of the simulator so that the particles moderately cluster and form somefixed structure.
Keywords: Amorphous computing, Morphogenesis, Viscoelasticity, Three-dimension.
by biological systems [6]. Thus there has beendeveloped an application of amorphous computing inwhich cells such as E coli are targeted [7]. Needless tosay, one of the goals of such applications is to constructshapes and structures composed of particles, just as
biological systems realize morphogenesis.
In this paper, we present a new approach to threedimensional morphogenesis in a model of amorphous
computing. We give particles slight motility that
depends on interactions among them, and implement thegeneration and destruction operations in addition to theoperations in the original model of amorphous
computing. Two-dimensional morphogenesis in a modelof amorphous computing was investigated in apreliminary attempt [8]. Although a three-dimensional
approach to morphogenesis in amorphous computing
was also proposed [9], motility of particles was notbased on their physical interactions but due topredefined background signals, and particles did not
execute an identical program. In our approach, we
assume that each particle executes an identical programand moves automatically by viscoelastic force. The
viscoelasticity of particles is generated according to
distance from the neighbors and density of theneighbors. We refer to smooth particle hydrodynamics(SPH [10]) in order to simulate the viscoelastic
behaviors of particles and implement calculation of

I. INTRODUCTION
Models of amorphous computing consist of acollection of small computational particles distributed
irregularly over space where each particle has a small
computing power and a small amount of memory. These
individual particles are entirely identical and notsynchronized. Each particle can communicate with a
few nearby neighbor particles. Although all particles areprogrammed identically, each particle is distinguishedfrom other particles by its local state which includes thelist of neighbor particles, distance from each neighbor,and local variables. The study of amorphous computing
was introduced by Abelson et al. in [1] and thecomputational models based on the idea of amorphous
computing were summarized in [2]. Among the models,that of programming amorphous medium wasestablished by Beal et al. [3]. Amorphous medium is thecontinuous limit of models of amorphous computing inwhich particles are infinitesimal and distributed densely
over space. Proto [4] is a Lisp-like programming
language which targets such amorphous medium. As an
example of programming amorphous medium, a sensor
network was implemented in Proto [5]. The advantage
of amorphous computing is its robustness comparedwith other general computational paradigms because theoriginal model of amorphous computing was inspired©ISAROB 2011625

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011neighbor gradient values and the local maximal value ofranges to its neighbors which ranges are known as apriori information. Sum of these local values is stored as
new gradient value in single communication step of theoperation. By repetition of the local communication, allparticles know their distance values from the specified
particle.
C. Generation and destruction operations
Inspired by cell division and death, we implement
the generation and destruction operations which
increase and decrease particles, respectively. In thegeneration operation, a particle reproduces itself with its
own state at an adjacent point. On the other hand, in thedestruction operation, a particle kills itself. In our model,
we do not consider the energy consumption withreproduction of particles and the remains of particles inthe destruction operation.

viscoelastic force using a method established by Clavet
et al. [11]. We need to update neighbors of each particle
at every simulation step, because particles have motility
and change their positions. Thus we have implemented
efficient neighbor search strategy of particle-based
fluids [12].

II. THE MODEL
Our simulation model is based on the original
amorphous computing model. The originality of ourwork is roughly divided into four important points. In
this section, we explain these points individually.
1. Motility of computational particles
We introduce motility of particles, because we
assume that particles are like biological organisms or
biomolecules. Thus it is natural that there are physicalinteractions among particles such as viscosity andelasticity. We use the well known method for particlebased viscoelastic fluid simulation in order to add
viscoelastic force to particles. This force is assumed to
be produced by interactions among nearby particles andaffects them. For each particle, we need to calculate
force by all interactions with its neighbors at every
computation step. Although we introduce motility, we
do not add to particles the ability to move freely
because we only allow particle to be affected byexternal force.

3. Mechanism of our simulator
We built a simulator which executes the particle
operations just as we introduced above. Afterinitialization, the simulator repeats updating states ofparticles and visualizing them. The single updateprocess executes the necessary tasks as follows. At first,
the simulator executes the neighbor search for allparticles, calculates viscoelastic force between allparticle pairs, and then moves all particles by thecalculated force. Next, the simulator executes theneighbor search again because neighbor particles arechanged by their movement. Finally, each particle
communicates with its neighbors and updates its state
which includes some values related to the operations.
Only one communication step for one particle isallowed in a single update. Thus, by repetition of the
update process, the value for each operation converges.
In our simulation, we use modestly converged values
for a cascade of operations.

2. Implementation of particle operations
To execute programs written in the style ofamorphous computing in our model, it is necessary to
implement several basic operations and build them into
particles. We can execute complex programs bysystematically combining these operations. Concretely,
we implement the following operations.
A. Minimum and maximum operations
In the minimum operation, each particle obtains theminimum of the specified values owned by all neighborparticles by repetition of local communication. A
particle refers to the specified values of neighbors, finds
the minimal value of them and then stores this value
into itself in a single communication step. Similarly, themaximum operation allows each particle to find the
maximal value of those owned by all neighbors.
B. Gradient operation
The gradient operation is a way for each particle toknow distance from a specified particle as the gradientvalue. Each particle finds the local minimal value of©ISAROB 2011

4. Dynamic neighbor search in 3D space
In our system, it is necessary for each particle toalways know its neighbor particles so that it can contact
and interact with each other. Since we assume thatparticles move by some operations, we have todynamically update neighbors at every computation stepof each operation. To search for neighbors, we need not
check all particles, but need to check only someparticles which are included in particular divisions ofspace called cells. Each cell is a 3D grid with size of theneighborhood radius of a particle. We divide space intosuch cells and index them depending on coordinates of626

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011of these particles “path”. For a particle on path, the sumof its gradient values from the source and from thedestination is almost equal to distance between thesource and the destination. Neighbors of particles on thepath comprise a channel which connects the source andthe destination with slight width. In Figure.2, it isshown how to make the path and channel in the cluster.
After determination of the particles in channel, eachparticle in the channel executes the destruction
operation and disappears from the space.

their minimal corner. Furthermore we associate eachparticle with its corresponding cell index. For searchingfor neighbors of a particle, it is only necessary to checkparticles inside 26 cells around the cell containing theparticle. In addition to this cell indexing approach, we
use also subdivision of the cell in order to further reducesearch space.

III. EXAMPLE
In this section, we demonstrate the process to form atubular structure as an example of our approach. We
explain an algorithm for modeling tubular structure andthen show simulation results of this example using oursimulator.
1. Algorithm for modeling tubular structure
We assume that there is one computational particle
at the initial condition. This particle begins thegeneration (divide) operation and keeps on dividing
until the number of particles is sufficient to form any
structure. After the particles are clustered, one particle
on the surface of the cluster is selected as a source.
By searching for the particle which has the minimal
number of neighbors in all particles, we can find thesource on the surface. We then execute the gradient
operation from the source and choose the destination
particle which has the maximum gradient value, that is
to say, the farthest particle from the source. The
destination is also assumed to be on the surface of thecluster. Figure.1 shows the state of choosing the sourceand destination particles in the cluster.

Fig.2. Creating tubular structure
An important issue in this algorithm is that eachparticle executes an identical program andcommunicates with only its neighbors. Although eachparticle has no a priori knowledge about other particles
except for its neighbors, it is distinguished from otherparticles by its internal state such as the gradient value.
Therefore, we can control identically programmed
particles in order to model desired structures like thisexample.
2. Simulation results
Using our simulator we could construct a cluster of
500 particles which is shown in Figure.3. There wasonly one particle at the initial condition and then the thenumber of particles started increasing by the generation
operation. To cluster particles as shown in Figure.3, weinvestigated and optimized some parameters of thesimulator by trial and error. These parameters include
neighborhood radius, spring constant, density constantand other values related to calculate viscoelastic force.

Fig.1. Choosing specific particles
After both source and destination particles are found,each particle starts the gradient operation towards thedestination to determine the specific particles lying
between the source and the destination. We call the line

©ISAROB 2011627

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

・We need to increase efficiency of each simulation
step because we want to enlarge the size parametersof our model such as the simulation space and thenumber of particles. To this end, we will attempt to
optimize the dynamic neighbor search and thecalculation process of local interactions.
・Although we currently assume a spherical clusterof particles in our model, we need to consider acluster of any shape like a distorted one.
・In this paper, we demonstrated construction of atubular structure. We want to extend our study toestablish a new method for constructing arbitrarily
complex structures.

Fig.3. A cluster of 500 particles
After clustering particles and finding the source,each particle executed the gradient operation from thesource and then could determine the destination particle
which had the maximum gradient value. Each particle
also could judge whether it was on the path andfurthermore in the channel. The particles in the channel
executed the destruction operation and disappeared. As
a result, we obtained a tubular structure like a bead (seeFigure.4).

REFERENCES[1] H. Abelson, D. Allen and D. Coore et al. (2000),
Amorphous computing. Communications of the ACM,
43:74-82[2] H. Abelson, J. Beal and G.J. Sussman (2007),
Amorphous Computing. MIT Technical Report 2007030.
[3] J. Beal (2006), Amorphous Medium Language. MIT
Technical Report 2006-040.
[4] J. Beal and J. Bachrach (2006), Infrastructure forengineered emergence in sensor/actuator networks.
IEEE Intelligent Systems:10-19[5] J. Bachrach and J. Beal (2006), Programming a
Sensor Network as an Amorphous Medium. MIT
Technical Report 2006-069.
[6] J. Beal and G. Sussman (2005), BiologicallyInspired Robust Spatial Programming. MIT Technical
Report AI Memo 2005-001.
[7] J. Beal and J. Bachrach (2008), Cells Are Plausible
Targets for High-Level Spatial Languages. SelfAdaptive and Self-Organizing Systems Workshops,
2008: 284-291[8] A.Kondacs (2003), Biologically-Inspired SelfAssembly of Two-Dimensional Shapes Using Globalto-Local Compilation. IJCAI'03 Proceedings of the 18th
international joint conference on Artificial intelligence.
[9] A. Bhattacharyya (2006), Morphogenesis as an
Amorphous Computation. Proceedings of the 3rd
conference on Computing frontiers:53-63[10] J.J. Monaghan (1988), An introduction to SPH.
Computer Physics Communications 48(1):89-96[11] S. Clavet, P. Beaudoin, and P, Poulin (2005),
Particle-based
Viscoelastic
Fluid
Simulation.
Proceedings
of
the
2005
ACM
SIGGRAPH/Eurographics symposium on Computer
animation:219-228[12] J. Onderik and R. Durikovic (2008), Efficient
Neighbor Search for Particle-based Fluids. Journal ofApplied Mathematics, Statistics and Informatics (4):2943

Fig.4. A tubular structure

IV. CONCLUSION
In this paper we have presented a new approach forconstructing a specified structure from a cluster ofcomputational particles which are programmed to
execute an identical operations. To cluster the particles
we implemented the generation operation and gavethem slight motility based on viscoelasticity among
particles. As an example of our approach, wedemonstrated construction a tubular structure by eachparticle’s executing an identical program. Although we
do not have actual implementation of suchprogrammable particles at present, we expect that wecan possibly use living cells for our research in future,
because a cell behaves according to its program, that isDNA.

V. FUTURE WORKS
Our future work can be considered in several ways
as follows.

©ISAROB 2011

628

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Demand level investigation for future domestic system
Shuki Inoue1,

Eiji Mimura1,

Shingo Aoki2 and Hiroshi Tsuji2

1) Department of Energy Use R&D Center
The Kansai Electric Power Co., Inc. Hyogo, Japan
2) Department of Computer Science and Intelligent Systems
Osaka Prefecture University, Osaka, Japan(Tel : 81-72-254-9353; Fax : 81-72-294-9915)(inoue.shuuki@d4.kepco.co.jp)

Abstract: Although the future domestic system has been visualized as smart house, it is difficult to decide if eachproduct or service in smart house is useful or not because value senses in our daily life are diversified. To forecast thestrength of demand on future domestic system, this paper describes questionnaire analysis. Creating six kinds of future
domestic systems which innovate our daily life, we design questionnaire which consists of question on personal
attributes, value senses and requirement on designed systems. This paper also introduces original analysis methods andillustrates findings in the responses of questionnaire.
Keywords: Future Assessment, Home Energy Consumption, Value Analysis, Market Analysis

future plan.

I. INTRODUCTION

To assess the future domestic system, experts have
II. FUTURE DOMESTIC SYSTEM
tried to create future living environment called smart
house [1]. However, experts often created their ideas not
First, in order to design questionnaire on future
based on consumers’ requirement but on technicaldomestic system, let us imagine future artificial life.
possibilities. Therefore, some ideas may fall into fans but
Referring to the conventional future assessment
do not contribute to new product and service
described in [2] [3], we have conducted brain storming.
development. In fact, our daily life is diversified based
It took about three months.
on our variety of value senses: convenience, housework
Under the assumption that our daily life consists ofsupport, health case, security, comfort and global
communication, input, work, learning and entertainment,
environment.
中水・汚泥処理（54）
水利用
In this research, we will create life
掃除ロボット（9・10）
掃除不要（12）domain map. Introducing six daily life
節水（7）
清掃
トイレ
Smart Security
Smart Bath-room
scenes (inventory control for household,
洗濯
防災
防犯
全自動で掃除（11）洗濯機（5）
media database, health care by an
ネットワーク管理（25）
温度自在調整（50）
ルーフ（6）
風呂
給湯
electric dietician, home security service,
個人情報管理（36・52）
リラックスバスシ
湯量・湯加減調整（21）ステム（22・24）
在庫管理
娯楽
daily case for children and aged
買い物
エンターテイメント
persons, total control for illumination
物品管理（16）
バスシステム（23）Home Media
Database エネルギー
録画機能（26）
andair-conditioning),
wewill
食品注文（15）
データ
illustrated future domestic images.
マスメディア 電子新聞（27）
キッチン
ベース
仕事
ピックアップ機能（28）Each image allows us to create future
Smart Kitchen
生態情報収集（51）
冷蔵庫（2・3）
インターネット会議（17・19）
domestic items. To find out the relation
臨場感（29）調理器（1）
在宅医療
among personal attributes, value senses
コミュニ
ケーション
ゴミ箱（4）
質感分かる（18）and their items, we will design
健康チェックシステム（8・32）
健康状態管理（35）Intelligent
Communication
家で聴講（20）questionnaire.
Then,
we
will
炊事
子供の世話
ペットの世話
介護支援ロボット（33）学校
demonstratefindings
fromthe
外出時監視システム（30・31）
HealthCare at Home 徘徊監視システム（34）questionnaire and finally describes
Fig. 1 Idea Map for Future Domestic System©ISAROB 2011

629

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

the future domestic system shall satisfy our values suchas healthy life, comfortable life, sustainable environment,
and secure living space. The household support andconvenience are another value senses in our life.
Based on value senses in our daily life, we depict
imaginary future scenes in home security, intelligent air
conditioner, smart kitchen and adaptive illumination. The
idea on product domain has been induced by analyzing
relation between place and action as shown in Figure 1.
While there are six categories in the product domain
map, the example for home security service is shown in
Figure 2. This figure implies that intelligent sensors withimage recognition and voice recognition should connect
with external network. We have collected a lot ofsuggestion from the product domain map.

Question on Persona/Home Attributes

Initial Question on Value sense

6 service images
Question
Question
about future
imagesQuestion
about future
imagesQuestionWith listof product
items
about
futureimagesQuestionWith listof product
items
about
futureimagesQuestionWith listof product
items
about
future images
With listof product
items
about
future images
With list of product items
With list of product items

Second Question on about Value sense

Fig. 3 Constitute of Questionnaire
Crime
Prevention
Net.
Connect local security network
Emergency
Call Unit

1. Future domestic items
From six kinds of future life scenery, we imagine
future domestic items. In principle, these future domestic
items are illustrated in scenery figures as shown in
Figure 2.
Our demand level investigation has two layers:whether you like to use the illustrated items or not, andwhether you think if most people like to use them or not.
On the other hand, it has three requirement strengths: itwill be in five years, in ten-twenty years or needless.

Disaster
Info. Net.

Smart sensor finds our intruders
not only at window/door but
also in garden

通報装置

安否判定器
Smart light and soundthreaten intruders

侵入検知Invisible body
空気圧力センサSensors
検知センサ

Invisible energy

侵入検知センサ
微振動検知センサ
Sensors

Various types of sensors detect intruders
before break-in, and alert the security
company and the police/fire station

2. Value sense and life style
As introduced in the previous chapter, we firstconsider on four kinds of value senses: healthy life,
comfortable life, sustainable environment, and secureliving space. Because cost expresses the strength of
requirement, we do not consider it as value. Then basedon the introduced future domestic items, two are added:
housework support and convenience.
Because life style as well as personal attributes andhouse attributes may affect on the strength of demandlevel, the questionnaire includes questions on behaviourof responders and their family.

Fig. 2 Future Home Security System

III. QUESTIONNAIRE DESIGN
Paying attention on daily value senses, we shouldfind out relation between them and future domestic items.
Because we prepare six kinds of value senses, there are
fifteen pair wise comparison. Because we are afraidsome consumers are sensitive for questionnaire (in otherwords, they may not have confirmative opinion on value
senses), the questionnaire asks preference among value
senses twice. The overview of questionnaire is shown in
Figure 3 where some items are borrowed from [4] and[5].

©ISAROB 2011

3. Response collection
We conducted strata sampling for response collection.
The stratum consists of gender and age. The age strata
has five segmentation: 20’s, 30’s, 40’s, 50’s and elders
than 60. Each stratum includes 103 samples and thentotally the sample number is one thousand and thirty.

630

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

IV. RESULT OF ASSESSMENT

From the result of age strata in Table 2, importanceon value sense strongly depends on the age of responders.
In fact, there are clear differences as follows: Forconvenience, 20s is the highest. For housework support,
30s is the highest and for health care, 50S is the highest.
On the other hand, 60 and more put their importance onsustainable environment.
The pair wise comparison can be described in VSmap [6] where the drawing algorithm is shown in Figure
4. The VS-map visualizes preference on value senses as
bird-eye where health care and home security have large
circle and housework support and sustainable
environment have smaller circle totally.

1. Overview
On occupation, twenty two percent of responders arehousekeepers. There are also sixteen percent for officeclerks and thirteen percent for engineers/ technicians.
On the demand levels for six kinds of future life
scenery, home security service is the highest: 60 percent
of responders like to live in the illustrated secure homeenvironment in five years. As another remarkable result,fifty percent of responder requests to improve weakpersons (children and aged persons) support in five years.
The rate of responders who like to use adaptive
illumination living space is fifty three percent.
On the other hand, most responders are afraid theexcessive service in another scene, especially ininventory service. The detail rate of responders is shownin Table 1.
Table 1Left

Future Assessment for New Home Service
In Five Years

In 10-20
years

Worthless

Total

Inventory
S1 control forhousehold

354
(34%)

346
(34%)

330
(32%)

1030
(100%)

S2 Media database

415
(40%)

335
(33%)

280
(27%)

1030
(100%)

438
(43%)

339
(33%)

253
(25%)

1030
(100%)

619
(60%)

296
(29%)

115
(11%)

1030
(100%)

564
(55%)

343
(33%)

123
(12%)

1030
(100%)

550
(53%)

295
(29%)

185
(18%)

1030
(100%)#

Health care by
S3 an electric
dieticianHome security
S4
service
Daily care forS5 children andaged persons
Total control for
S6 illumination andair-conditioning©ISAROB 2011

Age

Convenience

20
30
40
50
60Total

18
11
14
11
9
63

Housework
Support
9
17
10
10
7
53

B
C
D

E

F

A

B

C

D

E

A
B
C
D
E
F

Step 1: Conduct a paired
comparison testfor value senses

F

Step 2: Make a cross-tab ofvalue senses

Housework
supportUser
friendliness

ComfortEcology

Health

2. Value sense
The result is summarized in Table 2. Many
responders put their importance on their health care(32%) or home security (18%). Then Table 2 also showsthat there are so many responders (26.5%) who cannot
put importance on the unique value sense.

Table 2

Right

A
A
A

Step 3: According to across-tab, changea size of each
Security
circle and eachline weightFig. 4 Procedure for generating Value Sense Map

Important Value Sense by Age
Healthy
SecureSustainable
ComfortUnclear
Life
Living Space
Environment
58
39
17
9
56
41
50
21
9
57
64
33
17
14
54
81
26
22
9
47
74
29
18
18
51
318
177
95
59
265

631

Total
206
206
206
206
206
1,030

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

V. CAUSAL RELATION(2) To find out the relation among personal attributes,
value senses and their items, we have introduces
four general values (health case, security, comfort
and global environment) and two life-related values(convenience, housework support).
(3) As a result of pair-wise comparison on value senses,
there are not always ordinal preferences on value
senses. Then we have proposed visualizing method
called VS-map.
(4) To analyze the causal relationship among personal
biography, house attributes, responders’ value senses
and future domestic systems, we have constructed
Bayesian network and found its possibility.

As found in the previous chapter, there is different
importance on value sense among ages. Then it isexpected that there is causal relationship among
responder’s demographics (gender, age, family
occupation, living environment, etc.), their value senses
and demand level for future domestic items in thecreated scenes.
To verify the existence of causal relation, we havebuilt three layers Beyesian network. The network
structure shown in Figure 5 allows us to examine
sensitivity by assigning evidence on nodes in the
network. The detail discussion of BN is presented
elsewhere [7].

REFERENCES

Personal Biography & House Attributes
EnergyGenderAge

Occupation(1) R. L. Smith (1987), The Coming Revolution in
Housing. Go Courseware(2) Yano Research Institute (2006), Digital Home
Appliance Network Consumer Needs Survey (inJapanese). Yano Research Institute(3) NISTEP Report No.82 (2005), The 8th Science andTechnology Foresight Survey (in Japanese)(4) NHK Broadcasting Culture Research Institute (2006),
NHK Data book 2005 National Time Use Survey (inJapanese).
(5) Architectural Institute of Japan, Energy consumption
for residential buildings in Japan (in Japanese) (2006).
Architectural Institute of Japan ISBN978-4-81894202-8(6) H. Tsuji, M. Kojoma, A. Takahashi, M. Nakano, S.
Aoki, S. Inoue, K. Asari, E. Mimura (2008),
Preference Mining on Future Home Energy
Consumption, IEEE International Conference on
Systems, Man & Cybernetics (IEEE/SMC 2008),
pp.3697-3701(7) A. Takahashi, S. Aoki, H. Tsuji and S. Inoue(2008), Bayesian Network for Future Home Energy
Consumption, in Lecture Notes on Artificial
Intelligence (LNAI 5243: Ed by A. Dengel et al.
(Eds)), Springer-Verlag Berlin Heidelberg, pp.372379

Area

Family Constitution

Relation
House Type

Responder’sValue Sense
Important Value

Product A

Slight Value

Product B

Product C

Product D

Etc.

Future Domestic Systems

Fig. 5 Basic structure for Bayesian Network

VI. CONCLUSION
This paper has investigated the demand level for
future domestic systems. Especially, this paper has
contributed to find out the causal relationship between
our daily life style and preference on future items
generated from future life scenes. The remarks are
summarized as follows:(1) To create life domain map, we have introduced six
daily life scenes. Further, based on the scenes, wehave illustrated future domestic items.

©ISAROB 2011

632

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Visualizing language evolution as an emergent phenomenon based on
biological evolution and learning
Tsubasa Azumagakito, Reiji Suzuki and Takaya Arita
Graduate School of Information Science, Nagoya University
Furo-cho, Chikusa-ku, Nagoya 464-8601, Japan
Tel : +81-52-789-4266; Fax : +81-52-789-4258
E-mail : azumagakito@alife.cs.is.nagoya-u.ac.jp
Abstract: From the artiﬁcial life perspective, language can be viewed as a complex adaptive system emerging fromlinguistic interactions between individuals. Language and the human brain have evolved in parallel and interactingwith each other. In this study, we propose a model of language evolution based on biological evolution and learning. In
our model, the linguistic space is expressed in the polar coordinate system in which each possible language is expressed
as a point. We conduct evolutionary experiments based on the model and visualize the results in the linguistic space.
The trajectory of distribution of innate linguistic abilities shows the diversiﬁcation and complexity growth of language.
In the extended experiment, in which the angular coordinate represents the additional eﬀect on cost for the plasticity,
we observe a general tendency that the cost of plasticity evolves to become smaller. However, it never evolves to bezero, which might suggest that some cost of plasticity producing the Baldwin eﬀect is adaptive in language evolution.
Keywords: language evolution, visualization, learning, artiﬁcial life.

1

Introduction

Language distinguishes humans from other animals.
Language allows us to accumulate knowledge and transmit
it across both space and time. This has led to a greater understanding of the world and accelerated cultural achievement. The evolution of language has been the subject ofnumerous debates and speculations. Nevertheless, it is difﬁcult to study in a scientiﬁc manner and remains an open
research question.
Recently, a constructive approach has been adopted toinvestigate language evolution [1]. It is characterized by
the use of computational models from the viewpoint that
language is a complex adaptive system emerging from linguistic interactions between individuals. Another viewpoint states that language and the human brain have
evolved in parallel and interacting each other. In otherwords, they have coevolved. If we focus on the evolution of the human brain, there are two typical adaptive
processes at diﬀerent time scales: biological evolution andlearning (phenotypic plasticity) [2].
Based on these viewpoints, in previous work [3], we investigated the coevolution between communication ability
and phenotypic plasticity to clarify whether and how leaning can facilitate evolution in dynamic environments arising from communicative interactions among individuals.
To do this, a simple computational model was devised to
do this. The levels of adaptive communication of signaling
and receiving processes are determined by diﬀerent sets oftraits. Each level represents the expected value of ﬁtness
contribution for a successful communication. A communication is successful only when the levels of the signaler andthe receiver are the same. The agents try to improve theircommunication levels through learning in which the values of plastic traits can be modiﬁed from their geneticallydetermined values. The evolutionary experiments showedthat the population with a learning ability successfully increased its shared level of communication while the popu©ISAROB 2011

lation with no leaning was not able to increase the level.
It was also shown that the Baldwin eﬀect (typically interpreted as a two-step evolution of the genetic acquisitionof a learned trait without the Lamarckian mechanism [4])repeatedly occurred and facilitated the evolution.
The purpose of this research is to study the general rolesof biological evolution and learning in the evolution of language. For this purpose, we construct a generalized modelfor the coevolution between the communication ability andphenotypic plasticity. It is a generalization over the model
devised in the previous work [3] in the following two aspects. 1) The linguistic space is expressed in the polar
coordinate system in which each possible language is expressed as a point, and the success in the conversation isdetermined geometrically (instead of using a speciﬁc task
as in the previous work). Therefore, we can observe thecoevolution as trajectories of innate linguistic abilities of
agents in the linguistic space. 2) The ﬁtness can be deﬁned
by adjusting the beneﬁt from the communicating agents,
the beneﬁt from the complexity of the used language, andthe cost of learning, independently.

2

Model

2.1

Agent and communication

N agents in the population communicate with eachother using their language capacity. The linguistic spaceis expressed in the polar coordinate system in which eachpossible language is expressed as a point (Figure 1). The
distance from the origin (r) represents the complexity of
the language, and the angle from the positive x-axis ()represents a language type. Each agent is represented asa point and a ﬁeld surrounding the point in the linguistic
space. The former corresponds to the agent’s innate language, and the latter corresponds to linguistic plasticity
as an innate attribute of the agent. The plasticity is expressed as a fan-shaped ﬁeld with area determined by rp633

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

α=-20

Benefit

5000

α=0

2500

630

α=20

0

Figure 1: Linguistic space and agents.

1250
Number of Communicating Agents

Figure 2: The eﬀect of2500

on the beneﬁt.

and p as shown in Figure 1.
Overlap of two plasticity ﬁelds means that corresponding two agents can communicate with each other by learning (using their plasticity). Agents with large plasticity
can communicate with many agents although they incur alarge cost proportional to the area of their plasticity ﬁeld.
The polar coordinate system reﬂects the situation where
the diﬃculty and cost of communication is proportional
to the complexity of the language.

2.2

Fitness Evaluation

The ﬁtness of each agent depends on the number ofcommunicating agents, the complexity of the innate language, and the area of the linguistic plasticity. The ﬁtness
function is deﬁned as:

(a)

= 20

(b)

= −20

Figure 3: Results of language evolution.

 and variation σ 2 . Note that the range of a random3
L w1 w2
F itness = (L −) · r − (4p rp r)w ,(1) value for , p is inversely proportional to r of the parent.
2GThis property keeps the amount of displacement of the
innate language or change in its plasticity due to mutation
where wi (i=1, 2 and 3) are weights for three compoconstant independent of the location of the agent in thenents of the ﬁtness function. The ﬁrst term represents the
linguistic space.
beneﬁt from the number of communicating agents. L isnumber of communicating agents and G is the population
Resultssize.
is a parameter that determines the change in the 3
beneﬁt of communications with the increase in the numWe conducted evolutionary experiments for 1000 genber of communicating agents. There are three possible erations and visualized the results in the linguistic space.
situations: a) a linear increase ( =0), b) an exponential The following parameters were used: N =25000, w1 =1,
increase ( < 0), c) the existence of an optimal number w2 =1, w3 =1.1, Pm =0.8. The initial values of the genoof communicating agents for the best beneﬁt ( > 0), types of the agents (r, rp , , p ) were all zero.
as shown in Figure 2. Case b) corresponds to the situation in which there is a synergetic eﬀect in information 3.1 Evolution of linguistic diversity andsharing, and c) corresponds to the situation in which thecomplexity
beneﬁt of the information decreases if it is shared by too
We conducted two experiments in which
was 20 or
many agents due to some restrictions (e.g., the limitation−20.
The
resultsusing
thetwo-dimensional
polar
coordiof resources). The second term represents the beneﬁt ofnatesystemareshownin
Figure3.
The
innatelinguistic
communications depending of the complexity of the language. The more complexity will bring about the greater abilities of all agents were plotted with a unique color for
beneﬁts. The third term represents the cost of learning. every 10 generations. Figure 3(a) and (b) show the trajecWe assume that it is proportional to the area of plasticity tory in the case of =20 and −20 respectively. Both ﬁg(4p rp r), as it is probable that more complex and diﬀerent ures indicate that the agents formed some linguistic clusters from the initial population at the origin, then theylanguages are diﬃcult be learned.
increased their complexity of language gradually. However, the clusters converged to one large cluster with high
2.3 Evolution
complexity by the end of experiments. It is notable that
The agents are selected using roulette wheel selection distant clusters coexisted for a longer generation whento reproduce according to their ﬁtness to form the next was positive (Fig 3(a) vs. (b)). This means that the negageneration. Mutation is performed with probability Pm . tive eﬀect of information sharing on the ﬁtness caused by
The genotypes of oﬀspring are mutated by adding a small the excess number of communicating agents contributed
random value: R(0, 12) for rp , r, and 5/r·R(0, 1) for , to the maintenance of high linguistic diversity.
p , where R(, σ 2 ) is a normal random number with mean
2

©ISAROB 2011634

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011(a) The average complexity oflanguage.

(b) The average number ofcommunicating agents.

(c) The average plasticity of agents.

Figure 4: The comparison of agent abilities between experiments with =20 and =-20.
Figure 4 compares some properties of agents in bothcases. Figure 4 (a) shows the evolution of the average
innate complexity of language. The horizontal axis represents the generation. We see that the average complexity
in the case with =20 was higher than the one in the casewith =20 after around the 550th generation. This implies that the negative eﬀect of information sharing could
also accelerate the later evolution of the complexity of language.
Figure 4(b) and (c) shows the evolution of (b) the average number of communicating agents and (c) the average
plasticity for each agent respectively. We deﬁne the plasticity of each agent as its area (4 · p · rp · r). Figure 4(b)showed a sharp increase in the number of communicating agents for about 20 generations and its subsequent
decrease to the small value (about 1250) in the case of=20. In contrast, the number of communication in thecase of =-20 sharply increased to its maximum 25,000(equal to the population size). Figure 4(c) shows that thesuﬃcient amount of plasticity increased drastically fromthe initial generation in the case of =−20. In the case of=20, the plasticity evolved to the relatively smaller value
12694 at 1000th generation, although it is too small to see
its value in Figure 4(c). This indicates that agents in thecase of =20 formed linguistic clusters that keep the mostbeneﬁcial size (about 1250) by controlling their plasticity.
Note that, although Figure 3(a) shows that the linguistic
clusters seem to have converged to one cluster in the lastgeneration, that cluster is composed of small sub-clusters
that kept the most beneﬁcial size. Furthermore, the linguistic clusters that were close to each other were morerobust against a mutation of . This is an explanationof why the linguistic clusters converged to one cluster inspite of =20.
As a whole, the results indicate that the evolutionary
scenario of linguistic diversity as follows. 1) In the earlystages of evolution, there was an increase in the number of
agents with more plasticity to communicate each other. 2)
Then, in the case of =20 (means there is negative eﬀect
of information sharing), the plasticity is adjusted to smallvalue to keep population size in optimal value.
In addition, we conducted experiments without learningin which agents have no plasticity. Speciﬁcally, a communication between two agents results in success when theyshare the same r and  perfectly. The results with =20
are shown in Figure 5. We see that the complexity of language rapidly converged to about 6.5 in early generations.
©ISAROB 2011(b)Complexity of language.
(a)Trajectory of evolution.
Figure 5: Results of experiments without learning.
It shows that the evolutionary process of the complexityof language tended to constant without learning.

3.2

Eﬀects of learning cost

We conducted further experiments to understand theeﬀects of the leaning cost on the evolution of the population. We used various weights for learning cost (w3 )ranging from 0 to 2 at intervals of 0.05. Here we focusedon the case of =20. Figure 6 shows four typical trajectories of the population when w3 = 0, 1.2, 1.4 and 1.6
respectively. It shows that as the weight of learning cost
increased, the increasing rate of the linguistic complexity
decreased, and the linguistic clusters tended to converge
around the origin.
The average plasticity and complexity of language inthe last generation in these cases are shown in Figure 7.
When the weight was relatively small (w3 < 1.0), thecomplexity of language reached a high value around 4800.
The plasticity value has a wide distribution between low
value and signiﬁcant high value comparatively. In thesecases, the increase in the complexity of language brings
about the higher beneﬁt compared with its eﬀect on thecost. Thus, the higher complexity of language was essential for survival of agents. Besides, due to the smaller costof learning, the plasticity often became high value. As aresult, the language tended to become more complex andthe plasticity often reached high values.
When 1.0  w3 < 1.4, both indices tended to become smaller as w3 increased due to the increased costof learning. There is a possibility that the smaller plasticity retarded the evolution of the complexity because
agents might not be able to communicate with their own
mutants or not be able to keep the optimal number ofcommunicating agents.
When w3 ≥ 1.4, the complexity converged to the small

635

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Figure 6: Results of experiments for learning cost.

Figure 8: Results of extended experiments.
in the second and third quadrant. Thus, the languagethat had low cost to learn was selected in the evolution of
language. The results show that excessive cost of learning
prevents evolution of language.

4(a) The average plasticity (b) The average complexityof agents.
of agents.
Figure 7: The results comparison of learning cost.
value around 200 as w3 increased. In these conditions, theincrease in the complexity of language yields much larger
cost. Thus, the population was supposed to evolve to use
the less complex languages. The gradual increase in the
average plasticity when w3 ≥ 1.4 was due to the slower
convergence to the small values caused by the higher cost.
As a whole, it turned out that the degree of learning
cost strongly aﬀected the diversiﬁcation of languages andtheir properties.

3.3

Additional cost experiments

Conclusion

In this paper, we conducted experiments with a comprehensive model of language evolution. The results of ourexperiments showed some implications for language evolution as follows. First, the linguistic complexity and diversity can emerge through interactions between evolution
and learning. Second, the negative eﬀect of information
sharing on the ﬁtness caused by the excess number of communicating agents contributed to the maintenance of highlinguistic diversity. Third, agents may not be able to advance their linguistic complexity without the plasticity of
linguistic abilities. Finally, the excessive cost of learning
can prevent evolution of language because the plasticity
is not enough to cover the mutation range. It is not necessarily the case that the low cost of leaning accelerate
evolution of language due to the plasticity covering thewhole mutation range.
Future work includes considering geographical factors
in linguistic diversiﬁcation and the asymmetric aspects of
beneﬁt of communications by extending our model.

Finally, we conducted experiments to consider the evolution of learning cost. In these experiments, the angularcoordinate of an agent  was associated with an additional
cost of learning Cost, which was added to the ﬁtness value References
calculated as follows:[1] Luc Steels. The synthetic modeling of language origins.

w3
Evolution of Communication, Vol. 1, No. 1, pp. 1–34,
2(4θp rp r)−(| π2 − | ·) if 0 <   /2,

π

1997.

3

 (| π − | · 2(4θp rp r)w ) if /2 <   ,2π
Cost =
3[2] Yusuke Watanabe, Reiji Suzuki and Takaya Arita.
 (| π − | · 2(4θp rp r)w ) if  <   3/2,Language evolution and the Baldwin eﬀect. Artificial
2
π

3


2(4θp rp r)w
Life and Robotics, Vol. 12, pp. 65–69, 2008.
π−(| 2 − | ·) otherwise.
π[3] Reiji Suzuki and Takaya Arita. How learning can guide
When  is 0 or 2, there is no eﬀect of the additional cost.
evolution of communication. Proceedings of Artificial
As  gets closer to , the additional cost increases. We
Life XI, pp. 608–615, 2008.
used 20 as .
The typical results are shown in Figure 8. Each ﬁgure [4] Peter Turney, Darrell Whitley and Russell Anderson.
shows the result of a trial with a diﬀerent random seed.
Evolution, learning, and instinct: 100 years of the
They illustrate that linguistic diversiﬁcation occurred in
Baldwin eﬀect. Evolutionary Computation, Vol. 4,
the ﬁrst or the fourth quadrant due to the higher cost
No. 3, pp. 4–8, 1996.
©ISAROB 2011

636

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Self-organizing stability of food web that emerges from the evolution of restrictions
on speciation
Hirofumi Ochiai
Reiji Suzuki
Takaya Arita
Graduate School of Information Science, Nagoya University
Furo-cho, Chikusa-ku, Nagoya 464-8601, Japan
E-mail: ochiai@alife.cs.is.nagoya-u.ac.jp, reiji@is.nagoya-u.ac.jp, arita@nagoya-u.jp
Abstract
The food chain length has been considered as a key characteristic of food webs, and thus understanding its
determinants is becoming increasingly important for ecosystem management and biodiversity conservation. For
this purpose, we propose an evolutionary network model of food webs that captures the essential features of theones in the real-world. The results show some universal features of food webs including the fractions of top,intermediate and basal species in the webs, which are in good agreement with empirical data. We will discuss how
this structure can emerge in the simple evolutionary model of food webs.
Keywords: Ecosystem, Evolution, Extinciton, Restriction, Food chain, Scale-free network.

1

introduction

the strength of the restrictions, which implies that theevolution of restriction on speciation events itself is a
key factor that can aﬀect the self-organization of foodwebs.
It has been the subject of debates and speculations
among ecologists why the food chain length is short[2, 6]. Some researchers ague that they are productivity, system size or their combination [11, 12]. The
understanding of its determinants has recently become
important for ecosystem management and biodiversity
conservation [4].
In this paper, we focus on this subject and investigate it by using an evolutionary network model offood webs in which the restriction of speciation eventsis evolvable [9].

A food web is a highly complex network which deﬁnes prey-predator relationships of species. Understanding the structure and functioning of ecosystems
by exploring the network topology has long been acentral topic of ecological research. Various modeling levels and types of models have been proposed byecologists, mathematicians and physicists for understanding the mechanisms of ecological dynamics.
Amaral and Meyer’s model [1] is a well-known example based on a dynamic growth structure to clarify universal features of food webs. They constructeda network model for large-scale extinction and evolution of species, in which there exists a strong restriction that limits the number of the species on each
trophic level and the establishment of prey-predator
relationship between distant trophic levels. The results showed a power-law distribution of extinction
avalanche sizes, in good agreement with available datafrom fossil records. However, they did not discuss on
the inﬂuence of such restriction on the global behaviorof the network.
In the previous work, we clariﬁed how the restriction based on the trophic level can aﬀect the evolution and extinction of food webs [8], by expanding
their model so that the strength of the trophic level
restriction on evolution can be adjusted by a single
parameter. We found that the network structure andthe stability of the ecosystem strongly depended on

2
2.1

Network representation

Figure 1 shows an example of food webs in ourmodel. There is one special node representing the sunin an abstract form, which is the permanent energy
source. The other nodes represent species. The directed links represent the energy ﬂow from one species(prey) or the sun to another species (predator). The
trophic level of the sun is deﬁned as 0, and the trophic
level of each species is deﬁned as the minimum distance from the sun. The species at the level 1 correspond to the autotrophic species, and cannot survive
1©ISAROB 2011

The Model

637

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

S

level is smaller than the saturation point ofeach level NL .

speciation with µRestricted
speciation• Omnivorous species: An omnivorous species
creates a new omnivorous in an available
niche at any levels, and then make Kl links
from randomly-selected species in all levels.
This event occurs only when the number ofnodes in the system NS is smaller than thesystem size S.

k links3

speciation with µ
Unr estricted
speciation

2

2

2

3
k links

extinction with p

1

1

1

1

Trophic Level

3

The number of prey links Kl is loosely inheritedfrom the one of the original species Ko,l . Speciﬁcally, Kl is chosen randomly from Ko,l−1 , Ko,l orKo,l+1 . However, the speciation does not happen
if Kl is 0.

NL

In addition, the type of the new species is mutated(ﬂipped) with a probability ϕ.

Sun

2. Extinction.
Only autotrophic species can trigger an avalanche1
as is the case with Amaral and Meyer s model.
When a species goes extinct, all the links fromit to other species are removed. The extinction
occurs on all species which have lost all incoming
links recursively.

Figure 1: Schematic representation of the model. Circle and square nodes represent omnivorous species andrestricted species, respectively.

without the link from the sun. The other species correspond to heterotrophic species, and cannot survive
without incoming links from the other species. There
are two types of species, which are determined genetically. The ﬁrst type is a restricted species preying
on species at lower trophic levels and speciating at aneighboring trophic level. The second is an omnivorous species preying on any species and speciating at
any trophic level.

2.2

3

We used the system size S = 1000, the saturation
point of each level NL = 100, the the extinction probability p = 0.01, the probability of speciation µ = 0.02
and mutates ϕ = 0.01. Those parameter values are
based on the previous studies [5, 7, 10].
We adopted a food web composed of 10 restricted
species and 10 omnivorous species at the level 1 thatreceive a link from the sun as the initial state of thissimulation.

Algorithm

The dynamics of the web is driven by the speciation and extinction of species. The model starts with
N0 species at the level 1 and evolves according to thefollowing rules:

3.1

Basic Dynamics

Figure 2a shows a typical result of the experiments.
The number of entire species NS tended to ﬂuctuate
around the maximum value 1000 while it often decreased sharply. We observed the extinction of entire
species as seen at the 21000th step in the ﬁgure. It isalso shown that there is a strong correlation between
the number of speciation and extinction, which is ingood agreement with empirical data [1].
Figure 2b shows the transition of the length of thefood chain of restricted and omnivorous species. The

1. Speciation.
Every existing species at the level l tries to speciate with a probability µ.
• Restricted species: A restricted species creates a new restricted species in an available
niche at the same or neighboring levels l-1, l
or l+1, and then make Kl links from species
at the lower level. This event occurs only
when the number of nodes at the speciating©ISAROB 2011

Experiments

1 The avalanche means chains of extinction.

638

Speciation
Extinction

1000
500

NS
Nr

b

7.5
5

3000
1500
0

so
sr

2.5

c

0.9

Rate

4500

0.6

I
T
B

0.3
8000

10000

12000

14000

Time

16000

18000

20000

22000

Figure 2: Time sequence of the number of species, speciation and extinctions events. The number of speciation
or extinction is the total number of speciation or extinction events during consecutive non-overlapping intervals of
512 time steps.

values of the length was oscillating at around level1 ∼ 7 with the number of species NS , and the lengthof omnivorous species kept longer level than the one ofrestricted species. This indicates that the system wascomposed of two layers; lower layer: restricted species,
higher layer: omnivorous species, and the food chain
length was oscillating around the lower level.
Cohen and Briand [3] found a fundamental property of food webs that the ratio among basal species
B (predators without predators), intermediate species
I (predators and prey species) and top species T (preyspecies without prey) is roughly constant: B : I : T =0.19 : 0.52 : 0.29. Figure 2c shows the fraction of
the BIT at each step. The average of the fraction in
the time sequence for 50 runs is B : I : T = 0.104 :
0.513 : 0.383. The fraction shows the same tendencywith empirical data [3].

3.2

The results show that the length of food web does
not get longer compared to the increasing of the system size. Here, we explain the reason of the relatively
short length of food web.
The system has two tendencies. The length of food
web tends to increase with the system size increasing.
The system also tends to be unstable as the system size
becomes large. This instavility is supposed to derivefrom the inherent property concerning the ratio: I >T > B, which appears when the system size is large.
The smaller number of basal species could be the causeof mass extinction.
For these reasons, the length of food web is oscillating around lower level without reaching a higher level.

4Eﬀect of the system size

It has long been discussed among ecologists what
the determinant factors of the food web length are.
Some researchers ague that they are productivity, system size or their combination [11, 12]. However, fromthe results of the simple constructive food web model,
it is strongly suggested that the determinative factorof the short length is the general property cocerning
the ratio among basal species B (predators without
predators), intermediate species I (predators and preyspecies) and top species T (prey species without prey):
I > T > B. This tendency coupled with the basic tendency of the system to grow is supposed to control thehigh frequency of extinction, which leads to a limited

Table 1 summarizes the results of the simulationsfor 50 runs with the three diﬀerent size of system
S = 500, 1000 and 2000. As the system size increased,
the survival time2 and the rate of the number of all
species and restrected species decreased. On the other
hand, the length of the food chain and the fractionof intermediate species increased. This means thatthe system has same features of partial sphere which
tended to increase the ratio of volume area to surface
and to be unstable in larger system.
2 The elapsed time before all the species went extinct.

©ISAROB 2011

Conclusion

639

Speciation and Extinctiona5

Level N S and N r

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 1: Eﬀects of system size on the behavior which was the averages taken over 50 runs.
S
500
1000
2000

Survival
time
5677.04
4460.163714.26

Rate of speciesssrso
74.67 ％ 35.74 ％ 64.26 ％66.17 ％ 22.12 ％ 77.88 ％59.94 ％ 17.15 ％ 82.85 ％Average chain lengthssrso
3.75 2.98
4.04
4.27 2.92
4.44
4.62 2.71
4.76

length of the food-web. It should be noted that thesenario presented in this paper is in good agreementwith empirical data.

Software Science and Technology (in Japanese),(in press).
[10] A. Pekalski, J. Szwabiński, I. Bena, and M. Droz.
Extinction risk and structure of a food web model.
Phys. Rev. E, 77:031917 (8 pages), 2008.

References[11] S. L. Pimm and J. H. Lawton. Number of trophic
levels in ecological communities. Nature, 268:329– 331, 1977.

[1] L. A. N. Amaral and M. Meyer. Environmental
changes, coextinction, and patterns in the fossil
record. Phys. Rev. Lett., 82(3):652–655, Jan 1999.

[12] D. M. Post, M. L. Pace, and N.G. Hairston.
Ecosystem size determines food-chain length inlakes. Nature, 405(6790):1047–1049, 2000.

[2] M. Begon, J. L. Harper, and C. R. Townsend.
Ecology: Individuals, Populations and Communities [3rd edition]. Blackwell Seientiﬁc Publieations, Oxford, 1996.
[3] Joel E. Cohen. Food Webs and Niche Space.
Princeton University Press, Princeton, 1978.
[4] Michio Kondoh and Kunihiko Ninomiya. Foodchain length and adaptive foraging. Proceedings of the Royal Society B: Biological Sciences,276(1670):3113–3121, 2009.
[5] M. Lässig, U. Bastolla, S. C. Manrubia, andA. Valleriani. Shape of ecological networks. Phys.
Rev. Lett., 86:4418 – 4421, 2001.
[6] K. Matsuno and N. Ono. How many trophic levels are there? Journal of Theoretical Biology,
180:105 – 109, 1996.
[7] M. E. J. Newman and P. Sibani. Extinction,
diversity and survivorship of taxa in the fossil
record. Proceedings: Biological Sciences, 266:1593– 1599, 1999.
[8] H. Ochiai, R. Suzuki, and T. Arita. The eﬀects
of the trophic level on the stability of food webs.
Artificial Life and Robotics, 14(3):374 – 383, 2009.
[9] H. Ochiai, R. Suzuki, and T. Arita. A constructive model for the evolution of food web structure
based on the trophic constraint. Japan Society for©ISAROB 2011

Fraction of BITssrso
0.112 ％ 0.494 ％ 0.394 ％0.104 ％ 0.513 ％ 0.383 ％0.095 ％ 0.530 ％ 0.375 ％640

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Modeling Electrosensory System of Weakly Electric Fish
Miyoung Sim and DaEun Kim
Biological Cybernetics Lab
School of Electrical and Electronic Engineering,
Yonsei University, Schinchon, Seoul, 120-749(Tel : 82-2-2123-5879)(daeeun@yonsei.ac.kr)

Abstract: Weakly electric fish have specialized sensory system to detect the electric field. They generate the electricfield with the electric organ and sense the intensity change of the electric field with their electroreceptors. If there is atarget object in the environment, the electric fish detect the distortion of electric field, caused by the target object. A setof sensor readings in the rostrocaudal line or the dorsoventral line provides the localization information of a target
object as well as the object features such as the material and shape. We model the electrosensing mechanism of the
weakly electric fish for possible application to the underwater robot. It will be shown how the electric pole distribution
in electric organ affects the electrolocation mechanism. The agent can bend the tail for the body movement and a seriesof sensor readings in the temporal domain and spatial domain are observed. The temporal change with tail-bending
movements can be used to estimate the accurate position of a target object. The relative slope with the bent tail issimilar to the original relative slope when the fish body is in the neutral line. Regardless of the position, we can apply
the same electrolocation rule. This study helps to develop the electrosensor system and the biomimetic sensory system
can be applied to the underwater robotic fish.
Keywords: weakly electric fish, electrolocation, distance measure, tail bending, robotic fish.

It is shown that some species of weakly electric fish
can use spatiotemporal information to localize a target
object, especially with tail-bending movements [6-9].
The relative slope is known as a distance measurement
regardless of the size and conductivity of a target [1012].
Normally the electric organ is composed of manypoles [5] and in this paper a simplified electric organ
model will be introduced with two electric poles. The
two-pole electrosensory system can be a practical model
which can be easily realized in the electric fish robot.
We study two different electric organ models for theelectrosensory system and see how the models influence
the electrosensory performance. It is useful to study the
temporal sensor response with the electric organ models.
The relative slope will be tested for distance estimation,
and we will show the change of relative slopes with tailbending movements. The developed electric organ
structure and object localization features can be applied
to the electrosensory system of a robotic fish.

I. INTRODUCTION
Weakly electric fish are very specialized toelectroreception. They have three types of sensor
systems, mechanosensory lateral line, ampullaryelectroreceptors, and tuberous electroreceptors [1-2].
Among the three types of sensors, a large number of
sensors belong to tuberous electrosensors which aresensitive to the change of electric field. The tuberous
system is used for active sensing and weakly electric
fish can detect distortion of self-generated electric field[3-5].
The electrosensory system can be applied to thebiomimetic robotic fish. Weakly electric fish canidentify the location and characteristics such as size andconductivity of a target object [3-4]. When theelectrosensory system is equipped in robotic fish, it ispossible to localize and identify the characteristics of atarget. The electrosensory system is composed of theelectric organ discharge and electroreceptors along thebody. In this study, two types of electric organ structure,fish-like multiple-pole distribution and a simplified twopole model, will be tested for electrolocation.

©ISAROB 2011641

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

rostrocaudal line is known as a distance measurement[10-12] and it will be introduced in the next section.

II. MATERIALS AND METODS
1. Modeling of the electric field
The electric organ of weakly electric fish can bemodeled as a composition of many electric poles. The
electric potential at position x is calculated as the sum ofpotential differences generated by each electric pole [3].

q /(n - 1)qix - x npi =1 x - x p

n -1

V ( x) = å(1)where n-1 positive poles and one negative pole exist [3].
This electric field model is based on Gymnotiformes. In
the rostrocaudal line, n positive poles are arranged inrows and one negative pole is located in the last point of
the electric organ [3-5]. The electric field is the gradientof the electric potential.
n -1

q /( n - 1)

i =1

i 3
p

E ( x) = åx-x(x - x ) i
p

q
x-x

n 3
p

(x- x )
np

Fig.1. Electric organ models(a) fish-like multiple pole system and (b) two-poleelectric organ(2)

The electric perturbation of a simple sphere object isderived asd V ( x) = c

a 3 E ( xobj ) × ( x - xobj )x - xobj

3

2. Relative slope
The sensor readings can be represented as an electric
image. The electric image has been studied tounderstand the electrolocation mechanism. In this study,
the box model is used for electrolocation [13]. The
electroreceptors are distributed along the rostrocaudal
line parallel to the mid-line of the fish, which is called abox model. The normal vector of the electroreceptor,(3)

where a is the radius and xobj is the center of a target andχthe electrical contrast - 1 for perfect conductor and 0.5 for perfect insulator [5].
When we consider the component normal to theelectroreceptor, the transdermal potential isr
Vtd ( xs ) = E ( xx ) × nˆ ( xs ) skinr waternˆ ( xs ) , is vertical to the mid-line of the fish.

In three-dimensional space, weakly electric fish
have to estimate the position of a target in the
rostrocaudal (from head to tail), dorsoventral (fromventral to dorsal area), and lateral (form fish to side)axis with respect to the fish body. The rostrocaudal anddorsoventral position can be extracted from the maximal
point of the electric image [3,5]. The lateral distancechanges the intensity and width of the electric image.
The relative slope is the ratio of the maximal slope tomaximal slope (equation 5). It is possible to estimate thelateral distance regardless of the size and conductivityof a target object.

(4)

where nˆ ( xs ) is a normal vector at the measured pointxs and ρ resistivity. Apteronotus albifrons belonging to
Gymnotiformes bends the tail from side to side in arange from 45○ to -45○ [5]. The arrangement of theelectric organ is transformed according to tail-bending
movements.
In this study, we established two types of electricorgan structures, fish-like multiple pole model and asimplified two-pole model as shown in Fig.1. The
multiple pole model is based on the model for realelectric fish. The simplified electric organ model has
two electric poles, positive and negative. The
arrangement is different from the multiple-pole modeland sensor readings are affected by the electric organ
structure.
The relative slope with electroreceptors along the©ISAROB 2011

Relative slope =max i {I ( xi +1 ) - I ( xi )}max i {I ( xi )}(5)

The relative slope is a localization feature of a target
object in the environment and it is one of essential
properties in weakly electric fish as well as artificial
robotic fish.

642

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

slope is largely independent on the tail-bending
movements [8]. The multiple-pole electric organ
structure changes the arrangement of the electric poles
with tail bending movements. The caudal area of
weakly electric fish is approximately 65% of the bodylength and bends drawing circular portion around thepivot. The radius of the arc is R=L/2θ where θ isthe bending angle. The simplified two-pole model has
bending area, 50% of the body length, and this area isbended in a straight line. These two electric organ
models have a bending-angle range from 45○ to -45○.

III. Experiments
We applied the relative slope to two electric organ
models for distance estimation. Fig.2 and Fig.3 showchange of relative slope when the lateral distance androstrocaudal position of a target object change.

Fig.2. Relative slope for multiple-pole electric organ
when the rostrocaudal position of a target changes

Fig.4. Change of relative slope when the weakly electric
fish bend the tail with the multiple-pole modelFig.4 shows the relative slope with tail-bending
movements with the multiple-pole electric organ model.
When weakly electric fish bend the tail to the left andright, the relative slope is about the same. Weakly
electric fish might use relative slope with bent tail toassure the estimated distance in noisy environment [8].
Fig.3. Relative slope for two-pole electric organ when
the rostrocaudal position of a target changes
The relative slope is affected by both lateral distanceand rostrocaudal position of a target object. When the
rostrocaudal position changes from 3cm to 9cm, theshape of the electric image and relative slope areaffected. However, the relative slopes in the two-polesystem have similar patterns with those in the multiplepole system when the rostrocaudal position changes.
The rostrocaudal position of a target object can be
estimated with the peak amplitude in the electric image.
Then the relative slope curve can be used for the lateral
distance estimation.
We also test the temporal change of electrosensor
readings with tail-bending movements. The relative©ISAROB 2011

Fig.5. Change of relative slope when the weakly electric
fish bend the tail with the two-pole model

643

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig.5 shows change of the relative slope when thesimplified two-pole model is used. The relative slopes
with the two-pole model have a little change, but thepatterns are quite similar. We can use relative slope toestimate the distance regardless of the bending angle.

system for more exact estimation of target distance. This
study shows that we can use a simple electrosensory
model with two poles for a robotic fish instead of thecomplex fish-like multiple-pole model.
V. ACKNOWLEDGMENTS
This work was supported by Mid-career Researcher
Program through an NRF grant funded by the MEST
(No. 2010-0000460)

REFERENCES[1] Nelson, M. and MacIver, M. (1999), Prey capture in
the weakly electric fish Apteronotus albifrons. Journalof Exp. Biology 202(10):1195-1203[2] Nelson, M., MacIver, M., and Coombs, S. (2000),Modeling electrosensory and mechanosensory images
during the predatory behavior of weakly electric fish.
Brain, Behavior and Evolution 59(4)199-210.
[3] Rasnow, B. (1996), The effects of simple objects on
the electric field of Apteronotus, Journal ofComparative Physiology A 178(3):397-411.
[4] Assad, C. and Rasnow, B. (1999), Electric organ
discharges and electric images during electrolocation,
Journal of Experimental Biology 202:1185-1193.
[5] Chen, L., House, J., Krahe, R., and Nelson, M.
(2005), Modeling signal and background components ofelectrosensory scenes, Journal of Comparative
Physiology A 191(4):331-345.
[6] Sim, M. and Kim, D. (2009), Distancediscrimination of weakly electric fish with a sweep oftail bending movements, 10th European Conference on
Artificial Life.
[7] Sim, M. and Kim, D. (2009), Electrolocation of
weakly electric fish with differential electric image intail-bending movement (in Korean), Korea Robotics
Society Annual Conference, 348-350.
[8] Sim, M. and Kim, D. (2010), Electrolocation using a
relative slope with a distortion of an electric field in
robotic fish (in Korean), Korea Robotic Society Annual
Conference, 182-183.
[9] Sim, M. and Kim, D. (2010), Estimating distance ofa target object from the background objects with electric
image (in Korean), Journal of the Institute ofElectronics Engineers of Korea-SC, 56-62.
[10] von der Emde, G. (1999), Active electrolocation of
objects in weakly electric fish, Journal of Experimental
Biology, 202(10):1205-1215.
[11] Schwarz, S. and von der Emde, G. (2001), Distancediscrimination during active electrolocation tasks in the
weakly electric fish Gnathonemus petersii, Journal ofComparative Physiology A 186(12):1185-1197.
[12] Sicardi, E., Caputi, A., and Budelli, R. (2000),
Physical basis of distance discrimination in weakly
electric fish, Physica A 283(1-2):86-93.
[13] Babineau, D., Longtin, A. and Lewis, J. (2006),
Modeling the electric field of weakly electric fish,
Journal of Experimental Biology 209(18):3636-3651.

Fig.6. Relative slope with simplified electrosensory
model (the marker ‘o’ is the original relative slope andthe marker ‘x’ the relative slope in integrated electric
image with tail-bending)Fig.6 shows the original relative slope with the fishin a straight line and the relative slope in integrated
electric image generated by tail-bending movements.
The integrated image calculates the summation of theelectric image for each bending phase. In the integrated
electric image with the two poles, these relative slopes
are about the same to original one. The integrated
electric image with tail-bending can more exactly
estimate the distance of a target object, which can be
useful in noisy environment.

VI. CONCLUSION
In this study, we compared the fish-like multiplepole model and a simplified electrosensory model withtwo poles for electrolocation. From the relative slope
pattern, we can estimate the location of a target object.
Interestingly, the two-pole model shows similar
performance of electrolocation as the multiple-polesystem, although the two-pole system has slightly morevariations in electrolocation.
Weakly electric fish can use temporal structures with
tail-bending movements to estimate the estimateddistance. When they bend the tail, it is possible toextract relative slope from the electric image and also
the temporal integration of sensor readings during thetail-bending movement can be applied to the two-pole©ISAROB 2011644

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Simultaneous cognitive origin of life and information
Koji Ohnishi1,21Institute for Theoretical and Evolutionary Sciences (ITES),
c/o Koji Ohnishi, 8712-30, Ikarashi-2, Nishi-ku, Niigata, 950-2102, Japan(Tel : 81-25-262-5237; Fax : 81-25-262-5237)(ohnishi@sc.niigata-u.ac.jp, ohnishik@ma.tlp.ne.jp)
2
Department of Biology, Faculty of Science, Niigata University,
Ikarashi-2, Nishi-Ku, Niigata, 950-2181, Japan

Abstract: Shannon’s information quantity, I(E) = log (1/P(E)), is defined under the assumption of the existence of “cognitive subjective entity” capable of judging yes/no or occurred/non-occurred of an event E (which occurs with aprobability P(E)). Final acceptor/user of information is a living individual, although first and/or intermediate sender(s)and/or acceptor(s) of information may be either living individual(s) or non-living element(s) or man-made machine(s).
Thus we can conclude that information is a most essential character of living individuals, and that information and life
must have had simultaneously emerged as “minimum cognitive system (MCS)”. Since then, living individuals/lives
must have evolved as “self-improving learning neural network machine” capable of “active evolution”. How MCS
could have emerged was discussed. Not only cognitive origin and evolution of life, but also active evolution of
cognitive organisms were concluded as basic, general evolutionary principles.
Keywords: origin of information, semiogenesis, minimum cognitive system, active evolution
rhodopsin. The sender of the information is a nonliving thing (the sun), and the last receiver (acceptor)of the information is the living individual of an archaebacterial uni-cellular organism which can use a protein-tool or protein-machine for receiving the hν0 −information, which is further converted to bioenergy
inside the uni-cell individual via proton-pum
p. The hν0 energy was not “information” before theemergence of bacterio-rhodopsin-possessing archaea,
and the end-user of this information is the living unicell organic individual. Therefore the origin of the hν0information occurred when archaea first possessed or
made the hν0-utilizing bacterio-rhodopsin.
Example 2: A unicellular individual of a eubacteria, E.
coli, receives DNA codon-information from its intracellular DNA derived from the previous generation, byusing intracellular information-processing molecularmachines called mRN A s . The sender of codoninformation is the DNA derived (via cell division)from the uni-cell individual of the previous generation, and the final user/receiver of the DNA-informationis the living uni-cell organism capable of using mRNAand other RNA-machines and protein-enzymemachines.
Example 3: A multicellular human individual utilizes
solar hv00 light-energy(v= v00) by using green-sensitive
rhodopsin molecules embedded in the membrane ofgreen-sensitive optic cells in human’s optic organ. The
final user of the hv00-energy information is the living
multicellular human individual. Note that optic cells aswell as green-sensitive rhodopsi n s a r e not living
individuals, and do not actively use the hv00-photon
information.
Example 4. In vocal conversation between two persons,both of the first sender of vocal information and thelast receiver/user of the vocal information are living
human individuals, who can actively use the finallyI. INTRODUCTION
Biotic systems are full of “information” [1][2], but
the question, “what is information ?” is not fully answer-ed in biology as well as in other areas of information
sciences. In this paper, the origin and evolution of information systems and semiotic systems were analyzed
from evolutionary and cognitive viewpoints. The results
strongly suggest that information-accepting ability is an
essential character of life, and information and life seemto have simultaneously occurred by the emergence ofearliest cognitive life. Semiogenesis would have generated efficient biomachines which are bio-individuals.

II. ORIGIN OF INFORMATION
Shannon’s “amount of information”, I(E) = log(1/P(E)), is defined under the assumption of theexistence of “cognitive subjective entity” capable ofjudging yes/no or occurred/non-occurred (= 1/0) of anevent E (which occurs with a probability P(E)). Finalreceiver/user of information is a living individual,
although first and/or intermediate sender(s) (=outputter(s)) and receiver(s) of information may be either living
individual(s) or non-living element(s), or man-made/organism-made machine(s).
Let us consider some examples of information
systems.
Example 1. Solar light energy (photon) hν ( v = v0 )is received as information for driving a bioenergyaquiring proton-pump by bacterioro-dhopsin of anarchean,Halobacterium salinarun [1]. In this case,initial sender of hν0 information is the sun, a nonliving entity, and the information is recognized by thelivin
g archean with using its information-receiving
molecular tool or machine (receptor), bacterio-©ISAROB 2011645

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

received information.
Example 5. In radio-broadcasting of various sounds,
the first sender(= outputter) of the sound-information iseither non-living entity/element or a living organismic
individual, and inter-mediate information-receivers andintermediate information-senders are also either nonliving entity or living organic individuals, as well as
man-made information-processing machines which arebiotic (or human) cultural products. The last information receivers/acceptors being end-users of the soundinformation are living human individuals possessing
acoustic organ, “ears”.
From all of these E xamples 1 - 5 , end - user s of
information a r e u n e x c e p ti o n a l l y l i v in g organic
individuals, every of which is the unit of Darwinian
natural selection. Thus, “information” is a kind ofvalue for the living individual in increasing Darwinian
fitness. Before the emergence of life, any end-user of
information did not exist, and therefore, any“information” cannot be reasonably defined for prebiotic world. Information-using is an actively cognizing
behavioral process of living organic individuals.
In conclusion, “information” first emerged simultaneously with the origin of life (= the origin of living
individuals capable of evolving via Darwinian natural
selection). This means that “information” is a most essential feature of life itself, and that life and information
have had emerged simultaneously. A most important
problem for finding the origin of life is, therefore, how
to know the earliest “minimum cognitive system”
which possesses information-using faculty. How“minimum cognitive machine” could have emerged is
an important problem remaining to be solved.

II.

Fig. 1. Hierarchical neural network-type biomachinogenesis generating upper-leveled individuals via Altruistic
sociogenesis (Modified from [3]). Bee-superorganisms andmulticellular individuals are compared. DNA-flows makea hierarchical neural network machine possessing feedback DNA flows. “Parental manipulation” such as Polytes
maternal manipulation and Drosophia bicoid-mRNA seems
tofunction as “teacher information” in man-made learning neural network machines.

level of hierarchical individuality [3].
Paralell hierarchical evolution of individuality has
occurred in generating bee-super-organismic individuals and multi-cellular animal individuals, as illustrated in Fig.1. Queen-worker-type hierarchical (eu)society such as bee society is known to have evol-ved by kin selection [7]. Since relatedness (r) between workers and queens(=worker’s younger sisters)
is 3/4 of DNA (r=3/4), such altruistic behavior isconsidered to be equivalent to DNA-information
flow from workers to queens, as shown in Fig. 1.
DNAs outputted by gametes of queens are further
inputted to the workers and queens of the next generation via fertilization, which makes feedback
DNA-flows from the queens of present generation
to the workers and queens in the next generation.
Similar DNA-flows can be found in multicellularanimals in which each animal is originally an altruistic queen-worker-type hierarchical society of unicell animals consisting of queen-type germ-line cells(=queen-cells) and worker-type somatic cells (=worker-cells). Hamilton considered multicellularanimals have had evolved via altruistic behaviors ofworker-cells (= originally, uni-cell animals) to queen-cells [7].
In animals, multicellular individuals are considerd
to have evolved via an early super-organismic kinsociety of uni-cell animals. However, the evolutionary results show that upper-leveled (multi-cellular)
individuals really live and behave as active evolving organisms, and that germ-line cells and somatic cells are elements or parts of upper-leveledliving individuals. This is very similar to beesuperorganisms, which strongly suggests that“worker-bees and queen-bees are not living individuals, and are elements or parts of super-organisms.
The living individuals are bee-superorganisms.

ORIGIN AND HIERARCHICALEVOLUTION OF INDIVIDUALITY

Biotic individuals such as unicell bacteria andmulticellular animals are actively behaving entities
which are units capable of evolving via natural
selection. Throughout evolution different levels of
individualities have had evolved, and every behaving individual is well-made biomachine having
cognitive information-processing systems [3][4].
In modern living organic individuals, unicell bacteria and haploid uni-cell organisms are lowest
levels of individuality, confirming that unicell organsmic individuals having one set of DNA genes are
earliest living-organisms immediately after the emergence of DNA-type genetic machinery. Any ofliving organic individuals before the emergence of
DNA has not yet been known to date. Emergenceof unicell diploid individuals is the first hierarchicalevolution of individuality, as has been discussed by
Maynard Smith [5] and Dan-Sohkawa [6]. Multicellularization of unicell diploid and/or haploid individuals is the next step of hierarchical evolution.
The so-called super-organism of bee or other hymenopterran eusociety is the most highly evolved©ISAROB 2011646

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Parallelisms between bee-superorganisms and multicellular animal individuals are much more striking,as shown in Fig.2, where the well-known bee-dancesystem for collecting pollens and honey is compareed with the Aplasia simple neural system. Workerbees coming back to hive and making dance (=sensory bees) well correspond to sensory neurons (=worker-cells) in Aplasia siphon system, and bees
receiving dance-language correspond to Aplasia
motor-neurons (worker-cells). The dance language
being a semiotic signal corresponds to synaptic molecular signals evolved as semiotic signals. Thus,
dancing bees and dance-recognizing bees are “sensory bees” and “motor-bees”, respectively. Semiogenesis generating bee-dance and Aplasia synaptic
molecular signals must have had important roles forthe genesis of biomachine consisting of queens(or queen-cells) and workers(or worker-cells). Neurons
are not living individuals, and quite similaely, bees
are not living animal individuals. Bee-superorganisms are “genuine living bee-individuals”.
In Fig. 2, Aplasia neural system is evidently a“cognitive system” for accepting water-flow informa-tion and making adaptive muscle-moving. Very similarly, bee-dance system is also a “cognitive system” of bee-superorganism for accepting polleninformation and making adaptive pollen-collecting
movements of motor-bees. The parallelism between
these two cognitive systems tells us “how biotic
cognitive systems could have had evolved.” Semiogenesis is very essential for making efficient biomachinogenesis.

biomachine suggest that biotic individuals would bea cognitive machine similar to a hierarchical neuralnetwork machine. In the case of the eusociety ofPolystes (hymenopteran), maternal manipulation todaughter wasps makes daugtters be worker-wasps.
Similarly, earliest determination of embryonic cells
in Drosophia depends on the gradient of the concentration of maternal mRNAs (such as bicoidmRNA). These maternal effects are very similar tothe so-called “teacher-information depending on feed-back information” in learning neural network
machines, as shown in Fig. 1. Thus the scheme in
Fig 1 suggests that these superorganismic biomachines would have evolved as “hierarchical learning
neuiral network biomachines”. Such biomachines
can input enviomental and intra-body information,and output DNA information as schematized in
Fig.1. Accordingly, repeating of generation is a“thinking process” of the individual, which is acognitive neural network machines. Thus “thinking”
of the well-made cognitive bio-machines must have
improved biomachines in generating the present-day
well-made bioorganisms. Repeting of generations ofsuch learning biomachines is thinking processes oforganisms. Now we have reached to a clear answer
to the important question,“Who made the well-madebiomachines ?”, proposed by Dawkins [8]. The answer is that “Biotic individuals have had “actively”
made and improved the well-made biomachines by“thinking” via repeating generations.
The essential difference between autopoietic/active
bio-machines (= biotic individuals) and man-madelearning neural network machines (NNwMs) are thedifference of teacher-information (TI). TI is given
from outside by man-made program in man-madelearning NNwMs. However, as shown in Fig.1, IT
is included in the biotic system, and therefore, bioindividuals can actively self-improve their own individuals (=bio-machines) by repeating generations.
Thus, bio-individuals are cognitive self-improving
leaning NNwMs. The NNwM in Fig.1 is not a
simple 2-layered NNwM capable of performing“linearly separable” cognitions, but a multi-layered
complex NNwM capable of performing “non-linearly separable” cognitions, because every element of
the both layers (= queen-layer and worker-layer) in
Fig.1 is originally a cognitive (uni-cell or multi-cellular) living individual possessing intra-individual
neural networks of information-flows. Hierarchicalsociogenesis is thus considered to make an efficient
multi-layered neural-network bio-machine via semiotic function.
Uni-cell organisms also have intracellular complex
information networks capable of inputting environmental and intracellular information, and adaptively
outputting various kinds of information. Thus unicellorganisms are also considered to be cognitive machines which are complex thinking NNwMs.
Accordingly we have reached a conclution thatFig.2. Close similarities between the Aplasia nerve
system and the bee dance-language system.

II. BIOTIC INDIVIDUALS AS THINKINGMACHINES
Returning to Fig.1, we can find that the DNA-information flows in the hierarchical superorganismic©ISAROB 2011647

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

machines, semiogenesis must have been important
throughout evolution.
Early evolution having generated “minimum cognitive system(MCS)” needs to be analyzed from various aspects. An interesting approach is to analyze
the emergence of MCS from simple harmonic oscilator, F2x = -ω2x, where x = (x1, x2)T, F = (aij)2,2,
in which a22 = -a11, a21 = -(a112 + ω2) / a112, andwhere xi and aij (i,j = 1,2) are real numbers orreal functions. This model gives a MCS-like oscilla-tor, as shown in Fig.5, as has been recently discussed. See legend of Fig.5 and ref. [8] for details.

Fig. 3. Generalized model of biotic individual possessing
intra-individual cognitive complex neural networks of info
rmation flows and feedback information-flows.

living individuals are cognitive NNw bio-machines
which can actively think/consider and self-improving.
Accordingly, evolution is “active thinking process” ofliving organisms, resulting in “active evolution”.
A most generalized bio-individual consisting ofcomplex NNwM possessing feed-back information
flow from present to the next generation is schematically shown in Fig. 3.
Furthermore, diploid “species” can be considered
as a “hierarchical super-neural network machine
shown in Fig. 4, where multi-cellular male and female diploid individuals are input layer, and haploid gametes (ova/sperms) are output layer elements
which make feedback DNA-flows to the next generation. The “species really exists as a probabilisticneural network machine, which is also a cognitive
biosystem capable of thinking and improving to
make more adaptive bio-systems and eco-systems.

Fig. 5. Three-layerd neural network-like structure ofgeneralized harmonic oscillator system, where F = t((a11,a12), (a21, a22)), where a22 = - a11, a21 = - (a112 +ω2 )/a12 . By letting Qs = F4, and w = ω4, we find that Qs
and w satisfy a self-replication equation, Qs x = w x. If
ω2= 1, then F2 is a complete self-replication operator.
x = F0x = (F0x1,F0x2) T = (x1,x2)T ; Fx = F1x =(F1x1, F1x2)T = (a11x1+a12x2, a21x1+a22x2)T ; Fkx =(Fkx1, Fkx2)T, where Fkx1 = a11 Fk-1x1 + a12, andFkx2 = a21 Fk-1x1 + a22 Fk-1x2.
REFERENCES[1] Albert B, et al.(2008) Molecular Biology of the Cell,
5th ed., Garland Science, New York, NY.
[2] Rowe,GW(1994), Theoretical Models in Biology.
Oxford Univ.Press, Oxford.
[3] Ohnishi K, et al.(2002): Neural-network-like biomachinogenesis via semeiogenesis : A unified theory on theorigin of genetic codes and other semeiotic systems. Viva
Origino 30(2), 63-78.
[4] Ohnishi K., Shutou H., Sawamura H., and Goda M.
(2001): Neural-Network-like machinogenesis via
semeiogenesis: Origins of genetic codes and othersemeiotic systems. SIG Technical Reports 2001(37), 3-6.
[5] Maynard-Smith J (1989), Evolutionary Genetics,
Oxford Univ. Press, Oxford.
[6] Dan-Sohkawa M(1996), Reading the complexity ofLife. Biology of hierarchy (In Japanese),Heibon-sha,Tokyo.
[ 7 ] H a m i l t o n W D ( 1 9 6 4 ) , The genetical evolutionof social behaviour I, II. J. Theor. Biol. 7: 1-16, 17-52.
[8] Dawkins R (1996) The Blind Watchmaker. Norton &Co., New York.
[8] Ohnishi, K. (2008): A preliminary consideration on theorigin of life as a cognitive system.Proc. of the 13th Int.
Conf. on Artificial Life and Robotics, pp.686-689, Oita.

Fig.4. Hierarchical probabilistic neural network
machine model of diploid species.

III. EMERGENCE OF “MINIMUM COGNITIVE SYSTEM” AS A FIRST LIFEThe question, “What is first life ?”, needs to be
re-considered from the aspect of above-mentioned
cognitive life. “Information” emerged simultaneouslywith the emergence of cognitive individuals as thelast receiver(or acceptor)/user of information. For
efficient bio-individuals to evolve as thinking bio-

©ISAROB 2011648

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Study on Discrete Adiabatic Quantum
Computation in 3-SAT Problems
Mohamed El-fiky

Satoshi Ono

Shigeru Nakayama

Department of Information and Computer Science,
Faculty of Engineering, Kagoshima University
1-21-40, Korimoto, Kagoshima 890-0065, Japan
TEL: +81-099-285-8453

FAX: +81-099-285-8464

E-mail: {mh fiky77, ono, shignaka }@ibe.kagoshima-u.ac.jp
Abstract
This paper proposes a new variation method for the phase functions of the adiabatic quantum
algorithm, quadric variation method. Experiments are carried out solving 3-SAT problem withdiscrete adiabatic quantum algorithm to compare the proposed two formulation of quadric variation
method performance with the previously proposed methods linear and cubic.
Keywords:
Quantum Computation; Adiabatic Theory; Hamiltonian; k-SAT Problem.
II. k-SAT P ROBLEMS

I. I NTRODUCTION

The k-satisfiability problem (k-SAT) is a combinatorial
search problem, whose instance is a Boolean expression
written using AND, OR, NOT, n variables, and m clauses. A
clause is a logical OR of k variables, each of which may be
negated. Given an expression, the solution is an assignment,i.e., a value of TRUE or FALSE values for each variable
that will make the entire expression true,i.e., satisfying all
the clauses [7]. An example 2-SAT instance with 3 variables
and 2 clauses is (v1 OR(N OT v2 )) AN D (v2 OR v3 ), whichhas 4 solutions, for example, v1 = v2 = f alse and v3 = true.
For a given instance,the cost c(s) of an assignment s is thenumber of clauses it does not satisfy. For k ≥ 3, k-SAT is
NP-complete, i.e., among the most difficult NP problems in
the worst case [8].

The field of quantum computation have gone through anamazing development in past decade, it has rise as one of
the hot area of research after existence of many quantum
algorithms showing that the quantum algorithms greatly
enhance the efficiency of solving problems believed to be intractable on classical computers. as Peter Shor’s polynomial
time quantum algorithm for factorizing integers [1], Groveralgorithm for unstructured search with quadric speed up overany classical algorithm [2] and Hogg algorithm [3].
The Model of adiabatic quantum computation is a newparadigm for designing quantum algorithms proposed by
Farhi et al.[4]. The Adiabatic model is based on quantum
adiabatic theorem, where the quantum computer evolves thequantum system slowly to switch gradually from an initial
Hamiltonian with ground state easy to construct, to a final
Hamiltonian whose ground state encodes the solution of the
problem being solved.
In recently published articles, the Adiabatic quantum algorithm, was shown to give polynomial average cost growthfor some NP combinatorial search problems as Satisfiability
problems [5], and set partitioning problem [6].
This paper compares the adiabatic algorithm performancein solving 3-SAT problems using three different variation
methods Linear,cubic, and the proposed method, as result
the corresponding search costs and probability of finding
the solution are shown.

©ISAROB 2011

III. T HE Q UANTUM A DIABATIC T HEOREM
The adiabatic theorem states that, if the Hamiltonian of
any quantum system H(t) varies slowly enough, the state of
the system will stay close to the instantaneous ground stateof the Hamiltonian at each time t [3]. Assume we can build a
Hamiltonian H (c) with ground state encodes the solution of
the problem instance to be solved, and prepare the system inthe known ground state of another Hamiltonian H (0) . Thenthe adiabatic algorithm can continuously evolve the state of
the quantum computer using
H(f ) = (1 − f )H (0) + f H (c)

649

(1)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table IT HE PARAMETERS USED IN DISCRETE AND CONTINUOUS VARIATION METHODS .

Parametersphase functions∆
Phase shift function
Phases Mix function

Variation methods√Linear
1/ j
ρ(f ) = f
τ (f ) = 1 − f

Quadric 1
const(=1)
ρ(f ) = f 2
τ (f ) = 1 − ρ(f )Quadric 2
const(=1)
ρ(f ) = 2f − f 2
τ (f ) = 1 − ρ(f )with f ranging from 0 to 1 [3]. Under suitable conditions,
i.e., with a nonzero gap between relevant eigenvalues ofH(f ), and with sufficiently slow changes in f , the adiabatic
theorem guarantees that the evolution maps the ground stateof H (0) into the ground state of H (c) , so a subsequent
measurement gives a solution.

2) For Steps h = 1 through j repeat the matrix multiplication :|ψ (h) ⟩ = Uh (f )|ψ (h−1) ⟩(6)
where |ψ (h−1) ⟩ is the amplitude state vector at steph − 1, and Uh (f ) is unitary evolution operator for hth step which can be represented as

IV. T HE DISCRETE A DIABATIC ALGORITHM

Uh (f ) = e−iτ (f )HIn this paper, we use the algorithmically equivalent discrete formulation of the adiabatic algorithm acting on theamplitude state vector initially in the ground state of the(0)
Hamiltonian H (0) , which can be represented as |ψs ⟩ =1T√ [1, 1, ..., 1] . Consider a discreet Hamiltonian H(f ) ofN
the general formH(f ) =  (f )H (0) + (f )H (c)

(2)

∆/2

·e−i (f )H(c)∆·e−iτ (f )H

(0)

∆/2

V. E XPERIMENTS AND C ONSIDERATIONS
A. Variation Methods and parameter ∆Recently several variation methods for phase shift function (f ), and phase mixing function  (f ) were presented
as an attempts to decrease the overall search cost [3]. In
this paper we present monotonic (means + =1) versionof quadric variation method in two formula of quadricpolynomial in f . Table .1. summaries this methods , linear, cubic and the two quadric formula we presents, and also
it shows the values of the parameter ∆ as used with eachmethod.
Figures 1, 2, 3, amd 4 shows the phase fuctions for eachvariation methoud vs. the f value from 0 to 1, foucasing in
the figures we can note that the quadric variation methoudshown in fig. 3, has smalle diversity area than the others
whcih gives a sign for expecting better results. A good
performance of the discrete adiabatic algorithm requires
an appropriate choice of parameter ∆. The experiments
have shown that the performance of the algorithm remains
good for moderate value of j provided that ∆ is below
some threshold value. The experiments for solving 3-SAT
problems with n ≤ 20 has shown this threshold to be
somewhat near 1.

(4)

Although, the two functions  (f ), and (f ) are not necessary to be monotonic (i.e. obey the constraint  (f )+(f ) =1), we consider only the monotonic functions [?].
In matrix form [3], the Hamiltonian H (c) is a diagonal
matrix{
1 if r=s(c)
Hr,s = c(s)δr,s , where δr,s =(5)0 otherwise
This Hamiltonian introduces a phase shift factor in theamplitude of assignment s depending on its associated costc(s), where the higher cost results in more phase shift.
The Hamiltonian H (0) can be implemented with elementary
quantum gates by use of the Walsh-Hadamard transformwith elements Wr.s = 2−n/2 (−1)r.s [3], where H (0) =W DW and D is a diagonal matrix with the value for stater given by the sum of the bits, i.e, the element Dr,r is justa count for the number of bits equal to 1 in state r.
A single trial of the algorithm consists of j steps, parameter ∆ and can be described as1) Initialize the amplitude state vector to the ground state(0)of H (0) giving equal values for all states as |ψs ⟩ =√1 [1, 1, ..., 1]TN©ISAROB 2011

(0)

(7)
3) Measure the final system After the j steps take place,theto find a solution is given by Psoln =∑ probability(j) 2|| with the sum over all solutions s.
s ||ψAs a choice for the evolution, f is chosen to vary linearly
from 0 to 1. Specifically, we take f = h/(j + 1) for step h,where h is ranging from 1 to j.

where  (f ) and (f ) are phase mixing and phase shiftfunction, respectively. both of them are arbitrary functionsof f where (0 ≤ f ≤ 1), see Table.1, subject to the boundary
conditions
 (0) = 1, (0) = 0(3)
 (1) = 0, (1) = 1

Cubic
const(=1)
ρ(f ) = 1.921f − 2.665f 2 + 1.782f 3
τ (f ) = 1 − ρ(f )B. Search behavior
In this section we compare the search behavior of thediscrete adiabatic algorithm using the variation methods assummarized in Table.1. First is the linear variation method

650

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

1

1phase shiftphase shiftphase mixing

0.9

0.9

0.8

0.8

0.7

0.7
0.6

0.6

sn
iot
nc 0.5
uf
es
ah
p 0.4

nso
it
cn 0.5
uf
sea
0.4
ph

0.3

0.3

0.2

0.2

0.1

0.1

0

phase mixing

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0

1

0

0.1

0.2

0.3

0.4

0.5

Figure 1. Phase shift and phase mix functions V s f for the linear variation
method.

0.8

0.9

1

1

phase mixing

phase shift

0.9

phase mixing

0.9

0.8

0.8

0.7

0.7

0.6

0.6
nso
it
cn
uf 0.5
sea
hp

nso
it
nc 0.5ufsea
hp 0.4

0.4

0.3

0.3

0.2

0.2

0.1
0

0.7

Figure 3. Phase shift and phase mix functions V s f for the quadricvariation method with ρ = 2f − f 2 formula.

1

phase shift

0.6

f

f

0.1
0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0

f

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

f

Figure 2. Phase shift and phase mix functions V s f for the quadricvariation method with ρ = f 2 formula.

Figure 4. Phase shift and phase mix functions V s f for the Cubic variation
method.

corresponds to the √continuous adiabatic algorithm, thismethod uses ∆ = 1/ j, phase shift (f ) and phase mixing (f ) functions varies monotonically as linear function of f ,and number of steps j grows as cubic number of bits n3 .
Second method is the cubic variation [?] with (f ) and  (f )varies monotonically as cubic polynomial in f , constant ∆,and uses j grows as square number of bits n2 , and finallythe quadric variation method with two formula (f ) = f 2
and (f ) = 2f − f 2 .
Figures 5, and 6 compare the search behavior of thealgorithm in solving 3-SAT problem with n = 8, and16,
respectively. The figures show that the quadric variation
method with number of steps j only as 2 ∗ n can achieve
Psoln near 0.5 , which shows faster search behavior when
it is compared with the algorithm behavior using cubic©ISAROB 2011variation with number of steps at least n2 and the linear
variation with j = n3 .
C. Search cost
The search cost is defined to be the expected numberof steps required to find a solution C = j/Psoln . Fig. 7
compares the average search cost C for the Linear, cubic,
and the two formulation of quadric variation methods for the
adiabatic algorithm. it shows that using quadric variation
with just enough number of steps j = 2 ∗ n to achieve
moderate Psoln as shown in Figures. 5, and 6, reduces thesearch cost below the other methods. However the algorithmwith linear variation could achieve Psoln near 1 in most of
the trials as shown, the number of steps required j grows as
n3 giving a large search costs, far higher than those of other651

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

1

10000

Linear Variation
Cubic Variation
Quadric 1
Quadric 2

Linear0.9

Cubic
Quadric1

0.8

Quadric2

4379.66

1000

0.7
0.6

Ct

so
c
hc100
ra
e
S

nl
os 0.5P
0.4

424.29561.47115.72
78.03

58.07 56.22
29.27 31.27
19.67

0.3

9.72

10

10.22

0.2
0.1
0

1

10

Steps

100

1

1000

Figure 5. Psoln vs number of steps h for the 3 methods averaged over
10 random instances of 3-SAT problems with n = 848
Number of bits n

16

Figure 7. Log plot of the average search cost vs the number of variables
n using the same instances in Figures 5 and 6.

1
Linear

SAT problems with the discrete the adiabatic algorithm,
using a new monotonic variation method for the phase shift
and phase mixing function , quadric variation with two
different formula. The experiments have revealed that using
the quadric variation method improves on other variation
methods, in resulting Search cost and search behavior.

Cubic

0.9

Quadric1
0.8

Quadric2

0.7
0.6

lno0.5s
P

R EFERENCES

0.4[1] P. Shor, ”Polynomial-Time Algorithms for Prime Factorization on a
Quantum Computer”, SIAM J. Computing, Vol. 26, 1997.

0.3
0.2

[2] L. Grover, ” Fast Quantum Mechanical Algorithm for Database
Search”, ACM Symp. Theory of Computing, ACM Press, New York,
1996, pp. 212-219.

0.1
0
1

10

100

1000

[3] T. Hogg, Adiabatic Quantum Computing for Random Satisfiability
Problems, quant-ph/0206059, 2004.

10000

Steps

[4] E. Farhi, J. Goldstone, S. Gutmann, and M. Sipser, Quantum Computation by Adiabatic Evolution, quantph/0001106 .

Figure 6. Psoln vs number of steps h for the 3 methods averaged over
10 random instances of 3-SAT problems with n = 16

[5] E. Farhi. A quantum adiabatic evolution algorithm applied to random
instances of an NP-complete problem. Science, 292:472?476, 2001.

method. For comparison the figure shows that the mediansearch cost at n = 16 for the quadric1, quadric2, cubic,
and linear methods to be 58.07, 56.22, 424.29 and 8749,
respectively. which shows improve in the resulting cost
reduction using quadric variation method, due to reduced
number of steps .

[6] V. N. Smelyanskiy, and D. A. Timucin. Simulations of the adiabatic
quantum optimization for the set partition problem. Los Alamos
preprint quant-ph/0112143, 2002.
[7] M. R. Garey and D. S. Johnson. Computers and Intractability:A Guide
to the Theory of NP-Completeness.W. H. Freeman, San Francisco,
1979.

VI. C ONCLUSIONS[8] S. Kirkpatrick and B. Selman, Critical behavior in the satisfiabilityof random boolean expressions. Science, 264:1297-1301, 1994.

The quantum adiabatic algorithm is a remarkable discovery because it offers new insights into the usefulness ofquantum resources for computational tasks. In this paper,
we have presented an experimental study in solving 3-[9] S. Das , R. Kobes, and G Kunstatter Energy and Efficiency of Adiabatic Quantum Search Algorithms, quant-ph/0204044v4; Phys.Rev.
A 57, (2003).
[10] W.van, Dam M. Mosca, and U. Vazirani, How Powerful is Adiabatic
Quantum Computation?, quant-ph/0206003 v1 (2002).

©ISAROB 2011652

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fault-tolerant Image Filter Design using Particle Swarm Optimization
Zhiguo Bao, Fangfang Wang, Xiaoming Zhao and Takahiro Watanabe
Graduate School of Information Production and Systems
Waseda University
Kitakyushu-shi, Japan
Email: baozhiguo@moegi.waseda.jp
Abstract

Start

This paper describes mixed constrained image filter design with fault tolerant using Particle Swarm Optimization(PSO) on a reconfigurable processing array. There may
be some faulty Configurable Logic Blocks (CLBs) in a reconfigurable processing array. The proposed method withPSO autonomously synthesizes a filter fitted to the reconfigurable device with some faults, to optimize the complexity and power of a circuit, and signal delay in both CLBs
and wires. An image filter for noise reduction is experimentally synthesized to verify the validity of our method.
By evolution, the quality of the optimized image filter ona reconfigurable device with a few faults is almost same asthat with no fault.

Definecircuitfunction
UsePSOtofindoptimalsoluti
Removeredundanciesfromthesolu

tion

Getthefinalcircuit
End

Figure 1: The overview of our method.

Keywords

netic Algorithm) [8], and the experimental results showedthat the resultant image filter was surely fault-tolerant. But
the problems of quality and processing time still remain,
and which evolutionary method (such as GA, GP (Genetic
Programming), PSO (Particle Swarm Optimization) andACO (Ant Colony Optimization)) is most suitable has notbeen investigated.
This paper describes an efficient image filter design fornoise reduction using PSO on a reconfigurable processing array, where some faulty Configurable Logic Blocks(CLBs) may exist at random. The mixed constrained on
circuit complexity, power and signal delay in both logic
blocks and wires are optimized. In this design, first, theevaluating value about correctness, complexity, power andsignal delay are introduced to the fitness function. Then
PSO autonomously synthesizes an image filter which issimple and has better performance and fits to the reconfigurable processing array with some faults. To verify thevalidity of our method, an image filter for noise reduction
is experimentally synthesized.
The organization of this paper is as follows: a brief
overview of PSO is described in the next section. Section
3 describes fault-tolerant design optimization for an image
filter using PSO. Section 4 shows the experimental results.
Finally, Sect. 5 concludes this paper.

Particle Swarm Optimization, Mixed Constrained Image Filter Design, Fault Tolerant

1 Introduction
The image filter design problem is often approached bymeans of evolutionary design techniques. In addition to an
optimization of filter coefficients (for example, [1]), evolutionary approaches are applied to find a complete structureof image filters. In [2], Gaussian noise filters were evolved
using a variant of Cartesian Genetic Programming in whichtarget filters were composed of simple digital components,
such as logic gates, adders and comparators. A few years
later, image filters for other types of noise and edge detectors were evolved using the same technique [3, 4, 5]. But
there were few discussions about fault-tolerance for an image filter design.
Recently chip integration is higher and higher, so thatit increases the probability of faulty components and thecomplexity of designs increases the probability of humanerrors. The tolerance for faults is diminishing as the systems are demanded for high reliability. Therefore, theneeds for fault-tolerant designs are stated as the long-term
grand challenges in [6]. To solve this problem, we proposed a fault-tolerant image filter design using GA (Ge-©ISAROB 2011

on

653

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011yColumn:
36
34
26
24

2 Particle swarm optimization
PSO is an algorithm model on swarm intelligence thatfinds a solution to an optimization problem in a search
space.
In PSO, a particle represents a candidate solution to
the problem. Each particle is treated as a point in the
D-dimensional problem space. The i-th particle is represented as Xi = (xi1 , xi2 , · · · , xiD ). The best previous position (the position giving the best fitness value)of the i-th particle is recorded and represented as Pi =(pi1 , pi2 , · · · , piD ). The index of the best particle among
all the particles in the population is represented by the symbol g. The rate of the position change (velocity) for particle
i is represented as Vi = (vi1 , vi2 , · · · , viD ). The particle isupdated according to the following equations:
(t+1)

vid

=

(t)

6
4
2

0 2

(1)

(t)
(t+1)
xid + vid .

(2)

4

5

6

CLB9

CLB13

CLB17

CLB21

CLB25

CLB29

Row:
0

CLB10

CLB14

CLB18

CLB22

CLB26

CLB30

1 Out

CLB11

CLB15

CLB19

CLB23

CLB27

CLB31

2

CLB12

CLB16

CLB20

CLB24

CLB28

CLB32

3

28

32

8 12

18 22

38 42

48 52

58

x

3.1 Image Filter
Every image operator is considered as a digital circuitwith nine 8-bit inputs and a single 8-bit output, which processes gray-scaled (8-bit/pixel) images.
Every pixel value of the filtered image is calculated using a corresponding pixel and its eight neighbors in the processed image [4, 5].

n: number of particles in a group.
D: number of members in a particle.

3.2 Reconfigurable Processing Array for Image
Filtert: pointer of iterations (generations).

The reconfigurable image filter is implemented as a Virtual Reconfigurable Circuits (VRC) (Fig. 2) originally proposed in [4]. As a new pixel value is calculated using nine
pixels, the VRC has got nine 8-bit inputs and a single 8-bit
output. The VRC consists of two-input Configurable Logic
Blocks (CLBs) placed in an array. In our proposed, a 6 ∗ 4array is need because its size is enough for our image filter
from the results in paper [8]. Any input of each CLB may
be connected to either a primary circuit input or the output
of a CLB in the preceding column. Any CLB can be programmed to implement one of the functions given in Table
1 [8, 9], all these functions operate with 8-bit operands andproduce 8-bit results.
The CLB with different function, has different complexity, power and signal delay. We newly define the valuesof complexity (FC), power (FP) and signal delay (SD) for
each function in a CLB, as in Table 1.
As shown in Fig. 2, the coordinates of inputs and output
of each logic block are defined based on VRC, so that we

w: inertia weight factor.
c1 , c2 : acceleration constant.
rand(), Rand(): uniform random value in the range [0,1].
(t)

vid : velocity of particle i at iteration t, Vidmin ≤ vid ≤Vidmax .
(t)

xi : current position of particle i at iteration t.
The inertia weight factor w is employed to control theimpact of the previous history of velocities on the current
velocity, thereby influencing the trade-off between global(wide-ranging) and local (fine-grained) exploration abilities of the “flying points”. A larger w facilitates global
exploration (searching new areas) while a smaller w tends
to facilitate local exploration to free-tune the current searcharea. Good values of w are usually slightly less than 1 [7].

©ISAROB 2011

3

PSO is applied to search good solutions to optimize theimage filter design on a reconfigurable processing arraywith some faults.
The resultant image filter has an identical functional behavior with less circuit complexity, less power and less signal delay.

where,
0 ≤ i ≤ (n − 1), 1 ≤ d ≤ D.

(t)2

3 Image filter design using PSO(t)+c2 ∗ Rand() ∗ (pgd − xid ),1

Figure 2: A reconfigurable processing array with faults.

w ∗ vid + c1 ∗ rand() ∗ (pid − xid )
(t)

(t+1)
xid
=

16
14

0

I0
I1
I2
I3
I4
I5
I6
I7
I8

654

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

are ignored, and thus remain unfiltered. The fitness valueof a candidate filter is obtained as follows:(1) the VRC is configured using a candidate chromosome,(2) the created circuit is used to produce pixel values in
the image f i, and(3) the fitness value is calculated as

Table 1: Functions implements in a CLB.
ID
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
-

Function255x255 − xx∨yx∨yx∧ynot(x ∧ y)
x⊕y
xÀ1
xÀ2
(x ¿ 4) ∨ (y À 4)x+y
x +s y(x + y) À 1
max(x, y)min(x, y)(wire)

Description
Constant
Identity
Inversion
Bitwise OR
Bitwise x OR y
Bitwise AND
Bitwise NAND
Bitwise XOR
Right shift by 1
Right shift by 2
Swap
+ (addition)+ with saturation
Average
Maximum
Minimum(wire)

FC
8
16
24
32
40
32
40
64
15
14
16
358
367
350
240
240
16

FP
5
10
15
20
25
20
25
38
9
8
10
215
220
210
145
145
10

SD
1
2
3
3
4
3
4
4
2
2
2
18
19
18
16
16
2

F itness = (−1) ∗ (F1 ∗ β + F2 ).

(3)

where,
F1 and F2 are defined as follows and β is the weight on
F1 .

F C: function complexity.
F P : function power.
SD: signal delay.

F1 =

nc−2
X nr−2
Xi=1(|f i(i, j) − oi(i, j)|).

j=1

(4)
where,

can calculate the critical length of connection wires.
There may be some faulty CLBs in a reconfigurable processing array at random. The output of a faulty CLB is avalue in the range [0,255] at random.

nc: the number of columns of the pixels in the image.
nr : the number of rows of the pixels in the image.

3.3 Genetic encoding

fi (i , j ): the pixel (i, j) in filtered image f i, the value range
is [0,255].

The chromosome (particle) is a string of integers where
each three continuous integers constitute a logic block.
Each triplet in the chromosome encodes the two inputs andthe function type of a logic block, respectively, such as:oi (i , j ): the pixel (i, j) in original image oi, the value
range is [0,255].

(Input 1, Input 2, F unction type).

F2 =A typical chromosome then can be a sequence of
triplets, such as:

SD ∗ αsd + P g ∗ αpg + Cg ∗ αcg+P w ∗ αpw + Cw ∗ αcw .
(5)where,
1i
Xi = ((IN11 , IN21 , Ftype) · · · (IN1i , IN2i , Ftype)···)

SD: signal delay of a circuit individual, determined by acritical path.

Here, IN1i and IN2i mean positions of the correspondi
ing input signal. Ftypemeans function type of logic block.
iP g: power of logic blocks in a circuit, calculated by summation of all logic block’s power.

For primary input, 0 ≤ IN ≤ 8. For input from output ofa logic block CLBm shown in Fig. 2, IN i = m. Function
in a CLB is defined as shown in Table 1.

Cg: complexity of logic blocks in a circuit, calculated bysummation of all logic block’s complexity.

3.4 Fitness function

P w: power of all wires in a circuit, calculated by summation of all wire’s power.

The pixels of corrupted image ci are used as inputs of
VRC. Pixels of filtered image f i are generated, which arecompared to the pixels of original image oi.
The design objective is to minimize the difference between the filtered image and the original image. The image
size is nc∗nr pixels, but only the area of (nc−2)∗(nr −2)
pixels is considered, because the pixel values at the borders©ISAROB 2011

Cw: complexity of wires in a circuit, calculated by summation of all wire’s complexity.
αsd , αpg , αcg , αpw , αcw : the weights on SD, P g, Cg,
P w, Cw, respectively.

655

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 2: Conditions for evolution.
Number of Generation : 300
Population Size : 1210
Inertia weight factor w : 0.9.
Limit of change in velocity of each member in an individual:min
Vidmax = 0.5 ∗ pmax
= −0.5 ∗ pmax
id , Vid
id .
Acceleration constant : c1 = 2, c2 = 2.

Figure 3: Elite fitness of PSO (Y-axis) vs. the number ofgenerations (X-axis).
The priority of evaluating values in Eq. (3) is: F1 > F2 .
In this experiment, β is set to 0.1 ∗ 109 . The priority ofevaluating values in Eq. (5) is: SD > P g > Cg > P w >Cw. In this experiment, αsd is set to 0.1 ∗ 109 , αpg is set
to 0.1 ∗ 106 , αcg is set to 1 ∗ 103 , αpw is set to 100, and
αcw is set to 1. All α’s and β are empirically assigned inour experiment.

4 Experimental results
Table 2 shows the parameters of the evolution of PSO
used in this experiment. Some preliminary experiments
were performed in advance to decide parameters suitablefor our experiment.
The proposed method was implemented in Eclipse SDK
3.5.1 with Java Runtime Environment(JRE) 1.6.0; andtested on a PC with Inter(R) Core(TM) i7 CPU at 3.33 GHz
and 9.0 GB RAM.
The image filter is evolved for a 512 ∗ 512 Lena image
corrupted by 5% salt-and-pepper noise, shown in Fig. 5(a).
Fig. 3 shows the elite fitness of PSO vs. the numberof generations during the image filter evolution. The elite©ISAROB 2011

fitness is increasing during evaluation time.
Table 3 shows the results on a reconfigurable processing
array with different faults. For each case, we execute over
10 independent trials. “Available” means the number ofavailable CLBs. “Best one” means the best optimized image filter in terms of Mean Difference Per Pixel (MDPP)[10] among 10 trials, “Average” the average values of 10
individuals, and “Worst one” the worst optimized image
filter among 10 trials. The number of used CLBs, the valueof the MDPP and running time are listed. “Ratio” is therelative value of MDPP of each case compared to that of
no fault F aults(0). The larger the fitness is, the better thequality of image filter is. The less the MDPP value is, thebetter the quality is. The less the ratio is, the better thequality is.
The quality of the optimized image filter on a reconfigurable processing array with a few faults (2-4 faults) isalmost same as that on a reconfigurable processing arraywith no fault, that is less than 12.6% in the item of different value of MDPP of the best one.
The quality of the optimized image filter of F ault(2)is only 1.2% less than that of F ault(0) in the item of the
MDPP value of the best one.
An example of chromosome of best one of f ault(2) is
as follows:(0, 0, 0)(0, 0, 0)(1, 4, 15)(8, 0, 0)(0, 0, 0)(0, 0, 0)(11, 12, 11)(11, 0, 0)(0, 0, 0)(0, 0, 0)(0, 0, 0)(15, 7, 5)(0, 0, 0)(0, 0, 0)(0, 0, 0)(4, 20, 14)(0, 0, 0)(0, 0, 0)(0, 0, 0)(0, 0, 0)(16, 24, 11)(0, 0, 0)(0, 0, 0)(0, 0, 0)
The graphical representation of this chromosome isshown in Fig. 4.
As the used Lena image size is relatively large, we cansay that the resultant evolved filter is general purpose forthe same type of noise, that is the filter is able to remove thesame type of noise also from other images. Therefor, theimage filter was first evolved using Lena image and thentested on other images.
Figures in Fig. 5 show the input image with 5% saltand-pepper noise, the MDPP value of these images are
6.33, 6.39 and 6.28, respectively. Figures in Fig. 6 show
the output image by the image filter of Fig. 4, the MDPP
value of these images are 1.74, 1.42 and 1.76, respectively.
Obviously, this image filter could reduce noise for all cases,
even if there are two faulty CLBs.

5 Conclusions
This paper described mixed constrained image filter design with fault tolerance for noise reduction using PSO on
a reconfigurable processing array. By evolution, the qualityof the optimized image filter on a reconfigurable processing array with a few faults is almost same as that on a reconfigurable processing array with no fault. Consequently

656

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Figure 5: The input images with noise.

Figure 6: The output images by the evolved filter of Fig. 4.

©ISAROB 2011

657

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 3: Results on a reconfigurable processing array with different faults.
Item

CLBs
Available
21
19
17
15
13

Faults(0)Faults(2)Faults(4)Faults(6)Faults(8)

Used
8
7
12
6
3

MDPP
1.722
1.742
1.939
2.008
3.117

Ratio
1.000
1.012
1.126
1.166
1.810

Best one
SD
Pb
71
645
88
695
85
1070
50
550
48
435

Cb
1064
1150
1774
910
720

Pw
372
263
393
227
198

Cw
593
419
625
362
316

Used
6.6
5.7
5.6
5.4
4.8

Average
MDPP
Ratio
2.365
1.000
2.390
1.011
2.955
1.249
2.965
1.254
3.624
1.533

Time
405.2
411.5
416.7
425.2
432.0

Used
8
7
5
4
2

Worst one
MDPP
2.442
2.537
3.453
3.525
3.889

Ratio
1.000
1.039
1.414
1.443
1.593

Available: In a 6*4 CLBs array, only one CLB is used in the last column, so the max number of available CLBs is 21.
F aults(x): A reconfigurable processing array with x faulty CLBs at random position, x = 0, 2, 4, 6, 8.
U sed: the number of the used CLBs.

(0,0,0)0 1 2(0,0,0)13

9(0,0,0)10(0,0,0)17(0,0,0)14

18(0,0,0)(0,0,0)22(0,0,0)

EvoIASP 2002), vol. 2279 of LNCS, Springer, 2002,
pp. 255-266.

+

21(0,0,0)(0,0,0)

25

26

29

(16,24,11)30(0,0,0)[3] L. Sekanina (2004), Evolvable components: From
Theory to Hardware Implementations, Springer.

3 4 5
6 7 8

+

min
(1,4,15)

11(11,12,11)

12(11,0,0)19(0,0,0)255

255(8,0,0)15

16(0,0,0)^(15,7,5)

23

max
20

(4,20,14)(0,0,0)24(0,0,0)27

28(0,0,0)(0,0,0)31[4] T. Martinek and L. Sekanina (2005), “An Evolvable
Image Filter: Experimental Evaluation of a Complete
Hardware Implementation in FPGA,” Evolvable Systems: From Biology to Hardware, vol. 3637 of LNCS,
Springer, 2005, pp. 76-85.

32

Figure 4: The optimized image filter of F ault(2).

[5] Z. Vasicek and L. Sekanina (2007), “Evaluation of a
New Platform For Image Filter Evolution,” Proc. theSecond NASA/ESA Conference on Adaptive Hardware and Systems, 2007 (AHS 2007), Aug. 2007, pp.
577-586.

our proposed design method is effective for fault-tolerant
optimization.
We will also apply PSO to autonomous design circuits
for more complex functional requirements, and enhancemore practical information about circuit to fitness function.
Future works are to find better genetic encoding method toapply PSO to large sized circuits, and to improve PSO to
reduce the processing time.

[6] International
Roadmap
Committee(2009),“Executive
Summary,
International
Technology
Roadmapfor
Semiconductors,”
http://www.itrs.net/Links/2009ITRS/Home2009.htm.

Acknowledgements
This research was supported by Waseda University
Global COE Program ‘International Research and Education Center for Ambient SoC’ sponsored by the Ministry of Education, Culture, Sports, Science and Technology (MEXT), Japan, and partially by the Regional Innovation Cluster Program 2nd stage of MEXT, Japan and
the Core Research for Evolutional Science and Technology(CREST) of Japan Science and Technology Agency (JST).

[7] Jacob Robinson and Yahya Rahmat-Samii (2004),“Particle swarm optimization in electromagnetics”,
IEEE Transactions on Antennas and Propagation, Volume 52, Issue 2, Feb. 2004, pp. 397 - 407.
[8] Z. Bao and T. Watanabe (2009), “Evolutionary Designfor Image Filter using Genetic Algorithm,” Proc. IEEETENCON 2009, Singapore, Nov. 2009, pp. 1-6.
[9] Z. Bao and T. Watanabe (2010), “Mixed Constrained
Image Filter Design Using Particle Swarm Optimization,” Proc. The Fifteenth International Symposium on
Artificial Life and Robotics 2010 (AROB 15th 2010),
Beppu, Oita, Japan, pp. 230-235.

References[1] J. Dumoulin, J. Foster, J. Frenzel, et al. (2000), “Special Purpose Image Convolution with Evolvable Hardware,” Real-World Applications of Evolutionary Computing, vol. 1803 of LNCS, Springer, 2000, pp. 1-11.

[10] B. Rajan and S.Ravi (2006), “FPGA Based Hardware
Implementation of Image Filter With Dynamic Reconfiguration Architecture,” IJCSNS International Journalof Computer Science and Network Security, Vol. 6,
No. 12, pp. 121-127.

[2] L. Sekanina (2002), “Image Filter Design with Evolvable Hardware,” Applications of Evolutionary Computing, (the 4th Workshop on Evolutionary Computation in Image Analysis and Signal Processing,©ISAROB 2011

658

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A most simple, non-computer-aided proof of the Four-Color Theorem
Koji Ohnishi1,21Institute for Theoretical and Evolutionary Sciences (ITES),
c/o Koji Ohnishi, 8712-30, Ikarashi-2, Nishi-ku, Niigata, 950-2102, Japan(Tel : 81-25-262-5237; Fax : 81-25-262-5237)(ohnishi@sc.niigata-u.ac.jp)
2
Department of Biology, Faculty of Science, Niigata University,
Ikarashi-2, Nishi-Ku, Niigata, 950-2181, Japan

Abstract: An alternative brief proof of the Four Color Theorem without using a computer is described. The proof isessentially similar to the brief proof very recently described by the author as a bird’s-eye view, but is much more simple
than that, and more importantly, differs from that in the way of vertex-reducing of complete triangulation graph. This
new, most simple proof is much shorter than the recently described one, and both of these two proofs are far moreeasily understood than Appel and Haken’s proof in 1977. These new findings of non-computer-aided proofs clearly tell
us about what the essential portion of the enormous complexity of the Four Colour Theorem is.
Keywords: Triangulation, Vertex reduction of colored triangulation graphν-colorable; ν-colored graph written as colν (G):I. INTRODUCTION

see Definitions 8 in ref.[3].
colν (P)= a,(where P ∊ G), as denoting that vertex
P is colored with a in colν (G): see
Definitions 8 in ref.1.
Kempe-block, ab-Kempe-blocks written as Kab(Pi),
Kab(Pi, Pj) in col4 (G): see Definition 10 inref.[3].

The Four-Colour Conjecture (FCC) concerning
map-coloring first proposed by Guthrie [1], had long
been one of the most difficult unsolved problems inmathematics. In 1977, Appel and Haken (A&H) [2]
succeeded in proving the FCC using a computer. The
question whether or not the Four-Colour Theorem(FCT) could be proved without using a computer has
since arisen as a most important problem remaining tobe solved. Very recently, I have presented the details ofa bird’s-eye view of a very plausible brief proof of the
FCT without using a computer [3]. In this paper, analternative brief proof is described which is much more
simple, although it is essentially similar to the brief
proof described in Ohnishi (2009) [3], but considerably
differs from that in the way of vertex-reducing ofcomplete triangulation graph on sphere, S2.

Definition 1: edge, adjacent: Edge is defined as a
Jordan curve connecting and excluding two vertices(which are end-points) P and P’. An edge e, connecting
two vertices P and P’ is written as e = [P,P’]. A vertex P
is called to be adjacent to P’, if a graph Γ has an edge,[P,P’]. ▌Note that Definition 1 differs from the definition of“edge” given in the previous paper (see Definition 2 inref.[3] ), and is identical to the definition of Berge [4].
Accordingly, some terms need to be re-defined based on
Definition 1.
The next theorem is well-known as given in Ore [5].
Theorem 1: If C is a closed Jordan curve on a sphere S2,then we have S2 = int C + ext C + C = Int C + Ext C –C. ▌Definition 3: s-cycle, s-gon, s-path: An (s-)cycle isdefined by a s-vertex-graph, C = Cs = Cs(e12,e23,…,es,1)= P1 + e12 + P2 + e23 + … + es-1,s + Ps + es,1. An (s)path is defined by U(P1,Ps) = Us(P1,Ps)=
Cs(e12,e23,…,es,1) – es,1. Cs is also called s-gon (= shedron)(e.g.,
poly-gon,di-gon,tri-angle,
tetrahedron=quadrilateral, pentagon, etc.). ▌Definition 4: face: If G(S2) has a s-cycle (= s-gon), Cs,
where int Cs = ø, then int Cs is called face (or s-gonface). ▌Thus we find S2 = int Cs + Cs + ext Cs.

II. PRELIMINARIESIn this section, some basic definitions useful forachieving a most simple proof of the FCT are described.
“▌” denotes the end of each definition or theorem,
whereas “█” denotes the end of each proof.
The following terms are defined as given in theprevious paper [3].
Jordan curve QQ’and its endpoints Q and Q’: see
Definition 1 in ref. [3].
Internal and external domains ( int C, ext C, Int C,
Ext C, where C is a closed Jordan curve.):see Theorem 1 in ref.[3].
Graph, vertex, spherical graph G(S2), connected
graph: see Definitions 2 and 5 in ref.[3].
Valency (degree) of a vertex P, written as val P :see Definition 3 in ref.[3].

©ISAROB 2011

659

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Definition 5: (complete) triangulation: If G(S2) is aconnected graph dividing S2 into exclusively triangularfaces, G is called “complete triangulation (of S2)”. If“G(S2) = Cs” satisfies ext Cs = ø, and if G divides Int Cs
into exclusively triangular faces, G is called“triangulation of s-gon, Cs”. ▌The next theorem is well-known [6,7,11], as describedin [3].
Theorem 2: Let T(S2) be an arbitrarily selectedcomplete triangulation of S2. The four-color theorem(FCT) is equivalent to the statement that “Proposition A
is true”, where Proposition A is given by;
Proposition A: T(S2) is vertex four-colorable. ▌Definition 6: Two-faced quadrilateral ( Fig. 1): “Twofaced quadrilateral with a diagonal edge e13” is defined
as a subgraph of G(S2), and is given by Q2f = C40 + e13⊆ G, where C40 = C4(e12,e23, e34, e41), eij = [Pi, Pj], e13⊂ int C40, and e13 is a boundary edge dividing int C4
into two triangular faces. Q2f is written as Q2f =Q2f(C40;e13). If Q2f in G(S2) has any edge, e’13 or e’24,
satisfying e’13 = [P1,P3] ⊂ ext C40，or e’24 = [P2,P4]⊂ ext C40, then the Q2f is called “incomplete
quadrilateral”, whereas it is called “complete
quadrilateral” if there is none of such edges. G(S2)having its sub-graph Q2f0 =Q2f(C40;e13) is written as G =G(Q2f0;C40 ). ▌Theorem 4: Let Tk(S2) be a complete triangulation of
S2, having k vertices (k ≥ 4). Then there exists at least
one quadrilateral given by Q2fk,0 = Q2f(C4k,0;e13)⊂ Tk,
where C4k,0 = C4(e12,e23,e34,e41), e13⊂ int C4k,0, and eij=[Pi, Pj]. Furthermore, Q2fk,0 satisfies val P1 ≥3, val P3≥3, val P2 ≥2, and val P4 ≥2. ▌[Proof] Let e13 (= [P1,P3]) be an arbitrarily selectededge of Tk(S2), then we find two cases (i) and (ii), as
below;(i) In the case where one or more 2-cycles exist as
subgraph(s) of Tk (S2)（k≧4 as shown in Fig. 2[A]. Let
an arbitrarily selected 2-cycle be C2k(e13, e’31), where eij= [Pi,Pj], then one finds that the edge e13 is a boundaryof two 3-gon (triangular) faces, whose boundaries are 3cycles, C3(e12, e23, e31) and C3(e13, e34, e31) (where eij =[Pi,Pj].), as shown in Fig.1 [A]. Thus there exist aquadrilateral, G = G(Q2f0;C40 ), where C40 = C4(e12, e23,e34, e41), and e13 is a diagonal edge. Since C2k(e13, e’31) isa closed Jordan curve, there does not exist e34 = [P3,P4].
Then it follows that val P1≧4，val P3≧4，val P2≧2，
val P4≧2．
(ii) In the case where any 2-cycle does not exist as
subgraph of Tk(S2). By letting an arbitrarily selectededge be denoted by e13 = [P1,P3], P2 and P4 are found to
be different two vertices (See Fig 2[B]), since anidentical single vertex of P2 and P4 means the existenceof a 2-cycle, C2(e12, e21), which is in conflict with thecase (ii), and belongs to the case (i). In case (ii), we thus
find a quadrilateral, G = G(Q2f0;C40 ), where C40 =C4(e12, e23, e34, e41), and e13 is a diagonal edge, as shownin Fig. 2[B], satisfying val P1≧3，val P3≧3，val P2≧
3，val P4≧3．
Thus we find that there exist a quadrilateral, G = G(Q2f4440;C 0 ), where C 0 = C (e12, e23, e34, e41), and e13 is a diag
onal edge, and val P1≧3，val P3≧3，val P2≧2，
val P4≧2．█

Fig.1 Two-faced quadrilateral, Q2f(C40;e13), where C40
is a 4-cycle (= quadrilateral) given by C40 =C4(e12,e23,e34,e41), eij = [Pi,Pj], e13 ⊂ int C40. {Q2f} isan unavoidable (one-element-)set of subgraphs in Tk(S2),a complete triangulation of S2 withk vertices (k≥4). see
Definition 6.

III. BASIC THEOREMSThe following basic theorems are useful for proving the
FCT.
Theorem 3: For an ab-Kempe-block, Kab(Pi, Pj) ∊G(S2), where Pi and Pj ∊ G(S2), there exists a 2coloured path Uab(Pj,Pj) as a subgraph of the 2-coloured
graph, Kab(Pi, Pj). ▌[Proof] Evident from the definitions of connected
graph (Definition 5 in ref.[3].) and vertex 2-coloredgraph (Definition 8 in ref.[3].). █This theorem means that Pi and Pj are connected by a
2-coloured Jordan curve, Uab(Pj,Pj), which is a 2coloured path.

©ISAROB 2011

Fig. 2. Proof of Theorem 3.1 showing the existence of twofaced quadrilateral in Tk(S2)（k≧4）[A] The case (i) where a 2-cycle, C2k(e13, e’31), exists as a
subgraph of Tk (S2)（k≧4）.（val P1≧4，val P3≧4，val
P2≧2，val P4≧2 ）． [B] The case where any 2-cycle does
not exist as a subgraph of Tk (S2)（k≧4）,（ val P1≧3，val
P3≧3，val P2≧3，val P4≧3 ）.

Lemma 4.1: In Theorem 4, a set, {Q2fk } is an

660

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

unavoidable set (See [6,8] for definition.) of subgraphs in Tk (S2), and consists of only one element being
a quadrilateral. ▌[Proof] Evident from Theorem 4. █Theorem 5 : For an arbitrarily selected four-colorable
complete triangulation graph with k vertices, Tk(S2)(k≥4 ), let col40(Tk) be a 4-coloured graph of Tk.
Concerning an arbitrarily selected quadrilateral Q2fk,0=Q2f (C4k,0 ; e(k)13) ( ⊂ Tk ), where C4k,0 =C4(P1(k),P2(k),P3(k),P4(k)) and e(k)13 = [P1(k),P3(k) ] ⊂ int
C4k,0, col40 (Tk) belongs to either one of the two types,
type I and type II, defined by:type I: Kac(P1(k),P3(k)) (⊂col4(Tk; Q2f0 ) exists.
type II: Kac(P1(k),P3(k)) (⊂col4(Tk; Q2f0 ) does notexist. ▌[Proof] Evident. █Definition 6 : (case I and case II 4-colorations) Let
col4Ι(Tk;Q2f0) and col4ΙΙ(Tk;Q2f0) denote 4-coloredcomplete triangulation graph of case I and that of case II(in Theorem 5), respectively. ▌Figure 3. Reversible vertex-reduction and its reverse
operation of a 4-coloured complete triangulation graph,
col40(Tk ; Q2fk,0) in case I, where there exists an acKempe block, Kac(P1, P3), in Ext C4k,0. The left and right
schematized col40,I(Tk; Q2f0 ) and col40,Ι(Tk-1; Q2fk,0),respectively. (From Fig. 4 in [3] )III. FINAL PROOF
Definition 7 (Vertex-reducing operations f1 and f2 ):Definitions, f1 and f2, (and there reverse operations f1-1
and f2-1 ), are given by Definition 12 in ref.[3].
Suboperations f1a, f1b and f2a f2b satisfying f1 = f1b(f1a),
and f2 = f2b(f2a) are defined as schematized in Fig.3 andFig.4 ▌
Theorem 6. Let a complete triangulation Tk(S2) be
four-colorable (k≥4 ), then Tk-1(S2) defined by Tk-1 = Tk2f
2f
2f
1( Tk; Q k,0) = f (Tk; Q 0 ) is four-colorable, where Q k,0
(k)
(k)
(k)
2f
4
(k)
4
4
=Q (C k,0 ; e 13) ⊂ Tk, C k,0 = C (P1 ,P2 ,P3 ,P4(k)),
and e(k)13 = [P1(k),P3(k) ] ⊂ int C4k,0, and further, f
denotes a operation given by f = f1 or f2 for type I ortype II 4-coloration of Tk(Q2f0, e(k)13), respectively. Then
Tk-1( Tk; Q2fk,0) is 4-colourable. ▌[Proof] Let col4(Tk; Q2f0 ) be a 4-coloured graph ofTk(S2). Vertex-recucing operations of col4(Tk; Q2f0 ) are
as below;
Case I: If col4(Tk; Q2f0 ) is type I in Theorem 5, thenoperation of graph modification f1(Tk; Q2f0 ) = f1b( f1a(Tk;Q2f0 ) ) gives a reduced triangulation graph Tk-1( Tk;
Q2fk,0) and its 4-coloured graph, col4Ι(Tk-1; Q2fk,0), as
shown in Fig.3. Note that Tk-1( Tk; Q2fk,0) and col4Ι(Tk-1;
Q2fk,0) are defined by Tk(S2) and col4(Tk; Q2f0 ),respectively. Thus Tk-1 = f1(Tk; Q2f0 ) is 4-colourable.
Case II: If col4(Tk; Q2f0 ) is type II in Theorem 5,then operation of graph modification f2(Tk; Q2f0 ) =f2b( f2a(Tk; Q2f0 ) ) gives a reduced triangulation graph Tk2f
4
2f
1( Tk; Q k,0) and its 4-coloured graph, col ΙΙ(Tk-1; Q k,0),
2f
4
as shown in Fig.4. Note that Tk-1( Tk; Q k,0) and col Ι(Tk2f242f1; Q k,0) are defined by Tk(S ) and col (Tk; Q 0 ),2f
respectively. Thus Tk-1 = f1(Tk; Q 0 ) is 4-colourable.
Acordingly, Tk-1( Tk; Q2fk,0) is 4-colourable. █©ISAROB 2011

Figure 4. Reversible vertexreduction and its reverse ope
ration of a 4coloured complete triangulation graph, col40(Tk ; Q2fk,0) in case II, where there does not exist any
acKempe block, Kac(P1,P3), in Ext C4k,0. The left andright schematized col40,II(Tk; Q2f0 ) and col40,ΙΙ(Tk-1;
Q2fk,0), respectively. (From Fig.5 in [3])Theorem 7. Let Tk(S2) be 4-colourable, then there
exists a series of 4-coloured complete triangulationgraphs,{ col40(Tk ; Q2fk,0), col40(Tk-1; Q2fk,0), … , col40(T4 ; Q2f5,0), col40(T3 ; Q2f4,0) }, where col40(Tk-i ;Q2fk-i+1, 0) ( i=1,2, … , k-3) is defined by Tk-i =f(Tk-i+1; Q2fk-i,0 ), where f = fI (for type I coloration of
Tk-i+1 ) or fII (for type II coloration of Tk-i+1 ), andQ2fk-i+1,0 is an arbitrarily selected quadrilateral of
Tk-i+1. ▌[Proof] From theorm 6, col40(Tk-i ; Q2fk-i+1,0) =f (col40(Tk-i+1 ; Q2fk-i+1,0) ). Repeated applications of
this equation to Tk(S2), for i = 1,2, …, k-3, we obtain aseries of 4-coloured complete triangulations, { col40(Tk ; Q2fk,0), col40(Tk-1 ; Q2fk,0), … , col40(T4 ; Q2f5,0),col40(T3 ; Q2f4,0) }. Note that col40(T3 ; Q2f4,0) is a 4coloured 3-gon graph (3-vertex complete graph)
whose coloration depends on col40(Tk ; Q2fk,0). █Lemma 7.1: The necessary condition for acomplete The necessary condition for a complete triang
ulation graph, Tk(S2), (k≥4 ), to be four colorable is the existence of col40(T3 ; Q2f4,0), defined in Theorem 7. ▌[Proof] Evident from theorem 7. █661

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011graph, Tk(S2), (k≥4 ), to be four colorable is theexistence of col40(T3 ; Q2f4,0). █Theorem 9. The necessary and sufficient condition for
a complete triangulation graph, Tk(S2), (k≥4 ), to be fourcolorable is the existence of col40(T3 ; Q2f4,0), defined in
Theorem 7. ▌[Proof] Evident from Lemma 7.1 and Lemma 8.2. █Lemma 9.1. The necessary and sufficient condition(defined in Theorem 9) for a complete triangulationgraph, Tk(S2), (k≥4 ), to be four colorable, is satisfied. ▌[Proof] In theorem 9, col40(T3 ; Q2f4,0) really exist as
one of the 4x3x2 = 24 colorations with 4 colours. Thus
we have reached Lemma 9.1. █Theorem 10. (The Four-Colour Theorem): Everycomplete triangulation graph, Tk(S2) (k≥3) is fourcolourable. ▌[Proof] Evident from Lemma 9.1. █Lemma 7.2: The necessary condition for a complete
triangulation graph, Tk(S2), (k≥4 ), to be four colorable,
defined in Lemma 7.1 is satisfied. ▌[Proof] col40(T3 ; Q2f4,0) is a 4-coloured 3-gon graph(3-vertex complete graph) whose coloration depends on
col40(Tk ; Q2fk,0). Therfore, col40(T3 ; Q2f4,0) really exists
as one of the 4x3x2 = 24 4-colorations of the 3-gongraph. █Theorem 8. For a given coloration, col40(T3 ; Q2f4,0),
which is one of the 24 possible coloration of trigon,
col40(T4 ; Q2f5,0), given inTheorem 7 can bereconstructed from col40(T3 ; Q2f4,0). ▌[Proof] Since col40(T3 ; Q2f4,0) really exists from
Lemma 7.1, then col40(T4 ; Q2f5,0) can be reconstructedby col40(T4 ; Q2f5,0) = f-1 (col40(T3 ; Q2f4,0)), because theoperation f defined in Fig. 3 is reversible graphmodification. Thus, by applying col40(T4 ; Q2f5,0) = f-1(col40(T3 ; Q2f4,0)) for all of the possible operations of f-1,
there exists at least one operation generating col40(T4 ;Q2f5,0), whether or not T4, Q2f4,0, and col40(T3 ; Q2f4,0) are
known. █Lemma 8.1. For a given coloration, col40(Tk-i ; Q2fki+1,0) ( i=k-3, k-4, …, 2, 1), defined by Theorem 8,
col40(Tk-i+1 ; Q2fk-i+2,0) given inTheorem 7 can bereconstructed from col40(Tk-i ; Q2fk-i+1,0). ▌[Proof] Since col40(Tk-i ; Q2fk-i+1,0) really exists, from
Lemma 7 .2, for i = k-4, col40(T5 ; Q2f6,0) can bereconstructed by col40(T5 ; Q2f6,0) = f-1 (col40(T4 ; Q2f5,0)),because the operation f defined in Fig. 3 is reversible
graph modification. Thus, by applying col40(T5 ; Q2f6,0)= f-1 (col40(T4 ; Q2f5,0)) for all of the possible operationsof f-1, there exists at least one operation generating
col40(T5 ; Q2f6,0), even if T5, Q2f6,0, and col40(T5 ; Q2f6,0)
are unknown.
Similarly, for k-4 ≤ i ≤ 2, when col40(Tk-i ; Q2fk-i+1,0)
can be known to exist by col40(Tk-i ; Q2fk-i+1,0) = f-1(col40(Tk-i-1 ; Q2fk-i,0)), col40(Tk-i+1 ; Q2fk-i+2,0) can bereconstructed by col40(Tk-i+1 ; Q2fk-i+2,0) = f-1 (col40(Tk-i ;Q2fk-i+1,0)), because the operation f defined in Fig. 3 isreversible graph modification. Thus, by applying
col40(Tk-i+1 ; Q2fk-i+2,0) = f-1 (col40(Tk-i ; Q2fk-i+1,0)) for allof the possible operations of f-1, there exists at least one
operation generating col40(Tk-i+1 ; Q2fk-i+2,0), even if T5,
Q2f6,0, and col40(T5 ; Q2f6,0) are unknown.
The final similar application of col40(Tk-i ; Q2fk-i+1,0) =-1f (col40(Tk-i-1 ; Q2fk-i,0)), for i = 0, it follows thatcol40(Tk ; Q2fk,0) = f-1 (col40(Tk-1 ; Q2fk,0)), as shown in
Fig.4 and Fig.5. Thus the initial coloration of col40(Tk ;Q2fk,0) has now been reconstructed. █Lemma 8.2. The sufficient condition for a complete
triangulation graph, Tk(S2), (k≥4 ), to be four colorableis the existence of col40(T3 ; Q2f4,0), defined in Theorem
7. ▌[Proof] If there exists col40(T3 ; Q2f4,0), defined in
Theorem 7, by applying col40(Tk-i ; Q2fk-i+1,0) = f-1(col40(Tk-i-1 ; Q2fk-i,0)), for i = k-4, k-5, …, 1, col40(Tk-i ;Q2fk-i+1,0) can be reconstructed even if Tk-i, Q2fk-i+1,0, andcol40(Tk-i-1 ; Q2fk-i,0) are unknown. This means that The
sufficient condition for for a complete triangulation©ISAROB 2011

III. REFERENCES[1] Guthrie F (1880), Notes on the colouring maps.
Proc.R. Soc. Edinburgh 10: 727-728.
[2] Appel K et al. (1977), Every planar map is fourcolorable. Parts I, II. Ill. J. Math. 21: 429-489, 490-597.
[3] Ohnishi K (2009), Towards a brief proof of theFour-Color Theorem without using a computer. Artif.
Life Robotics 14(4):551-556.
[4] Berge C (1970), Graphe et hypergraphes. Dunod,
Paris.
[5] Ore O (1967), The Four-Color Problem. Academic
Press, New York.
[6] Hitotsumatsu S (1978), Yon-shoku Mondai (TheFour Colour Theorem). Kodansha, Tokyo (in Japanese).
[7] Saaty TL and Kainen PC (1977), The Four Color
Problem. assaults and conquest. McGraw Hill
International Company, New York.
[8] Wilson R (2002), Four colours suffice: how the map
problem was solved. Penguin Books, London.
Acknowledgements: Valuable discussions with Dr.
Shigeki Akiyama (Department of Mathematics, Niigata
University) are acknowledged. This study was partly
funded by the financial support of the Uchida Energy
Science Promotion Foundation, No. 131520.

662

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Autonomous Decentralized Control Scheme for Large Scale and Dense
Wireless Sensor Networks with Multiple Sinks
Masahito Maki†, Akihide Utani‡, and Hisao Yamamoto‡†Graduate School of Engineering, Tokyo City University‡Faculty of Knowledge Engineering, Tokyo City University
1-28-1, Tamazutsumi, Setagaya, Tokyo 158-8557, Japan{autani; yamahisa}@tcu.ac.jp
Abstract: A wireless sensor network has great potential as a key network to facilitate ubiquitous environments. A wireless sensor network is generally made up of many sensor nodes limited resource. Therefore, a data gathering scheme saving and balancing energy
consumption of each sensor node is needed to prolong the lifetime of a wireless sensor network. This paper proposes an autonomous decentralized control scheme for a large scale and dense wireless sensor network, which is a new data gathering scheme with transmission power control that adaptively reduces the load of load-concentrated nodes and facilitates the long-term operation of a large scale and dense wireless sensor network. The proposed scheme is evaluated by computer simulations.
Keywords: Data gathering, Autonomous load-balancing, Transmission power control, Wireless sensor networks.
eless sensor network difficult. To solve this communication load concentration problem, a data gathering scheme for a wireless sensor network with multiple sinks has been proposed [3]. Each sensor node, in this scheme,
sends sensing data to the nearest sink node. In comparison with the case of a one sink wireless sensor network,
the communication load of sensor nodes around a sinknode is reduced. In each sensor node, however, the destination (sink) node cannot be selected autonomously and adaptively. In case original data transmission rate from each sensor node is not even, therefore, the load ofload-concentrated nodes is not sufficiently balanced. An
autonomous load-balancing data transmission scheme isrequired.
This paper proposes an autonomous decentralized control scheme for a large scale and dense wireless sensor network, which is a new data gathering scheme withtransmission power control that adaptively reduces theload of load-concentrated nodes and facilitates the longterm operation of a large scale and dense wireless sensor network with multiple sinks. The proposed scheme is
an autonomous load-balancing data transmission one devised by considering the application environment of awireless sensor network as a typical example of complex systems where the adaptive adjustment of the entire
system is realized from the local interactions of components of the system.

I. INTRODUCTION
A wireless sensor network has great potential as a key network to facilitate ubiquitous environments [1]. In
this study, a large scale and dense wireless sensor network made up of static sensor nodes with GPS is assumed.
In a large scale and dense wireless sensor network, generally, hundreds or thousands of static sensor nodes limited resource are placed in an observation area and sensing data of each node is gathered to a sink node by internode wireless communication. Therefore, a data gathering scheme capable of meeting the following two requirements is mainly needed to prolong the network lifetime.
1) Efficiency of data gathering2) Balance on communication load among sensor nodes
In a large scale and dense wireless sensor network, the communication load is generally concentrated on sensor nodes around a sink node during the operation process. In case sensor nodes are not placed evenly in a large scale observation area, the communication load is concentrated on sensor nodes placed in an area of low node density. As the scheme that satisfy the above two requirements, gradient-based routing protocol has attracted
attention [2]. However, this does not ease the communication load concentration to sensor nodes around a sinknode that is the source of problems on the long-term operation of a wireless sensor network. Intensive data transmission to specific nodes, such as sensor nodes arounda sink node and sensor nodes placed in an area of low node density, brings on concentrated energy consumptionof them and causes them to break away from the network early. This makes the long-term observation by a wir-

©ISAROB 2011

II. PROPOSED SCHEME
For facilitating the long-term operation of an actual
sensor network service, recently, it has been considered
to introduce multiple sinks in a wireless sensor network

663

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

ighbor nodes of node (i).
Before data transmission is started, each sensor node
computes initial connective value of each neighbor node
based on Eq.(1) and Eq.(2) and stores the computed connective value, which is used as the only index to evaluate the relay destination value of each neighbor node, ineach neighbor node field of its own routing table.
2. Data Transmission
For a while from starting data transmission, each sensor node selects the neighbor node with the greatest connective value from its own routing table as a relay node and transmits the data packet to this selected node byhigh power. In case more than one node shares the greatest connective value, however, the relay node is determined from them at random. In each sensor node, the data packet is not sent to a specified sink node. By repetitive data transmission to the neighbor node with the greatest connective value, data gathering at any of the available sinks is completed. In the proposed scheme, the connective value of each sensor node is updated by considering residual node energy. By repetitive data transmission to the neighbor node with the greatest connective
value, therefore, data transmission routes are not fixed.
To realize autonomous load-balancing data transmission, in the proposed scheme, the data packet from eachsensor node includes its own updated connective value.
Let’s assume that a node (l) receives a data packet at time (t). Before node (l) relays the data packet, it replaces
the value in the connective value field of the data packet
by its own renewal connective value computed according to the following connective value update equation[3]. In a wireless sensor network with multiple sinks, sensing data of each node is generally allowed to gather
at any of the available sinks. The proposed scheme is a
new data gathering one based on this assumption, which
can be expected to produce a remarkable effect in a multiple sink wireless sensor network. In this study, it is assumed that each sensor node can select either of high power and low power for packet transmission. In the proposed scheme, high power corresponds to normal transmission power and low power is newly introduced formoreover balancing the load of each sensor node.
1. Routing Algorithm
Each sink node has a connective value named a “value to self”, which is not updated by transmitting a control packet and receiving data packets. In the initial stateof a multiple sink wireless sensor network, each sink node broadcasts a control packet containing its own location information, ID, hop counts(=0), and “value to self”
by high power. This control packet is rebroadcast throughout the network with hop counts updated by high power. By receiving the control packet from each sink node, each sensor node can grasp the “value to self” of each sink node, the location information and IDs on its own neighbor nodes, and the hop counts from each sinknode of neighbor nodes.
Initial connective value of each sensor node, whichis the connective one before starting data transmission,is generated by using the “value to self” of each sink node and the hop counts from each sink node. The procedure for computing initial connective value of a node (i)is as follows:
1. The value [vij(0)] on each sink node (j = 1, … , S) of
node (i) is first computed according to the following
equationhops
vij (0 ) = vo j × dr ij ( j = 1,… ,S )(1)
where voj(j = 1, … , S) is the “value to self” of sink node (j), hopsij(j = 1, … , S) is the hop counts from sink node (j) of node (i), and dr represents the value attenuation factor accompanying hop determined within the interval [0,1].
2. Then, initial connective value [vi(0)] of node (i) is generated by the following equation(2)vi (0 ) = max vij (0 )( j = 1, ... , S )where this connective value [vi(0)] can be also conducted from the following equationvi (0 ) = vmi (0 ) × dr(3)
In the above Eq.(3), vmi(0) represents the greatest connective value before starting data transmission in ne-©ISAROB 2011e (t )vl (t ) = vml (t ) × dr × l(4)
where vml(t) is the greatest connective value at time (t)in the routing table of node (l), and el(t) and El represent
the residual energy at time (t) of node (l) and the battery
capacity of node (l), respectively.
In the proposed scheme, the data packet addressed to
the neighbor node with the greatest connective value isintercepted by all neighbor nodes. This data packet includes the updated connective value of the source node based on the above Eq.(4). Each neighbor node that has intercepted this packet stores the updated connective value in the source node field of its own routing table. The
proposed scheme requires the construction of a data gathering environment in the initial state of a wireless sensor network but needs no special communication for network control. The above simple system alone achievesautonomously adaptive load-balancing data transmissi-

664

El

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

on using multiple routes and sinks. The lifetime of a wireless sensor network can be extended by reducing thecommunication load for network control and solving thenode load concentration problem.
3. Transmission Power Control
For data packet transmission, the transmission power of each sensor node is switched to low power if its own residual energy is less than the set threshold [Te]. In
this case, each sensor node selects the neighbor node with the greatest connective value within range of radio
wave of low power as a relay node and transmits the data packet to this selected node by low power.
Fig.1 shows an example on the above transmission
power control, which means that the transmission powerof each sensor node is switched to low power according
to the above condition. In this example, node (m) is a load concentration node. Node (m) has autonomously transmitted the data packet to node (r) with the greatest connective value within low power range by low power because its own residual energy has become less than theset threshold [Te]. By switching to low power, the energy consumption of node (m) is saved, but node (k) andnode (l) may continue to transmit the data packet to node (m) because they cannot grasp the updated connective value of node (m). In the proposed scheme, therefore,
every tenth data packet from the node switched to low
power is transmitted by high power.

1. Condition of Simulations
In a wireless sensor network with three sinks consisting of many static sensor nodes placed in a large-scaleobservation area, the only sensor nodes that detected abnormal data set were assumed to transmit the measurement data. The condition of simulations, which were used in the experiments performed, is shown in Table 1. In
the initial state of simulation experiments, static sensor
nodes are randomly arranged in the set simulation area
and three sinks are placed on the three boundaries containing the two corners of this area. In Fig.2, the network
configuration is illustrated.
Table 1. Condition of simulations
Simulation size

2,400m× 2,400m

The number of sensor nodes 750, 1,000, 1,250
Range of radio wave
200m(high power),150m(low power)
The number of sinks
3

Sink3

Sink1
evaluation node

Sink1

Sink2
l

n
r

m

s

Fig.2 Wireless sensor network with three sinks: data packetq

2. Experimental Results
As the first experiment, it was assumed that the evaluation node marked in Fig.2 detected an abnormal value and transmitted the data packet with the detected abnormal value periodically. The routes used by applying
the proposed scheme are shown in Fig.3 and Fig.4, where the number of sensor nodes is 1,000. Of the 2,000 data packets transmitted from the evaluation node, the routes used by the first 500 data packets are shown in Fig.3
and Fig.4(1), those by the first 1,000 data packets are shown in Fig.3 and Fig.4(2), and those by a total of 2,000 data packets are shown in Fig.3 and Fig.4(3). From
Fig.3 and Fig.4, it can be confirmed that the proposed scheme enables autonomous load-balancing transmissionof data packets to three sinks using multiple routes andits effect is extended by early switching to low power.

k

Node m routing table
Next Hop
Node kNode lNode nNode qNode r

Node s

10.0

20.0

50.0

12.0

25.0

12.0

Fig.1 An example on transmission power control

III. SIMULATION EXPERIMENT
Through simulation experiments on a large scale wireless sensor network with three sinks, the performanceof the proposed scheme is investigated.

©ISAROB 2011

665

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

evaluation nodeevaluation node(1) 1 to 500 data packets(1) 1 to 500 data packets

evaluation nodeevaluation node(2) 1 to 1,000 data packets(2) 1 to 1,000 data packets

evaluation nodeevaluation node(3) 1 to 2,000 data packets
Fig.3 Routes used by applying the proposed scheme (Te= E×0.5J)(3) 1 to 2,000 data packets
Fig.4 Routes used by applying the proposed scheme (Te= E×0.9J)

IV. CONCLUSIONS

ong-term operation. Future works include evaluation onparameters introduced in the proposed scheme in detailand verification of its effectiveness on various network
environments.

In this paper, a new data gathering scheme with transmission power control that adaptively reduces the loadof load-concentrated nodes and facilitates the long-term
operation of a large-scale wireless sensor network withmultiple sinks, which is an autonomous load-balancing
data transmission one devised by considering the application environment of a wireless sensor network to be atypical example of complex systems, has been proposed.
Experimental results indicate that the proposed scheme
is superior to the existing one and has the development
potential as a promising one from the viewpoint of the l-©ISAROB 2011

REFERENCES[1]I. Akyildiz, W. Su, Y. Sankarasubramaniam and E. Cayirci, “Wireless sensor networks: A survey,” Computer
Networks Journal, vol.38, no.4, pp.393-422, 2002.
[2]L. Xia, X. Chen and X. Guan, “A new gradient-based
routing protocol in wireless sensor networks,” Lecture
Notes in Computer Science, vol.3605, 2004.
[3]E. I. Oyman and C. Ersoy, “Multiple sink network design problem in large scale wireless sensor networks,”
Proc. of the International Conference on Communications, 2004.

666

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Advanced Adaptive Communication Protocol for Ubiquitous Sensor Networks
Shuichi Osawa†, Akihide Utani‡, and Hisao Yamamoto‡†Graduate School of Engineering, Tokyo City University‡Faculty of Knowledge Engineering, Tokyo City University
1-28-1, Tamazutsumi, Setagaya, Tokyo 158-8557, Japan{autani; yamahisa}@tcu.ac.jp
Abstract: There is growing expectation for a wireless sensor network as a means of realizing ubiquitous information environments.
A wireless sensor network, which is a key network to facilitate ubiquitous information environments, has great potential as a meansof realizing a wide range of applications, such as natural environment monitoring, environmental control in office buildings and factories, object tracking, and precision agriculture. Control mechanism for a wireless sensor network should adapt to the variety of types of communication (one-to-one, one-to-many, many-to-one, many-to-many), depending on application requirements and the context. This paper proposes a new adaptive communication protocol for the long-term operation of a wireless sensor network. We evaluate the proposed protocol using computer simulations and discuss its development potential. In the experiments performed, the performance of the proposed protocol is investigated in detail to verify its effectiveness.
Keywords: Wireless Sensor Networks, Ubiquitous Environments, Adaptive Communication Protocols.
This paper proposes a new adaptive communication
protocol for the long-term operation of a wireless sensor
network. We evaluate the proposed protocol using computer simulations and discuss its development potential.
In the experiments performed, the performance of the proposed protocol is investigated in detail to verify its effectiveness. This paper is organized as follows. In Section II, the proposed protocol is outlined. In Section III, the experimental results are reported and the effectiveness of the proposed protocol is demonstrated by comparing the performance of it with those of the existing ones.
Finally, the paper closes with conclusions and ideas forfurther study in Section IV.

I. INTRODUCTION
Various network services have been provided. They
include inter-vehicle communication, which is a network service in intelligent transport systems, natural environmental monitoring, and emergent communication between mobile nodes in such the case of emergency as disaster. As a means of realizing the above network services, autonomous decentralized networks, such as a mobile ad-hoc network and a wireless sensor network, havebeen intensively studied with great interests. Especially,a wireless sensor network, which is a key network to facilitate ubiquitous information environments, has great
potential as a means of realizing a wide range of applications, such as natural environmental monitoring, environmental control in office buildings and factories, object tracking, and precision agriculture [1]. In a general
wireless sensor network, hundreds or thousands of micro sensor nodes, which are generally compact and inexpensive, are placed in a large scale observation area andsensing data of each node is gathered to a sink node byinter-node wireless multi-hop communication. Each sensor node consists of a sensing function to measure thestatus (temperature, humidity, motion, etc.) of an observation point or object, a limited function on information
processing, and a simplified wireless communication function, and generally operates on a resource of a limited
power-supply capacity such as a battery. Therefore, in the existing studies, the efficient and load-balancing datatransmission scheme to sink nodes has been proposed.

©ISAROB 2011

II. PROPOSED PROTOCOL OUTLINE
For facilitating ubiquitous information environments,
control mechanism for a wireless sensor network should
adapt to the variety of types of communication, i.e., one-to-one, one-to-many, many-to-one, and many-to-many,
depending on application requirements and the context.
Recently, many adaptive protocols for a large scale anddense wireless sensor network, which are “directed diffusion”[2,3] and “ant-based rendezvous communication
protocol”[4] have been proposed. “directed diffusion”,
that a representative scheme for adaptive communication consists of the following protocols: Two-Phase Pull
diffusion, One-Phase Pull diffusion, Push diffusion, where “Pull” adapts itself to the case that the number of source nodes is more than the number of sink nodes. “Push” adapts itself to the case that the number of sink nodes

667

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

©ISAROB 2011：sensor node：interest：sink：head sink
：sourceStep1：sensor node
：exploratory：sink：head source：sourceStep1：sensor node
：gradient：sink：head sink
：sourceStep2：sensor node
：gradient：sink：head source：sourceStep2：sensor node：reinforcement：sink：head sink
：sourceStep3：sensor node：reinforcement：sink：head source：sourceStep3：sensor node
：data
：sink
：head sink
：sourceStep4：sensor node
：data
：sink
：head source：sourceStep4：sensor node
：data
：sink
：head sink
：sourceStep5：sensor node
：data
：sink
：head source：source
Step5

Fig.1. From sink nodeFig.2. From source node

668

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

is more than the number of source nodes. However, theexisting adaptive communication protocols cause flooding storm problems. The proposed protocol improves flooding storm problems. In Fig.1 and Fig.2, the example
on the proposed protocol is shown.

IV. SIMULATION EXPERIMENT
Through simulation experiments, the performance ofthe proposed protocol is investigated in detail to verify
its effectiveness. A wireless sensor network consistingof many sensor nodes placed in a wide range is assumed.
The conditions of simulation, which were used in the experiments performed, are shown in Table1. In the simulation area, sensor nodes are randomly placed. In Fig.3,
the network configuration is illustrated.

Fig.4. Total energy consumption (from sink node)

Table1. Conditions of simulation
Simulation times

1,000sSimulation size

2,400m × 2,400m

The number of sensor nodes

1,000

Range of radio wave

150m

Data transmission interval

10s

Model change interval

250s

Fig.5. Total energy consumption (from source node)Fig.6. Transition of delivery ratio (from sink node)Fig.3. Network configuration
In the experiments performed, the proposed protocol
is evaluated through the comparison with “Pull” or “Push” of “directed diffusion”. Figs.4 and 5 show total energy consumption of a network. Figs.6 and 7 show the transition of delivery ratio, where the battery capacity ofeach sensor node has been set to 0.5J. Experimental results verify that the proposed protocol is substantially advantageous for the long-term operation of a large scaleand dense wireless sensor network.

©ISAROB 2011

Fig.7. Transition of delivery ratio (from source node)

669

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

IV. CONCLUSIONSREFERENCES

In this paper, a new adaptive communication protocol, which adapts itself to the variety of types of communication, has been proposed. This protocol can improve
flooding storm problems. In the experiments performed,
the performance of the proposed protocol was comparedwith those of the existing ones. Experimental results indicate that the proposed protocol is superior to the existing ones and has the development potential as a promising one from the viewpoint of the long-term operationof a wireless sensor network. In future studies, we evaluate the performance of the proposed protocol in detail.

[1]I. Akyildiz, W. Su, Y. Sankarasubramaniam, and E.
Cayirci, “Wireless sensor networks: A survey,” Computer Networks Journal, vol.38, no.4, pp.393-422, 2002.
[2]C. Intanagonwiwat, R. Govindan, D. Estrin, J. Heidemann, and F. Silva, “Directed diffusion for wireless sensor networking,” ACM/IEEE Trans. on Networking, vol.11, pp.2-16, 2003.
[3]F. Silva, J. Heidemann, R. Govindan, and D. Estrin,“Directed diffusion”, Tech. Rep., ISI-TR-586, 2004.
[4]M. Wakabayashi, H. Tada, N. Wakamiya, M. Murata,
and M. Imase, “Proposal and evaluation of a bio-inspired adaptive communication protocol for sensor networks”, IEICE Tech. Rep., vol.107, no.294, pp.89-94, 2007.

©ISAROB 2011670

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Lateral Control of Unmanned Vehicle Using PD Controller
Young Chul Cha*, Jong Il Bae**, Kil Soo Lee*, Dong Seok Lee*, Yun Ja Lee***, and Man Hyung Lee*
*School of Mechanical Engineering,Pusan National University,Busan, 609-735, Korea(Tel : 82-51-510-1456; Fax : 82-51-512-9835)(mecha-cha@pusan.ac.kr, melgibsoo@pusan.ac.kr, dslee@pusan.ac.kr, mahlee@pusan.ac.kr)**Department of ElectricalEngineering, PukyongNational University, Busan,608-739,Korea(Tel : 82-51-629-6314:)(jibae@pknu.ac.kr)***Department of Dance,Pusan National University,Busan, 609-735, Korea(Tel : 82-51-510-2949)(leeyj@pusan.ac.kr)
Abstract: In order to control an unmanned vehicle, steering, acceleration, braking etc.., an actuator control is required. This paper
proposes a lateral control system for an unmanned vehicle to improve the responsiveness of the system with a PD control. If a PD
controller is used in the system, angle errors from autonomous navigation can be stabilized and the system will improve the transient
response characteristics. Generally, when calculating a mathematical model of a vehicle, 4m/s2 the lateral acceleration in less as twodegrees of freedom bicycle model also shows better performance. In this paper, a mathematical modeling of a vehicle using two
degrees of freedom model was calculated with a controller designed by Matlab, and autonomous navigation simulations were carriedout. Path estimation method of autonomous navigation was done with the Point to Point algorithm, current position of vehicle gainedwith GPS. Performance of the designed controller was verified through autonomous navigation with a real vehicle.
Key words : Lateral control, PD Controller, Unmanned Vehicle, P to P algorithm

Ⅰ. INTRODUCTION

Ⅱ. 2WS MODEL

Today, society has become more convenient with the rapid
development of transportation; however, as a result, traffic
congestion and accidents are increasing the seriousness of thetraffic problems. The cause of most accidents occurs from thecarelessness of an individual. So safety-devices for preventing
accidents are being developed [1, 2], and research anddevelopment of unmanned vehicles are being conducted inadvanced countries as well as our country [3, 4].
Here, lateral control of unmanned vehicles and lateralcontrol method using a PD-controller from existing P to P
driving were studied. The original P to P driving is driving fromthe current location to the next location. The driving method
calculates the angle between the current position and the next toobtain the steering value. However, in the original method,stable driving is not guaranteed due to the rapid changes of thesteering value at the waypoints which are renewed along thepath.
A PD controlled driving can stabilize the occurring error
angles of the rapidly changing steering value which makes
reliable driving possible. In the main part of this paper, 2WS
modeling and PD-controller design is described and the original
P to P driving is introduced. And using the PD-controller, the P
to P driving and the original driving results are analyzed andcompared.

A 2WS(2 wheel steering) car has two front steering wheels
while the rear wheels of the vehicle are fixed. Analysis of thegeneral steering characteristics of the car when driving, that is, if
the lateral acceleration is less than 0.4G, a linear model of two
degrees of freedom can be used to obtain accurate results.
Degrees of freedom linear model uses lateral displacement andyaw. The purpose of this study is a 2WS vehicle steering control
so the roll, pitch motions were ignored, and experiments weredone assuming that the differences of the angle of yaw direction
were small.

©ISAROB 2011

Fig. 1 2WS Bicycle Model.
Slip angle  f ,  r can be calculated as shown below by
speed of vehicle  , each distance from the center of gravity to
the front-wheel and rear-wheel l f , lr , the center of gravity ofthe vehicle yaw rate  and lateral velocity  , rear-wheel
steering angle  r , front-wheel steering angle  f , Fig. 1 foreach of the front tire and rear tire slip angle  f ,  r .

671

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

   lf   lrr  r f f 

In this study, the necessary parameter is steering
angle  of the unmanned vehicle. Thus, Equation (11)is added, and then Equation (12) is expressed as theEquation of state [5].

(1)
(2)

 

Since a linear tire model is used, the cornering force   a11
    a   21   0f f and f r acts on the front tire and rear tire so theirrelevancy are shown as Equation (1) and Equation (2).
𝑓𝑓 = 𝑐𝑓 ∙ 𝛼𝑓
𝑓𝑟 = 𝑐𝑟 ∙ 𝛼𝑟

(3)
(4)

using Equation (3) and Equation (4), as shown in Fig. 1,
the equilibrium conditions of the vehicle's lateral andyaw moment are used to derive the Equations of motion.
(5)(6)

m is the mass of the vehicle, J is the vehicle's yaw
moment of inertia. When Equations (1) and (2) aresubstituted in Equations (5) and (6), then linear
Equations of the model are shown in the determinant (7)and then Equation (8) can be expressed in the form of adeterminant as.

   a11    a
   21

 f 
u (t )    r b b
B   11 12 b21 b22 

a12     b11 b12  
a22     b21 b22   f 

J (kgm2 )

193.25c f (kgf / rad )387.3~3690.6

cr (kgf / rad )848.8~9409.1

l f (m)

1.158

lr (m)

1.737

L(m)

2.895

Ⅲ. PD-CONTROLLER DESIGNPD-Controller reduces error signals effectively since
the feedback of the control signal is proportional to thechange of the differential value of the error signal, thedamping ratio is increased and the overshoot issuppressed. Considering the effect of these differential
controls in the controller design, the system can improve
the transient response characteristics. Transfer function
K ( s) of controller is as shown in Equation (13).

x(t )  Ax(t )  Bu (t ) x(t )    ,
 
a12 
a
A   11,
 a21 a22 

0    b11  b12 
0    b21  b22   fr  (12)0    0 

Table 1 Unmanned Vehicle Parameters.
ParameterValuem(kg )
2,055
 (m / s)
2.78~8.33

c f and cr are the cornering stiffness. Therefore,𝑚(𝜈̇ + 𝜐𝛾) = 𝑐𝑓 ∙ 𝛼𝑓 + 𝑐𝑟 ∙ 𝛼𝑟𝐽𝛾̇ = 𝑐𝑓 ∙ 𝑙𝑓 ∙ 𝛼𝑓 − 𝑐𝑟 ∙ 𝑙𝑟 ∙ 𝛼𝑟

a12
a22
1

(11)

(7)

K (s)  K p (1  Td s)

(8)

(13)

Elements of system matrix A and input matrix B aregiven in the below Equations (9) and (10).

b11  b21  J©ISAROB 2011cf

m
cr l fJ,,(cr l f  cr lr )m
(cr l 2f  cr lr2 ), b22  Jcb12  rmcl
b22   r r
J, b12  m
(cr l f  cr lr )

b11 
b21 

(cr  c f )Fig. 2 Block Diagram of PD-Controller.

(9)Fig. 2 is a block diagram of a simulation. Here, r isthe control input representing the destination, e issteering angle error which is the difference between thedestination and current location. G(s) is represented by
Equation (13) as a plant function. The maximum steering
angle of the unmanned vehicle was set to ±30 degrees,and control gain was set throughout the experiments.

(10)

672

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Proportional gain Kp = 0.8 and derivative gain Kd = 0.35
is calculated [6].

destination [6].

Ⅴ. EXPERIMENTS
In this study, a PD-controller is applied to a twosteering unmanned vehicle, general P to P path tracking
algorithm and P to P driving algorithm using a PD
controller are experimentally compared. When thegeneral vehicle's driving characteristics for analyze, inother words, when the maximum steering angle of 20
degrees at the lateral acceleration of 0.4G is less than the
degrees of freedom linear model, approximately 20km/h
speed is turning.
Fig. 4 is a vehicle used in the experiment, and driving
speed 10km/h, 20km/h, were carried out, respectively.
The experiment vehicle used was a Hyundai-Kia
MOHAVE, and the experiment was carried out in theschool field. GPS coordinates were obtained through P to
P driving experiment while turning on an ellipse course.

Ⅳ. LOCATION RECOGNITION ANDDRIVINGIn autonomous navigation, the vehicle's current
position and the heading of the vehicle have to be known.
In this study, location-aware system used DGPS
(Differential Global Positioning System). The DGPS
reference station and rover is composed. Rover's absolute
position is known, and then GPS measurements of Roverare calibrated based on this. If the destination station is
located within the station's range of about 2km, thesystem provides a precision of 20cm CEP (95%).
Novatel's GPS receiver ProPak-V3 was used, andantenna, GPS-701-GG was used. For the unmanned
vehicle experiments, the roof of the vehicle was
equipped with two GPS receivers, so the azimuth of thevehicle can be obtained from two GPS coordinates.

Fig. 3 Lateral Control Model.
Fig. 3 is model for lateral control of vehicle. Pp(i)represents the current position of the vehicle, destination
location Pp(i+1), azimuth(heading) of the vehicle 𝜃1 ,azimuth(heading) of the destination location 𝜃3 , 𝜃2 isthe steering angle of the vehicle. The vehicle's current
location is known by DGPS measurements, and the
waypoint on the path to the destination is also known.
Therefore, the vehicle’s steering angle can be calculated
by Equation (14).

Fig. 4. Equipment used in the experiment.
.

𝜃2 = 𝜃3 − 𝜃1

(14)

The steering angle is calculated from the differenceof the destination azimuth and the current position of thevehicle. When the steering angle is 0 degrees, the vehicle
is head in the destination allowing you to reach the©ISAROB 2011673

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

P driving shows the results of smooth and stable driving.

Ⅵ. CONCLUSION
Lateral control of an unmanned vehicle using PD
controller is proposed in P to P driving. To apply to theactual vehicle, DGPS was used. When using DGPS,
azimuth of the vehicle can be measured more precisely.
A PD controller was applied to lateral control, andresults of experiments using the PD controller confirmed
it to be stable than general P to P driving. However,when the original path and experiment result's GPS
coordinate were compared, the turning radius was larger
than that of the original path. This error occurred because
the dynamic elements of the vehicle were not considered.
Research design considering the dynamic elements need
to be done so that reliable path tracking can be
performed for variable state of velocity.

Fig. 5 P to P Driving with P controller.

REFERENCES[1] Y. R. Oh, H. J. Lee, and Y. P. Park, "A Study on
Validation of Braking Performance Considered ABS
Friction Characteristics," The Korean Society ofAutomotive Engineers, Vol. 2, pp. 884~889, 2007.
[2] J. H. Choi, "Wheel Slip Control of ABS Using
Adaptive Control Method," Journal of the Korean
Society of Manufacturing Process Engineers, Vol. 5,
No. 3, pp. 71~79, 2006.
[3] H. C. Moon, C. M. Kim, and S. G. Kim, "The
Research for the Vision System of Tele-operated
Unmanned Vehicle," The Korean Society ofAutomotive Engineers, Vol. 2, pp. 1316~1323, 2001.
[4] J. Y. Um and I. S. Lee, "A Study for DGPSNavigation System for an Autonomous Vehicle," The
Korean Society of Automotive Engineers, Vol. 3, pp.
1179~1188, 2004.
[5] Y. H. Lee, S. I. Kim, M. W. Suh, H. S. Son, and S. H.
Kim, "Linearized Dynamic Analysis of a Four-wheel
Steering Vehicle," The Korean Society of Automotive
Engineers, Vol. 2, No. 5, pp. 101~109, 1994.
[6] K. S. Lee, M. K. Kim, Y. H. Koh, J. I. Bae, and M. H.
Lee, "Control of Unmanned Container Transporters
and Management System Development," Korea
Association of Automatic Control, 2009.

Fig. 6. Using the PD Controller Driving.
The experiment was set up in the path by using
DGPS, and driving speed of 10km/h ~ 20km/h were
performed while changing the driving speed, and theresulting values were compared with those of the originalpath.
Fig. 5 is a result of normal P to P driving. GPS pathcan be tracked successfully while driving at the speed of
10km/h, however, the path was not successfully tracked
when at 20km/h.
In Fig. 6, using the PD-controller P to P driving, itshows reliable path tracking regardless of speed. The
biggest differences in the two experiments appear whendriving along the corners. When you track a straight line
path from the current location of vehicle to thedestination in the original P to P driving, arrival at thedestination causes rapid increase in the steering angle.
However, P to P driving with PD controller can reach thedestination without a major change in steering angle
because of stable driving. Therefore, in this study, usinga PD controller P to P road driving than the original P to

©ISAROB 2011674

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A Control Method with Pheromone Information forTransport System
Yoshitaka Imoto ∗1 , Yasutaka Tsuji ∗2 and Eiji Kondo ∗2∗1

Faculty of Engineering, Kyushu University, 744 Motooka, Nishi-ku, Fukuoka, 819-0395, Japan(Tel: 81-92-802-3167; Fax: 81-92-802-0001; imoto@zeus.mech.kyushu-u.ac.jp)∗2

Faculty of Engineering, Kyushu University

ABSTRACT

pheromone information deposited on the track of
products. These two types of information are updated in diﬀerent frequency. we examine validity
and usefulness of the proposed method through simulation experiments.

Since control methods of a transport system between
processing machines are specialized for each layoutof the system, it is diﬃcult to design prompt schedules and control logic for recent complicated systems, especially in case of machine troubles or restructuring of system to modify the layout. In thispaper, we propose a multi-agent transport control
system which utilizes pheromone information. Eachelement of the system acts as an agent and estimates
the appropriate path and time to drive out the products with information obtained by communication
with other elements and pheromone information deposited on the track of products.

II. RELATED STUDIES
Pheromone is a chemical material with which social
insects, such as ants, communicate with each otherindirectly through the environment. Ants deposit
pheromones on their path when they ﬁnd a food intheir search. By using the pheromone information,other ants can ﬁnd eﬃcient path to the food. The
deposited pheromone evaporates over time and oldinformation loses their eﬀect on the path selection
gradually. In the engineering area, the mechanism
is modeled as a “pheromone model” and utilized inmany applications[3][4].
In the pheromone model, agents deposit thepheromone information onto the environment. The
information decreases in its value gradually overtime and is also updated by agents, so that the information goes along with the changes of the environment. Especially, in a recent study of the ITS(Information Transport System), pheromones areused to predict traﬃc congestions [2]. In the study,some information is used as the pheromone information, such as number of the braking and distancefrom other cars. The information enables short term
prediction for frequent change of traﬃc congestion
which can not be realized by long term prediction
such as statistic.
On the other hand, there are some applications
which utilize multi-agent control method for transport system. A multi-agent control method baseon A*-algorithm is proposed in [4]. It considers asystem in which products have priority to be transported, and reduce relocation of products in buﬀers.
In [5], another multi-agent control method is introduced. It uses several types of virtual agent to ﬁnd
a path to the destination or to collect information.
In these studies, the controlled systems have only
one processing machine as a destination and need

Keywords : Swarm Intelligence, Pheromone Information, Transport System
I. INTRODUCTION
Since control methods of a transport system between
processing machines in manufacturing systems are
specialized for each layout of the system, it is diﬃcult to design prompt schedules and control logic ofrecent complicated systems, especially in emergent
situations such as machine troubles or restructuringof the system to modify the layout[1].
On the other hand, in recent study in Information
Transport System (ITS), pheromone model for prediction of traﬃc congestion has been proposed[2].
In the study, the pheromone model is used to makea short term prediction of the traﬃc congestion forfrequent change.
In this paper, we propose a multi-agent transport
control system which utilizes pheromone information. In the proposed system, the transport system
consists of processing machines, buﬀers which temporally store products and drive them out in appropriate time, belt conveyors and turntables. Eachelement acts as an agent and estimates the appropriate path to drive out products with information
obtained by communication with other elements and©ISAROB 2011675

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

tsend ,csend ,lsend trec ,crec ,lrec

some centralized control such as a server of virtual
space, so that they are not enough in the case thatthe system contains several processing machines andseveral types of products or the scale of the system
becomes larger.
In this paper, we propose a decentralized multiagent transport control method for the system whichcontains several types of products and processing
machines.

M₁

T₂

B₂

M₂

T₃

B₃

M₃

R₂

III. PROPOSED METHOD

Figure 1: Flow of information

1. Assumption of production system
The production system addressed in this work includes several kinds of processing machine and atransfer system, which consists of belt-conveyors,
turntables and buﬀers. A belt-conveyor can carrya product in only one direction and a turntable canchange a move direction of products. Buﬀers, located back and forth of a processing machine, cantemporarily storage products. In this work, it isassumed that each element of the production system can communicate with each other elements connected to it.
Each element of the transfer system sets themachines as candidate destinations，which perform
next process to a product on it, and can obtain
useful information of the candidates to transport
the product through the communication between
the elements. When a production system is large,
amounts of information sent and stored are alsohuge. In this work, therefore, the each element sends
only information of the next destinations, i.e. next
machines in downstream, and the communication
between them is executed on periodic interval.
Each element of the transport system can obtain
information of candidate destinations, which may be
a little past, but they do not have a current status ofits neighbor element. In order to estimate the local information of neighbor element, each turntablecounts number of products which have moved ineach direction as trail of a product move. We callthis number as “pheromone”. The pheromone isevaporated and the eﬀect of the past information
decreases gradually as the natural pheromones.
With these kinds of information, we construct themethod which determines the optimal path for thesituation on each element by considering the wholeof the system. As the information is obtained bycommunication between neighbor elements or an element and a product, the control can be decentralized. Thus, it is expected that the transport changes
ﬂexibly for the change of the system.

t : waiting/moving time to next elementc : vacant capability of a buﬀer
l : completion time of next processing
Processing machines send their own information
to the elements which connected to them in the upstream side. Each element receives the informationand sends the information added with its own status value to the upstream side, but the information
is not sent to the processing machines in upstreamside.
Each element receives three parameters trec , crec
and lrec from its downstream side, then sends tsend ,csend , lsend to its upstream side. When sending
these parameters, each parameter is updated as
Eqs.(1)-(3).
tsend = max(town , trec )csend = min(cown , crec )

(1)
(2)

lsend = lown + lrec

(3)

town , cown , lown are parameters of the element.
town is the process time in the processing the machine or time interval from input to output on the
buﬀer. The element sends bigger one of town andtrec . cown is value of empty space in the buﬀer. The
element sends smaller one of cown and crec . lownis the time for process completion on the destination. The element adds lown to lrec and sends it tothe upstream side. By using these parameters, theevaluate function of the path is deﬁned as:β
E = cαsend /(atsend + blsend ) .

(4)Here α , β , a , b are weighting coeﬃcients.
3. Pheromone information
The pheromone information is deposited when aproduct is transported from an element to the next
element, in the direction of the next element. The
large value of the information implies the congestion and the direction is avoided to transport. The
pheromone increases with constant value △ϕ fora product. The deposited pheromone evaporates
and decreases its eﬀect on path selection. The

2. Information about the path to the candidate destinations
In the communication, the following three types ofparameters (estimations) are used,©ISAROB 2011

R₁

B₁

T₁

676

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

BufferIN

Processing Machine

Turntable

Belt Conveyor

B1

M1

B4

B7

M4 B10B13 M7 B16

B19 M10 B22

B25 M13 B28

B2

M2

B5

B8

M5

B11

B14 M8 B17

B20 M11 B23

B26 M14 B29

B3

M3

B6

B9

M6

B12

B15 M9 B18

B21 M12 B24

B27 M15 B30

Process P3

Process P4

Process P5

Process P1

Process P2

OUT1

OUT2

OUT3

Figure 2: Layout of the production systempheromone ϕ(s) on time s evaporates in time period△s as given in the Eq.(5). Here, ρ is evaporativerate.
ϕ(s + △s) ＝ (1 − ρ)ϕ(s)(5)

4. Collision avoidance
In the system which contains some conﬂuent point,collisions must be considered. In this study, as theconsidered system is constructed by belt-conveyors,
it is diﬃcult to avoid collisions by adjacent avoidance. For the problem, we put a buﬀer in the upstream side of each conﬂuent point and make themcommunicate with each other to transport elements
one by one in order of input. Therefore, the system
in Fig.2 assumes that the each of buﬀers B4 -B6 , B10 B12 , B16 -B18 and B22 -B24 can communicate anddetermine the time of output.

By using these two types of information, next element of the transport is selected for each product(opt)
pk . The best element ek
is determined, which canperform the next process to pk and have small valueof the pheromone information ϕ and large valueof the evaluate function E. If the candidate elements have several destinations, they are evaluatedas the weighted sum of the evaluation function E(see Eq.(7)). The optimal element to transport isdetermined as shown in the followings.
(opt)= ei , i = arg min(ϕγj /Ek,j )j∈D∑∑
Ek,j =
Em cm δk,m /cm δk,mek{
δk,m =m∈S

1
0

IV. SIMULATION EXPERIMENT
We examine validity and usefulness of the proposed
method through simulation experiments. The parameters β , γ are varied to examine the eﬀectiveness of the proposed information. α is ﬁxed.
In the simulation experiments, we use the layout
shown as Fig.2. Mi (i = 1, ..., 15) represent processing machines, Bi (i = 1, ..., 30) represent buﬀers andlines between them are belt conveyors. Products arecarried into the system from IN with constant interval, performed each process Pi (i = 1, ...5) on processing machines M1 -M15 , and ﬁnally output fromOU T 1-OU T 3. The system has several processing
machines of same type. For example, process P1 isprocessed in M1 -M3 , but each processing machine
has diﬀerence in process time as shown in Table 1.
The maximum capability of the buﬀers is 20. The
type of products is only one type. Products are carried into the system for every 30 steps. The numberof the products is 200. The time from the output of
the 50th product to the output of the 150th product is measured, and the value is deﬁned as T(est) .
The work rate w is calculated as the ratio of working time of the processing machines to the T(est) .
The results of the experiments are shown in Table
2, Table 3 and Table 4.
Table 2 shows the work rate of each processing
machine in the condition of α = 1.0, β = 1.0,
γ = 1.0. Table 3 and Table 4 show T(est) in eachcondition. In each experiment, a = 1.0, b = 0.01.

(6)
(7)

m∈S

(pk can be processed)(pk can not be processed)(8)

D is the set of possible elements where pk can betransported. S is the set of path including elementj. δk,m is 1 when the destination element em of thepath can process the product pk , otherwise δk,m is
0.
In Fig.1, M1 -M3 represent processing machines
and they are set as candidate destinations for B1 -B3and T1 -T3 . For example, M1 sends the information
about its self such as process time t to B1 . B1 also
sends the information updated as in Eqs.(1)-(3) toT1 . In the same way, T1 gets information about thepath to M2 and information about the path to M3 .
The products on T1 are transported based on theinformation and pheromone. If the product pk on T1
can be processed on both M2 and M3 , the evaluationof the direction R2 is calculated as weighted sum of
evaluation of the path to M2 and that of the pathto M3 . Finally, the product is transported to the
optimal direction and the pheromone information isdeposited on the direction.

©ISAROB 2011

677

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 1: Process time t of each processing machine
for a product
M1
P
t
P
t
P
t

60
M7
50
M13
30

M2
P1
50
M8
P3
30
M14
P5
25

M3

M4

35
M9

80
M10

25
M15

50

M5
P2
50
M11
P4
40

Table 3: Process time for 100 products (α = 1.0,
γ = 1.0)M6

βT(est)βT(est)

40
M12
40

w
w
w

M4
0.780
M5
0.466
M6
0.293

20

M7
0.156
M8
0.268
M9
0.396

γ
T(est)

M10
0.487
M11
0.339
M12
0.189

1.0
4366

1.5
4239

2.0
4208

3.0
3927

0.0
−

0.1
−

0.2
−

0.5
4424

1.0
4366

1.5
4351

2.0
−

V. CONCLUSIONSIn this paper, we have proposed the multi-agent
transport control system which utilizes pheromone
information and examined validity and usefulnessof the proposed method through simulation experiments. As a result of the experiments, although
there is some unbalance in work rates, the proposed
method can transport products properly in someconditions. For future works, we need to improve
the work rate by controlling the input time of product, adjust some parameters and introduce new information such as pheromone of distanced elements.

M13
0.117
M14
0.204
M15
0.209

The transport was ﬁnished properly except insome conditions. In the case that pheromone information has no eﬀect (γ = 0.0) or little eﬀect(γ =0.1, 0.2), products are transported to the buﬀer overits capability and the transport was not ﬁnishedproperly. Thus, it is considered that the pheromone
information plays important role in path selection.
On the other hand, the transport is also not ﬁnished properly, where the eﬀect of the pheromone
information is bigger (γ = 2.0). γ has to be adjusted to proper value. The small or the big eﬀectof the time information also results in overﬂow of
the buﬀers(β = 0.1 , 10.0). As T(est) changes remarkably for the change of β, it is considered thatthe information about waiting/moving time t has agreat inﬂuence on the utility.

ACKNOWLEDGMENT
This work has been supported in part by a financial
aid for Encouragement of Young Scientists of Faculty of Engineering, Kyushu University(2009).
REFERENCES[1] K. Uchiyama, T. Yoshikawa, K. Uchimura andZ. Hu: Construction of transport control systemusing congestion index, IEICE Technical Report,
ITS 105(166), pp.23-28, (2005), (in Japanese).
[2] Y. Ando, Y. Fukazawa, O. Masutani, H. Iwasaki
and S. Honiden: Performance of pheromone
model for predicting traﬃc congestion, AAMAS’06 Proceedings of the fifth international joint
conference on Autonomous agents and multiagent systems , Vol.3, No.1, pp.1-16 (2006).
[3] M. Dorigo and T. Stützle: Ant Colony Optimization, The MIT Press, (2004).
[4] T. Seidel, J. Hartwig, R.L.Sanders and
D. Helbing: An Agent-Based Approach toSelf-organized Production, Swarm intelligence,
Springer, pp.219-252 (2008).
[5] P. Valckenaers and T. Holvoet: The environment: An essential abstraction for managing
complexity in MAS-based manufacturing, In
Lecture notes in computer science, Springer Verlag, (2006).

In the experiments, the input time of products isconstant, so that the work rate should be improved
if the input time is controlled properly. However,considering some loss of time to transport products
from the buﬀers to the machines, the work rate of
M4 is nearly bound to improve. Thus, the unbalanceof work rates between processing machines needs tobe improved. For example, the work rate of M6
is less than that of M4 and M5 . It is caused bya lack of the information about products which isprocessed in M1 and M2 . Although some of the information about the products processed in M1 andM2 is given as waiting/processing time t and capability of the buﬀer c, which improve the unbalancein the condition of bigger β, additional introductionof some new information is needed.

©ISAROB 2011

0.5
4615
10.0
−

Table 4: Process time for 100 products (α = 1.0,
β = 1.0)

Table 2: Work rate w for each processing machine(α = 1.0, β = 1.0, γ = 1.0)
M1
0.419
M2
0.378
M3
0.335

0.1
−
5.0
3765

678

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Implementation of real-time distributed control for discrete event
robotic manufacturing systems using Petri nets
G. Yasuda
Nagasaki Institute of Applied Science, Nagasaki 851-0193, JAPAN(Tel & fax : 81-95-839-8973)(yasuda_genichi@pilot.nias.ac.jp)
Abstract: The paper deals with a systematic method of modeling and real-time control for discrete event robotic
manufacturing systems using Petri nets. Because, in the complex manufacturing systems, the controllers are distributed
according to their physical structure, it is desirable to realize real-time distributed control. In this paper, the task
specification of discrete event manufacturing processes is represented using a global Petri net. Then it is decomposedand distributed into the machine controllers, which are coordinated through communication between the coordinatorand machine controllers. The coordination algorithm was derived using formal expressions of transition firing and statechange so that the decomposed transitions fire simultaneously. Implementation of real-time distributed control using a
microcomputer network is described for an example robotic manufacturing cell. The demonstrations show that the
proposed method can be used as an effective tool for consistent modeling, simulation and real-time control of large andcomplex manufacturing systems.
Keywords: Discrete event robotic manufacturing systems, modeling, real-time control, implementation, Petri nets.

specification. Therefore, it has been applied to a variety
of system developments such as real-time systems,
production systems, communication systems, and so on.
However, in case of factory automation systems, thenetwork model becomes complicated and it lacks for thereadability and comprehensibility. Besides, only
specification analysis stage has been supported, and the
support for the control software coding stage isinsufficient [1]. Therefore, the flexibility andexpandability are not satisfactory in order to deal withthe specification change of the system.
Due to its complexity, a large and complex
manufacturing system is commonly structured into ahierarchy; factory, line, station, machine, and actuators.
The manufacturing system handles complicated tasks bydividing a task hierarchically in this structure, which is
expected to be effective in managing cooperation tasks
executed by great many machines or robots.
Conventional Petri net based control systems were
implemented based on an overall system model. Since
in the large and complex systems, the controllers are
geographically distributed according to their physical(hardware) structure, it is desirable to realize thehierarchical and distributed control. If it can be realized
by Petri nets, the modeling, simulation and control oflarge and complex discrete event manufacturingsystems can be consistently realized by Petri nets. In
this paper, the author presents a methodology forhierarchical and distributed control of large and

I. INTRODUCTION

0B

As the factory automation systems have been
becoming larger and more complex. The systemarchitecture has shifted to distributed and parallel
processing from centralized processing in order toreduce the development cost and to improve thereliability. In the field of factory automation systems, ademand for the automatic control has diversified and thecontrol logic has become extremely complicated,because the combinative complexity of the control
requirement, which comes from the non-deterministicfeatures of event driven systems such as manufacturingcontrol systems, is inevitable. To deal with thecomplexity, a new methodology on control systemdesign based on the concept of event driven system isnecessary. However, appropriate representation methods
and analysis methods for control mechanism have notsufficiently been established.
Programming paradigm modeled by a network, such
as Petri net, has been considered to be useful, because
the network model can describe the execution order ofparallel/sequential processes directly without ambiguity.
The Petri net is excellent in expression and analysis of
the dynamic behavior of event driven systems, because
it can model the system that consists of simultaneous
process elements that interfere to each other. The
programming technique makes it possible to realize
systematic and high-level description of system©ISAROB 2011

679

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

complex robotic manufacturing systems using extended
Petri nets, to construct the control system where thecooperation of each controller is implemented so thatthe aggregated behavior of the distributed system is thesame as that of the original system and the task
specification is completely satisfied.

M : input place set of transition j
p Ij ,m ( k ) : state of input place m of transition j
at time sequence k
N : output place set of transition j
p Oj ,n ( k ) : state of output place n of transition j
at time sequence k
Q : internal permissive gate signal set of transition j
g IP
j ,q ( k ) : internal permissive gate signal variable qof transition j at time sequence k
R : internal inhibitive gate signal set of transition jg IIj ,r ( k ) : internal inhibitive gate signal variable r oftransition j at time sequence k
U : external permissive gate signal set of transition j
g EP
j ,u ( k ) : external permissive gate signal variable uof transition j at time sequence k
V : external inhibitive gate signal set of transition j
g EI
j ,v ( k ) : external inhibitive gate signal variable v oftransition j at time sequence k

II. MODELING OF DISCRETE EVENTSYSTEMS USING EXTENDED PETRI NETS
1B

Discrete event systems such as robotic
manufacturing systems have the properties ofasynchronism, ordering, concurrency and conflict, sothat deadlock will be apt to occur in the systems. The
Petri net is one of the effective means to represent such
systems. For applying it to the design, analysis, andcontrol of the systems the guarantee of safeness and thenotation of input/output of signals from/to machines
should be required. A kind of graph deduced from thePetri net was proposed so as to satisfy the above
requirements [2].
The extended Petri net consists of the following six
elements: place, transition, directed arc, token, gate arc,
output signal arc. A place represents a condition of asystem element or action. A transition represents anevent of the system. A directed arc connects a place to atransition, and its direction shows the input and output
relation between them. Places and transitions arealternately connected using directed arcs. The numberof directed arcs connected with places or transitions is
not restricted. A token is placed in a place to indicatethat the condition corresponding to the place is holding.
A gate arc connects a transition with a signal source,and depending on the signal, it either permits or inhibits
the occurrence of the event which corresponds to theconnected transition. Gate arcs are classified aspermissive or inhibitive, and internal or external. An
output signal arc sends the signal from a place to an
external machine.
Formally, the firability condition and external gatecondition of a transition j are described as follows:
M

The state (marking) change, that is, the addition
or removal of a token of a place, is described as follows:

m =1

n =1

Q

R

(1)

I g (k ) ∧ I g (k )
IP
j ,q

q =1

II
j ,r

r =1

U

V

u =1

v =1

EI
g Ej ( k ) = I g EP
j ,u ( k ) ∧ I g j ,v ( k )(2)where,©ISAROB 2011(3)

p Oj ,n ( k + 1) = p Oj ,n ( k ) ∨ (t j ( k ) ∧ g Ej ( k ))(4)

An enabled transition may fire when it does not haveany external permissive arc signaling 0 nor any external
inhibitive arc signaling 1. The firing of a transition
removes tokens from all its input places and put a token
in each output place connected to it. The assignment oftokens into the places of a Petri net is called marking
and it represents the system state. In any initial marking,there must not exist more than one token in a place.
According to these rules, the number of tokens in aplace never exceeds one, thus the Petri net is essentiallya safe graph; the system is free from the bumping
phenomenon. If a place has two or more input
transitions or output transitions, these transitions may bein conflict for firing. When two or more transitions are
firable only one transition should fire using somearbitration rule.
In the proposed procedure of modeling of robotic
manufacturing systems, a global, conceptual Petri net
model is first chosen which describes the aggregate
manufacturing process. The places which represent thesubtasks indicated as the task specification areconnected by arcs via transitions in the specified ordercorresponding to the flow of subtasks and a workpiece.
For the example manufacturing cell with two robots,Nt j ( k ) = I p Ij ,m ( k ) ∧ I p Oj ,n ( k ) ∧p Ij ,m (k + 1) = p Ij ,m (k ) ∧ (t j (k ) ∧ g Ej (k ))680

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

one machining center, and two conveyors, where one isfor carrying in and the other is for carrying out, themain execution of the system is indicated as thefollowing task specification:

(CV1)
t2

Waiting(Robot)Moving(1) A workpiece is carried in by the conveyor CV1.
(2) The workpiece is loaded to the machining center
MC by the Robot R1.
(3) The workpiece is processed by the machining center
MC.
(4) The workpiece is unloaded to the conveyor CV2 by
the Robot R2.
(5) The workpiece is carried out by the conveyor CV2.

Grasp

Moving(MC1)
ForwardPut onWaiting

Waiting t3

FixingFig.2 Detailed Petri net representation of loading
operation

III. DECOMPOSITION AND
COORDINATION ALGORITHM
2B

The places representing the existence of machines
are also added to connect transitions which correspondto the beginning and ending of their subtasks. Thus at
the conceptual level the manufacturing process isrepresented as shown in Fig. 1 for the cell with two
robots.
1

Conveyor CV1

3

4

Conveyor CV2

For the manufacturing system, an example structureof hierarchical and distributed control is composed of
one station controller and three machine controllers. The
detailed Petri net is decomposed into subnets, which are
executed by each machine controller.
In the decomposition procedure, a transition may be
divided and distributed into different machine
controllers as shown in Fig.3. The machine controllers
should be coordinated so that these transitions fire
simultaneously, that is, the aggregate behavior of
decomposed subnets should be the same as that of theoriginal Petri net. Decomposed transitions are called
global transitions, and other transitions are called local
transitions.
By the Petri net model, the state of the discrete event
system is represented as the marking of tokens, andfiring of any transition brings about change to the next
state. So the firing condition and state (marking) change
before decomposition should be the same as those after
decomposition. If transition j is divided into stransitions j1 , j 2 , , , js , the firability condition of atransition after decomposition is described as follows:

6

Robot R2 5

2 Robot R1Carrying in Loading Processing Unloading Carrying out
MC

Fig.1 Petri net representation of example system with
two robots at the conceptual level
Next, based on the hierarchical approach, the Petrinet is translated into detailed subnets by stepwise
refinements from the highest system control level to thelowest machine control level. At each step of detailed
specification, a place of the Petri net is substituted by asubnet in a manner which maintains the structuralproperties. The detailed Petri net representation of theloading operation in the example system, which is atypical task at the subsystem coordination level in thefactory automation system, is shown in Fig.2. Loading aworkpiece to the machining center necessitates thecooperative or synchronized activities among the input
conveyor, the machining center, and the robot. Similarly,
unloading a workpiece from the machining center,
necessitates the cooperative or synchronized activities
among the output conveyor, the machining center, andthe robot.

©ISAROB 2011

Msub

Nsub

t jsub (k ) = I p Ijsub ,m (k ) ∧ I p Ojsub ,n (k ) ∧
m =1

n =1

Qsub

Ig
q =1

Rsub

IP
jsub , q(k ) ∧ I g
r =1

Usub

Vsub

u =1

v =1

II
jsub , r

(5)

(k )

EI
g Ejsub ( k ) = I g EP
jsub ,u ( k ) ∧ I g jsub,v ( k )(6)From eq.(1) and eq.(5),St j ( k ) = I t jsub ( k )sub =1

681(7)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

From eq.(2) and eq.(6),overall system is achieved by coordinating these Petrinet based controllers such that decomposed transitions
fire at the same time and the task specification iscompletely satisfied. The overall control structure of theexample manufacturing system was implemented on a
local area network of computers as shown in Fig.3.

S

g Ej ( k ) = I g Ejsub ( k )(8)sub =1

where,
S : total number of subnets
Msub : input place set of transition jsub of subnet

sub

p

I
jsub,m( k ) : state of input place m of transition jsub

Station controllerof subnet sub at time sequence k

Robotcontroller

Nsub : output place set of transition jsub of subnet

PC

MC
controller

serial bus

sub

p

O
jsub,n( k ) : state of output place n of transition

Robot R1

jsub of subnet sub at time sequence k

Machining Conveyor
Center MC
CV1

Fig.3 Implementation of real-time control system on
microcomputer network

Qsub : internal permissive gate signal set of transition
jsub of subnet sub

Each Petri net based machine controller on a
dedicated microcomputer (Renesas H8/3069, real-time
OS ITRON 4.0) executes real-time machine control, by

Rsub : internal inhibitive gate signal set of transition
jsub of subnet sub

Usub : external permissive gate signal set of transition
jsub of subnet sub

initiating the execution of the unit actions attached to thefired transitions. The machine controller in charge ofrobot control executes robot motion control through thetransmission of command and the reception of status
report with serial interface to the real robot controller.
The station controller implemented on another computeris also used to support modeling, simulation, anddebugging of Petri nets and to load the Petri net based(9)control program into the machine controllers.
(10)
Communications among the controllers are performed
using TCP/IP protocol. The coordinator sends thecommands based on the conceptual, global Petri net
model and coordinates the global transitions, which areaccessed by the controllers as a shared file, such that
decomposed transitions fire simultaneously.

Vsub : external permissive gate signal set of transition
jsub of subnet sub
The addition or removal of a token of a placeconnected to a decomposed transition is described asfollows:

p Ijsub ,m (k + 1) = p Ijsub ,m (k ) ∧ (t j (k ) ∧ g Ej (k )) (9)p Ojsub,n ( k + 1) = p Ojsub,n ( k ) ∨ (t j ( k ) ∧ g Ej ( k )) (10)From the logical formulation of firability condition
and marking before and after decomposition, it isproved that the firability condition of the originaltransition is equal to AND operation of firability
conditions of decomposed transitions. In case that atransition in conflict with other transitions isdecomposed, these transitions should be coordinated by
the station controller. The coordinator was introduced to
execute the coordination algorithm [2].

V. CONCLUSIONS

4B

An implementation of real-time distributed control
for discrete event robotic manufacturing systems was
described to realize consistent modeling, simulation andreal-time control of large and complex robotic systems.

IV. PETRI NET MODEL BASED CONTROLEXPERIMENTS

3B

REFERENCES

The control software is distributed into the station
controller and machine controllers. The station
controller is composed of the Petri net based controller
and the coordinator. The conceptual Petri net model isallocated to the Petri net based controller for
management of the overall system. The detailed Petrinet models are allocated to the Petri net based
controllers in the machine controllers. The control of the©ISAROB 2011

Robot R2

Microcomputers
H8/3069
ITRON 4
Conveyor
CV2

Conveyor
controller

5B[1] Hasegawa, K., Takahashi, K., and Miyagi, P. E.,(1988) Application of the Mark Flow Graph to represent
discrete event production systems and system control,
Trans. of SICE, 24, 69-75[2] Yasuda, G., (2009) Implementation of distributed
cooperative control for industrial robot systems using
Petri nets, Preprints of the 9th IFAC Symposium on
Robot Control (SYROCO ’09), 433-438

682

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Development of Motion Analysis System using Acceleration Sensors for
Tennis and its Evaluations
Takaya Maeda, Kenichirou Fuji, Hiroki Tamura, and Koichi Tanno
Dept. of Electrical & Electronic Engineering, University of Miyazaki, 1-1 Gakuen-kibanadai-nishi, Miyazaki-city,
Miyazaki, 889-2192, Japan(Tel : 81-985-58-7409; Fax : 81-985-58-7409)(Email address: htamura@cc.miyazaki-u.ac.jp)

Abstract: The term biological motion is often used by researchers studying the patterns of movement generated byliving forms and sport forms. We study the pattern recognition system of sport motion using the biological motion data.
Biological motion data are acquired using 3D motion capture system. However, 3D motion capture system is too
expensive. In this paper, the biological motion capture system was built using acceleration sensors. Our proposedsystem has the technique of Gaussian fitting and regression analysis. We tested our proposed system in the patternrecognition of the outdoor tennis and its evaluations.
Keywords: Motion Analysis System, Acceleration Sensors, Biological motion data, Gaussian fitting

12 acceleration sensors (WAA-006: ATR-promotions
Co.,Ltd) onto the subject’s wrist / elbow / shoulder /waist / knee / ankle of both the left and the right side [2].
However, we analyzed three places of data of the wrist /elbow / shoulder of the dominant arm. These
accelerations information are important place in thetennis form and the much clear than other datas changes
in acceleration information. A motion sensing system isacceleration and a sampling frequency of the system is237.5Hz. We show the photograph of Figure 1 when theplayer worn the acceleration sensors. The subjects are
eleven male university students. The tennis experienced
are five people, and the tennis inexperienced person are
five people. These experimental conditions are similar
to paper [3].

1. Introduction
The motion capture system is one of the techniques toretrieve the action of the human body and the animal in
the three-dimensional space in a computer as digital
data. We wear a marker and a sensor on parts becoming
the imperative construct of the movement such as thehead or a hand foot joint and record those movements inthe space. However, many cases were impossible tomeasure the outdoor sport. These systems were used acertain indoor. In addition, there are problems that theprice of machine parts is expensive [1].
Therefore, in this paper, we built the motion capturesystem which used an acceleration sensor. The proposal
system needs little load of the wearing, but it canacquire data fast outdoors. Because we can acquire
many data by using this system, it is easy to build a
database of outdoor sport. Therefore, in this paper, webuild the database of the sports that individual
difference is big and perform the operating analysis ofdata. We targeted the tennis. In this paper, we acquire
the database of experienced persons and inexperienced
persons form of tennis swing using the proposal system.
And, we examine the point of difference of experienced
person and inexperienced person for the form of tennis
swing. We analyzed the data and performed thecomputer experiment that estimated of the tennis career.
This paper used the technique of Gaussian fitting andmulti regression analysis. From these experiments, we
tried that find out a common knack from an experienced
person.

Figure 1.An acceleration sensor and the state that
player wore the acceleration sensors.
2.2 The proposal analysis system.
In this subsection, we propose a feature extraction
method for the estimates of the tennis career from theacceleration data. The proposed method is summarized
in Figure 2.
As a first step, the player wears the acceleration sensor
and play tennis to acquire acceleration data (X-axis, Yaxis, Z-axis). Next, one swing from acceleration data ofthe tennis swing were pulled out.

2. Feature Extraction from Biological Motion
2.1 Procedure to obtain biological motion data
To obtain the biological motion data of subjects, we put©ISAROB 2011683

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Figure 2. The summary of the proposal analysis system.

The STEP 2, Pi are computed by Eq.(1) [4]. Pi means
the amount of the change of the acceleration of threeaxes. Next STEP3 and 4, we perform a Gaussian fittingwith Eq.(2), and calculate A, B, C of Eq.(2). A means
power, B is timing, and C is sharp of tennis form. The
timing of B are calculated based on the shoulder. We
repeat ourselves from STEP1 to STEP4. And we
calculate the average and the standard deviation of theeach A, B, C. Finally STEP 6, we perform multiple
regression analysis and Leave-one-out cross validation
method with the average and the standard deviationfrom computed by this analysis system.

Pi 

x  y  z 
2
i

2
i

2
i

  Pi  B 2 f  Pi   A  e  
2C 2 
interlocking movement characteristics of the wrist /elbow / shoulder and calculate the degree of similarity E
between two joints using Eq. (3). N in Eq.(3) is thenumber of the samplings, Pi k is one of the data of thewrist / elbow / shoulder. Pil is a non- Pi k information. We
calculate the average and the standard deviation ofEq.(3). In this paper, we added the data of the degree ofsimilarity E to the STEP6.

(1)

(2)

2.3 The interlocking movement characteristics of thedominant arm

Figure 3. The interlocking movement characteristics ofthe dominant handWe guessed that the interlocking movement of the wrist/ elbow / shoulder of the dominant arm might be related
to the person who experienced tennis. And, we
presumed the career of tennis that uses only theinterlocking movement by the previous research [3]. Weshow the graph of the wrist / elbow / shoulder in Figure
3 to express the interlocking movement characteristicsof the dominant arm. As for the data, it normalized sothat the maximum value becomes 1.0 and the averagevalue becomes 0.0. We focus attention on the©ISAROB 2011E degree of similarity  



1 N k
Pi  Pi l

N i

 (3)

3. Experiment Results
In order to test the effectiveness of proposal analysis
system, we compared its performance with only used A,

684

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

B, C and only used the degree of similarity E. In thispaper, we perform multiple regression analysis andevaluate using Leave-one-out cross validation method.
3.1 The results of multiple regression analysis
We show the result of the multiple regression analysis
how we used only the degree of similarity E in Figure 4.
From t value and the P value of the analysis result, weunderstood that wrist / elbow (average) of Eq.(3) andwrist / shoulder (standard deviation) of Eq.(3) weremeaning for tennis career. We assume these twoelements E’. Next, we show the multiple regression
analysis how we used the A (Power), B (Timing), C(Sharp) of Eq.(2) in Figure 5. The simulation of Figure
5 was used 8 elements. 8 elements are the average value.
Finally, we show the multiple regression analysis how
we used E’ and the A (Power), B (Timing), C (Sharp) ofEq.(2) in Figure 6. The simulation of Figure 6 was used
9 elements. A value of R2 (a coefficient of correlation)
got results more than 0.8 from Figure 4 and Figure 5
and Figure 6. In three graphs, when a predicted value by
the multiple regression analysis assumes the smallest
person boundary value among the person who
experienced tennis, we can pull the boundary line of anexperienced person and the inexperienced person. But
only one person did wrong identification among
inexperienced people in Figure4. A graph of Figure 6
was the best in these computer simulation results.
Therefore, we selected the results of Figure 6, and toanalyze it in detail in the next subsection.

Figure 5. The result of the multiple regression analysis
using the A (Power), B (Timing), C (Sharp).

Figure 6. The result of the multiple regression analysis
using the degree of similarity E’ and the A (Power), B(Timing), C (Sharp).

3.2 The result of Leave-one-out cross validation
methodFrom many analysis results, we selected the threefollowing elements using t value and P value.
・The timing B of elbow (standard deviation)
・The degree of similarity of wrist / elbow (average)
・The degree of similarity of wrist / shoulder (standard
deviation)
We show the result of the multiple regression analysis
how we used three elements mentioned above in Figure
7. From Figure 7, value of R2 got results more than 0.8.
The result of the Leave-one-out cross validation
method using three elements are shown in Figure 8.
When a predicted value by the multiple regression
analysis assumes the smallest person boundary value
among a person who experienced tennis, we can pull theboundary line of an experienced person and theinexperienced person. As a result, only one person did
wrong identification among inexperienced people.

Figure 4. The result of the multiple regression analysis
using the degree of similarity E.

©ISAROB 2011685

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

In future work, we must increase the number ofsubjects. We think that it raises the reliability of thisstudy.

References[1] Hiroki Tamura, Daisuke Sasajima, Koichi Tanno,
Takako Toyama (2008), “Human Motion Recognition
System using Nintendo wii Remote Controller”, inJapanese, Memoirs of the Faculty of Engineering,Miyazaki University, 37:PP141-146[2] Hidenori Maruta, Masahiro Ishii (2007), “Feature
extraction from Biological motion of human gait
patterns for emotion discrimination”, IAPR Conference
on Machine Vision Application, 13-15, pp492-495

Figure 7. The result of multiple regression analysis.

[3] Kenichirou Fuji, Takaya Maeda, Hiroki Tamura,
Takako Toyama, Kouichi Tanno (2010), “Developmentof Motion Analysis System using Acceleration Sensors
for Outdoor Sports and its Applications (in Japanese)”,HINOKUNI information symposium[4] Takaya Maeda, Hiroki Tamura, Takako Toyama,
Kouichi Tanno, Masahiro Ishii (2009), “A study about
the feelings estimate from ambulation movement (inJapanese)”, Student Activity Committee, Kyusyu
Section IEICE

Figure 8. The result of Leave-one-out crossvalidation method.

4. Conclusion and Future WorkWe aimed for examining the point of difference ofexperienced person and inexperienced person for theform of tennis swing. In this paper, we built the motion
capture system which used an acceleration sensor. We
acquire the database of experienced persons andinexperienced persons form of tennis swing using the
proposal system. And we analyzed these data andperformed the experiment that estimated of the tennis
career. For sports, we were able to extract power andtiming and sharpness and the interlocking movement
characteristics of the joint from acceleration information.
As a result, we could discriminate the inexperienced
person from an experienced person from tennis swing
with an acceleration sensor and was able to find out athing such as a common knack from an experienced
person. It is thought that three points of the following
are common to an experienced person in conclusion.
1) The movement of the shoulder and the elbow is
stable.
2) The interlocking movement characteristic of theelbow and the wrist is strong.
3) The interlocking movement characteristic of thewrist and the shoulder is stable.

©ISAROB 2011

686

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Discriminate approach for data selection in data envelopment analysis
Akio Naito and Shingo Aoki
Osaka Prefecture University, 1-1 Gakuencho aka-ku Sakai Osaka(Tel : 81-72-254-9353; Fax : 81-72-254-9915)({naitou@mis., aoki@}cs.osakafu-u.ac.jp)

Abstract: DEA (Data Envelopment Analysis) is a well-known method for evaluating management efficiency of DMUs
(Decision Making Units). To calculate efficiency of DMUs, analytical data are necessary. However, there are not clear
criteria for data selection so that analysts have to choose the data on their own. Therefore, it is important to support data
selection by reasonable ways to let analysis be informative and beneficial. In order to deal with this matter, newmethods are proposed based on traditional ones. Support for data selection is realized by considering analyst’s intention.
Analytical data for making some specific DMUs efficient are obtained by reflecting knowledge or experience analysts
have. TDS-DEA (Tight Data Selection based DEA) reflects the analyst’s intention strongly and tries to make only
intended DMUs efficient. On the other hand, LDS-DEA (Loose Data Selection based DEA) reflects it loosely and at
least intended DMUs can be efficient. Then both methods should be examined more detail and how data selection iscarried out effectively. On this point, this study prepares the experimental data to clarify the effectiveness and drawbackof the methods. According to the experimental result, additional ideas such as discriminate approach or assurance
region method are considered to improve the quality of data selection.

Keywords: Data Envelopment Analysis, Linear Programming, Decision Making Support, Data Selection

Therefore, the purpose of this study is to examine thetraditional method in detail and find benefit anddrawback to improve the approach.

I. ITRODUCTIO
DEA (Data Envelopment Analysis) is a method formeasuring efficiency of DMUs (Decision Making
Units) like company, hospital, municipal government or
so. DMUs are evaluated by index called “efficiency
score”. Each DMU is classified as the state of efficientor that of inefficient based on the score [1]. DEA
calculates efficiency score of each DMU based on
Pareto optimal line which is called efficiency frontierconsists of efficient DMUs. Then DEA shows a plan forimprovement to inefficient DMUs.
DEA assumes activity of DMUs that produce output
from input. This mechanism is interpreted as production
function. DEA is a data-oriented method so that
efficiency score depends on analytical data. Then alldata related to DMUs can be possible to be selected foranalysis. Therefore, data selection is really important. It
is not easy to prove whether selected data correspond to
purpose of analysis or not. Criteria of data selection are
unclear practically. Hence, some methods were
proposed to support data selection. The basic idea of the
traditional methods is to utilize analyst’s intention suchas experience or knowledge regarding evaluated DMUs.
Though numerical experiment is carried out, but stillneed more trial to clarify the power and effectiveness.

©ISAROB 2011

II. DATA EVELOPMET AALYSIS
1. Outline
DEA was proposed by A. Charnes et al. in 1978 as amethod for management analysis [2]. DEA has room totreat a lot of data related to DMUs. Then necessary
elements (capital, employee, etc) for operation are
generally recognized as input and yielded elements(sales, profit rate, etc) are recognized as output. DEA
calculates efficiency by input and output so that lessinput and larger output is more preferable. And theefficiency of each DMU is evaluated relatively among
analyzed DMUs.
Efficient DMUs are regarded as best practice among
DMUs and they get efficiency score as “one”.
Inefficient DMUs have that score less than one.
Efficiency score is denoted as θ and calculated by
dividing virtual output by virtual input. Virtual input
and output are the useful for dealing with multielements. DEA puts weight to each element and it is not
fixed but variable. Therefore, it is possible to evaluateadvantages of DMUs as much as possible [1].

687The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Efficiency frontier

Output2 / Input

a

d’
b

Efficiency scoreof DMUddeoHence, v i x ij and u r y rj represent virtual input andoutput. One of the constraints works for virtual input of
DMUk to be “one”. Then virtual output is maximized on
condition that virtual input exceeds that of output foreach DMUs. If virtual output is equal to one, DMUk is
the state of efficient. On the other hand, DMUk is thestate of inefficient if virtual output is less than one. Thus
objective function, namely, virtual output signifies
efficiency score directly.
As a result, efficiency score and weight value areshown by solving linear programming. Then weight thathas value means corresponding input and output
elements are employed in analysis. That is why DEA
enables analyst to know strength of each DMU.

c

θd =

od
od'

Output1 / Input

Fig.1. Efficiency frontier and efficiency score
Fig. 1 demonstrates efficiency frontier andefficiency score. There are five DMUs (a~e) with one
input and two outputs. Each DMU is dotted based onscore that output is divided by input. Hence, DMU
located far from origin is more efficient. Here three
DMUs are the state of efficient and they form efficiency
frontier. DMUc has largest output1 and DMUa haslargest output2. In addition, DMUb has larger output1
and output2 with well-balanced. Characteristics of thesethree DMUs are evaluated well respectively. Thus DEA
is able to reflect strength of DMUs for evaluation.
Therefore, analytic data play an important role to extract
characteristic of each DMU.
Efficiency score of inefficient DMU is calculated bydistance to efficiency frontier that expresses ideal state.
In case of DMUd, ideal state of management is d’. Thenefficiency score θd is calculated by the ratio of
distance from origin to d and d’. In other words,
efficiency score is calculated based on efficient DMUs
among analytical objects.

III. TRADITIOAL METHOD
1. Outline
The previous study focused on data selection formaking intended DMUs efficient [3]. That is becausesome of the analysts are quite familiar with analyzed
DMUs and they can predict result roughly. For
beginners, deciding efficient DMUs they want is helpful
to have reasonability of analytic data elements. Of
course, it is possible to incorporate external information
to the data selection.
Traditional methods need to take inverse procedurecompared with fundamental DEA. It means result thatsome DMUs are efficient is assumed first and then useddata are calculated based on the assumption. Fig. 2
shows stream of traditional method. Dotted line is route
concerning fundamental DEA and procedure consists ofdata collection, analysis and result. However, traditional
method assumes the result in advance and then data
elements are calculated in order to guarantee assumed
result (analyst’s intention).

2. Formulization
When analysis is carried out, linear programming isutilized. Here formulization of CCR model is shown.
Assuming
that
there
are
n
DMUs
( DMU 1 , DMU 2 ,L, DMU n ) with m inputs and s outputs.
DMUk is characterized by inputs ( x1k , x 2 k ,L, x mk ) andoutputs ( y1k , y 2 k ,L, y sk ). Then efficiency score of
DMUk is calculated by following formula.

Datas

Max ∑ u r y rk

Input: x1,x2,x3
Output: y1,y2

r =1
m

s

i=1
m

r=1

s.t. − ∑ v i x ij + ∑ u r y rj ≤ 0 ( j =1,2,L, n )(1)Result
DMUk: efficientFig.2. Procedure of the method∑ v i x ik =1
i=1v i ≥ 0 (i =1,2,L, m), u r ≥ 0 (r =1,2,L,s)

While idea of traditional method is formed, we
consider possible users, namely, analysts who utilize themethods. There are mainly three types of users. Expert
who has knowledge to DMUs, decision maker who

Inputs and outputs are denoted as x ij , y rk whilev i , u r are weights for input and output elements.

©ISAROB 2011

DEA

688

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

belongs to analyzed DMU, and beginners. The merits ofthe methods are mentioned for every type of user.
For experts, they have forecast for result based on
their deep knowledge. That is why they can know
effective data elements to realize their intentional
analysis thanks to traditional methods. So it is effective
for analyzing DMUs. For decision makers, they can findstrengths of their company and competitive one byassuming those two DMUs as efficient. That is to say,management strategy can be planned by DEA since theyknow characteristic in detail for themselves and rival.
For beginners, they are often confused when they decidedata. Then traditional methods can show necessary data
elements for them and support smooth analysis. Thus
various analysts are able to get benefit by utilizing themethod.

Max
s.t.

i=1
m

r=1
s

i=1
m

r=1

(3)

∑ v i x ik =1
i=1v i ≥ 0 (i =1,2,L, m), u r ≥ 0 (r =1,2,L,s)
3. LDS-DEA
As long as data selection relies on TDS-DEA, weneed to care the accuracy of analyst’s intention. That is
because intended DMU surely get the state of efficient
after calculation. At the same time extension of themethod was necessary. It is to analyze DMUs by
employing common elements for intended DMUs.
 The number of DMUs : n The number of intended DMUs : α Combination for choosing two DMUs from
intended DMUs : h ( 1, 2,L, α C 2 ) DMU No. of t-th DMU among intended ones : qt
( t =1,2,L, α )

α C2

m

s

y+
y−
x+
x−
Min ∑ {∑ (d ih+ d ih) + ∑ (d rh+ d rh)}h =1 i=1
r=1
m
s
t
t
s.t. − ∑ v i x iq t + ∑ u r y rq t = 0 ( t =1,2,L, α)
i=1
r=1
m
s
− ∑ v it x ij + ∑ u rt y rj + s tj = 0 ( j =1,2,L, n : j∉t ) ( t =1,2,L, α)
i=1
r=1
m
t
v
x
=
iq
t
1 ( t =1,2,L, α )
∑ i
i=1
(4)
x+
x−
v ik x iq k − v il x iq l + d ih
− d ih=0y+y−u kr y rq k − u lr y rq l + d ih
− d ih=0(i =1,2,L, m), (r =1,2,L,s), (k , l∈t :k < l), ( h =1,2,L, α C 2 )v it ≥ 0, v it ≥ 0 (i =1,2,L, m)
u rt ≥ 0, u tr ≥ 0 ( r =1,2,L,s)(2)

Only slack variable of DMUk is zero and others havevalue. Traditional method enables analyst’s intention to
be reflected strongly with these conditions. And theused elements are found by the weights v i , u r . If
weight has value, corresponding element is used foranalysis. Therefore, weight is a key to know whichelements should be selected. Formula (3) is a linear
programming regarding TDS-DEA. Assume that there
are n DMUs with m inputs and s outputs. {J-k} signifies
set including DMUs except for DMUk. In order torealize concept of TDS-DEA, objective function are
maximized. It means TDS-DEA tries to decrease
efficiency of DMUs except for intended one as much as
possible with keeping intended DMU efficient. Whenmulti DMUs are assumed as efficient, elements arecalculated by repeating analysis for each DMU.

©ISAROB 2011s

− ∑ v i x ik + ∑ u r y rk = 0− ∑ v i x ij + ∑ u r y rj + s j = 0 ( j =1,2,L, n : j ≠ k )2. TDS-DEA
TDS-DEA (Tight Data Selection based on DEA)
calculates data elements that make only intended DMUs
efficient. Hence, it is possible to decrease the number of
elements in analysis. TDS-DEA introduces condition for
slack variable that treat surplus of input and lack of
output. If slack variable has value, the DMU is the stateof inefficient. On the other hand, the DMU is the ideal
state if slack variable is zero. Thus only specific DMUs
can be efficient and others can be inefficient. Hereanalyst would like to make DMUk efficient. Then slack
variable should be satisfied with following conditions;s j = 0 ( j= k)s j > 0 ( j =1,2,L, n : j ≠ k )∑s jj∈{J−k}mIV. UMERICAL EXPERIMET
To confirm the effectiveness, experimental data thathas some features is prepared. There are twenty DMUs
with six inputs and six outputs. The feature is that
DMU1 to DMU6 have strong points in input1, input2,
output1, and output2. DMU7 to DMU12 have strength in
input5, input6, output5, and output6. DMU15 to DMU20
has strength in input3, input4, output3, and output4.
Then TDS-DEA is applied and calculates the data
element to realize analyst’s intention. Table. 1 shows theexperimental data in this study.

689

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 1. Experimental data
Element

Input

Output

x1
x2
x3
x4
x5
x6
y1
y2
y3
y4
y5
y6

DMU1

DMU2

DMU3

0.111
0.778
0.897
0.545
0.311
0.317
0.778
0.625
1
0.351
0.899
0.214

0.222
0.667
0.691
0.364
0.378
0.293
1
0.500
0.188
0.485
0.528
0.827

0.444
0.444
0.918
0.852
0.333
0.610
0.889
0.875
0.800
0.897
0.876
0.459

DMU
DMU10 DMU11

・・・
・・・
・・・
・・・
・・・
・・・
・・・
・・・
・・・
・・・
・・・
・・・
・・・

0.444
0.778
0.804
0.523
0.200
0.244
0.556
0.500
0.350
0.472
0.270
0.663

・・・
・・・
・・・
・・・
・・・
・・・
・・・
・・・
・・・
・・・
・・・
・・・
・・・

0.556
0.333
0.763
0.511
0.244
0.512
0.222
0.375
0.800
0.485
0.506
0.867

DMU18

DMU19

DMU20

0.556
0.778
0.433
1
0.622
0.976
0.243
0.875
0.753
1
0.461
0.337

0.667
0.556
1
0.477
0.711
0.512
0.220
0.500
0.753
0.763
0.360
0.306

0.444
1
0.351
0.273
0.422
0.659
0.060
0.250
0.494
0.371
0.247
0.112

Table 2. Result
Weight
v1
v2
v3
v4
v5
v6
u1
u2
u3
u4
u5
u6

DMU
DMU1
9.000
0
0
0
0
0
0
0
1.000
0
0
0

DMU 2
4.500
0
0
0
0
0
0.464
0
0
0
0
0.648

DMU3
0
0
0
0
3.000
0
0
0
0
1.115
0
0

DMU4
1.446
0.804
0
0
0
0
0
1.143
0
0
0
0

DMU 9
0
0
2.243
0
0
0.626
0
0
0
0
1.000
0

DMU10
0
0
0
0
5.000
0
1.800
0
0
0
0
0

Table. 2 shows the result for the input and output
should be chosen. Shaded area is that each DMU’s
strong elements. According to the result, we find someknowledge through experimentation.
Let us focus on DMU1 to DMU4. They originallyhave strong input1, input2 and then TDS-DEA actuallycalculates those elements. The method signifies input5
is strong for DMU3. That is because DMU3 has larger
input compared with other DMUs and choosing input5
is inevitable for individual efficiency. However, data
selection for output is not enough since obtained result
does not reflect the actual strength DMUs have. This is
the drawback of the traditional method. That is to say, it
is possible to get necessary data roughly but sometimes
it is not reliable and accuracy. In this point, introducing
assurance region method is helpful to improve themethod. Also discriminate approach will have influence
to data selection better.
The traditional method calculates some unsuitableelement for other DMUs though they get desirable
result. Analyst is able to set the analytical data based on
certain reasonability and confidence thanks to themethod. And it is beneficial for not only analysts whoare expert but also analysts who don’t have enough©ISAROB 2011

DMU11
0
0
0
0
4.091
0
0
0
1.250
0
0
0

DMU12
0
0
0
0
0.285
2.854
0
0
0
0
0
1.000

DMU15
0
0
4.042
0
0
0
0
0
0
1.738
0.027
0

DMU16
0
3.000
0
0
0
0
0
0
1.149
0
0
0

DMU17
0
0
0
6.286
0
0
0
0.766
1.542
0
0
0

DMU18
0
0
2.310
0
0
0
0
0
0
1.000
0
0

knowledge regarding evaluated DMUs. if analyst haveproblem for data selection, they just utilize the method
and get the direction for their analytical procedure.

V. COCLUSIOThis paper examines the power of traditional methodfor data selection. The result shows how the method
calculates necessary data. However, it is important toimprove quality of the approach since some of the data
are not calculated well based on analyst’s intention.
Then we consider introducing assurance region method
or discriminate way in order to complement current
method.

REFERECES[1] W. W. Cooper, L. M. Seiford and K. Tone (2007),
Data Envelopment Analysis A comprehensive Text with
Models, Applications, References and DEA-Solver
Software Second Edition. Springer[2] A. Charnes, W. W. Cooper and E. L. Rhodes (1978),
Measuring the Efficiency of Decision Making Units.
European Journal of Operational Research, vol. 2, pp.
429–444[3] A. Naito and S. Aoki (2010), Support for Awarenessof Data Selection in Data Envelopment Analysis. The
2nd International Symposium on Aware Computing(ISAC 2010)

690

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Proposal of recommender system simulator based on small-world model
Ryosuke Saga*, Kouki Okamoto*, Hiroshi Tsuji**, and Kazunori Matsumoto**Kanagawa Institute of Technology, 1030 Shimo-ogino, Atsugi, Kanagawa, Japan**Osaka Prefecture University, 1-1 Gakuen-cho, Nakaku, Sakai, Osaka, Japan(Tel : +81-46-291-3235; Fax :+ 81-46-242-8490)(saga@ic.kanagawa-it.ac.jp)
Abstract: This paper proposes the development of a software simulator that allows a simulator’s users to evaluate
algorithms for recommender systems. This simulator consists of agents, items, Recommender, Controller, and Recorder,
and it locates the agents and allocates the items based on a small-world network. The agent plays a role in a user inthe recommender system and the recommender plays a role in the recommender system. Controller handles the simu
lation flow that (1) Recommender recommends items to agents based on the recommendation algorithm, (2) eachagent evaluates the items based on agents’ rating algorithm using each item’s and agent’s attribute, and (3)Recorder obtains the results of the rating and the evaluation measurement for the recommendation pertaining to such information as precision and recall. This paper discusses the background of proposal and the architecture of this simulator.
Keywords: Recommender System, Multi-agent simulation, evaluation, small-world model.

voting and rating, and access logs, etc. Two types offiltering algorithms are used for dynamic
recommendation:
content-based
filteringandcollaborative filtering [6]. Especially, collaborative
filtering is the most successful algorithms, and its
profile is based on relationships among users or items[7]. It has an advantage wherein collaborative filtering
is applicable for any items because the algorithm does
not need to analyze the content itself. Also, the hybridalgorithm combining collaborative filtering and contentbased filtering [8] has also been developed.
Generally, the recommender system algorithms work
better as the dataset that includes the user’s rating anditem information becomes more massive. However, thealgorithms do not work for a small dataset because thedataset is insufficient for calculating similarity andpredicting items (called cold-start problems).
Therefore, developing the algorithms has various
problems associated with it. The first problem is alimited dataset. To evaluate the algorithms, we need touse various environments by collecting various datasets.
However, collecting the various datasets is difficult
because we generally do not make use of recommendersystems and do not have the data source. Therefore,
many researchers have utilized limited dataset such as
MovieLens Dataset and EachMovie Dataset. theevaluation measurement may change according to thegoals of the algorithm. The algorithm is developed forspecified goals, and in order to evaluate the algorithm,
the proper evaluation measurements and data set areneeded [3]. Also, each method has strong/weak points,

I. INTRODUCTION
Recommender systems have been used for several
applications and systems such as news sites,
information sharing system, e-commerce and soon[1][2][3]. The systems offer benefits to consumer anditem providers. These recommender systems help
consumers in particular acquire new as well as
preferable items and users can expect effective
acquisition of the information. Therefore, an appropriate
algorithm is needed for the recommender system.
Several researchers have proposed and developed
many algorithms since collaborative filtering, one of themost successful technologies for recommender systems,
was introduced and attracted many attentions[4][5][6][7]. However, developing and applying thealgorithm of collaborative filtering are difficult. One
reason is based on the difficulty of evaluating thesealgorithms. For example, the algorithms were developed
for any purposes and have validated specified andlimited datasets in many cases. Therefore, the generalityof the algorithms is not clear. Another reason for thedifficulty is that the validation of the algorithm needs
massive dataset.
Therefore, we developed a multi-agent-likesimulator for evaluating the collaborative filtering and it
is described in this paper.

II. Motivation
Recommender systems aim to recommend
preferable items to users from user profile [1][2][3]. The
profile is constructed by analyzing the content, user’s©ISAROB 2011

691

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Simulator
Controller
Input parameters

Agent

Graph Library

Preferable
Attributes

Item
Items

Attributes

Parameters
Rating Algorithm

Rate recommended items

Recommendation

Recommender
Output Result

Evaluation

Filtering
Algorithms

RatingInformation

RecorderUser & Item
Information

Fig.1. Simulator architecture
2.

Controller initializes the status of the agents anditems and configures the simulator based on theparameters.
3. Recommender calculates the user similarity andrecommends items to agents.
4. The agents vote on the rating of recommended
items and update the status.
5. Recorder compiles the result of recommendationand evaluates the recommendation using several
measurements such as MAE, recall, precision,novelty, diversity, and discovery [9][10].
6. Controller updates agent status and preference.
The simulator regards step 3 to step 6 as one turn,and it goes through the steps and iterates the turns.

and if we apply the algorithms for other goal, we cannot
easily judge whether any of the algorithms are suitable.
In order to identify the suitable algorithm, to compare
the algorithms is useful; however, an experiment with alimited dataset and with different goals is difficult.
Therefore, we build the simulator to enable thecomparison of the algorithms. The requirements/goalsof simulator are defined as the followings.
1. The simulator can build the evaluation environment
for the recommender system.
2. The simulator can compare the filtering algorithms
of collaborative filtering and content-based filtering.
3. The simulator can output the results of evaluations
to compare the filtering algorithms.

III. OVERVIEW OF RECOMMENDERSYSTEM SIMULATOR

IV. ARCHITECHTURE OF SIMULATOR
1. Agents, Items, and Ratings
A. Definition of attributes
The key point of modeling recommender systems is thepreferences of agents for items. This paper assumes thatthe agents and items have many attributes. Here, we
assume that the agents have specified preferable genres
and domain, and in order to express the assumption, thesimulator lets agents have a positive real number for all
attributes for their degree of preference. The simulator
also defines the attributes whose degree of preference isover the preference threshold as preferable attributes. In
contrast, an item has the 0/1 flags for each attribute.
B. Rating items
In this simulator, the agents and Recommender
evaluate each item. For the agents, the rating rij of anThis simulator consists of agents, items,
Recommender, Controller, and Recorder as shown in
Fig. 1. In this simulator, a simulator user gives thenumber of agents, items, thresholds, and algorithms asparameters. Agent acts as the user of recommendersystems, and the algorithm of collaborative filtering ismodeled into Recommender. Recommender has
information on agents and items’ ratings for each user.
Controller receives parameters from the simulator userand handles the simulator such as initialization,
progression, and suspension. Controller accepts not only
the number of agents and items, but also thresholds forthe simulator environment, preference, and agent status.
The simulation steps are as follows:
1. The simulator user inputs parameters.

©ISAROB 2011

692

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Here, I is the set of recommended items from
Recommender, rij is the rating of item j by agent i which
is used for rating algorithm, ranki is the rank of the item
i, and ni is 0 if i has already been recommended;otherwise it is 1.

item j for an agent i is generally evaluated by theformulation,
rij = A(Ui,Ij, ui)
Here Ui is the preferable attribute set of agent i. Also, Ij
is the attribute set of Item j whose value is 1. ui is theinformation about agent. Also, we call the function an
rating algorithm. For an example of rating algorithm,
we can configure it as follows∑
max

2. Building Simulator environment
The configuration of the simulator environment is
one of the most important steps in the simulation
process because an inappropriate environment leads toinappropriate and wasteful result.
The simulator environment is built duringinitialization and is updated after the recommendation isdone. In order to configure the proper simulator
environment, we utilize the structural (topological)
features of the recommender system.
A. Initialize simulator environment from small world
network.
Generally, communities tend to follow complex
networks and, according to several references, they findthat the networks tend to be small-world networks. Here,a small world is a phenomenon in a real world network,and the model has several features such as stability andcompression of network. The structure was first
formulated by Watts and Strogatz [10]. The structureappears in several networks, and the trend also appears
in recommender system. Therefore, we create theenvironment based on the small-world network.
If we restrict the community to the recommendersystem, the network of the recommender system isscale-free network like it is in References [11][12][13].
Therefore, we regard an agent as a node and initialize
agents and items according to generating a scare-free
network. The algorithm in detail is as follows, given n
agents and m items as simulation parameters from theuser,
1. Create the k-core clique in order to generate thescale-free network, and allocate the C0 common items
among k nodes.
2. Add an agent (ai (i=1, 2, …, n)) to the network
according to the algorithm of BA(Barabasi-Albert)
model which is a scale-free network, and allocate the Cj
common items to the clique including ai. Note that  Cj=m.
3. Iterate step 2 by allocating an.
4. Allocate the attributes of items to agents who haveitems in common.
5. Let Recommender calculate the similarity between
the agents for recommendation.

where |E| indicates the number of the elements of set E,
u is the elements of attributes of Ui corresponding to theindex ofI and max(Ui) indicates the maximum ofUi, which is given by users as one of parameters.
On the other hand, Recommender, which is theimplementation of the filtering algorithm, predicts theratings of the items and recommends high-rated items to
the agents. For example, the predicted rating of an item
for the agent is generally shown in the following
formulation on collaborative filtering:
rij = R( sim(i, u) ,R),where rij is the rating of item j for user i, sim(u,v)
indicates the similarity between user/item u anduser/item v among a set of user U who evaluate the itemj, and R shows the rating information. For example,
GroupLens[4], which is a representative system using
collaborative filtering, has the following formula.

rij  ri uU( sim(u, i)(ruj  ru ))uUsim(u, i)is the average rating when a user i
In this formula,
has already voted.
C. Status of Agents
One of the problems that requires attention is thetiresome status. This simulator allows agents to change
the status in order to express the degree to which anagent is tiresome. In this simulator, agents have two
statuses, normal and tiresome, and the trigger ofchanging status occurs in recommendation. The
measurement for tiresome is calculated concretely by
the evaluation formula, and when the measurement isless than the tiresome threshold, the agent changes thestatus to tiresome, otherwise it remains normal. The
simulator implements the following formula, which isan example of the evaluation formula;©ISAROB 2011

693

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011[2] Adomavicius, G. and Tuzhilin, A., (2005). Toward
the Next Generation of Recommender Systems: A
Survey of the State-of-the-Art and Possible Extensions.
IEEE Transactions on Knowledge and Data Engineering,17(6), 734-749.
[3] Saga. R., Tsuji, H, and Onoda, J., 2005. Agent
System for Notifying Hotel Room Reservation
Alternatives, Proc. of 11th International Conference on
Human Computer Interaction (HCII2005), Vol. 5, pp.
1-10, July 2005[4]Resnick, P. et al., (1994). GroupLens: an open
architecture for collaborative filtering of netnews. In
Proceedings of the 1994 ACM conference on Computer
supported cooperative work.
Chapel Hill, North
Carolina, United States: ACM, pp. 175-186.
[5]Sarwar, B. et al., (2001). Item-based Collaborative
Filtering Recommendation Algorithms. Proc. 10th
International Conference on the World Wide Web, 285-295.
[6]Pazzani, M. (1999). A Framework for Collaborative,
Content-Based and Demographic Filtering. Artificial
Intelligence Review. 13(5-6) 393-408.
[7]Linden, G., Smith, B. and York, J., (2003).
Amazon.com
recommendations:
item-to-item
collaborative filtering. Internet Computing, IEEE, 7(1),76-80.
[8] Claypool, M., Gokhale, A., and Miranda, T. (1999).
Combining content-based and collaborative filters in an
online newspaper. In Proceedings of the SIGIR-99
workshop on recommender systems: algorithms andevaluation.
[9]Herlocker, J.L. et al., (2004). Evaluating
collaborative filtering recommender systems. ACM
TRANSACTIONS ON INFORMATION SYSTEMS,
22(1), 5--53.
[10]Watts, D.J. and Strogatz, S.H. (1998). Collective
dynamics of 'small-world' networks. Nature 393 (6684):
409-410.
[11]Albert, R. and Barabasi, A., (2002). Statisticalmechanics of complex networks. Reviews of Modern
Physics, 74, 47-97.
[12]Martin-Buldú, J., Cano, P., Koppenberger, M.,
Almendral, J., Boccaletti, S. (2007). The complex
network of musical tastes. New Journal of Physics. 9,[13] Cano, P., Celma, O., Koppenberger, M., MartinBuldú, J. (2006).
The Topology of music
recommendation networks. Chaos An Interdisciplinary
Journal of Nonlinear Science. 16,[14] Tabei, Y., Uno, T., Sugiyama, M., and Tsuda,K.,(2010). Single versus Multiple Sorting in All Pairs
Similarity Search, The 2nd Asian Conference on
Machine Learning (ACML)[15] Breese, J. S., Heckerman, D., and Kadie, C., (1998).
Empirical analysis of predictive algorithms forcollaborative filtering. In Proceedings of the fourteenth
conference on uncertainty in artificial intelligence,
Madison, Wisconsin.

B. Update simulator environment
The simulator needs to be update because the agents
may have new interests and because Recommender
identifies the new relationships between agents becauseof the rating of recommended items by agents. In thissimulator, the agent’s preferences are updated by asimple approach. When an agent regards an item as thepreferable items, then we can assume naturally that the
impression for the item is good. Therefore, when rij,,which is the rating of item j by agent i, is over the
preferable item threshold, then agent regards the item as
the preferable item and the simulator adds a constantvalue to the attribute of agents that corresponds to thoseof the item.
On the other hand, Recommender has trouble
indentifying new relationships between agents because
the cost of calculation of similarity is normally veryhigh. In order to constrict the complexity of calculating
similarity, SketchSort, which is software for all pairs
similarity search, is useful [14]. The basic idea of
SketchSort is to combine Locality Sensitive Hashing(LSH) and Multiple Sorting Method (MSM). SketchSort
takes as an input data points and outputs approximate
neighbor pairs within a distance. SketchSort is so quickthat the cost of calculating the similarity can be lower.
Here, In order to uses the SketchSort, the rating data ofagent is recorded by the matrix where the row indicates
items and column indicates the agents. However, mostagents have not evaluated many items yet so that thesimulator cannot build the matrix. Therefore, using
default voting [15] and filling the ratings of items whicha agent has not evaluated yet, we build the matrix andutilize SketchSort for similarity calculation.

V. CONCLUSION
This paper has proposed a simulator to allow a user
to evaluate the algorithms for recommender systems. In
order to evaluate the algorithms, the simulator builds anenvironment of a virtual recommender system based on
a complex network model from parameters;
Recommender makes recommendations to agents, andthe simulator evaluates and outputs the results though
Recorder. In future work, we will validate the simulator
by gaining a resurgence of the phenomenon ofcollaborative filtering, and we will test the usability.

REFERENCES[1] Resnick, P. and Varian, H.R., (1997). Recommendersystems. Commun. ACM, 40(3), 56-58.

©ISAROB 2011694

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Construction and analysis of purchase factor model by using creativity
methodKodai Kitami, Ryosuke Saga, Kazunori Matsumoto
Kanagawa Institute of Technology, 1030 Shimoogino, Atsugi, Kanagawa, Japan(Tel :+81-46-291-3235 ; Fax :+81-46-242-8490 )(saga@ic.kanagawa-it.ac.jp)

Abstract: This paper describes a purchase factor analysis for best-selling software games. Japanese game industry has
grown remarkably since 1983. Developers have to produce best sellers in order to get a profit. However, the concrete
factors for a best-selling game have not yet been qualified. Structural equation modeling (SEM) seems to have thebeneficial effect on causality analysis. However, the SEM results may lack the reliability because the model isconstructed based on the analyzers’ subjective assumptions. We need to construct a factor model for solving thisproblem by extracting the purchase factor from diverse viewpoints. Consequently, we use the KJ method, which is oneof the creativity methods, to do just that. There are four steps in the process for analyzing the factor model from theresults of KJ method: (1) extract the factors from the KJ method results, (2) refine the model by integrating
conceptualistic meanings, (3) assign the collected data to the model, and (4) construct and analyze the model byindentifying the variables. From the result of our analysis of the model, we could qualify the factors of best-sellinggames by using an objective purchase factor model that was mainly constructed of the “Contents”, “Advertisement”,
and “Brand”.
Keywords: creativity method, KJ method, Structural Equation Modeling, marketing, purchase factor analysis.

information about the games and the consumers’
expectation. However, this result may not be able toextract substantial paths or latent factors, and thus, thereliability and relevance of this model may beinsufficient. The reason for this is derived from thesubjective assumptions of the analyzers.
To solve this problem, we propose a process forconstructing a SEM model from the results when using
the creativity method and that analyzes the consumers’purchasing factor. In this paper, we use a bottom-up
type KJ method. We outline the investigative items andapply them to a factor model composed of the factors
when using the KJ method.
The rest of this paper is presented as follows. We
describe the problem with the approach when using
SEM in Chapter 2. Chapter 3 explains the process ofconstructing a model from the KJ method. Chapter 4
explains a model constructed according to the process
and the result of our analysis. The results are theconsidered and discussed in Chapter 5. Finally, Chapter
6 concludes with a summary of the key points.

I. INTRODUCTION
The Japanese game industry has grown remarkably
since 1983 beginning with the “Family Computer”
product released by Nintendo Co. The size of the market
in 2007 was twice as large as that in 1987, and now
software games play a central role in the industry [1].
However, the running costs of the developers havebecome compressed, and there is more absorbability
between the makers as a result of the ever-increasing
appreciation for the development of new and better
software games. Therefore, the developers are forced to
produce best sellers in order to make a profit. However,the concrete factors surrounding the development of abest seller and the latent factors have yet to be qualified.
There are many analysis methods for estimating theconsumers’ purchase factor, such as Factor Analysis [2~3],
Regression [4], and Baysian Modeling [5]. In particular,
Structural Equation Modeling (SEM) [6] has a more
beneficial effect on a causality analysis because thismethod can express the complex causal relationshipbetween the observational and latent factors.
We have analyzed the factors by using SEM in orderto qualify the factors surrounding best-selling software
games. As a result of our analysis, we found that amodel constructed by Kitami [7] proved that theconsumers’ purchasing factor was affected by the basic©ISAROB 2011

II. PROBLEM WITH SEM AND SOLUTION
1. Structural Equation Modeling
SEM is an analysis method that quantitatively
evaluates the causality relationship between the more

695

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

remarkable variables in statistic data and causality
information, and has been used in causality analysis
since the 1950’s in many domains, such as economics
and social science [8]. SEM can visualize and quantify
the complex causality relationships between variables.
The formula for SEM is as follows.

xi =∑a x + u ,ik ki

i = 1,..., nseller by using the KJ method, which is one of thecreativity methods.
3. Suggestion of model construction by using
creativity method
The creativity method is a systematic method thatthinks out and arranges many ideas in order tocreatively solve a technical problem. In this paper, we
systematized the consumers’ opinions by using the KJ
method. The KJ method is a method used for
information integration and can effectively organize
innovative ideas by arranging and refining segmentizedideas [9]. This method appropriates a causality analysis
because it can express the causal relationship between
the factors by using arrows. We expect the following
merits from using the KJ method.
(1) A subjective model constructed of diverse opinions.
(2) We can construct a model that has explanation
power.
(3) A causal relationship model is expressed.
In this paper, we construct the SEM model based on
the completed chart of the KJ method.
However, problems arise when constructing a factor
model from the KJ method results. The first problem isthe way the factors are decided. If we assume all thegroups to be factors, there is a possibility that a massive
latent factor will be embedded in the model. So, there
are problems with the goodness of fit and the relevanceof model. The second problem is the difference of
expression. There is a possibility that constructing amodel is difficult because the cards are written in a freeform language so that cards with similar meanings arefound throughout the model. Therefore, it is necessary
to construct a model while solving these problems.

(1)

xk ∈ paixi is a set of variables are considered direct factors.
pai corresponds to the variables that have coefficientsthat are not 0 on the right side. Also, ui shows the errorthat is not expressed. SEM presents the strength of thecorrelation and the covariance as the path coefficient.
Also, if the path coefficient between variables is large,
we assume the relationship between the variables isstrong.
2. Problem of approach by SEM
SEM has some advantages as stated above, and is aneffective method in causality analysis. However SEM
has the following problems.
(1) Experience needed: The analyzer in a targeted
analysis when using SEM needs a lot of priorknowledge, because the construction of the model
is only exploratory if the analyzer lacks sufficientknowledge, which may lead to bad results andcontradictions.
(2) The construction of model is subjective: Even
though the analyzer has sufficient knowledge
about the target of analysis, there is the possibility
that the objectivity of model is lacking.
(3) The accuracy is more weighted than the
explanatory power when constructing the model:
As previously noted, the accuracy is exploratorywhen the analysis model is not clarified. Therefore,
there is a possibility that the model which has low
explanatory power is constructed because theanalyzer too achieves the accuracy.
In order to solve these problems, we need toconstruct a SEM model that is objective and has a broad
range of explanatory power. Therefore, we constructed apurchase factor model and clarified the factors for a best

III. MODELING PROCESS FROM KJMETHOD
After experimenting with the KJ method, weconstruct a SEM model that contains the four following
processes (Fig. 1).
(1) Extract factors from the KJ method results.
(2) Refine graph by integrating expressions.

A’
BGM

Sound

A

Sound

(a) Run KJ method(b) Extract factors(c) Refine graph(d) Collect data and collate labels to data

Fig.1 The flow of construction of purchase factor model©ISAROB 2011696(e) Identify variables and analyze SEM

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

When constructing the model, we have to identify
the latent and observational variables. In this paper, we
assume the factors that cannot collate with theobservational data to be latently variable. In addition,
we assume the factors and the cards that can collatewith the observational data to be observational variables,
and we construct the model excluding cards that cannot
collate with the observational data.

IV. CASE STUDY

Fig.2

1. Target of experiment
We constructed a purchase factor model from theconsumers’ opinions by using the process proposed in
Chapter 3. In this paper, we ran the KJ method targeting
five university students as our case study. In thisexperiment, the theme was set to “the factors for bestselling software games”.
The target data for software games included 2381
products that were evaluated by consumers in
PlayStation mk2 [10]. There were 17 data attributes(maker name, platform, genre, price, rating of target age,
publish date, the number of players, game rank, thegame criterion (difficulty, originality, graphic, sound,
excitement and amenity), the number of reviews, thedegree of satisfaction, and comprehensive evaluation).
We also used R 2.10 [11] when analyzing the factors in
SEM.

Completed SEM model

Table.1 Fit indices of model
Goodness-of-fit index
Adjusted goodness-of-fit index
RMSEA index
SRMR
BIC

0.858
0.784
0.118
0.100
2061.4(3) Collate labels to data.
(4) Identify variables and analyze SEM.
The processes for each of the above listed step are as
follows.
(1) Extract factors: First, we identify the factor
settings for the threshold of the degree and exclude
the cards and arrows that adjoin the factors. By
using process, we can compress the model
excluding the cards and arrows that barely relate to
the final factor model.
(2) Refine graph: Second, we refine the model
conceptually by integrating the expressions among
the cards. In this step, we integrate cards thatmutually look alike and exclude cards and arrows
that barely relate to the final model.
(3) Collate labels to data: After the above processes, itis necessary to collate the labels to the analysis
data. In addition, as much of the data in which theconceptual meanings are approximated with thecards and factors is as collected as possible
regardless if its structured or semi-structured data.
(4) Identify variables and analyze SEM: Finally, we
identify the variables and analyze the factors for abest seller based on the graph completed by using
the above mentioned processes.

©ISAROB 2011

2. Experiment results
The KJ method results showed that the model
consisted of 63 cards and 73 arrows. According to theprocess proposed in Chapter 3, we set threshold to fourand constructed a SEM model consisting of five latentand 15 observational variables using this process.
The results from analyzing the completed model are
shown in Fig. 2. The fit indices of the model are listed
in Table. 1. The Goodness-of-fit index (GFI) of thismodel was calculated to be 0.858 and was unable toachieve a rough standard of 0.9. However, we could
express part of the consumers’ purchase factor [12].
This factor model was assumed to consist of thecontents of the games, the maker brand, andadvertisement because their degrees were the largest.
Next, in the calculated path coefficients, the contents
more relatively strongly influenced the “Sound”,“Excite”, and “Originality” than “Graphics”,“Difficulty”, and “Playnum” (the number of players).
“Brand” had the high influence on the “Advertisement”(17.135) and “Contents” (18.294). In addition, “Brand”

697

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

VI. CONCLUSION

was strongly affected by the “Graphics” (1.189).
“Advertisement” showed a strong relationship with
Multimedia (1.099).

We propose a construction process of a consumers’purchase factor model using a creativity method, andclarified the factors surrounding the best seller ofsoftware games. We constructed a model by using the
KJ method, which is one of the creativity methods, tosolve the problems with SEM concerning the objectivityof the model and the construal difficulty cased byindividual differences.
The results from constructing a model and analyzing
the factors concerning the best seller when using the KJ
method when the theme was set to “the factor of thebest-selling software games” showed a consumers’purchase factor that mainly consisted of the contents ofthe games and the brands of the makers.
However, the KJ method has some problems in thata difference in the height of an interest for a theme andthe degree of knowledge bias are caused by individual
differences. Therefore, some issues with the overall
reliability and adequacy of the factor model may remain
if we construct a factor model based on only one
experiment. Therefore, we will construct a factor modelby integrating multiple results from our experiments.
We can construct a factor model that has a higher levelof objectivity by improving this process.

V. CONSIDERATION
We understood that the consumers’ purchase factor
mainly depends on the contents of games, maker brandand advertisement from the results of our analysis.
“Contents” positively influenced “Sound”, “Graphics”,
and “Originality”. The reason for this is that theconsumers want novel games that can be played for
longer periods of time, and they want an aural
allurement that is yielded by next-generation hardware.
Next, “Brand” highly influenced “Advertisement” and“Contents”. Therefore, the maker brand for consumers
consists of the degree of advertisement and quality of
the contents. We also found that “Advertisement” wasaffected by “Environment” and has a strong relationshipwith “Multimedia”. So, we learned that the amount andquality of current information greatly influences theconsumers’ purchase factor.
Five male university students ran the KJ method forthis research paper. They had a lot of experience playing
games, so the KJ method results highly relevant and the“Brand” and “Advertisement” were taken into
consideration. However, the KJ method has a problem
in that the level of objectivity may be lacking due to theage-bracket of the test subjects, their gender, and thedegree of knowledge for the set theme. Therefore, itwould be more effective for us to run multiple KJ
methods targeting different test subjects and combine
these results to construct a more objective factor modelthat contained broader view points.
Next, we look into consideration the model
construction process. First, for the extraction method of
the factors, we set threshold for the degree and assumed
the cards that were over the threshold to be a factor. In
addition, we excluded 14 cards and 14 arrows that did
not adjoin the factor. If we extract too many factors,
there is a possibility that a model identification problem
may occur because a lot of latent factors are set in themodel. Therefore, when we set the threshold, it isimportant to have the cogitate attributes of the data andthe extracted factors. In integrating the expressions
between cards, we integrated the cards by confirming
their meanings one by one in order to avoid wrongly
classifying them during the automated process.

©ISAROB 2011

REFERENCES[1] Famitsu.com, http://www.famitsu.com/[2] H.Yanai, K.Shigemasu and S.Maekawa et al (1990),
Factor analysis –theory and process (in Japanese),
Asakura Publishing Co[3] S.Muto (1995), Handbook of statistics analysis (inJapanese), Asakura Publishing Co[4] D. Freedman, R. Pisani and R. Purves (2007),
Statistics, W. W. Norton & Company[5] N.Terui (2008), Marketing analysis by Bayesianmodeling (in Japanese), Tokyo Denki University Press[6] H.Toyoda (1998), Structural Equation Modeling[Beginner ver.] (in Japanese), Asakura Publishing Co[7] K. Kitami and R. Saga (2010), Causality Analysis
for Best Seller of Software Game by Regression and
Structural Equation Modeling, 2010 IEEE International
Conference on Systems, Man, and Cybernetics[8] J. Pearl: Causality (2000), Models, Reasoning, andInference, Cambridge University press[9] D. Silverstein, P. Samuel and N. Decarlo (2009),
The innovator’s toolkit, Breakthrough Management
Group International[10] PlayStation mk2, http://www.psmk2.net/[11] The R project for Statistical Computing,
http://cran.r-project.org/[12] H.Toyoda (2007), Structural Equation Modeling[Amos ver.] (in Japanese), TokyoTosho Co

698

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Developing a monitoring psychological stress Index system via
photoplethysmography
112

Jiann-Shing Shieh , Yu-Ren Chiou , Shou-Zen Fan
1

2

Department of Mechanical Engineering, Yuan Ze University, Taiwan
Department of Anesthesiology, College of Medicine, National Taiwan University, Taiwan(Tel :886-3-463-8800 ext. 2470; Fax : 886-3-455-8013)(jsshieh@saturn.yzu.edu.tw)

Abstract: The purpose of this paper is to assess the psychological stress index (PSI) by using oxyhemoglobin saturation
by SpO2 (i.e., plethysmograph (PPG) signal), which is measured easily and conveniently. We use plethysmographamplitude (PPGA) and heart beat interval (HBI) extracted from SpO2 waveform to obtain the stress index and quantify
it from 0 to 100. Also, the respiration rate can be extracted from heart rate interval according to our previous research.
Therefore, the PPG signal can display heart rate, SpO2 waveform, SpO2 concentration, respiration rate andpsychological stress index. This methodology has been tested in 9 volunteers under English presentation stress for
Taiwanese students. The experiment in 30 min for each student was separated into three stages: preparation,
presentation & discussion, and relaxation. The PSI values of these three stages are 49±10, 60±11, and 56±10,respectively. The results have been shown very successful for PSI value changing from low, high, and low during
these three stages. In the near future, we try to implement this system into robotic wheelchair in order to monitor thisPSI of elderly at nursing home to evaluate their psychological condition based on this method.
Keywords: Photoplethysmograph, pulse oximeter, plethysmograph amplitude (PPGA), heart beat interval (HBI),
psychological stress index (PSI).

I. INTRODUCTION

In order to monitor physical condition continuously,
people usually need to wear a lot of equipment. Basedon the patient's condition, they may have many
physiological signals that need to be monitored, likeelectrocardiography (ECG), pulse oximeter (SpO2),
respiration, etc. When people have many signals thatneed to measure, they wear multiple devices.
Considering patients only have limited ability,
complicated physiological monitoring systems areinconvenient to them. If people only wear one device tocapture one signal, with other signals derived by this, it
can get comprehensive physiological information easily.

In society, there are more and more factors that
affect people's health. Recently, a lot of people haveexcessive stress, which leads to physical disease [1].
There are many diseases that occur due to excessive
stress. Psychological stress leads to physiologicalchanges that help people cope with unexpected
situations and critical incidents. But frequent, strong,and sustained stress will create an adverse impact on the
body. In the past, there are several famous scholars who
research stress. One is Walter Cannon, who proposed“Homeostasis”[2]. He thought stress affects an
individual’s balance. If the body’s system cannot keepbalance, the person will “flight or fight”. This response
from epinephrine is trying to restore a stable body. The
other is Hans Selye, who found that stress response is a
“General Adaptation Syndrome” [3]. When feelingstress, the body will go through alarm, resistance andexhaustion where Glucocorticoid is playing a vital rolein this part. Both stress responses were similar no matter
where the stressors come. When people suddenly have athreatening situation or emotional stimuli, there is anincrease in heartbeat, blood pressure, adrenaline etc.
[4,5]. Hence, a lot of researchers try to measurephysiological signals to represent the stress.

©ISAROB 2011

In 2007, Huiku et al. [6] developed a method toassess stress index during surgical operations. The
experiment in the paper is to observe the effect ofvarious physiological signals such as plethysmographamplitude (PPGA), heart beat interval (HBI), pulse
transit time (PTT), response entropy (RE) when apatient suffers two opposite stimulus, incision andanesthetic drugs. They point out PPGA and HBI havethe most remarkable correlation with stress andestablish an equation to calculate surgical stress index(SSI). We analyzed the psychological condition basedon this method.

699

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

respiration rate. More details can be seen in ourprevious study [9].

Hence, the purpose of this paper is to assess thepsychological stress index (PSI) by using
oxyhemoglobin saturation by SpO2, which is measured
easily and conveniently. We use PPGA and HBI
extracted from SpO2 to obtain the stress index andquantify it from 0 to 100. Also, the respiration rate canbe extracted from heart rate interval according to ourprevious research. Therefore, the PPG signal candisplay heart rate, SpO2 waveform, SpO2 concentration,
respiration rate and psychological stress index.

4.

Psychological stress index
Surgical stress index (SSI) [10, 11] has been used insurgery. It can be used to determine a patient's pain
during surgery. SSI values near 100 corresponding to ahigh stress level, and values near zero corresponding toa low stress level. Firstly, we need to get PPG amplitude
and the heartbeat intervals. Secondly, the standard
deviations of PPG amplitude and heart rate variability
are needed to calculate for normalization as shown in
Fig. 3 (a). Finally, we can calculate the PSI value
according to the following equation (2) as shown in Fig.
3 (b).

II. METHODOLOGY
1.

2.

3.

Photoplethysmography
A plethysmograph sensor is a medical
instrument used mainly to determine variations inoxygen saturation. PPG [7] selects a skin area andmeasures the optical characteristics of that region.
Usually we send red and infrared light through theskin and blood vessels. The components of blood
Hb (reduced hemoglobin) and HbO (oxygenatedhemoglobin) [8] will have different degrees ofabsorption. Therefore, the amount of oxygen in the
blood can be measured due to the different
absorption of red and infrared light through the skinand blood vessels.

PSI= 100-(0.7 × PPGAnorm - 0.3 × HBInorm)…(2)where PPGAnorm is normalized continuous blood
oxygen pulse amplitude and HBInorm is normalizedheart beat interval.
We tried to use this algorithm to calculate
psychological stress index (PSI), but the volunteer did
not lay on an operating table during surgery. We
designed a pressure environment and measured thephysiological signals during this period. We canmeasure the signal and used the equation (2) to calculate
PSI.

Calculating heart rate from PPG
We mainly used maximum gradient to detect thepeak point of SpO2 signal that is similar to the R-R
interval in ECG signals as shown in Fig. 1. The first
derivative was calculated at each point of SpO2 in
the following equation (1):Y(n)=X(n+1)-X(n)(1)
If the gradient was larger than 60, we marked the X(n) point. Then, we found the local initial climbing
point. According to this point, we took the front andback 30 points as a domain and found the
maximum and minimum points. Then, PPGA is theamplitude between adjacent peak point and footpoint (Fig. 2). And, HBI is the time interval
between adjacent peak points (Fig. 2) that is similar
to the R-R interval in ECG signals so the heart rateis easily obtained from this P-P interval.

III. RESULTS
This study has been tested in 9 volunteers underEnglish presentation stress for Taiwanese students. They
are not use English as their mother language and asked
to discuss their research with two professors in English.
The experiment in 30 min for each student was
separated into three stages: preparation, presentation &discussion, and relaxation as shown in Fig. 4. Therefore,
the PPG signal can display heart rate, SpO2 waveform,
SpO2 concentration, respiration rate and psychologicalstress index as shown in Fig. 5. The PSI values from 9
volunteers of these three stages are 49±10, 60±11, and56±10, respectively as shown in Table 1. The results
have been shown very successful for PSI valuechanging from low, high, and low during these threestages.

Calculating respiration rate from PPG
The calculation of HBI from SpO2 signal inprevious section can be applied to calculate the©ISAROB 2011

700

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

IV. CONCLUSION
In conclusion, we confirm a new method to assess
the psychological stress index by using SpO2 signals. At
the moment, the method is limited to normal people andshort-term condition. The method is particularly usefulin situation where the SpO2 is the only available
information source. Using a pulse oximeter, the user canobtain both psychological and physiological information
at the same time.

(a)

REFERENCES(b)
Fig. 1. Definition of peak to peak signals (a) from PPG
signal (b) from ECG signal[1] Agid O, Kohn Y, Lerer B (2000), Environmental
stress and psychiatric illness. Biomedicine &Pharmacotherapy 54:135-141.
[2] Cannon W (1935), Stresses and strains of
homeostasis. The American Journal of the Medical
Sciences 189(1):1-25.
[3] Wortsman J, Frank S, Cryer PE (1984),
Adrenomedullary
response to maximal stress inhumans. The American journal of medicine 77:779-784.
[4] Crype PE (1980), Physiology and pathophysiologyof the human sympathoadrenal neuroendocrine system.
The New England journal of medicine 303:436-444.
[5] Munck A, Guyer PM, Holbrook NJ (1984),
Physiological functions of glucocorticoids in stress andtheir relation to pharmacological actions. The Endocrine
Society 5:25-44.
[6] Huiku M, Uutela K, van Gils M, Korhonen I,
Kymäläinen M, Meriläinen P, Paloheimo M, Rantanen
M, Takala P, Viertiö-Oja H, Yli-Hankala A (2007),
Assessment of surgical stress during general anaesthesia.
Br J Anaesth 98: 447–55.
[7] Allen J (2007), Photoplethysmography and its
application in clinical physiological measurement,
Physiol Meas 28:1-39.
[8] Sinex J (1999), Pulse oximetry: Principles andlimitations, The American Journal of Emergency
Medicine, vol. 17, 1:59-66.
[9] Singhathip R, Si-Hui Yang SH, Abbod M, Yeh RG,
Jiann-Shing Shieh JS (2010), Extracting Respiration
Rate from Raw ECG Signals. Biomedical Engineering Applications, Basis & Communications 22(4): 307314.
[10] Wennervirta J, Hynynen M, Koivusalo A.M,
Uutela K, Huiku M. and Vakkuri A (2008), Surgicalstress index as a measure of nociception/antinociception
balanceduringgeneralanesthesia,
Acta
Anaesthesiologica Scandinavica 52(8):1038-1045.
[11] Ahonen J, Jokela R, Uutela K, Huiku M (2007), Surgical stress index reflects surgical stress ingynaecological laparoscopic day-case surgery Br J
Anaesth. 98(4):456-61.

©ISAROB 2011

Fig. 2. Definition of PPGA and HBI
Table 1. The PSI in the experiment during skype
meeting
BeforeCase

701

DuringAfter

Presentation Presentation Presentation

A

46 ±1357±11

56 ±20B

33 ±1052 ±1762 ±12

C

59 ±1068 ±1057 ±9

D60 ±12

57 ±1657 ±14

E

45±12

57±11

51±2

F

60±10

60±10

64±10

G

34±10

62±11

53±8

H

N/A

62±12

61±6

I
Average
±SD

59±12

65±9

48±10

49±10

60±11

56±10

0.01
0

0
100
200
300
Individual distribution of PPGA

0.02
0.01
0

0
100
200
300
Combined distribution of PPGA

0.02
0.01
0

0

100
200
Persentage of PPGA300

Group distribution of HR
0.04

PPGA

0.02
0

0

200
400
600
800
Individual distribution of HR

0

150

2000

100

0

0

200
400
600
800
Combined distribution of HR

0

1

1

0.5

0.5

0

0

100

200

300

20

40
60
time (min)Normalized PPGA

80

50

1

1
0.5
0

20

40
60
time (min)
PSI

80

0

20

40
60
time (min)

80

0

0

0

20
40
60
time (min)Normalized Heart Rate

20

40
60
time (min)

100

0

200
400
600
Persentage of HR

800

50
0

0

0

0.5
0

0.05

Heart Rate

4000

0.05

index

Group distribution of PPGA
0.02

percentage(%) percentage(%) percentage(%)

percentage(%) percentage(%) percentage(%)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

0

200

400

600

800

(a)
(b)
Fig. 3 An example of (a) the process of normalization and (b) the variety of PSI in experimentFig. 4. The procedure of experimentFig. 5. An example of volunteer’s heart rate, SpO2 waveform, SpO2 concentration,
respiration rate and psychological stress index during skype meeting©ISAROB 2011

702

80

80

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A Study on Efficient Query Dissemination in Distributed Sensor Networks-Forwarding Power Adjustment of Each Sensor Node Using Particle Swarm OptimizationJunya Nagashima†, Akihide Utani‡, and Hisao Yamamoto‡†Graduate School of Engineering, Tokyo City University‡Faculty of Knowledge Engineering, Tokyo City University
1-28-1, Tamazutsumi, Setagaya, Tokyo 158-8557, Japan{autani; yamahisa}@tcu.ac.jp
Abstract: In a wireless sensor network, flooding is required for the dissemination of queries or event announcements. The original
flooding causes the overlap problems. In the original flooding, generally, all sensor nodes receiving a broadcast message forward it
to its neighbors by full forwarding power, resulting in a lot of collisions and duplicate messages. For a dense wireless sensor network, the impact caused by the original flooding may be overwhelming. The original flooding may result in the reduced network
lifetime. Therefore, an efficient query dissemination method is needed to prolong the lifetime of a wireless sensor network. This
paper proposes a new query dissemination method based on the particle swarm optimization method for the long-term operation of a wireless sensor network. We evaluate the proposed method using computer simulations. In simulation experiments, theperformance of the proposed method is compared with those of the existing ones to verify its effectiveness.
Keywords: Wireless Sensor Networks, Particle Swarm Optimization, Query Dissemination, Long-Term Operation.
warding nodes for the dissemination of queries or event
announcements has been studied to prolong the lifetimeof a wireless sensor network. The existing methods of[1-4] have been proposed to resolve or improve the overlap problems of the original flooding in disseminating
queries or event announcements in a wireless sensor network. However, the gossiping of [1] may result in some nodes not receiving queries. The existing methods of[2,3] can disseminate queries to the whole nodes construct a wireless sensor network, but can not select a mostefficient and optimum forwarding nodes set. The method of [4] can compute an optimum forwarding nodes set, but a more efficient query dissemination method is needed to prolong the network lifetime.
This paper proposes a new query dissemination method based on the particle swarm optimization method for the long-term operation of a wireless sensor network.
The rest of this paper is organized as follows. In Section
II, the proposed method is described. In Section III, experimental results are reported, and the effectiveness ofthe proposed method is demonstrated by comparing theperformance of it with those of the existing ones. Finally, the paper closes with conclusions and ideas for further study in Section IV.

I. INTRODUCTION
A wireless sensor network, which is a key network
to realize ubiquitous information environments, has attracted a significant amount of interest from many researchers. In a wireless sensor network, hundreds or thousands of micro sensor nodes, which are compact and inexpensive, are placed in a large scale observation area andsensing data of each node is gathered to a sink node byinter-node wireless multi-hop communication. Each sensor node consists of a sensing function to measure thestatus (temperature, humidity, motion, etc.) of an observation point or object, a limited function on information
processing, and a simplified wireless communication function, and generally operates on a resource of a limited
power-supply capacity such as a battery. Therefore, thesuppression of communication load is generally required for the long-term operation of a wireless sensor network.
In a wireless sensor network, flooding is required forthe dissemination of queries or event announcements. The original flooding causes the overlap problems. In theoriginal flooding, generally, all sensor nodes receiving abroadcast message forward it to its neighbors by full forwarding power, resulting in a lot of collisions and duplicate messages. For a dense wireless sensor network, theimpact caused by the original flooding may be overwhelming. The original flooding may result in the reduced
network lifetime. Therefore, the selecting method of for-©ISAROB 2011

II. PROPOSED METHOD
In this paper, an efficient query dissemination method based on the Particle Swarm Optimization (PSO) method, which adjusts the forwarding power of each node703

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011that constructs a wireless sensor network, is proposed.
In this section, the PSO method is first outlined. Then,the proposed method based on the PSO method is described.

x 2k −1
pbest 2

1. Particle Swarm Optimization
The PSO method belongs to the category of swarm
intelligence methods. It was developed and first introduced as a stochastic optimization algorithm [5]. Currently, the PSO method is intensively researched because itis superior to the other algorithms on many difficult optimization problems. The ideas that underlie the PSO method are inspired not by the evolutionary mechanisms
encountered in natural selection, but rather by the social
behavior of flocking organisms, such as swarms of birds
and fish schools. The PSO method is a population-basedalgorithm that exploits a population of individuals to probe promising regions of the search space. In this context, the population is called a swarm and the individuals
are called particles. In the PSO method, a multidimensional solution space by sharing information between a swarm of particles is efficiently searched. The algorithm
is simple and allows unconditional application to various optimization problems.
Assume an n-dimensional search space S, and a swarm consisting of N particles. Each particle (The i th particle) has a position vector
xi = (xi1, xi2, … , xin)┬ ∈S,
and the velocity vector
vi = (vi1, vi2, … , vin)┬ ∈S,
where the subscript i (i =1, … , N) represents the particle’s index. In addition, each particle retains the position
vector pbesti of the best evaluation value found by theparticle in the search process and the position vector gbest of the best evaluation value among all particles as information shared in the swarm in the search process.
In the PSO method, each particle produces a new velocity vector vik+1 by linearly coupling the best solution[pbestik] found by the particle in the past, the best solution [gbestk] shared in the swarm, and the previous velocity vector vik and moves to the next position xik+1, where the superscript k indicates the number of search iterations. At the k+1 search, the velocity vector vik+1 and theposition vector xik+1 of the i th particle is updated as follows (Fig.1):

v1k

v 2k +1
gbest

x2k +1

pbest1x1k −1
Fig.1. The movement of particles
where r1 and r2 are random numbers, uniformly distributed within the interval [0,1]. ω is a parameter called the inertia weight, and c1, c2 are positive constants, referred to as cognitive and social parameters, respectively.
2. Proposed Method
In this study, a general wireless sensor network consisting of static sensor nodes with Global Positioning System (GPS) placed in an observation area is assumed.
At the initial stage of the network, the sink node set requests the location information from every sensor nodeby broadcasting a Location Discovery Message. Each sensor node receiving this Location Discovery Message
sends a Location Response Message to the sink node. The sink node can grasp the location information of eachsensor node from the gathered Location Response Messages.
By applying the PSO method, this study detects anadjustment solution on the forwarding power of each sensor node that constructs a wireless sensor network based on the location information gathered by the sink node.
In this study, variables are the forwarding power of eachsensor node as follows:x = (ET(1), ET(2), … , ET(ntotal))┬(2)where ET(i) is the forwarding power of sensor node (i),and ntotal represents the number of all sensor nodes that
constructs a wireless sensor network.
To compute an adjustment solution on the forwarding power of each sensor node for disseminating queries
or event announcements to all sensor nodes in a wirelesssensor network and minimizing total energy consumption on forwarding of queries or event announcements, the objective function is set as follows:+ c2 ⋅ r2 ⋅ ( gbest k − x ik )©ISAROB 2011

v1k +1v 2k

x1k +1x1k

v ik +1 = ω ⋅ v ik + c1 ⋅ r1 ⋅ ( pbest ik − x ik )x ik +1 = x ik + v ik +1

x 2k

(1)

704

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

f (x ) =s − ( ntotal − nreceive )ood results in a preliminary investigation were adopted
for the comparison with the proposed method. In Fig.3,
a forwarding nodes set and the number of forwarding nodes selected by using the existing methods of [2-4] areshown. From the results of Fig.3, it is confirmed that theexisting methods of [2,3] can not detect an optimum forwarding nodes set. The method of [4] computes an optimum forwarding nodes set, but a more efficient query
dissemination method is required to prolong the lifetimeof a wireless sensor network.

n∑ E (i )T(3)
where nreceive represents the number of sensor nodes thatreceived queries or event announcements.
i =1III. EXPERIMENTAL RESULTS
Through simulation experiments, the performance ofthe proposed method is investigated to verify its effectiveness. The conditions of simulation and the values on
PSO parameters, which were used in the experiments performed, are shown in Table 1, where the selected values on PSO parameters are considered proper default values and they are used in the relevant literature on the PSO method. In Fig.2, an optimum forwarding nodes set
is shown when range of radio wave of each sensor node
is set to 25m, where static sensor nodes are randomly arranged in the set experimental area.

Forwarding
Forwarding nodes:
nodes: 31
31

Table1. Condition of simulations and settings on PSO parameters
Simulation size

100(m)×100(m)The number of sensor nodes

73

Range of radio wave

0 〜25(m)The number of particle

30

Cognitive parameter c1

2.1

Social parameter c2

0.8

Inertia weight ω1.0(a) Results by the method of [2]Forwarding
Forwarding nodes:
nodes: 11
11

Forwarding Node(b) Results by the method of [3]Forwarding
Forwarding nodes:
nodes: 77

Sink NodeFig.2. Simulation model
In experimental results reported, the proposed method is evaluated through the comparison with the existing ones of [2-4]. On the existing methods, range of radio wave of each sensor node was set to 25m, and the parameter settings that were used in [4] and produced g-©ISAROB 2011(c) Results by the method of [4]Fig.3. Forwarding nodes set computed by using the existing methods of [2-4]705The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table2. Total energy consumption
Total energy consumption
The method of [2]100.6μJ

The method of [3]78.1μJ

The method of [4]73.6μJ

Proposal Best value(4searches) Mean value

71.7μJ
72.2μJ

Ranges of radio waves of all sensor nodes computedby using the proposed method are illustrated in Fig.4. In
Table 2, total energy consumption on query dissemination of all sensor nodes that construct a network is shown.
From Fig.4 and Table 2, it can be confirmed that the proposed method is a promising query dissemination one
for the long-term operation of wireless sensor networks.

Sink node(a) The first search

IV. CONCLUSIONSIn this paper, an efficient query dissemination method based on the PSO method for the long-term operation of a wireless sensor network, which adjusts the forwarding power of each sensor node for disseminating queries or event announcements to the whole nodes in awireless sensor network, has been proposed. Experimental results indicate that the proposed method has the development potential as an efficient query dissemination
one. In an actual sensor node, however, detailed adjustment of the forwarding power is difficult. In future studies, we want to improve the proposed method by considering actual sensor nodes.

Sink node(b) The second search

REFERENCES[1]Lin MJ, Marzullo K, and Masini S (2000), Gossip
versus deterministic flooding: Low message overhead and high reliability for broadcasting on small networks. DISC, pp.253-267[2]Adjih C, Jacquet P, and Viennot L (2002), Computing connected dominated sets with multipoint relays.
Technical Report 4597, INRIA:1-16[3]Nagashima J, Utani A, and Yamamoto H (2009), A
study on efficient flooding in wireless sensor networks. IEICE Technical Report 108(443):49-52[4]Nagashima J, Utani A, and Yamamoto H (2010), Efficient flooding method for wireless sensor networks based on discrete particle swarm optimization computing multiple forwarding node sets. Proc. Fifteenth International Symposium on Artificial Life andRobotics, pp.244-248[5]Kennedy J and Eberhart RC (1995), Particle swarm
optimization. Proc. IEEE International Conference
on Neural Networks, pp.1942-1948--------------------------------------------------------------------

Sink node(c) The third search

Sink node(d) The fourth search
Fig.4. Results obtained by using the proposed method©ISAROB 2011706

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Evaluation of a competitive particle swarm optimizer in multimodal
functions with complexity
Yu Taguchi†, Hidehiro Nakano†, Akihide Utani†, Arata Miyauchi†
and Hisao Yamamoto††Tokyo City University
1–28–1, Tamazutsumi, Setagaya-ku, Tokyo, 158–8557 Japan(taguchi@ic.cs.tcu.ac.jp)
Abstract: In this paper, we present a simple competitive particle swarm optimizer (CPSO) for finding plural solutions.
In the CPSO, particles are divided into groups corresponding to the required number of solutions. Each groupsimultaneously searches solutions having a priority search region. This region affects to prohibit that different groups
search the same solutions. The CPSO can effectively find desired plural acceptable solutions with a high accuracy andwith a low computation cost, and can easily control combinations of these solutions by adjusting a parameter. This
paper evaluates the CPSO in complex global optimization benchmarks. Through the numerical experiments, searching
performances of the CPSO are clarified.
Keywords: Particle Swarm Optimization, Optimization Problems, Plural Solutions.

I. INTRODUCTION

In our previous works, a simple Competitive PSO(CPSO) for finding plural solutions has been proposed[10]. In the CPSO, particles are divided into groups
corresponding to the required number of solutions. Eachgroup simultaneously searches solutions having apriority search region. This region affects to prohibitthat different groups search the same solution. The
CPSO can effectively find desired plural acceptable
solutions and can easily control combinations of thesesolutions by adjusting a parameter. Also, quality ofsome solutions obtained from the CPSO are almost thesame as or better than that obtained from the original
PSO. This means that the competitive search in the
CPSO does not suppress the searching performances ofthe original PSO, and realize to effectively search plural
acceptable solutions. In addition, the CPSO can fast findplural solutions without repeating many trials.
The CPSO has been evaluated for basic global
optimization benchmarks and has been applied to a
problem in sensor networks [10]. However, in practicalproblems such as the sensor networks, objectivefunctions can be complex shapes and can include
complex dependencies between design variables. The
detailed evaluations of the CPSO in such functions havenot been sufficient so far. This paper evaluates the
CPSO in complex global optimization benchmarks.
Through the numerical experiments, searching
performances of the CPSO are clarified.

Particle Swarm Optimizers (PSOs) are known as akind of metaheuristic algorithms [1]-[4]. Swarms suchas birds and fishes decide actions to consider not only
status information of each individual but also status
information as whole of their swarms. The PSO
expresses such actions by simple arithmetic operations.
In the PSO, particles search solutions in an objective
problem. Each particle has velocity and position
information, and moves in a multidimensional search
space considering a personal best solution which eachparticle memorizes and a global best solution which allparticles share. The PSO can fast solve various
optimization problems with a low computation cost.
On the other hand, in the actual engineering
optimization problems, there exist a lot of design
variables and constrained conditions to be considered.
Then, the exact modeling for these problems can be
hard. Also, solutions obtained from approximatedmodels are not always available in the actual problems.
Therefore, it is needed that plural acceptable solutions
as design candidates can be provided in reasonable
computation time. Many methods along this line havebeen proposed [5]-[9]. These methods can sequentiallyfind plural acceptable solutions by behaving like generaltabu search. However, by the effect of competitive
search, the quality of each solution obtained from thesemethods is often lower than that obtained from theoriginal PSO. In addition, these methods have many
parameters; it is hard to control them.

©ISAROB 2011

707

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

II. PARTICLE SWARM OPTIMIZERS
Swarms such as birds and fishes decide actions
depending on not only status information of eachindividual but also status information as whole of theirswarms. PSO is an optimization method that imitates
behavior of the swarms. In PSO, particles efficiently
search solutions in a target problem, by updating theirpositions and velocities based on personal best solutions
which each particle has and a global best solution which
all the particles have. Basic algorithm of PSO is
described as follows (see Fig.1).
Fig.2. Groups of particles with priority search regions(step6) Repeat from step2 to step5 until the number of
iterations or an evaluation value of a solution reaches apredetermined value.

III. COMPETITIVE PSO
In general, it will be difficult to model design
variables and constraint conditions exactly when
optimization algorithms are applied to real problems.
Then, it is more practical to obtain plural acceptable
solutions as design candidates and to select the bestsolution from them rather than to obtain a single exact
optimum solution. This paper presents a competitive
PSO (CPSO) that can efficiently find the pluraldifferent acceptable solutions by dividing particles into
plural groups. In the original PSO, it is difficult to findplural solutions because all the particles converge to asingle solution by moving toward a global best solution.
In the CPSO, it is considered that particles are divided
into arbitrary m groups. In addition, these groups have
own local best solution instead of global best solution asshown in Fig.2. As a result, plural solutions can befound because particles move toward each own local
best solution. But, only dividing into plural groups,
they may converge to the same solution. Therefore,priority search regions from each local best solution are
introduced. In the region, the particles of a group cansearch more preferential than other groups. If a particle
moves into the regions of the other groups, the CPSO
excepts for the particle from the candidate in updating
own local best solution. Then, it is possible to search
the plural different solutions efficiently because eachgroup does not approach to the regions of the otherFig.1. Movements of particles(step1) Set positions and velocities of each particle at
random.
(step2) Update the positions of each particle by
Equation (1).

xik 1  xik  vik
k

(1)

k

where xi and vi are position and velocity of the i th particle at the k -th iteration, respectively.
(step3) Calculate evaluation values of each particle andupdate each personal best solution ( pbesti ).
(step4) Update global best solution ( gbest ).
(step5) Update the velocities of each particle by
Equation (2).

vik 1  w  vik  c1  r1  ( pbesti  xik ) c2  r2  ( gbest  xik )(2)where w is an inertia coefficient for the previous
velocity vector. c1 is a weight coefficient for the
personal best position vector. c2 is a weight
coefficient for the global best position vector. r1 andr2 are uniform random numbers from 0 to 1.

©ISAROB 2011

708

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Varying the parameter of range r and the number ofdimensions n , typical results are presented.
The CPSO is applied to Modified Rastrigin function
defined by Equation (4).

groups to each other. The algorithm of the CPSO is
described as follows.
(step1) Set positions and velocities of each particle at
random and set the parameter r corresponding topriority search region.
(step2) Divide p particles into arbitrary m groups.
(step3) Update positions of all the particles regardlessof groups by Equation (1).
(step4) Calculate evaluation values of each particle andupdate each personal best solution ( pbesti ).
(step5) Let lbestj be the local best solution which
particles in the j -th group have. Calculate Euclidean
distances between positions of each particle in the j -thgroup and positions of lbestj ' in the j ' -th group( j '  j ).
(1)Euclidean distance is shorter than the range r:
The particle is excepted from a candidate of lbestj .
(2)Euclidean distance is longer than the range r:
The particle is included as a candidate of lbestj inthe same way as gbest in the original PSO.
(step6) Update each lbestj that are chosen from theparticles of each group.
(step7) If each position of lbestj and lbestj '( j '  j ) overlaps to each other for the priority searchregions, values of lbestj and lbestj ' are compared.
If lbestj is better, the positions of the particles in thej -th group are left. Otherwise, their positions andlbestj are reset at random.
(step8) Update the velocities of each particle by
Equation (3).

vik 1  w  vik  c1  r1  ( pbesti  xik ) c2  r2  (lbestj  xik )nF ( x)   ( zi2  10 cos(2zi )  10)

zi  R( )(xi  xi* )xi*  U (4.0,4.0),    4 ,  5.0  xi  5.0
x*

is a random optimal solution and U (a, b)
denotes uniform random numbers from a to b . And
rotation matrix R is given by Equations (5a) and (5b).

R  T 12 T 13 T 1n T 23 T (n1)n (5a)
 cos
 sin  sin Tklij  
 cos
 1
 0k  i, l  i
k  i, l  j
k  j, l  ik  j, l  jk  l  i, j
not above(5b)This function has a complex shape. This function*
gives minimum value 0 when design variables are x .
It has many suboptimum solutions in arrangement like alattice around the optimum solution.
First, we show the simulation results for n  2 . Asr  0.8 , the CPSO can find the optimum and better
suboptimum solutions as shown in Fig.3. Therefore,
when the range is set appropriately, it is possible to finddesired plural acceptable solutions. As r  1.3 , the
CPSO can find optimum and better suboptimum
solutions as shown in Fig.4. In addition, the discovered
suboptimum solutions are more distant from theoptimum solution than those obtained in r  0.8 . This
reason is that the suboptimum solutions obtained inr  0.8 are contained in priority search region of thegroup at the optimum solution as r  1.3 . As a result,
the CPSO can accurately search for better solutions
outside the range r . This means that the CPSO caneasily control the distance of each solution by adjusting
the range r .

(3)(step9) Repeat from step3 to step8 until the number of
iterations or an evaluation value of a solution reaches apredetermined value.
When a group always overlaps priority searchregions of other groups, this group can obtain nosolution because local best solution of the group is reset
every time.

IV. SIMULATION RESULTS
The CPSO is applied to some benchmark problems
and the performances are confirmed. In all theexperiments, some parameter values are fixed asfollows: p  500, w  0.9 , c1  c2  1.0 , m  5 .

©ISAROB 2011

(4)

i 1

709

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 1. Evaluation values ( n  20, r  0.8 )

F2

Group

Fig.3. Simulation results ( n  2, r  0.8 )

1
2
3
4
5

average
23.10
29.42
34.96
40.49
51.63

best
9.95
17.91
22.88
26.86
28.85

worst
38.80
50.74
60.69
61.69
95.52

Original PSO

37.08

13.93

71.58

REFERENCES[1] J. Kennedy and R. C. Eberhart, “Particle Swarm
Optimization,” Proc. of IEEE Int. Conf. on Neural
Networks, pp.1942-1948, 1995.
[2] R. C. Eberhart and J. Kennedy, “A New Optimizer
Using Particle Swarm Theory,” Proc. of the Sixth
International Symposium on Micromachine and Human
Science, pp.39-43, 1995.
[3] R. C. Eberhart and Y. Shi, “Comparing Inertia
Weights and Constrictions Factors in Particle Swarm
Optimization,” Proc. of IEEE Congress on Evolutionary
Computation, pp.84-88, 2000.
[4] J. Kennedy and R. C. Eberhart, “SwarmIntelligence,” Morgan Kaufmann Publishers, 2001.
[5] A.Kumamoto, A.Utani and H.Yamamoto,“Advanced Particle Swarm Optimization for Computing
Plural Acceptable Solutions,” Int. J. Innovative
Computing, Information and Control, vol.5, no.11,
pp.4383-4392, 2009.
[6] Y.Yoshimura, H.Nakano, A.Utani, A.Miyauchi and
H.Yamamoto, “An Effective Allocation Scheme for
Sink Nodes in Wireless Sensor Networks Using
Suppression PSO,” ICIC Express Letters, vol.3, no.3
(A), pp.519-524, 2009.
[7] G.G.Yen and M.Daneshyari, “Diversity-based
Information Exchange among Multiple Swarms in
Particle Swarm Optimization,” Proc. IEEE Congress on
Evolutionary Computation, pp. 6150-6157, 2006.
[8] S.Janson and M.Middendorf, “A HierarchicalParticle Swarm Optimizer and Its Adaptive Variant,”
IEEE Trans.Systems, Man, and Cybernetics, Part B,
vol.35, No.6, pp.1272-1282, 2005.
[9] S.Nakano, A.Ishigami and K.Yasuda, “Particle
Swarm Optimization Based on the Concept of Tabu
Search,” Proc. IEEE Congress on Evolutionary
Computation, pp.3258-3263, 2007.
[10] Y.Taguchi, Y.Kanamori, H.Nakano, A.Utani and
A.Miyauchi, “A Competitive PSO Based on Evaluation
with Priority for Finding Plural Solutions” Proc. IEEE
Congress on Evolutionary Computation, pp.102-107,
2010.

Fig.4. Simulation results ( n  2, r  1.3 )Next, simulation results for higher dimensional
Modified Rastrigin function ( n  20 ) are shown. The
trials are repeated 100 times and the average values arepresented. Table 1 shows the results for r  0.8 .
Although the optimum solution cannot be found, plural
acceptable suboptimum solutions can be found as
shown in Table 1. Therefore, the CPSO can obtainplural solutions easily also in high dimensional andcomplex problems. In addition, quality of somesolutions obtained from the CPSO are almost the sameas or better than that obtained from the original PSO.

V. CONCLUSION
This paper has been evaluated a simple competitive
PSO for multimodal functions with complexity.
Adjusting a parameter of a priority search range, desired
plural acceptable solutions can be effectively found andcan be easily controlled. In the practical engineering
optimization problems, not only a single exact optimum
solution for the problems but also plural acceptable
solutions for them are often required. The CPSO will bea simple and powerful tool to effectively solve these
problems. Future problems include (1) adaptive controlof parameters in the CPSO, (2) investigations of
performances for various higher-dimension problems,and (3) applications to various engineering optimization
problems.

©ISAROB 2011710

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

An effective allocation method of ZigBee sensor nodes using a discrete
particle swarm optimizer
Ryota Saito, Hidehiro Nakano, Akihide Utani, Arata Miyauchi
and Hisao Yamamoto
Tokyo City University
1-28-1,Tamazutsumi,Setagaya-ku,Tokyo,158-8557 Japan(saito@ic.cs.tcu.ac.jp)

Abstract: This paper proposes a method to find effective allocations of ZigBee coordinators using the Discrete Particle
Swarm Optimizer (DPSO). In the proposed method, an observation area is represented as a grid space. For a given
ZigBee end device location, it is decided whether each ZigBee coordinator is allocated or not on each intersection of
the grid. Such binary state variables are optimized by the DPSO. In the simulation experiments, the proposed method is
evaluated to some of given ZigBee end device locations. It is shown that the proposed method can provide effective
allocations of ZigBee coordinators.
Keywords: ZigBee sensor networks, Coordinator, Discrete Particle Swarm Optimization
wireless communication, since each end device does nothave routing functions. Therefore, effective allocationsof coordinators in an observation area should beconsidered. That is, the number of coordinators andtheir locations should be optimized. This is also
regarded as a problem to design a kind of cluster tree
networks.
This paper proposes a method to find effective
allocations of coordinators using a discrete particle
swarm optimizer (DPSO) [5]. In the DPSO, eachparticle having binary state variables represents asolution of an objective function, and moves in a
multidimensional search space based on its own andother particles' experiences. As each particle effectively
interacts to each other, an optimum solution for theobjective function can be found. The DPSO can fastsolve various optimization problems although thealgorithm uses only simple and fundamental operations.
In the proposed method, an observation area isrepresented as a grid space. For a given end device
location, it is decided whether each coordinator isallocated or not on each intersection of the grid. Suchbinary state variables are optimized by the DPSO.
Desired allocations of coordinators are defined asfollows: all end devices can communicate directly to
one of coordinators with the minimum number ofcoordinators. The objective function is designed byconsidering them. In the simulation experiments, theproposed method is evaluated to some of given end
device locations. It is shown that the proposed method
can provide effective allocations of coordinators.

I. INTRODUCTION
ZigBee sensor networks are known as a wirelesstechnology of an open global standard [1] [2]. The
ZigBee sensor networks have a wide range ofapplications, such as voice services [3] and vehicularenvironments [4]. Also, many protocols for ZigBee
sensor networks have been proposed. ZigBee sensor
nodes are classified into a Full-Function Device (FFD)
and a Reduced-Function Device (RFD). The RFD is a
low-cost device and can be a ZigBee end device whichmonitors status information around it, such astemperature, light intensity, and moving objects. The
FFD can be not only a ZigBee end device but also a
ZigBee coordinator or a ZigBee router which gathers
sensing information transmitted from RFDs via wirelesscommunication.
The basic network topologies of the ZigBee sensor
networks are star networks, cluster tree networks, meshnetworks, and so on. This paper focuses on the star
networks. In the networks, one coordinator and end
devices exist; routers do not exist. Each end device does
not have routing functions; it only transmits its own
sensing information directly to a coordinator and does
not relay sensing information from the other devices.
Since multi-hop wireless communication is not required
to the end devices, energy consumption of each end
device can be saved. Hence, long-term observation ispossible. In addition, constructing cluster tree networksof plural coordinators, large scale observation is also
possible. However, it is needed that all end devices cancommunicate directly to one of coordinators via©ISAROB 2011711

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

II. ZIGBEE SENSOR NETWORKS
ZigBee is one of the world standards on a shortdistance wireless sensor network [1]. The ZigBee
belong to the WPAN (Wireless Personal Area network),
and can construct low-cost and low-power networks.
Specification of the ZigBee in a basic part isstandardized as IEEE 802.15.4 [2]. The typical datatransfer speed is from 20kbps to 250kbps. Each ZigBee
sensor nodes can operate during several years bybatteries. ZigBee sensor nodes are classified into a Fullfunction Device (FFD) and a Reduced-function Device(RFD). The RFD is a low-cost device and can be a
ZigBee end device which monitors status information
around it, such as temperature, light intensity, andmoving objects. The FFD can be not only a ZigBee end
device but also a ZigBee coordinator or a ZigBee routerwhich gathers sensing information transmitted from
RFDs via wireless communication.
The basic network topologies of the ZigBee sensor
networks are star networks, cluster tree networks, meshnetworks, and so on (see Fig.1). In all the networks, one
coordinator controls whole of their networks. In the star
networks, a root node becomes a coordinator and leaf
nodes become end devices; routers do not exist in the
networks. In the cluster networks, a root node becomesa coordinator, leaf nodes become end devices, and theother nodes become routers. In the mesh networks, one
node becomes a coordinator and all the other nodes
become routers.
This paper focuses on the star networks. In the
networks, each end device does not have routing
functions; it only transmits its own sensing information
directly to a coordinator and does not relay sensing
information from the other nodes. Since multi-hop
wireless communication is not required to end devices,energy consumption of each end device can be saved.
Hence, long-term observation is possible. In addition,
constructing cluster tree networks of plural coordinators,
large scale observation is also possible as shown in
Fig.2. However, it is needed that all end devices cancommunicate directly to one of coordinators via
wireless communication, since each end device does nothave routing functions. Therefore, effective allocationsof coordinators in an observation area should beconsidered. That is, the number of coordinators andtheir locations should be optimized. This is also
regarded as a problem to design a kind of cluster tree
networks.

©ISAROB 2011

Fig.1. Basic topologies of ZigBee sensor networks.

Fig.2. Effective allocations of plural coordinators.

III. DISCRETE PARTICLE SWARM OP
TIMIZERThe Particle Swarm Optimizer (PSO) is known as akind of metaheuristic algorithms, and can fast solve
solutions in various optimization problems, comparedwith the other optimization methods [6]. The PSO ismodeled by particles with positions and velocities in
multidimensional search space. Each particle has apersonal best solution as a search history of its particle
and shares a global best solution as a search history of
all particles. The Discrete Particle Swarm Optimizer(DPSO) is a discrete binary version of the PSO [5]. The
DPSO can be applied to various combinational
optimization problems. Basic algorithm of the DPSO is
described as follows.
(step1) Set positions and velocities of each particle at
random.
(step2) Update the positions of each particle by
Equation (1). They are decided as binary values by
substituting the current velocities to the sigmoid
function (2), and comparing them with uniform random
numbers.

712

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

if   sig(vik 1 ) then xik 1  1else xik 1  0
1
sig(vik 1 ) 
1  exp(vik 1 )kwhere F is an evaluation value. f1 is the number ofcoordinators, f 2 is the number of end devices which
can directly connect with one of coordinators, and W
is a weight parameter.
End devices (nodes) are allocated in the observation
area at random. The size of the observation space is
20×20. The radio range of nodes is 5. The number ofparticles in the DPSO is 10. The number of cycles for asingle trial is 100. In all the experiments, the DPSO uses
the fixed parameter values:w  1.0 , c1  c2  1.0 , W  1.0 .
First, we show the simulation results for 10 nodes.
Fig.3 shows the example allocation of coordinators
obtained by the DPSO when the grid is 5×5. In the
figure, obtained minimum number of coordinators is 5.
Fig.4 shows the results when the grid is 9×9. In the
figure, obtained minimum number of coordinators is 4.
As compared with the case of 5×5, the number ofcandidate locations of coordinators increases. Then, theobtained minimum number of coordinators decreases.
Table 1 shows the number of allocation patterns andtheir obtained times for 100 trials.
Next, we show the simulation results for 20 nodes.
Figs.5 and 6 show the example allocations when thegrid is 5×5 and 9×9, respectively. Table 2 shows thenumber of allocation patterns and their obtained times
for 100 trials.
Depending on the number of nodes and theirlocations, total number of effective allocation patternsof coordinators change. However, it should be notedthat the proposed method based on the DPSO can findacceptable solutions for all trials.

(1)

(2)

k

where xi and vi are the position and velocity of thei -th particle at the k -th iteration, respectively.
sig() is the sigmoid function, and  is a uniform
random number from 0 to 1.
(step3) Calculate evaluation values of each particle.
(step4) Update each personal best solution ( pbesti ).
(step5) Update global best solution ( gbest ).
(step6) Update the velocities of each particle by Eq
uation (3).

vik 1  w  vik  c1  r1  ( pbesti  xik ) c2  r2  ( gbest  xik )(3)

where w is an inertia coefficient for the current
velocity vector. c1 is a weight coefficient for personal
best position vector. c2 is a weight coefficient for
global best position vector. r1 and r2 are uniform
random numbers from 0 to 1.
(step7) Repeat from step2 to step6 until the numberof iterations or evaluation value of a solution reac
hes a predetermined value.

IV. PROPOSED METHOD
In this section, the proposed method is explained
and typical simulation results are shown. We apply the
DPSO to the ZigBee coordinator allocation problem asfollows. The observation area is delimited as the grid
space. Each intersection of the grid represents a
candidate location of coordinators, and the combination
whether coordinators are allocated is optimized by the
DPSO. The purpose of this problem is that all end
devices are connected directly by one hop with one ofcoordinators via wireless communication. In such a
constraint condition, the number of coordinators is
minimized. The evaluation function is given by
Equation (4).

F©ISAROB 20111
 Wf 2
f1

Fig.3. Allocation result (10 nodes, 5×5 grid).

(4)

713

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

networks," IEEE Commun. Mag., vol.46, no. 1, pp.121127, 2008.
[4] H.M.Tsai, O.K.Tonguz, C.Saraydar, T.Talty,
M.Ames, and A.Macdonald, "ZigBee-based inter-car
wireless sensor networks: A case study," IEEE WirelessCommun., vol.14, no.6, pp.68-77, 2007.
[5] J.Kennedy & R.Eberhart, "A discrete binary versionof the particle swarm optimization algorithm," Proc.
SMC '97, pp.4104-4109, 1997.
[6] J. Kennedy and R. C. Eberhart, “Particle Swarm
Optimization,” Proc. ICNN, pp.1942-1948, 1995.

Fig.4. Allocation result (10 nodes, 9×9 grid).
Table 1. Number of allocation patterns (10 nodes).

Grid

5×5
9×9

Number ofcoordinators

Obtainedfrequency

Number ofpatterns

5
6
4
5
6

79
21
35
57
8

28
20
35
57
8

Fig.5. Allocation result (20 nodes, 5×5 grid).

VI. CONCLUSION
This paper has proposed an application of the DPSO
to ZigBee sensor networks. The proposed method canprovide effective allocations of ZigBee coordinatorssuch that all ZigBee end devices can connect directlywith one of coordinators via wireless communication, as
minimizing the number of coordinators. Also, theproposed method can provide various allocation
patterns by changing initial values. If the grid becomes
finer, the problem becomes more difficult. However, thepossibility that a better solution can be discovered
increases. This means that allocations of thecoordinators can be effectively optimized by the DPSO
by appropriately setting the grid considering the radio
range and the scale of problems.
Future problems include (1) more detailed analysisof searching performances, (2) application of themethod for finding plural acceptable solutions.

Fig.6. Allocation result (20 nodes, 9×9 grid).
Table 2. Number of allocation patterns (20 nodes).
Grid

5×5

REFERENCES[1] ZigBee Alliance, http://www.zigbee.org, 2007.
[2] 802.15.4-2003 IEEE Standard for Information
Technology-Part 15.4, 2003.
[3] C.Wang, K.Sohraby, R.Jana, L.Ji, and
M.Deneshmand, "Voice communications over ZigBee©ISAROB 2011

9×9

714

Number ofcoordinators

Obtainedfrequency

Number ofpatterns

6
7
8
5
6
7
8

55
39
6
3
26
50
21

14
28
6
3
26
50
21

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

An Implementation of Probabilistic Model-Building Coevolutionary Algorithm
Takahiro OTANI and Takaya ARITA
Graduate School of Information Science, Nagoya University
Furo-cho, Chikusa-ku, Nagoya 464-8601, Japan(Email: t-otani@alife.cs.is.nagoya-u.ac.jp, arita@nagoya-u.jp)

Abstract: We propose an extended coevolutionary algorithm (CA) with probabilistic model-building (CA-PMB) in
order to improve search performance of CA. This paper specifically describes an implementation of CA-PMB called
coevolutionary algorithm with population-based incremental learning (CA-PBIL), and analyzes behavior of thealgorithm through computational experiments using intransitive numbers game as a benchmark problem. The
experimental results show that desirable coevolution may be inhibited by the over-specialization effect, and that thealgorithm shows complex dynamics caused by the game's intransitivity. However, further experiments show that the
intransitivity encourages desirable coevolution when a different learning rate is set for each population.
Keywords: Coevolutionary algorithm, Probabilistic model-building genetic algorithm, Intransitive numbers game.

PMBGA can explicitly deal with dependence ofvariables in a problem, and therefore it shows bettersearch performance on various optimization problems
than standard GAs. This paper shows an implementationof CA-PMB called coevolutionary algorithm withpopulation-based incremental learning (CA-PBIL).
This paper analyzes behavior of CA-PBIL throughcomputational experiments using intransitive numbers
game as a benchmark problem. Intransitive numbers
game is an abstract model devised by Watson and
Pollack [3] to study intransitivity which refers to cyclic
dominance such as the Rock-Paper-Scissors game, andhas been seen as a substantial obstacle to progress incompetitive coevolutionary algorithms. Experimental
results show that desirable coevolution may be inhibitedby the over-specialization effect, and that the algorithm
shows complex dynamics caused by the game's
intransitivity. However, further experiments show thatthe intransitivity encourages desirable coevolution whena different learning rate is set for each population.

I. INTRODUCTION
Competitive coevolutionary algorithm is anextension of standard evolutionary algorithms in which
each solution is not evaluated by a fixed objectivefunction (or fitness function), but is evaluated based oninteractions between other solutions.
The first application of the algorithm is design of asorting network proposed by Hillis [1]. He showed CA
can design a feasible network structure which has fewer
comparators than a network designed by a standard
genetic algorithm. CA has also been successfully
applied to various problems including evolution of
artificial neural network for classification problems,
function approximation, evolving strategies of gameplayers, coevolving predator-prey robots, and so on.
In this study, we propose an extended CA called
coevolutionary algorithms with probabilistic modelbuilding (CA-PMB) in which probabilistic modelbuilding genetic algorithm (PMBGA) [2] is used as asearch heuristics, and analyzes behavior of thealgorithm through computational experiments.
Standard CA uses the search mechanism of thegenetic algorithm to evolve candidate solutions. CAPMB adopts PMBGA as a search heuristics in order toimprove search performance of standard CA. PMBGA(also called estimation of distribution algorithm) is a
generic name given to a class of evolutionary
algorithms in which genetic operators, crossover andmutation, are replaced by building a probabilistic model
which represents the distribution of promising solutions
and generating new solutions based on the model.

©ISAROB 2011

II. COEVOLUTIONARY ALGORITHM
CA is an extension of standard evolutionary
algorithms. An important difference between(competitive) CA and standard evolutionary algorithms
is how to define an evaluation function. In CA, eachsolution is not evaluated by a fixed objective function as
in the standard evolutionary computations, but isevaluated based on interactions between other solutions.
A model of CA dealt in this paper uses two solution sets
S 1 and S 2 . A solution s in a solution set is evaluatedbased on interactions with all of solutions in the other715

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

solution set. An evaluation function F s  is defined
as follows,

probability vector as a probabilistic model which
represents the probability of assigning value “1” to thecorresponding bit for a new candidate solution.

  E s, t  if s  S 1
 2(1)
F s   tS2 ,
 1 E s, t  if s  S
tS
where E s, t  denotes a payoff of solution s
depending on an interaction with solution t , which isdefined based on a targeted problem.

III. COEVOLUTIONARY ALGORITHM
WITH PROBABILISTIC MODEL-BUILDING
1. Scheme of Proposed Algorithm
PMBGA (also called estimation of distribution
algorithm) is a generic name given to a class ofevolutionary algorithms in which genetic operators,
crossover and mutation, are replaced by building aprobabilistic model which represents distribution ofpromising solutions and generating new solutions basedon the model.
In this study, we propose coevolutionary algorithms
with probabilistic model-building (CA-PMB) in which
PMBGA is used as a search heuristics. A scheme of thealgorithm is described in Fig. 1. This algorithm
generates each solution set using probabilistic model
which represents the distribution of promising solutions,and evaluates each generated solution according to theequation (1). Then, the model is updated based on goodsolutions.
Probabilistic
model

Solution 2

IV. INTRANSITIVE NUMBERS GAMEThis paper analyzes the behavior of CA-PBIL
through computational experiments using intransitive
numbers game (ING) [3] as a benchmark problem. ING
is an abstract model to study intransitivity (as in thewell-known Rock-Paper-Scissors game) which has been
seen as a substantial obstacle to progress in competitive
coevolutionary algorithms, and is usually used tocompare the performance of coevolutionary algorithms.

Probabilistic
model

Generate Update

Solution 1

Fig. 2. Pseudocode of CA-PBIL.

1. Problem Definition
A solution of ING is a n -dimensional vector, and arange of each dimension’s value is [0,k ] . The payoff
function E ( s,t ) is defined as follows,

Generate Update

Evaluation

Solution 1 n
E s, t   sign  g i  , i 1 

Solution 2

Solution N1

Solution N2

Candidate solutions

Candidate solutions

wheresi  ti if hi  min h jjgi  
0otherwisehi  si  ti .

Fig. 1. Scheme of CA-PMB.
2. CA-PBIL: An Implementation of CA-PMB
This paper shows an implementation of CA-PMB
called coevolutionary algorithm with population-based
incremental learning (CA-PBIL). A pseudocode of thealgorithm is described in Fig. 2. CA-PBIL uses PBIL
algorithm [4] as a search heuristics. PBIL is a simple
variation of PMBGA. This algorithm deals withcandidate solutions as bit-strings, and uses and updates©ISAROB 2011(2)

The function sign returns 1 when an input value ispositive value, returns -1 when an input value isnegative value, and returns 0 when an input value is 0.
Therefore, the winning solution is the one with higher
magnitude in the dimension of least difference between
the two solutions.

716

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

difference is the first, and b gets positive payoff. Now,a' 3,6 is a small variation from a . The dimensionof least difference when a ' and b is the second
dimension and a ' gets a positive payoff. Therefore,a should be replaced by a ' . Then, b' 4,4 gets a
positive payoff depending on the interaction with a ' ,and b should be replaced by b' . These solutions
evolve to low values in this manner repeatedly.

Furthermore, a modified ING is also used forexperimental analysis of the algorithm. In this problem,the function sign in equation (2) is modified asfollows. The function returns 1 when an input value ispositive value, otherwise returns 0. These settings cause
algorithm’s complex dynamics more frequently.
2. Bit-strings Solution Representation
A bit-string b is converted to a vector solution s
as follows. The length of the bit-string is set toL1  L2  nk , and i -th dimension’s value of avector solution is calculated as follows,ksi   bi 1k  j ,j 1

where bi denotes i -th bit value of the bit-string.

V. COMPUTATIONAL EXPERIMENTS
In this section, we analyze the behavior of CA-PBIL
using ING. The number of dimension and the maximum
value of each dimension were set to n  2 andk  100 respectively. Optimal solution is 100,100 .
The length of a bit-string was set to L1  L2  200 .

(a) Stagnation (99 times).

1. Dynamics on the Standard ING
First of all, we carried out following experiments
using the standard ING to study how the average
evaluated value of generated solution progresses.
Algorithm’s parameters were set as follows, the learning
rates: LR1  LR2  0.01, the numbers of solutions
generated every iteration: N1  N 2  100 , and themutation probabilities: MP1  MP2  0.0 . Thismeans mutation procedure was not executed. The
number of iteration was set at 10000, and theexperiment was trialed 100 times.
The experimental results show that a stagnation
pattern occurred in almost all trials while a convergence
to a low value pattern rarely occurred. Typical examplesof each occurred pattern are described in Fig. 3. Fig.
3(a) describes the stagnation pattern which is called theover-specialization effect [3]. If two values in one
dimension become closer to each other than in the otherdimension enough, the other dimension does not play arole in decision of payoffs. Therefore, the former will
evolve desirably, however the latter will stagnate.
Fig. 3(b) shows a rare case in which the convergence
to a low value pattern occurred. This pattern is causedby the intransitivity of the game, and is called relativism
effect in the literature [3]. For example, considera  4,7 and b  5,5 . The dimension of least©ISAROB 2011(b) Convergence to a low value (1 time).
Fig. 3. Typical examples of progress patterns on thestandard intransitive numbers game.
2. Dynamics on the Modified ING
We carried out the same experiments using themodified ING. Algorithm's parameters were set to sameas previous experiments. In this experiments, four typesof patterns occurred. Typical examples of each pattern
are described in Fig. 4. Dynamic patterns described inFig. 4(b)(c)(d) were caused by the intransitivity of thegame.
3. Coevolution under a Different Learning Rate
Finally, we carried out experiments to analyze thebehavior of the algorithm when a different learning rate
was set for each solution set. We examined the number2of reached optimal solutions varying S 's learning rate
LR2 from 0.01 to 0.03. And other parameters were set
to same as the previous experiments.
Fig. 5 shows the experimental results. According tothese results, the search performance was increased

717

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

when a different learning rate was set for each
population on the modified ING. The reason is supposed
to be that these settings could encourage convergence tohigh values as shown in Fig. 4(c).

Fig. 5. The number of reached optimal solutions.

VI. CONCLUSION
In this paper, we proposed an extended CA called
CA-PMB,
showedCA-PBIL as an
initialimplementation, and examined the algorithm's behavior
through experiments using ING. Experimental results
showed that desirable coevolution may be inhibited byover-specialization effect. We also observed aninteresting coevolutionary behavior caused by thegame's intransitivity especially on the modified ING.
Further experiments showed that the intransitivity
encourages desirable coevolution when a different
learning rate is set for each population. This feature has
a possibility to provide new ways to solve various
problems in coevolutionary domain more effectively
and efficiently.
Future work includes the analysis of the proposed
algorithm by conducting further experiments. In
particular, the effectiveness of a different learning rateshould be evaluated in more detail. Another direction
would be to compare the performance with otheralgorithms and to evaluate using other more practicalproblems. It might be also interesting to implement CA
combined with other PMBGA.

(a) Stagnation caused by the over-specialization(38 times).

(b) Converges to a low value in one dimension(49 times).

REFERENCES(c) Convergence to high values in both dimensions(4 times).

[1] W. D. Hillis (1990), Co-evolving parasites improve
simulated evolution as an optimization procedure.
Physica D: Nonlinear Phenomena 42:228-234[2] M. Pelikan, D. E. Goldberg, and F. G. Lobo (2002),
A survey of optimization by building and using
probabilistic models. Computational Optimization andApplications 21:5-20[3] R.A. Watson and J.B. Pollack (2001),
Coevolutionary dynamics in a minimal substrate. Proc.
of the Genetic and Evolutionary Computation
Conference:702-709[4] S. Baluja (1994), Population-based incremental
learning: A method for integrating genetic search based
function optimization and competitive learning.
Technical Report CMU-CS-94-163, Computer Science
Department, Carnegie Mellon University, Pittsburgh, PA(d) Oscillation starting in one dimension(9 times).
Fig. 4. Typical examples of progress patterns on themodified intransitive numbers game.

©ISAROB 2011

718

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Modeling and solution for optimization problems
with incomplete information– A general framework and an application to cruising taxi problems –Makoto O HARA and Hisashi TAMAKIKobe University, 1-1 Rokkodai-cho, Nada-ku, Kobe
(Tel: +81-78-803-6102; Fax: +81-78-803-6102)(ohara@al.cs.kobe-u.ac.jp)
Abstract: The purpose of this study is to construct a general model for optimization problems with incomplete information,and to assist designing solutions. The term “incomplete information” means that information of target system, e.g. the dynamics and the status variables, is not obtained enough to optimize. Before description of proposed model with incomplete
information, an optimization model with “complete” information is structured. The occasions with “incompleteness” of information are explained on this optimization model, and approaches to resolving them are described. Then, the optimization
model with incomplete information is defined. Moreover, in order to investigate the validity of the proposed framework, it isapplied to cab-dispatching (of cruising taxis) problems, where the objective is to distribute a set of cabs efficiently by indicating
the adequate location to each taxi driver. Through some computational examples, the effectiveness and the potential of the
proposed approach is confirmed.
Keywords: Uncertainty, Optimization Model, Distributing Taxi Problem
mize/minimize evaluation values about behaviors of systems.
Components of the optimization problems are referred as; “Target System” is the target of evaluation for optimization, andis controlled directly, “Relevant System” interacts with Target
System, in other words, Relevant System may be possible tobe controlled indirectly, “Environment” acts Target System, andcan’t be conrolled, “Controller” decisions a direction of Target
System’s behaviors, “Supervisor” affects the decision-makingof Controller.
In order to optimize, these components have to convey decisions for others, i.e., “Control”: Controller sends operations,which indicate what to do, to Target System, “Supervision”: Supervisor sends orders for decision-making to Controller.
For making decisions, the components know about others,i.e., “Observation”: The states of Target System, Relevant System and Environment are sensed by Controller, “Evaluation”:
Controller sends results, which are clued to judge the effectiveness of decision-making, to Supervisor. These components
and information-exchanges are collectively called “Optimization Model” in this paper.
To understand easily, Optimization Model is identified as ahierarchical manner; “Physical Layer” shows the physical sideof the problems, and includes continuous-time systems, i.e. Target System, Relevant System and Environment, “Information
Layer” shows the information side of the problems, and includes short-rate discrete-time system, i.e. Controller, “Supervisor Layer” shows an optimizer for the problems, and includes
long-rate discrete-time system, i.e. Supervisor.
Definition of Time Constant: Target System is optimized
during T (= [0, T )). Controller evaluates behaviors of Target
System every time interval τ E , and Physical Layer is observed
each sampling cycle τ O . The time horizon T is discretized tonE terms by the interval τ E , and each k-th term T (k) is discretized to nO periods by the cycle τ O . Then, T (k) corresponds
the time interval [ kτ E , (k+1)τ E ), and m(k) th period correˆ´
sponds kτ E +mτ, kτ E +(m+1)τ . The set of periods T D is

1. INTRODUCTION
This research deals with a class of optimization problems
with incomplete information, in which the information about thestructure as well as the dynamics of the target system is not sufficient. To this class of the problems, in this paper, a generalframework for both modeling the problem formally and designing a solution structure systematically is newly proposed. This
class includes such types of optimization problems as containing some uncertainty in systems, e.g. due to the spatially partial
or temporarily late observability, and/or some unobservability indecision-making.
So far, lots of researches have been presented on several
types of optimization problems with uncertainty, from the viewpoints of the description of the problem as well as the designof solutions [1∼3]. However, there have been few studies on
general frameworks of both optimization models and their solutions. By introducing informational viewpoints, a variety ofuncertain aspects may be dealt with in a uniform way, which is
the keynote point of our research.
For confirming validity of the proposal aproach, it is applied
distributed cruising taxi problems. In these cab-dispatchingproblems, there are many unobservable state variables, e.g., theoccurrence of passengers and the positions of the cabs of competitive companies. It is difficult or it might be impossible tooptimize the dispatching beforehand due to incompleteness. In
designing the solution, a set of rules (rule-set) for indicating thepriority areas in the information layer, and a genetics-based machine learning (GBML) method is adopted to adjust a rule-set in
the supervisor layer. As a result of the computational examples,the effectiveness and the potential of the proposed approach is
confirmed.

2.

OPTIMIZATION PROBLEMS

2. 1 Optimization Model with Complete Information
Optimization problems are defined as problems to maxi-©ISAROB 2011

719

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

defined as below;
T O(k) = {0, . . . , nO }(nO = (τ E /τ O )−1).

2. 2. 2 Completion
Completion functions are required for each incompleteness;• for incompleteness ( i ) : f C(1)

If notation “(k)” isn’t be needed, it is omitted in following. The
information-exchanges on the time axis are illustrated in Fig. 1.
Elements: The elements of the optimization model (andcorrespondent elements of Dispatching Cruising Taxi Problems
which are described later) are shown in Table 1.

y[m] = f C (x(mτ O ) + σ(mτ O )) = x(mτ O ),(6)

• for incompleteness (ii) : f E
y[m] = f E (x(mτ O − τ )) = x[m],(7)• for incompleteness (iii) : f P
y[m] = f P (∗) = x[m].

The functions f C , f E and f P each are called “Correction”, “Estimation” and “Prediction”.
In order to implement completion of incompleteness, information as follows are needed. “History”: an history about astatus variable to observe is effective for Correction and Prediction. “Same kind of a status variable”: if a system is consistsof subsystems, e.g. multi-agent, status variables of each subsystem, which are same kind of a state to observe, help Estimation.
“Other kind of a status variable”: if a status variable can’t beobserved, other state values of a same system might be valid for
Estimation. “Knowledge”: knowledge about a system or dynamics assist all completions.

Fig. 1 Information-exchanges between each layer.

Input/Output:
Each component’s inputs/outputs withcomplete information is defined as below;aS (t) = oR (t),eS (t) = oV (t),aR (t) = oS (t) ,2. 3 Optimization Model with Incomplete Information
Optimization Model with incomplete information has completion functions defined in Section 2. 2. 2. Completion Module
converts observed values, and sends complementary values toController. An overview of Optimization Model with incomplete information and an inside of Completion Module are illustrated in Fig. 2.

eR (t) = oV (t),u(t) = v [bt/τ c],` S´
y[m] = x (mτ ), xR (mτ ), xV (mτ ) ,p[0(k) ] = c[0(k) ], r[nE(k) ] = Z[nE(k) ].
Optimization Problems:
restriction is formulated as;

The objective function and themin Z = F {y[0], . . . , y[nO ]}+P (G){y[0], . . . , y[nO ]}s.t. v[m] ∈ H[m](8)(∀ m ∈ T O )Controller obtains the states of Target System by observations.
Then, the state restriction is used in a penalty P .
2. 2

Incompleteness2. 2. 1 Classification
A variable y is observation quantities of a status variable x.
The relation of y and x with complete information is as follow;
y[m] = x(mτ O ).

(2)

The classes of incompleteness in the observations are;( i ) Accuracy (error): An observed value includes an error
σ(t) [%].
y[m] = x(mτ O ) + σ(mτ O ).

Completion Module

Overview

Fig. 2 Hierarchical model of optimization problem with incomplete information(3)(ii) Time delay: A status value of time τ (t) ago is observed.
y[m] = x(mτ O − τ ).

(4)

3.

(iii) Unobservable: A status value is unobservable.
y[m] = ∗.

For confirming validity of the proposal optimization model,
the model applies Dispatching Cruising Taxi Problem.

(5)

3. 1 Outline
Since the revision of Road Transportation Law in Japan, thetaxi business is becoming saturated by new entries. In particular, rookie drivers have a lot of trouble finding passengers,

The symbol ∗ means uncertain value.
The observations are classified completeness, incompleteness( i ), (ii), (iii), or combination of theirs.

©ISAROB 2011DISPATCHING CRUISING TAXI PROBLEMS

720

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 1 Elements of optimization problems and corresponding with Dispatching Cruising Taxi Problems
Symbol‡ Element†
• xS
state
(position, assigned passenger and number of assignment)+ucontrol input(dispatching area)+ aS
relevant action(position, destination and assigned cab)Target System+ eS
environment action (traffic jam)
(Cooperative Company)? TS
transition function (move rule)− oS
output action(position and assigned passenger)? US
output function(take rule)
• xR
state
(position, destination, appearance time and assigned cab)
+ aR
relevant action(position and assigned passenger)
Relevant System (1)+ eRenvironment action (weather)(Passengers)? T R transition function (appear rule)
Physical− oR
output action(position, destination and assigned cab)
? U R output function(ride rule)
• xR
state
(position and assigned passenger)+ aR
relevant action([Not Available])
Relevant System (2)+ eRenvironment action (traffic jam)(Competitive Company) ? T R transition function (move rule)− oR
output action(position and assigned passenger)? U R output function(take rule)
• xV
state
(states about weather and traffic jam)
Environment? TVtransition function (weather and traffic translation)(Environment)− oV
output action(weather and traffic jam)
? UVoutput function(weather and traffic action rule)+yobserved value(cabs states, passengers actions and environment actions)• Mobserved record(observed record)−v
operation(dispatching area)+p
operation parameter(weight coefficients)
Controller
Information? G
state restriction([Not Available])(Dispatch Controller)? H
operation restriction(area permitted business)? C
operation function (priority calculation)−Zevaluated value(total of assignment)? F
evaluation function (sum of assignment)+rresult(total of assignment)• R
result record(assignment record)
Supervisor
Supervisor−corder(weight coefficients)(Supervisor)? oorder function(GBML)• O
order record(weight coefficient record)† Words in parentheses are corresponding components or elements of Dispatching Cruising Taxi Problems.
‡ Header symbols mean +:input, −:output, • :state, and ? :other, e.g. dynamics.
Layer

Component/System†

because taxi business has relied on individual experience ofdrivers. The taxi companies need systems which assist lessexperienced drivers for finding passengers. There have beenstudies based on statistical models [4), 5], in contrast, few studies have been carried out to analyze Dispatching Cruising Taxi
Problem agent-models.
3. 2

Priority Rule: A priority rule calculates priorities of eacharea for every cab. Each cab is dispatched to the area which ishighest priority for the cab. The rule calculates priority which isa weighted summation of preparing equations.
Learning Method: In this paper, condition is defined as
partially feature quantity space of whole areas, and action is defined as weight coefficients in the priority rule [6].
Elements: The elements of dispatched cruising taxi problems have been shown in Table 1.

Approach

Positions of passengers, targets of taxis, are unobservable inthese problems. Thus, taxis are dispatched regions. This paper
suggests an optimization framework for taxi dispatching by therule-set which calculates target areas with observable state variables. The corresponds of the proposed model and Dispatching
Cruising Taxi Problems have been described in Table 1.
Definition of Space: A region A is intended in a problem,
and a cooperative company is permitted business in a region
AB . The region A is divided to regions Aa (a = 1, . . . , nA ),and each cab of the cooperative company is distributed to any
region Aa . The region Aa is called “Area a” below.

©ISAROB 2011

4. COMPUTATIONAL EXAMPLE
Simulations which learn the rules and evaluate them are performed to verify the validity of proposed approach for distribution cruising taxi problems. The Rules learned by GBML arecompared with a heuristic rule by evaluated values.
4. 1 Setting
Simulator: The cooperative company’s cabs go to its own
dispatching area by the shortest distance, and move randomly

721

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

in the dispatching areas after arrival. The competitive company
uses a proportion rule, this rule dispatches proportional number
cabs to the ratio of estimated number of passengers in each area
to whole areas. The passengers appear only in permitted business region AB . The number of emerging passengers in wholeareas is 3 per a time unit on average. The number of the areas is100, and the number of the permitted business areas is 64.
Priority Rule: The priorities are calculated by weighted
equations with properties as below. The areas have more priority, in which more passengers appear probably and more nearly.
The areas have more priority, in which less cabs of the cooperative company, The areas have more priority, in which morepassengers appear probably, The prior areas are decided at random. The weight of equations have been normalized in refer toresults of preliminary experiment.
GBML: The number of individuals is 30, and the numberof generations is 100. Each individual is simulated 2 times ineach condition (I) ∼ (III), about random variables for passengers, weather and traffic jam, and the average of evaluated value
are treated as a fitness of the individual. The conditions of passenger appearance are combination of 4 patterns. The patterns
are illustrated in Fig. 3, and the combinations are (I): pattern 1
in the noon and pattern 2 in the night, (II): 3 and 3, (iii) 4 and 4
which are made by different seeds. GBML searches the weight
coefficients from 0, ±0.25, ±0.5, ±0.75 or ±1. The feature
quantity space is composed of variables defined as, deviation of
the number of cabs in each area, estimated number of appearing
passengers in whole areas, ratio of cabs carrying no passengers,
time slot. The space is divided by small or large of each variable.

Pattern 1

plotted in Fig. 4. The result for evaluation the acquired rule andthe proportion rule is plotted in Fig. 5.

(a)(b)
Fig. 4 Learning processes of GBML

(a)Fig. 5 Evaluation results(b)

From the figures, the acquired rules are valid as well as the
proportion rule in all conditions, and better than substantially inparticular situations.

5. CONCLUSION
The optimization problems and incompleteness on observation are described. By using them, the optimization model withincomplete information is defined. In addition, in order to confirm utility of the proposed model, it applies Dispatching Cruising Taxi Problems. As the result, the proposed model has potential to express the taxi problems, and to acquire the good solutions. The following are left for future study: to improve theoptimization model focused on incompleteness of information,and to apply the proposed model to other problems for future
works.

Pattern 2

REFERENCES

Pattern 3[1] Ian Jordaan: Decisions under Uncertainty, Cambridge University
Press (2005).
[2] Zhiang Lin: Environmental determination or organizational design:
An exploration of organizational decision making under environmental uncertainty, Simulation Modeling Practice and Theory, 14,
438–453 (2006).
[3] Enrique Alba, Christian Blum, Pedro Asasi et al.: Optimization
Techniques for Solving Complex Problems, Wiley (2009).
[4] Wong K.I., Wong S.C. and Yang H.: Modeling urban taxi services
in congested road networks with elastic demand, Transportaion Research Part B, 35, 819–842 (2001).
[5] Yu Jianxin, Zhou Xiaomin and Zhao Hongyu: Design and Implementation of Taxi Calling and Dispatching System based on GPS
Mobile Phone A Research for LBS Application Teaching Case,
Proceedings of 2009 4th International Conference on Computer
Science & Education, 1163–1169 (2009).
[6] Sakakibara K., Tamaki H., Murao H. et al.: Genetics-Based Machine Learning Approach for Flexible Shop Scheduling Problems: Rule Acquisition for Real-time Scheduling (in Japanese), Journalof Information Processing Society of Japan, 45, 2, 91–98 (2004).

Pattern 4

Fig. 3 Examples of appearance patterns of the passengers

Example Problem
The simulations for learning are performed in 2 cases; (a) the
cooperative company has 10 cabs, and the competitive company
has 20 cabs, (b) the cooperative company has 80 cabs, and thecompetitive company has 160 cabs, and each case are learned 3
trials. Moreover, the best solutions of each trail and the proportion rule are evaluated 4 times in each condition (I) ∼ (III) andall conditions.
4. 2 Result
The learning process of the best solution in every trial are©ISAROB 2011

722

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Prime Number Generation Using Memetic Programming
Emad Mabrouk1, Julio César Hernández Castro2 and Masao Fukushima3
1

Dept. of Applied Mathematics and Physics, Graduate School of Informatics, Kyoto University, Kyoto 606-8501, JAPAN.
Dept. of Mathematics, Faculty of Science, Assiut University, Assiut, EGYPT.
(hamdy@amp.i.kyoto-u.ac.jp)
2
School of Computing, University of Portsmouth, Buckingham Building, Lion Terrace, Portsmouth PO1 3HE, UK.
(Julio.Hernandez-Castro@port.ac.uk)
3
Dept. of Applied Mathematics and Physics, Graduate School of Informatics, Kyoto University, Kyoto 606-8501, JAPAN.
(Tel: +81-75-753-5591; Fax: +81-75-753-4756)(fuku@i.kyoto-u.ac.jp)

Abstract: For centuries, the study of prime numbers has been regarded as a subject of pure mathematics in number
theory. Recently, this vision has changed and the importance of prime numbers increased rapidly especially in
information technology, e.g., public key cryptography algorithms, hash tables, and pseudorandom number generators.
One of the most popular topics that attract attention is to find a formula that maps the set of integers into the set ofprime numbers. However, up to now there is no known formula that produces all primes. In this paper, we use a hybrid
evolutionary algorithm, called the Memetic Programming (MP) algorithm, to generate mathematical formulas that
produce distinct primes. Using the MP algorithm, we succeeded to discover an interesting set of formulas that produce
sets of distinct primes.
Keywords: Hybrid Evolutionary Algorithm, Iterated Local Search, Memetic Programming, Prime Number

I. INTRODUCTION
A natural number grater than 1 is called a prime if it
is only divisible by 1 and itself. The study of prime
numbers and their properties have attracted
mathematicians for several centuries. Questions related
to prime numbers have puzzled mathematicians formany years, e.g., “is there a formula that maps the set of
integers into the set of primes?” Recently, several
applications in the field of information technology haveincreased the importance of prime numbers, andchanged the vision that classifies the study of primes aspure mathematics.

II. MEMETIC PROGRAMMING

The Memetic Programming (MP) algorithm is a new
evolutionary algorithm that hybridizes the well-known
Genetic Programming (GP) algorithm, Koza [1], withsome local search procedures over a tree space tointensify promising programs generated by the GP
algorithm. The aim of this paper is to use the MP
algorithm to generate some mathematical formulas
which produce distinct primes for a set of consecutive
integers.

The Memetic Programming algorithm is a hybrid
evolutionary algorithm that searches for desirable
computer programs as outputs. Computer programs
treated in the MP algorithm are represented as trees inwhich leaf nodes are called terminals and internal nodes
are called functions. Depending on the problem at hand,the user defines the domains of terminals and functions.
In the coding process, the tree structure of a solution
should be transformed to an executable code. Fig. 1
shows two examples of MP programs represented astrees, along with their executable codes.

The paper is organized as follows. In the next
section, we introduce the MP algorithm briefly. In
Section III, we report the results of three experiments
for generating formulas which produce distinct primes.
Finally, conclusions make up Section IV.

©ISAROB 2011

The main loop of the MP algorithm consists of twophases; diversification phase and intensification phase.

723

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

In the diversification phase, the MP algorithm
guarantees the diversity in the current population byusing the GP strategy. Specifically, the MP algorithm
selects some programs using a suitable selection
strategy, and generates a new population from thecurrent one by using crossover and mutation operators.
Fig. 2 illustrates an example of applying thecrossover and mutation operators for some trees. In theintensification phase, the MP algorithm uses a set of
local search procedures to intensify elite programs ofthe current population. These local search procedures
will be described in the following subsection. Using
these procedures, the Local Search Programming (LSP)
algorithm will be introduced in Subsection 2, which isused in the intensification phase of the MP algorithm.
Finally, the whole picture of the MP algorithm will besketched in Subsection 3.
changing its structure. The altered nodes are chosenrandomly and replaced by alternative ones, i.e., aterminal node is replaced by a new terminal value and afunction node is replaced by a new function of the same
number of arguments as the original one. Grafting
search generates a new tree X̃ from a tree X by replacingsome of its terminals, chosen randomly, by branches ofdepth   1, where these branches are generated
randomly. In contrast to grafting search, pruning search
generates an altered tree X̃ from a tree X by cuttingsome of its branches of depth   1 and replacing themby new terminals, where these branches and terminals
are chosen randomly. One may note that X and X̃ havedifferent tree structures in case of applying the graftingor pruning procedures. Fig. 3 shows examples ofgenerating a new tree X̃ from a tree X by applying

1. Local Searches over Tree SpaceThe local search procedures aim to generate newtrees in a neighborhood of a given tree X. In thissubsection, we discuss two types of local searches;static structure search and dynamic structure search,
Mabrouk et al [2, 3]. Static structure search explores theneighborhood of a tree by altering its nodes withoutchanging its structure. On the other hand, dynamic
structure search changes the structure of a tree by
expanding its terminal nodes or cutting its subtrees.
Shaking operator is used as a static structure search
procedure, while Grafting and Pruning operators are
introduced as dynamic structure search procedures.
Shaking search generates a new tree X̃ from thecurrent one X, by altering some nodes of X without©ISAROB 2011724

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

function. The addition function “+” is used as thelinking function in this paper. To generate a gene in theinitial population, we generate a temporary random geneconsisting of two parts, head (functions and terminals)
and tail (terminals only). Then, we adjust the final formof the gene by deleting unnecessary elements, based on
the functions and terminals that are generated randomly
within the gene. For more details see Mabrouk et al [2,3].
Once the initial population is generated, it will be
evolved and improved using the MP operations; i.e.,crossover, mutation, shaking, grafting and pruning. For
each problem to solve, the function set, the terminal set,
the set of representation parameters, the set of searchparameters, and the fitness function must be determined
before calling the algorithm. The set of representation
parameters contains the head length hLen of an initialgene, the maximum length MaxLen of a gene, thenumber of genes nGenes of each program. On the other
hand, the set of search parameters, which guide thealgorithm during the search process, consists of the
population size nPop, the number of generations nGnrs,
the number of programs nLs that are selected to apply
local search procedures, the number of trial programs
nTrs that are generated in the neighborhood of aprogram using a local search procedure, and the
maximum number of non-improvements nFails. In
particular, nFails is used to terminate the LSP
algorithm. For more details about these parameters see
Mabrouk et al [2, 3].

shaking, grafting and pruning procedures. For moredetails see Mabrouk et al [2, 3].
2. LSP Algorithm
In this subsection, we introduce the LSP algorithm
to discover the best program in the neighborhood of thecurrent program X. This algorithm uses the local search
procedures described in the previous subsection to
generate trial programs in the neighborhood of thecurrent one. This process iterates until the termination
condition is satisfied, and then the algorithm returns thebest program found. Fig. 4 shows the flowchart of the
LSP algorithm, where it returns the original program X
in case of no improvement.

III. NUMERICAL EXPERIMENTS

3. MP Algorithm
The main target of the MP algorithm is to improve
the results of the GP algorithm by performing a local
search for some promising programs. If the search
process succeeds to reach the area near an optimal
solution, then a simple local search algorithm cancapture that optimal solution easily. Fig. 5 shows the
flowchart of the MP algorithm that behaves like the GP
algorithm if the LSP algorithm fails to improve theselected programs. However, in this case the MP
algorithm will be more costly than the GP algorithm,
because of the computational effort spent through theintensification phase.

In this section we report the results of three different
experiments for the MP algorithm to generate formulasthat produce primes. The parameter values for the MP
algorithm during all experiments are hLen = 3, MaxLen= 40, nGenes = 3, nLs = 4, nTrs = 4, nFails = 1, nPop
= 100 and nGnrs = 100. In addition, the selection
strategy for the diversification phase is the tournament
selection of size 4. The fitness value for each program iscomputed as the maximum number of consecutive
integers in the interval [100, 100] for which theprogram produced distinct primes.
1. Polynomials
In this experiment, we used the set of binary
functions    as the function set, i.e., eachprogram generated by the MP algorithm represents apolynomial. In addition, we used x 2 3 5 7 9 as theEach program treated in the MP algorithm consistsof one or more gene(s), where a gene represents a subtree consisting of terminal and function nodes. Genes ina program are linked together by using a suitable linking©ISAROB 2011725

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

example, (8x3  69x2  461x  176) / (8x  3), withthe fitness value 42.

terminal set, where x is an integer. We performed 1000
independent runs for the MP algorithm, and we got anumber of polynomials with the fitness values up to 40.

3. Composition Functions
Since we have already got new independent
polynomials that can generate different sets of distinct
primes, we can use these polynomials to composite new
formulas. In this experiment, the output of a program
evolved by the MP algorithm is expressed as a linear
composition of its genes with some independent
polynomials that produce distinct primes. Suppose that
G1, G2 and G3 are the genes of a program evolved by the
MP algorithm. Then, the output formula of this program
is composed as f x  G1  P1  G2  P2 G3  P3,
where P1, P2 and P3 are independent polynomials. Using
this strategy, we got new formulas that produce distinct
primes up to 59, for example, 7 / (81x  27)  (x2  x 41) + 9 / (5  45x) (8x2  22x  647), with the fitness
value 59.

Table 1 shows some of polynomials which generated
by the MP algorithm. The first three polynomials in
Table 1 have already been found in the literature.
Specifically, the first two polynomials are the Euler andLegendre polynomials, and the third one is thepolynomial generated by the CGP algorithm, Walker[4]. During our experiments, these three polynomials
were found frequently. To the best of the authors’knowledge, the other polynomials seem to be newpolynomials.
Table 1. Polynomials generated by the MP algorithm toproduce distinct primesx
Polynomial
Fitness
1 x2  x  41
2 x2  x  41
3 x2  3x  43
4 9x2  33x  71
5 4x2  50x  197
6 8x2  22x  647

40{1,…,40}

40
40
40
40
40

{0,…,39}{2,…,41}{-28,…,11}{-13,…,26}{-19,…,20}

IV. CONCLUSION
The MP algorithm has succeeded to generate several
new formulas that produce sets of distinct primes. Someof the new formulas are polynomials that are able toproduce up to 40 distinct primes for a set of consecutive
integers. Other rational functions are also generated andthey are able to produce up to 59 distinct primes for aset of consecutive integers.

In the literature, researchers consider the first threepolynomials in Table 1 to be different polynomials.
However, all of these polynomials produce the same setof primes for different values of the independent
variable x. Specifically, one can generate those entire
polynomials one after another by using x  x   forsome integers . On the other hand, the last threepolynomials in Table 1 produce different sets of distinct
primes for different sets of consecutive integers.
Therefore, we consider these three polynomials to be
the best results for the current experiment since all ofthem are different and independent.

REFERENCES[1] Koza JR (1992) Genetic programming: On the
programming of computers by means of natural
selection. MIT Press, Cambridge[2] Mabrouk E, Hedar A, Fukushima M (2008) Memetic
programming with adaptive local search using tree data
structures. In: Chbeir R et al (eds.), Proceedings of the
5th International Conference on Soft Computing asTransdisciplinary Science and Technology (CSTST08),
Cergy-Pontoise, Paris, France, October 27–31, 2008, pp.
258–264[3] Mabrouk E, Hedar A, Fukushima M (2010) Memetic
programming algorithm with automatically defined
functions. Technical Report 2010-015, Department ofApplied Mathematics and Physics, Kyoto University,
Japan[4] Walker JA, Miller JF (2007) Predicting prime
numbers using Cartesian genetic programming. In:
Ebner M et al. (eds.), Proceedings of the 10th European
Conference on Genetic Programming (EuroGP),
Valencia, Spain, April 11–13, 2007, pp. 205–216

2. Rational Functions
We performed another experiment to find formulasthat produce primes with fitness values greater than 40.
In this experiment, we modified the function set in theprevious experiment to include the protected division
operator , where x  y  1 if y  0, and x  y  x  yotherwise. In this case, programs of the MP algorithm
will produce real values. Therefore, we let the nearest
integer less than or equal to the produced real value be
the output of the program. Using the new function set,
we got several new formulas that produce up to 42
distinct primes for a set of consecutive integers, for©ISAROB 2011726

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

EMG control of a pneumatic 5-fingered handusing a Petri net
Osamu FUKUDA*, Jonghwan KIM*, Isao NAKAI**, and Yasunori ICHIKAWA***

National Institute of Advanced Industrial Science and Technology,
Measurement Solution Research Center, 807-1 Shuku-machi, Tosu, Japan**SQUSE Inc., 67-21 Nishinokyousyokushi-chou, Nakagyou-ku, Kyoto(Tel : 81-942-81-3633; Fax : 81-942-81-3698)(fukuda.o@aist.go.jp)

Abstract: This paper presents a control method of a 5-fingered artificial hand using EMG signals. Our targeted artificial
hand is driven by pneumatic actuators to reduce its weight. Also we do not use electro pneumatic regulators but use
ON/OFF solenoid valves to simplify the control system. The pneumatic hand has 15 degrees of freedom, and it seems
difficult to discriminate all finger motions only from the EMG signals. Therefore, in this paper, we describe typical
hand motions using a Petri net, and efficiently control the finger motions based on this model. Each state of the Petrinet indicates a step of the hand posture to complete the intended motion. Simultaneously, this state corresponds to the
ON/OFF pattern of the 15 solenoid valves. This enables the operator to control the 5-fingered dexterous hand smoothly,transiting the state in the Petri net according to the EMG motion discrimination. We conducted experiment to verify
validity of the proposed method. In the experiment, the typical 5 motions (Spherical grasp, Power grip, Hook grip, Key
grip, Precision grip) were successfully performed using the 6-channel EMG signals measured from the operator's
forearm.
Keywords:

EMG, pneumatic hand, Petri net, motion discrimination, neural network

actuator can be driven by a lower air pressure than theprevious one. Also, the system becomes compact usinga small compressor. But, it is considerably difficult tocontrol the multi-channel finger joints precisely,
because non-linear relationship exists between the input
air pressure and the output tension.
This paper proposes a novel control method for a 5fingered prosthetic hand using EMG signals. We adopt
pneumatic actuators to control the multiple joints to
reduce the hand weight. Also, we use ON/OFF solenoid
valves instead of electro pneumatic regulators tosimplify the control system. The pneumatic hand has 15
degrees of freedom, so that it appears to be difficult todiscriminate the motion of all joints. Therefore, we
design the state transition model for the typical handmotions using a Petri net. This model enables theoperator to control dexterous finger motions based on
the simple discrimination of the discrete hand motions,such as hand opening, hand closing, and so on. Eachstate of the state transition model corresponds to thehand posture completing to the intended motion, and the
ON/OFF pattern of the 15 solenoid valves are changed
according to this state. Transiting the state in the Petrinet, the operator can skillfully control the 5-fingered
dexterous hand smoothly.

I. INTRODUCTION
Until now, many researchers have designed EMGcontrolled artificial hands for amputees. Its motion waspreviously limited only the opening and closing hand,but novel 5-fingered dexterous hands have been
emerged, due to the progress in robotics andmechatronics fields [1]-[3]. For example, Otto Bock Inc.
[1] and TOUCH BIONICS Inc. [2] have launched
Michelangelo Hand and i-LIMB Hand, respectively.
These hands are controlled by EMG signals and realize
human-like finger movements.
However, a servo motor is used as the actuator ofeach finger joint, it is inevitable that its weight raises aphysical burden of the operator during a long time ofoperation, with increasing number of degrees of
freedom. Accordingly, a pneumatic actuator was tried touse as the substitute for the servo motor, and several
researchers developed the robotic hands driven by thepneumatic actuator [4], [5]. The pneumatic actuator has
a great advantage because it has a very light weight sothat it can be expected to reduce the hand weightsignificantly. For example, Tsujiuchi et al. and SQUSE
Inc. [6] newly designed the small pneumatic actuator,
which can be installed into finger joints, and developed
5-fingered artificial hand [4], [5]. This pneumatic©ISAROB 2011

727

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig. 1 Control strategy
This paper is organized as follows. The control
strategy of the 5-fingered pneumatic hand is introducedin Section II, the structure of the developed system is
explained in Section III, the experiment is reported in
Section IV, and Section V concludes the paper.

the places where the token exists, 15 solenoid valves arecontrolled depicted in (b).
In this paper, we select 5 motions for thediscrimination targets: 1. Spherical grasp, 2. Power grip,
3. Hook grip, 4. Key grip, 5. Precision grip. Thesemotions were frequently used in the previous research,and have possibility to discriminate with high accuracy
using the EMG signals. It is noted that the proposed
control method is not restricted within these 5 motions.
We can choose appropriate motions according tooperator's operability.

II. Control strategy
This paper proposed a novel control strategy of the
5-fingered pneumatic hand. The pneumatic hand used inthis study has the 15 pneumatic actuators, and eachactuator is controlled by the ON/OFF solenoid valve.
Therefore, we have to consider how ON/OFF patterns
are designed using 15 solenoid valves, and when thesepatterns are changed. Fig. 1 explains the control strategyof the hand motion based on the proposed method. (a)expresses the state transition model described by the
Petri net, and (b) depicts the ON/OFF pattern of the 15channel solenoid valves. The Petri net is composed ofsome places, transitions and arcs, that denote the statesof the hand postures, the conditions based on the EMG
discrimination, and the flows of the hand postures,
respectively. At the start of the operation, a token is set
on the initial place. The transition means that the
duration time of a discriminated motion exceeds theprespecified threshold. For example, the token is moved
from I to V, if Spherical grasp is discriminated in thesystem and its duration time exceeds the prespecified
thresholds. Conversely, if No motion is discriminatedand its duration time exceeds the threshold, the token is
moved from V to I and the hand is opened. According to©ISAROB 2011

III. Components of the system
Fig. 2 indicates the components of the system. The
EMG signals are measured by the EMG amplifiersystem (DELSYS Inc., BAGNOLI EMG SYSTEM),
and are transferred to the personal computer (HewlettPackard Inc., HP Pavilion, Intel(R)Core(TM)2 Duo
CPU 2.27GHz) via the multi function DAQ (National
Instruments Inc., NI USB-6229).
The signal processing, which is conducted in the
personal computer, is composed of 3 parts: 1. Feature
extraction, 2. Motion discrimination, 3. patterngeneration of the solenoid valves. Each process is
explained in the following sections.
1. Feature extraction
First, the measured EMG signals are rectified andfiltered out through the 4th order Butterworth filter (cutoff frequency: 1.0[Hz]) to extract the amplitude
information. The filtered signals are resampled with asampling frequency of 50.0[Hz], and defined as 𝐸𝑙 (𝑛).
728The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig. 2 Components of the system
Then, the pattern vector 𝑥𝑙 (𝑛) for the motion
discrimination is calculated as𝑥𝑙 (𝑛) =signals are returned to the control unit through the Multifunction DAQ. The compact compressor (SQUSE Inc.,
MP-2-C) is installed in the control unit, and it drives thepneumatic actuators in the finger joints.

𝐸𝑙 (𝑛)/𝐸𝑚𝑎𝑥𝑙∑𝐿𝑙′ =1 𝐸𝑙′ (𝑛)/𝐸𝑚𝑎𝑥𝑙′IV. Experiment
We conducted experiment to verify the validity ofthe proposed method. Subject controlled the 5-fingered
pneumatic hand using his EMG signals. Before theexperiments, we explained the purpose of theexperiments and obtained informed consents from thesubject. Subject was briefly trained for about 30 minutes
before the measurements. The EMG signals were
measured from 6 electrodes, and the number of thediscrimination target were 5 motions: 1. Spherical grasp,
2. Power grip, 3. Hook grip, 4. Key grip, 5. Precision
grip, which frequently arise in daily activities. The 6
electrodes were placed on the right forearm surrounding
regular intervals. The operator does not have to set theelectrodes on the associated muscles strictly, due to thelearning ability of the neural network. The neuralnetwork can learn the pre-extracted EMG patterns toadapt the system to individualities, electrode locations,and so on. The parameters in the signal processing wereset as follows: the number of the learning data was 50
samples for each motion. The threshold of the duration
time of the transitions was 0.5[sec].
Fig. 3 shows examples of the motion control using
the developed system. Subject performed the 5 motions.
The figure indicates the 6-channel of EMG signals, themuscular contraction information, the changes of theplace in the Petri net, and the discrimination results.
During the operation, no discrimination error wasobserved and the token in the Petri net appropriatelytransited based on the discrimination result. We

is the maximum value of 𝐸𝑙 (𝑛) while thewhere 𝐸𝑚𝑎𝑥𝑙operator executes the maximum voluntary contraction.
Also, the muscular contraction level 𝐹(𝑛) is calculated
to recognize the motion.
𝐿1
𝐸𝑙 (𝑛)
𝐹(𝑛) = � 𝑚𝑎𝑥
𝐸𝑙
𝐿
𝑙=1

If 𝐹(𝑛) exceeds the prespecified threshold 𝛼, thesystem considers motion has occurred.
2. Motion discrimination
In the proposed method, we use a neural network todiscriminate the operator's intended motion. The neuralnetwork can learn feature patterns and adapt the system
to the differences due to the individuality, the electrodelocations, and so on. Before the motion discrimination,
the neural network has to be trained using learning
samples. The initial value of the weight coefficient is arandom number from 0 to 1. The mean square error isused as the energy function, and the learning is carriedout to minimize this function. The back propagation
method is adopted for the learning algorithm.
3. Pattern generation of the solenoid valves
Finally, the ON/OFF patterns of the solenoid valves
are generated. In this process, we use the state transition
model which describes typical grasp motions using a
Petri net. The control signal for the ON/OFF solenoid
valves is generated, transitioning the state in the Petrinet according to the EMG motion discrimination. These©ISAROB 2011

729

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig. 3 Control example
confirmed that all 5 motions were successfully carriedout in regular order.

performance. Also, we will optimize the mechanism of
the pneumatic hand for amputee's daily activities. A partof this work was supported by Adaptable and Seamless
Technology Transfer Program through Target-Driven R& D of Japan Science and Technology Agency.

V. CONCLUSION
This paper proposed the novel EMG control method
for 5-fingered pneumatic hand. We introduced thetypical hand motion model described by a Petri net into
the control system, because it was difficult to control 15
degree-of-freedom hand precisely using only the EMG
signals. Each place in the Petri net corresponds to the
ON/OFF pattern of the solenoid valves, and the token is
moved based on the result of the motion discrimination.
The pneumatic actuators installed in the finger joints arecontrolled according to places where the token exist.
Consequently, the operator can control 5 finger motions
smoothly and naturally discriminating only the discrete
motions, e. g. hand grasping, hand opening, and so. on.
To verify the validity of the proposed method, theexperiment was conducted. In the experiment, we
examined the control performance of 5 typical grasping
motions: 1. Spherical grasp, 2. Power grip, 3. Hook grip,
4. Key grip, 5. Precision grip. The experimental result
confirmed the validity of the proposed method.
In future study, we would like to re-design
discrimination algorithm to improve the discrimination©ISAROB 2011

REFERENCES[1] http://www.ottobock.com[2] http://www.touchbionics.com[3] M. C. Carroza, G. Cappiello, S. Micera,
B. B. Edin, L. Beccai, C. Cipriani:
Design of a cybernetic hand for perception and action,
Biological Cybernetics, Vol. 95, No. 6, pp. 629644, 2006.
[4] Nobutaka TSUJIUCHI, Takayuki KOIZUMI,
Shinya NISHINO, Hiroyuki KOMATSUBARA,
Tatsuwo KUDAWARA, Masanori HIRANO:
Development of Pneumatic Master-Slave Hand andJoint Control, The Japan Society of MechanicalEngineers, Vol. 74, No. 741, pp. 223-228, 2008.
[5] Hiroyuki Takeda, Nobutaka Tsujiuchi, Takayuki
Koizumi, Hiroto Kan, Masanori Hirano, Yoichiro
Nakamura: Development of Prosthetic Arm with
Pneumatic Hand and Tendon-Driven Wrist, Proceedingsof the 31st Annual International Conference of the IEEE
Engineering in Medicine and Biology Society, pp. 50485051, 2009.
[6] http://www.squse.co.jp/index.html

730

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Depth Calculation by using Face Detection ASIC
Seung Min Choi, *Jae-chan Jeong , Jaeil Cho
Electronics and Telecommunications Research Institute*University of Science and Technology{ccsmm, jicho }@etri.re.kr*channij@ust.ac.kr

Abstract - Real-time processing at mobile devices is always a hot issue. Although face detection and stereo matching,specially, are core elements for HRI application, they are one of the processes with heaviest loads at real-timeprocessing. In this paper, we present novel methods of calculating disparity and detecting face with one chip, named
Altair, which can detect 32 frontal faces/frame at 30 fps. Stereo image can be inputted into Altair, because it is designed
to detect face on frame by frame and has two input ports. If the left image is inputted on N-th frame time and then theright image is inputted on (N+1)-th frame time, we can get face information of left and right images with one frame
delay. Then we can calculate the disparity of face with this information and we can easily estimate distance from robot
to human with it. When one or more faces are detected, we classify various cases to identify correspondent faces. We
can detect faces and calculate the distances of them at 15 fps and can adjust the frame rate.
Keywords:

HRI, HCI, face detection, stereo vision, disparity, real-time

I.

INTRODUCTION

II. Stereo Camera and face detector
Disparity can be calculated by finding correspondenceof stereo cameras. Fig.1 shows geometry of stereo
camera system which is consists of two camera andfield of object models. Because in fig.1 there are 4
triangles which have properties of triangle similarity,
depth of disparity equations can be derived from themlike table 1.

We introduce a novel method of calculating
disparity of stereo camera with face detection chip.
Because, real-time processing at mobile devices has
been always a hot issue, many researchers have studied
about speedy and light image processing algorithms foryears. Although, face detection and stereo matching,specially, are core elements for HRI application, they
are one of the applications with heaviest loads at
processing.
In this paper, we present novel methods ofcalculating disparity and detecting face with one chip,
named Altair, which can detect 32 frontal faces/frame at
30 fps. Stereo image can be inputted into Altair, because
it is designed to detect face on frame by frame and has
two input ports. If left image is inputted on N-th frame
time and then the right image is inputted on (N+1)-thframe time, we can get face information of left and rightimages with one frame delay. Then we can calculate thedisparity of face with this information and we can easily
estimate distance from robot to human with it. Whenone or more faces are detected, we classify various
cases to identify correspondent faces.

©ISAROB 2011

Fig.1. Geometry of Stereo Camera Model
Table 1. Equation of Stereo Camera Model

731

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

In generally finding correct disparity of stereo images
is key point of stereo correspondence problem. So,
many stereo vision algorithms are suggested over theyears. According to [1], however, there are few realtime algorithms (over 30 fps) as shown in taxonomy.
Because of difficulty of real-time processing and huge
power consumption for stereo matching, in some
application where disparity of whole image pixel are notneed, only disparity or depth of ROI(Region Of Interest)
is important value in the camera image. For example,
when a service robot has to keep a distance and serves
some convenience to human, it only needs to know the
distance from human to it. If it has a face detection
function with its platform, it’s easy to know the distancefrom human.
We present the way that calculates the distance withface detection chip. Fig.2 shows the outline of the novel
method of calculating disparity of stereo camera withface detection chip. Two of camera image enters facedetection unit in time division manner. Then facedetector finds face and calculate the difference of centerof face in left and right images.

III. Disparity Calculation with Face Detector

Fig.4. Flow Chart of our new method.
Fig.4 shows the flow chart of our novel method for
calculating disparity of stereo camera with facedetection chip. After start left and right camera images
input into the multiplexer (MUX), then left or rightimage outputs from MUX in regular sequence by‘Select’ signal. Then this interlaced image with frameorder enters face detection block whose output isinformation of faces that consists of following table 2.

Fig.2. Outline of novel method.
We use the Altair as a face detection unit in fig. 3.
Altair is the ASIC chip( 1.2 ´ 1.2cm 2 ) for face detectionthat can detect 32 person per frame at 30 fps of
QVGA(320x240) image[3]. It covers frotal and± 15o slant face. The chosen method of face detection is
the AdaBoost algorithm, which consist of MCT(Modified Census Transform) confidence

Table 2. Face information register of Altair

Fig.3. Altair (Frontal face detect ASIC)©ISAROB 2011

After getting face information from Altair, next step isto compare information of faces on left frame and onright frame as shown in table 3.

732

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

We multiply the constant ‘k’ to depth, because
B(baseline),f(focal length) and wpp(width per pixel)
may have measurement error, and it’s hard to knowcorrect parameters for the combination of lens andCMOS sensor used in this experiment(‘k’ is aexperience value).

Table 3. Comparing face and Calculating disparity
item

Informationnote

On the same
epipolar line ?
Has the same
size of face‘x’ coordinate ofcenter point

Verticaldirection

Has the similarenergy value?

Sum ofConfidence

Calculate
Disparity‘x’ coordinate ofcenter point

Size of face
Sum of featureof Adaboost
algorithm(1/probability)
Horizontal
direction

B´ f
B´ f
= k´
d
disparity ´ wpp
where, our system has
Depth =

B = 0.24m
f = 0.0045m
wpp = 0.0000123m
k = 0.85

Another test set consists of 7 frames where a human has
a paper written the real measured distance from camera
to him as shown in fig.6. The distances are 0.5m ~ 3.5m
with step of 0.5m. After gaining disparity with thissecond test set by our new method, we compared it toreal measured distance as shown fig. 7. and table 4. We
can detect faces and calculate the distances for human inrange of 0.5m ~ 3.5m with 0.5~ 8.9% error

VI. Experiment
We capture two sets of stereo image with system in fig.
4. One is the continuous image set where a humanmoves around camera as shown in fig. 5

Fig.6. test image set with real distance value (0.5~3.5m)

Fig.5. depth calculation for moving human
A human is near camera at first and move far from itthen comes to near again. Because face was not detected
for a while at around about 350-th frame, face occlusion
occurred and disparity and depth could not be estimated.
Depth can be converted from disparity by using
following equation which is modified version of table 1.

©ISAROB 2011

Fig.7. our method’s accuracy of depth calculation

733

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 4. Result of Depth calculation(average of tests in 5 times)
Frame

Depth(m)(Inference)

Depth(m)(Real Measurement)

Error(m)(Difference of two)

1
2
3
4
5
6
7

0.54480.96930.14972.01712.48783.10983.2450

0.5
1.0
1.5
2.0
2.5
3.0
3.5

0.0448(8.9%)0.0307(3.1%)
0.0073(0.5%)
0.0171(0.9%)
0.0122(0.5%)
0.1098(3.7%)
0.2550(7.0%)

Because Altair has two ports for input image andstereo camera inputs are multiplexed with time divisionmanner, frame rate for depth result is half (15 fps) of
Altair’s that. By configuring Altair chip, we can adjust
the frame rate of face detection and depth calculation.

V. CONCLUSION
We can calculate the disparity of face with thisinformation and we can easily estimate distance fromrobot to human with it. When one or more faces aredetected, we classify various cases to identify
correspondent faces. We can detect faces and calculate
the distances for human in range of 0.5m ~ 3.5m with
0.5~ 8.9% error
We are planning to do experiment for the case where
there are many people a frame

Acknowledgement
This work was supported by the R&D program of the
Korea Ministry of Knowledge and Economy (MKE)and the Korea Evaluation Institute of Industrial
Technology (KEIT). [Project KI001813, Developmentof HRI Solutions and Core Chipsets for u-Robot].

REFERENCES[1] Scharstein, Daniel and Richard Szeliski., “Ataxonomy and evaluation of dense two-frame stereo
correspondence algorithms”, International Journal ofComputer Vision. 2002[2] Jiho Chang, Seung Min Choi, at el, ” Design and
Implementation of Real-Time stereo vision ASIC
and embedded system”, IEEK 2010 Summer
Conference, CFP-326, June, 2010.
[3] Seung Min Choi, at el, ” Implementation of Robust
Real-time Slant Face Detection ASIC”, IEEK 2010
Summer Conference, CFP-326, June, 2010.

©ISAROB 2011734

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

An experimental study on interactive reinforcement learning
Tomoharu Nakashima,Yosuke Nakamura, Takesuke Uenishi and Yosuke Narimoto
Osaka Prefecture University, 1-1 Gakuen-cho,Nakaku,Sakai,Osaka(Tel : 81-72-254-9351; Fax : 81-72-254-9351)(nakashi@cs.osakafu-u.ac.jp)

Abstract: Q-Learning is a popular non-supervised reinforcement learning (RL) technique that learns an optimal actionvalue function characteristic of a learning problem. Due to the complexity of some problems, the number of training
episodes to reach the convergence of the learning problem can increase drastically. In order to fasten the learning speedof an agent on a particular problem, researchers have been exploring interactive reinforcement learning (IRL), i.e. a way
to interact with an agent so that it does not learn to solve a problem only by itself. This paper proposes an interactive
reinforcement learning to try to fasten the learning speed of an agent. Especially, the combination of an agent asking
for advice and getting advice from supervisors was explored. A simple way to experiment this combination is an agent
evolving on a maze (a gridworld problem) trying to find its path to a fixed goal point. Experiments shows how aninteractive learning agent solve the problem compared to a classical learning agent.
Keywords: Q-Learning, interactive reinforcement learning, human machine cooperative systems

I. INTRODUCTION

which is the proposed method in this paper. After presenting the results of the experiments and their results,
we give the conclusions and future works.

Reinforcement learning methods [1] are quite popularin the robotic field but classical reinforcement learning
does not work so good in many cases on real environment. In order to overcome this fact, new reinforcement
learning algorithms, such as policy gradient [2], havebeen developed. Another idea was to transform classical reinforcement learning into interactive reinforcement learning to speed up the learning of behaviors. Interactions have been stated in many terms like interactions with a human teacher [3], [4], or incorporation ofadvice [5].
There are of several kinds of interactive learning. Theinteractions can come from an agent asking for advice[6] — When it does not know what to do — to an advisor that completely knows the environment. Anotherone is a human advisor or a computed advisor that gives
advice to a learning agent to correct its behavior whilelearning by giving variable amount of reward/penalty or
directly the action to perform [7], [8]. In multi-agent
systems, advice can come from the other agents as a wayof sharing experience by observing their actions [9], by
being able to evaluate the weight of the other agents advice [10], by requesting episodic advice to other agents
solving the same kind of problems [11], or by getting information from other agents about the environment thatthe learning agent is not (yet) aware of [12]. On theother hand, advice can be seen as a way of transferring
information from previous experience to a new similar
situation [13], [14], [15].
In all of those explorations in IRL, the learning was
fastened by the interactions between the learning agent
and the outside world.
In the following chapters, an overview of reinforcement learning will be shown, followed by the presentation of the interactive reinforcement learning method,©ISAROB 2011

II. OVERVIEW OF TRADITINAL
REINFORCEMENT LEARNING
1. Traditional reinforcement learning
A. Definition
Traditional reinforcement learning is the third modeof learning where an agent learns to act by itself in anenvironment by interacting with this environment andgetting feedback from it. The agent can choose to explore its environment by acting randomly or to exploit its
knowledge of the environment it got from previous experiences through trial and error. In addition, the learning agent reinforces the knowledge it gets from experiences by getting a numerical reward when doing a goodaction or a penalty when doing wrong or forbidden actions. The learning agent, by iterating experiences, aims
at taking actions that maximize the sum of the rewards
over the time to find an optimal policy.
B. Algorithm
There exists many kind of reinforcement learning algorithms such as Temporal Difference (TD) learning,
actor-critic learning, Q-Learning.
The basic reinforcement learning model is the following one:
• A set of possible states of the environment• A set of possible actions• Transition rules to go from the current state to thenext state735

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

• Rules to determine when an agent receives a reward
or a penalty

Algorithm 2 Q-Learning
Initialize Q(x, a) arbitrarily
Initialize T at 0
repeatInitialize x randomly
repeatChoose action a from S using policy derived
from Q (here the probability action selection)
Take action a and observe r and x0 (the next
state)
Update Qx←x
until x is terminal
Decrease T
until

Algorithm 1 shows a basic reinforcement learning algorithm.
Algorithm 1 Basic RL algorithm
Initialize the value function arbitrarily
Initialize the policy to evaluate
repeatInitialize x randomly
repeatChoose action a from S given by the policy forx
Take action a and observe r and x0 (the next
state)
Update algorithm’s value functionx ← x0
until x is terminal
until

III. INTERACTIVE REINFORCEMENTLEARNING
1. Proposed
methodinteractive

reinforcement

learning

In this research, two types of interactive reinforcement learning were combined: asking for advice modeand getting advice mode.

2. Q-Learning
A. Principle

A. Asking for advice

Q-Learning is an online reinforcement learning technique where the agent evolve in a completely unknown
environment. In Q-Learning, the agent approximates thepolicy function directly. Q-Learning algorithm is guaranteed to converge under some circumstances.
For this paper, the Boltzman distribution probability
was used for action selection. The Boltzman temperature factor T included in this distribution is used for the
trade-off of the exploration/exploitation rate.

When the probability for a learning agent to take anaction is almost the same for all the possible actions,an agent can ask for advice to an advisor. This advisor is completely aware of the environment of the agent
and is considered to know the action that can lead to a
maximum reward. When receiving an answer from thesupervisor on what action to perform, the agent does theadvised action and updates its state-action value as if it
was its own decision.

B. Detailed algorithm
B. Getting advice
The value function of a state and an action is calculated by the following equation:

While learning, an agent can receive advice from anadvisor agent that is resolving the same problem. The
supervisor agent can have different level of expertness:
BEGINNER, INTER., and EXPERT. The learning agent
acts differently in function of the expertness level of its
advisor.

Q(x, a) ← (1− )Q(x, a)+ (r +γ max Q(y, b)) (1)b∈Awhen x is the state of the agent, A the set of possible
actions, a, b ∈ A an action, r the reward, the learning
rate, and γ the discount rate. The probability of taking
an action is calculated by the Boltzman distribution as
follows:px (a) = ∑exp Q(x,a)
Tb∈A exp

Q(x,b)T

C. Constraints
As being in possession of advice from several sourcesat the same time may be confusing, to simplify the algorithm it has been decided that an agent cannot ask foradvice and get advice at the same time, with the following priority: asking advice is predominant over getting
advice. Moreover, in one episode, only one advisor canobserve an agent to avoid confusing advice.

(2)

At the end of each episode, the Boltzman temperaturefactor T is divided by a fixed step, to decrease the exploration rate. The algorithm is run in the following way:

©ISAROB 2011736

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

• INTER.: the learning agent update the state-value
related to the advisor advice, but take its own decided action.

2. Overview of the framework
At each step the learning agent calculates the probability of taking an action. When the agent does not knowwhat to do, it asks for advice to an advisor, in this case
the A* algorithm, then the learning agent takes the advised action and updates its map of the state-action values. After calculating the probability of taking an action,
if the learning agent does not ask for advice, the learning agent chooses the action it performs. The advisor isnotified of the chosen action the learning agent wants toperform, and from time to time decides to give an advice
knowing this action and the state of the learning agent.
Finally, when getting an advice, the learning agent acts
and updates its map of the state-action value in functionof the expertness level of the advisor giving the advice.

• EXPERT: the learning agent take the advisor action and update its state-action value as if it was its
own decision.

IV. EXPERIMENT
1. Maze problem

Fig. 1. System components
Fig. 2. Maze problem
3. Details on the framework
To take into account the advice in its policy, a learning
agent gets a variable reward from its advisor every time
it receives an advice. The advisor agent that gives advice
to the learning agent is chosen at the beginning of each
episode randomly. An agent can receive advice only
from advisor agent whose level of expertness is equal
or superior to its own. An agent’s expertness level isdefined the following way:

A. Principle
For this research, IRL was applied to the grid world
problem. In the case of this research, the grid world isa simple maze made of impassable walls. The principleof this problem is the following one: an agent is positioned on a randomly chosen starting point on a maze,and tries to find its path into the grid world to reachthe goal. The agent can perform four possible actions:
Direction={UP, DOWN, LEFT, RIGHT }. The following rules are applied: the agent receives a reward whenreaching the goal and a penalty when trying to go on awall or outside of the maze, otherwise nothing. If theselected action is a valid action (not a wall, or not theoutside of the maze),the agent performs the selected action, otherwise it remains at the same place. The agent
updates its action-state values after each move. Whenthe agent reaches the goal the episode is interrupted andthe agent is ready to begin a new episode. During an
episode the maximum number of move is limited to thenumber of squares of the maze, that is to say, the maze
width multiplied by the maze length.

• BEGINNER: success to resolve the problem <33%• INTER.: 33%  success to resolve the problem< 67%• EXPERT: success to resolve the problem ≥ 67%
Knowing the expertness level of an advisor agent, thelearning agent acts the following way when getting anadvice:• BEGINNER: the learning agent take the advised
action but does not update its state-action value.

©ISAROB 2011

737

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

B. Parameters details

3. Case study

A transition is a pair of (agent-state, direction). An
advice consists of a reward or penalty and a transition.
To determine when an agent can ask for advice, insteadof asking randomly, it calls the advisor only when the
difference between the highest and the lowest probability of taking a direction is lower than threshould δ. A
agent gives to a learning agent an advice the following
way: If the learning agent selected action correspond to
the agent highest state-action value, then it gives an extra reward to the learning agent for this action. If the selected action correspond to the lowest state-action value,
it gives an extra penalty to the learning agent. Otherwise, the agent gives the learning agent the next action
to perform with an extra reward. The framework has
been tested on different sizes of maze: 10x10, 15x15,
20x20, 25x25. The numbers of walls and their placein the maze are set randomly. Three difficulty types ofmaze were created. The simplest type of maze is covered by 0 to 25% of wall squares, the normal type by 25
to 50% of wall squares, and the most difficult one by 50
to 75% of wall squares.
The exploration rate was also tested by using different
coefficient steps to decrease the exploration rate during
the learning (step = 2, 5, 10).
Each mode has been run separately before being combined as a base of comparison.

The performance of the proposed method is comparedwith traditional Q-Learning method. The interactive
reinforcement learning with asking for advice method
and getting advice method was employed for the performance evaluation.
A. Influence of size
The influence of the size was evaluated by increasing
the size of the maze. The results show that the proposed
method as well as the other type of interactive reinforcement learning increases the learning speed of the agent
and that the success rate converge quicker than with traditional Q-Learning.

2. Evaluation index
The success rate of an agent to resolve the gridworld
problem is evaluated as the number of time an agent
succeed in reaching the goal from each possible starting
points divided by the number of possible starting points.
At the end of each episode, the success rate of the agent
is calculated by proceeding to an offline turn where theaction decision is greedy and where there is no update of
the state-action values. The evaluation index is definedas the number of episodes it takes to reach a high successrate. When the learning speed is high then the number ofrequired episode to reach the convergence point is low.
On the contrary, with a low learning speed, the numberof required episodes is high.

Fig. 4. Influence of the size on the success rate

B. Influence of difficulty
The influence of the difficulty of the maze was tested
by increasing the number of walls in the maze. The
results show that by increasing the number of walls,
the traditional Q-Learning agent needs more episodes toconverge to a 100% success rate, but with the proposed
IRL the learning speed remains higher than with classical Q-learning, at least at the beginning for the mostdifficult mazes.
C. Size evaluation
We can observe that for the most difficult and biggest
maze after showing good results at the beginning of thelearning there is a decreasing peak in the success rate forthe proposed method and the ask for advice mode after
around a hundred of episodes. This phenomenon can beconsidered as a loss of memory since the agent forgets
what he learned with its advisor.

Fig. 3. Illustration of evaluation index©ISAROB 2011

738

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig. 7. Size evaluation for a big and difficult maze

Fig. 5. Influence of the difficulty on the success rate

REFERENCES[1] R. S. Sutton and A. G. Barto. Reinforcement learning : An Introduction, The MIT Press Cambridge,
Massachusetts London, 1998.
[2] S. Vijayakumar, T. Shibata, and S. Schaal. Reinforcement learning for humanoid robotics, Proceedings of the Third IEEE-RAS International Conference on Humanoid Robots (Humanoids2003), pp.120, 2003
[3] A. L. Thomaz, G. Hoffman, and C. Breazeal. Reinforcement learning with human teachers: Understanding how people want to teach robots, Proceedings of the 15th IEEE International Symposium on
Robot and Human Interactive Communication, pp.
352-357, 2006.

Fig. 6. Size evaluation for a small and difficult maze

[4] A. L. Thomaz and C. Breazeal. Learning from human teachers with socially guided exploration, Proceedings of the 2008 IEEE International Conferenceon Robotics and Automatio, pp.3539-3544, 2008.

V. CONCLUSION
A method combining two types of interactive reinforcement learning to speed up behavior learning was
proposed. The first one is an agent asking for advice
and the second one an agent getting advice from otheragent. According to the numerical experiments, the proposed method showed good result at the beginning of
the learning but still has rooms to improve performance.
In future work, the following actions should be done:[5] R. Maclin and J. W. Shavlik. Incorporating advice
into agents that learn from reinforcements, Proceedings of the 12th National Conference on Artificial
Intelligence, pp. 694-699, 1994.
[6] J. A. Clouse. An introspection approach to querying
a trainer, University of Massachusetts, 1996.

• To have stable performance which does not dependon the number of episodes, the proposed methodshould be combined with other methods.

[7] A. L. Thomaz and C. Breazeal. Reinforcement
learning with human teachers: evidence of feedback
and guidance with implications for learning performance, Proceedings of the 21st national conference on Artificial intelligence, Vol.1, pp.1000-1005,
2006.

• The candidate complementary methods are collaborative reinforcement learning where several
agents on the same maze can spy on each other actions©ISAROB 2011[8] V. N. Papudesi and M. Huber. Learning from reinforcement and advice using composite reward func-

739

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

tions, Proceedings of the 16th International FLAIRS
Conference, pp.361-365, 2003.
[9] W. J. Gutjahr. Interaction dynamics of two reinforcement learners, Central European Journal ofOperations Research, Vol.14, pp.59-86, 2006.
[10] M. N. Ahmadabadi, M. Asadpour, and E. Nakano.
Cooperative Q-learning: the knowledge sharing issue, Advanced Robotics, Vol.15, pp.815-832, 2001.
[11] L. Nunes and E. Oliveira. Advice-Exchange Between Evolutionary Algorithms and Reinforcement
Learning Agents: Experiments in the Pursuit Domain, Lecture Notes in Computer Science, Vol.3394,
pp.185-204, 2005.
[12] S. Kalyanakrishnan, Y. Liu, and P. Stone. Half fieldoffense in robocup soccer: A multiagent reinforcement learning case study, RoboCup 2006: Robot
Soccer World Cup X, pp. 72-85, 2007.
[13] L. Torrey, T. Walker, R. Maclin, and J. Shavlik.
Advice taking and transfer learning: Naturally inspired extensions to reinforcement learning. AAAI
Fall Symposium on Naturally Inspired AI, 2008.
[14] L. Torrey, T. Walker, and R. Maclin. Skill acquisition via transfer learning and advice taking. Proceedings of the 17th European Conference on Machine Learning, pp.425-436, 2006.
[15] L. Torrey, J. Shavlik, T. Walker, and R. Maclin.
Advice-based transfer in reinforcement learning,
University of wisconsin machine learning group
working paper 06-2, 2006.

©ISAROB 2011740

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Crutch Gait Pattern for Robotic Orthosis by the Feature Extraction
I.H. Jang, J.Y. Jung, D.Y. Lee, D.W. Lee, H.G. Lee, H.S. Park*
Korea Institute of Industrial Technology, Republic of Korea(Tel : 82-31-8040-6292; Fax : 82-31-8040-6370)
hsubpark@kitech.re.kr

Abstract: In this paper, we present the method to generate crutch gait pattern for robotic orthosis to walk using motor.
In first we define the features to characterize gait pattern and extract features from the normal gait pattern, which isexperimental encoder data of hip and knee being taken from motion capture device. We also present the selectedfeatures containing physical meaning in gait pattern enable us to vary speed, step size and foot clearance easily. Finally,
we make crutch gait patterns using these features and apply them to our exoskeleton robot ROBIN-P1 to verify our
method.
Keywords: Exoskeleton, Robotic Orthosis, Gait Pattern, Feature Extraction.

size cannot be changed in real time, once it has been
determined. ReWalk[2] is a robot for Spinal Cord
Injury(SCI) patients but there is no paper or patents
describing their control technology. However the
analysis of the various video clips on internet showedthat ReWalk has several gait patterns depending
patient’s condition but it also seems not to change its
walking speed and step size once the gait pattern isselected. I.H. Jang et al. [3] also have made wearable
robotic orthosis ROBIN-P1 for assisting gait of SCI
patients. ROBIN-P1 also memorize reference gait
pattern in the form of lookup table and use thecorresponding gait pattern to control motors according
to user intention.
It is not proper for those kinds of methods shown inthe case of HAL, ReWalk, ROBIN-P1 to be used indynamic environments of day life. Therefore this paper
describes how to generate gait pattern from the several
features and shows that we can change the gait patternonline with the several features.
The rest of this paper is structured as follows.
Section II defines features from normal gait pattern anddescribes the algorithm to generate gait pattern for our
exoskeleton robot platform ROBIN-P1. Experimental
results are provided in Section III, and finally, Section
VI concludes the paper and suggests future research.

I. INTRODUCTION
The exoskeleton type of wearable robots can be divided
into power suits for enhancement of a healthy person’s
activities and rehabilitation robots for support of aphysically challenged person.
Of two categories, rehabilitation robots are noted
more and more recently as the Aging Society is goingon quickly. Robotic orthosis draws special attention
recently because the application of rehabilitation robot
is extended to a development of rehabilitation robot
assisting gait for spinal cord injury patients and theirclinic.
In view of control aspects, the movement ofwearable robot should be synchronized with a user forsafety and natural movement. Therefore we assumed
and defined role sharing as follows.
Assumption 1: user has to play a role in theawareness of environment and thedecision of movements.
Assumption 2: robot has to control its motion
according to user intention.
Assumption 3: user has to play a role in shifting
their weight and balancing for robot
movements.
Suzuki et al. [1] have applied HAL-3 for paraplegia
patients to walk again. This paper describes how to
detect user intention by robot for solving the problem
concerning assumption 2. For the gait pattern, they use
the one extracted from a healthy person’s walking.
However it can be applied only to a hemiplegia patient
and the gait parameters such as walking speed and step

©ISAROB 2011

II.DESCRIPTIN OF THE ALGORITHM FOR
GENERATING GAIT PATTERN
1. Definition of the feature for gait pattern
Previous works for gait pattern say that normal
people has a gait pattern like Fig.1. The (a) (b) and (c)741The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011using this encoder system. In previous case, we have torenew the whole gait pattern and restore that in a lookup
table whenever the gait speed or step size changes.
Therefore, ROBIN-P1 cannot change its gait pattern by
online manner and then it is very difficult for robot toadapt itself in dynamic environment of day life. In orderto solve this problem, we have developed new algorithm
to generate gait pattern and this new algorithm use thefeatures which is defined at previous chapter.
Our new algorithm consists of totally 6 steps asfollows:
Step1: sample one period of gait pattern, heel striketo heel strike, from the encoder data on Fig.2.

represent the range of Ankle, Hip and Knee motion
respectively. We can find several peaks and both end
points from the gait patterns. Meanwhile, if we canknow those points, we may be able to recover whole
gait pattern by connecting those points with spline
method. Therefore we can select those points as thefeatures for gait pattern.

(a)

(b)
Fig.2. The raw data being acquired from the encoder
system in the case of healthy person
Step2: Make the time-axis of sampled gait pattern tonormalize with the range of 0 to 100.

(c)Fig.1. (a) Ankle Range of Motion (b) Hip range of
motion and (c) Knee range of motion

Step3: Select peaks, both end points and points
being changed rapidly on a gait pattern as the features.

2. Algorithm for generating gait pattern
In order to get the crutch gait pattern of normal
people, we have designed encoder system that is theexoskeleton type of dummy robot having encoders at
hip and knee joint replacing DC motor of ROBIN-P1.
The encoder system has same constraint as our platform
ROBIN-P1.
ROBIN-P1 actually has DC motors at hip and knee
joints and each joint has full range of motion as shownin Table 1.
Fig.3. Feature extraction from hip and knee encoder

Table 1. The range of motion at each joint for the
ROBIN-P1
Joint
Range (degree)Hip-10 ~ 125

Knee
Ankle

0 ~ 115
Fixed

data
Step4: Repeat step1~3 and make an average of eachfeature point over the sets of feature point.
Step5: Connect each averaged feature point by using
spline method and prune graph out of joint movement
range as shown in Fig.4

In order to generate a gait pattern for ROBIN-P1, we
have edit normal people’s gait pattern being acquired©ISAROB 2011742

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

operating area by robot means the phase of swing for
gait. One leg is always at stance phase whenever theother is at swing phase, so the robot motions on thesetwo phases occur at the same time for the different twolegs alternately. Therefore we can capture these twophases of operating are by robot from a period of gait
pattern and make them overlap as shown in Fig.6.
Step6: Finally, Correct the gait pattern for use themas DC motor input values. In other words, we need toadjust the starting point of the stance and the end
point of the swing at hip and knee joint respectively
for recurring use.

III. EXPERIMENTAL RESULTS

Fig.4. Gait pattern made by connection of featurepoints through cubic spline and correction

The gait pattern was generated from only 8 features
if we count both end points as one because both end
points are same on Fig.5. And then the speed can bevaried according to the desired time value being
converted from the normalized time value.
Fig.6 represents that it takes 4seconds in aswing/stance phase time while the step size is fixed at
250mm. We have experimented with our exoskeleton
platform ROBIN-P1 varying its swing/stance time from
2seconds to 5seconds. We also changed the step size byvarying the step size in angle domain that is defined as
the difference between the starting angle of swing hip
and the starting angle of stance hip on Fig.6. However
there are more things we have to take into accountbecause the peak angle values at hip and knee on swing
phase may also have some correlation with the step size.
In our experiments, ROBIN-P1 did not show natural
gait pattern when we varied the step size in angle
domain beyond 5 degree.

Fig.5. The gait pattern can be divided two parts:
operating area by human and that by robot.

Fig.6. The DC motor input data at hip and knee joint fora stance and a swing phase respectively.
From the assumption being described at introduction,
one period of gait pattern has to be divided into twoparts for controlling DC motor: Fig. 5 shows these twoparts that are operating area by human and operatingarea by robot. The operating area by human indicatesthe area that users shift their weight and balancethemselves for next step. The operating area by robot
indicates the area that all motors at joints are actuallyoperated. The first operating area by robot in Fig.5
means the phase of stance for gait and the second©ISAROB 2011

Fig.7. Results of experiment assisting gait using
ROBIN-P1 with a gait pattern which is generated
through feature extraction.

743

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

VI. CONCLUSION
We defined the feature in a gait pattern anddescribed our algorithm to regenerate gait pattern using
several features in Section II. Our experimental resultsin Section III also showed that it is possible for the
ROBIN-P1 to walk not by lookup table type of control
data but by a gait pattern being generated from several
features in real time. This means that it is possible tochange the gait parameters (speed, step size) in online
manner. However, we find it necessary to analyze thecorrelation the features and the step size in order not tolose natural of original reference gait pattern. What gait
pattern do we select as a reference gait pattern? These
are for our future work.

REFERENCES[1] Suzuki K, Kawamura Y, Hayashi T, et al. (2005),
Intention-based walking support for paraplegia patient.
IEEE International Conf. on Systems, Man andCybernetics; 2005: 2707–2713.
[2] [Online] Available: http://www.argomedtec.com/[3] I.H. Jang et al.(2010), Development of therehabilitation robot assisting gait for spinal cord injury
patients (in Korean). J. Institute of Control, Robotics
and System 16(9): 860-865©ISAROB 2011744

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Design of Personal Support System in Telephone Correspondence
using Smart Phone
Hiroyuki Nishiyama and Fumio Mizoguchi
Faculty of Sci. and Tech. Tokyo University of Science
Yamazaki 2641, Noda-shi, CHIBA, 278-8510, Japan(Tel: +81-47122-1106; Fax: +81-47122-1106)
Abstract: In this paper, we design a personal support system in telephone correspondence using smart phone.
This system consists of a phone-monitor tool that checks the use state of the smart phone, and a personal support
tool that gives the user advice on the personal computer. The phone-monitor tool recognizes and manages thestate of the smart phone, including outgoing and incoming calls, and sends information via wireless communications to the personal support tool. These tools can select the suitable type of wireless communication (Wi-Fi
or Bluetooth), depending on the situation. Our system determines the user’s state by using the sensor systemon a smart phone and achieves secretarial personal support, including advice on the response confirmation, for
telephone correspondence using smart phone.
Keywords: Smart Phone, Personal Support System, Telephone Correspondence, Bluetooth1

Our design enables user support by cooperation between a smart phone and other computer terminals (e.g.,
laptop personal computer and desktop personal computer).
Cooperation includes dynamically gathering information
on a smart phone and transmitting the information to acomputer terminal, depending on the situation. The telephone call situation and sensor information can be collected from the smart phone. This system recognizes theuser’s situation and the state of the smart phone from obtained sensor information, and decides the content thatshould support the user.
Our support system dynamically manages outgoing andincoming calls on a smart phone, and enables management
by wireless communications on a specific desktop or laptop personal computer. Thus, this system achieves user
support for the response confirmation through the telephone correspondence in a smart phone.

INTRODUCTION

Recently, many multifunctional cellular phone terminals,
such as the smart phone (e.g., Android cellular phone andiPhone), have been developed as a result of the evolutionof the computer, network infrastructure, and lightweighttechnology of the battery; thus, the number of users is
rapidly increasing [7]. Many use the smart phone as adaily support tool by taking advantage of various services(e.g., downloading free or paid applications), in addition totalking on the telephone through the Internet connection.
However, these applications are limited to the functionsof one smart phone, and few applications cooperate withother terminals (e.g., smart phones or laptop computers).
Those that do exist temporarily enable intelligence sharing through a connection between the smart phone and thecomputer terminal, using the USB. Additionally, general
research on user support intended for the cellular phone
specializes in the function of the portable terminal that canconnect with the Internet [1, 2, 4, 5]. In such studies, when
the portable terminal is used by the office and the home ina private network, an external server machine is needed[2, 5].
Considering this background, we have designed a system for wide user support by enabling cooperation between a smart phone and other computer terminals (e.g.,
the laptop personal computer that the user carries, anddesktop personal computers that the user uses in the homeor the office). This system automatically selects the communication facility that is appropriate for the situation
when cooperating with the various communication facilities of a smart phone (e.g., wireless LAN, Bluetooth, and3G), and dynamically monitors the use state of the smart
phone. By using these functions, we have designed a system to enable casual support without disturbing the use of
the smart phone. For example, we achieve a secretarial
personal support system that manages the sending and receiving of calls on a smart phone (Android phone).

©ISAROB 2011

2

PERFORMANCE OF THE SMART
PHONE

In our study, we use HTC Desire SoftBank X06HT as asmart phone (Android phone). The smart phone is similar
to a small laptop personal computer. However, a smart
phone has a telephone call function, and it is easy to carrybecause it is small and light. Its display is rather small, andit does not have a keyboard. Moreover, the development ofan Android smart phone is comparatively easy because thedevelopment language basically conforms to Java in the
Android application. In our research, we regard a smart
phone as a general portable computer terminal that has atelephone call function, and enable cooperation with thevarious personal computer terminals that the user uses.
2.1 Communication Facility of the Smart Phone
Three kinds of communication facilities can be used withthe smart phone. The wireless LAN is a wireless communications function that can communicated at a maximum

745

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

speed of 54Mbps, which the computer terminal uses. Its
use requires a connection to a wireless LAN router. Bluetooth is a wireless communications function that connects
to a computer terminal by a PtoP connection and can communicate at a maximum speed of 2.1Mbps. Connection
requires paired setting with a connected terminal (only
once).3G high speed is basically a communication facility
intended for the cellular phone, and the wireless communications function can communicate at a speed of 7.2Mbps
or less. Its use requires a contract with the communicationenterprise, and the cost corresponds to the wire traffic.
The use of a wireless LAN is considered suitable, considering the transmission rate and the cost, based on theperformance comparison above. However, the use of awireless LAN is difficult because the connected accesspoint changes when the smart phone is used at many bases.
2.2

WirelessLAN

Event
Recognition
SensorMonitorＤＢ

Wireless

Communication
Communication
Bluetooth

Telephone CallManagement

Situation
Recognition

AdviceDisplay

User Support
Interface

Smart Phone

Figure 1: System configuration

API for Telephone Use
sensor), besides the GPS function. Each sensor can confirm a change of sensor value by using the sensor manager
API (android.hardware.SensorManager). For example, theacceleration sensor can measure acceleration in three directions (X axis, Y axis, and Z axis).
When a user is carrying an Android phone, the user’s
behavior can be detected with the use of the acceleration information [1, 3]. Moreover, a brightness sensor can detect whether the smart phone is put awayin the bag at the office. Additionally, the API (android.media.AudioManager) of the phone’s manner modeand silent mode sets and confirms it exists, too.

By using API [8], the Android application can call anotherphone and recognize the use state of its own phone. Additionally, API (android.telephony.TelephonyManager) dynamically identifies calling and receiving, and the following information can be collected.
• Ringing State (CALL STATE RINGING):
The ringing state is the state by which a telephone callis received. The calling person’s telephone number
can be identified.
• Off-hook State (CALL STATE OFFHOOK):
When the telephone receiver is raised in response toa received call, the ringing state changes to the offhook state. It can be assumed that the smart phone iscalling another phone when the idle state changes tothe off-hook state.

3

A change from the ringing state to the idle state of
the standby mode means no response to a call. Achange from the off-hook state to the idle state means
the telephone call ended.
The API is a function basically intended for receiving;when sending, not much information can be obtained. The
obtained information involves calling and ending only,
and the telephone number and whether it was possible
to talk over the telephone cannot be confirmed. Therefore, it is necessary to acquire information about the calling telephone number and the calling person’s response
after the end of telephone call by using the API (android.provider.CallLog.Calls) to acquire the vita information prepared for the Android phone.
Sensor System and API of the Android Phone

The Android phone has a variety of sensor systems (e.g.,
acceleration sensor, brightness sensor, and temperature©ISAROB 2011

SYSTEM ARCHITECTURE

In our study, we have designed a system that enables user
support through cooperation between a smart phone (Android phone) and computer terminals (e.g., laptop personal
computer and desktop personal computer). We achieveuser support as cooperation by dynamically gathering information on the smart phone and sending the information to the computer terminal, depending on the situation.
Information collected on a smart phone includes the telephone call situation and sensor information described in
the preceding section 2. This system decides what content should be supported by recognizing the user’s situation and the state of the smart phone, based on the sensor
information.
The system configuration is depicted in Fig. 1. The
smart phone consists of four modules that monitor the stateof the phone. The user-support interface consists of threemodules that support the user with a GUI that displays advice.
In the smart phone, each module monitors the telephone
call situation and the sensor situation. When a change isrecognized, information is sent to the event recognition
module. This module requests the transmission of information from the wireless communications module when
it judges that the recognized change should be transmit-• Idle State (CALL STATE IDLE)：

2.3

Wireless

746

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

ted to the user-support interface. The wireless communications module sends information to and receives information from the personal computer that is operating theuser-support interface by wireless LAN or Bluetooth.
In the user-support interface of the personal computer,when event information is received through the wirelesscommunications module, the situation recognition module
recognizes a change in the situation. When it is judgedthat the change requires user support, information is transmitted to the display module, and advice is displayed on
the GUI.

• Step 4: Repeat Step 3 as long as it is in the range ofBluetooth communication.

in section 2.2. When a change in the situation is recognized, this module sends information to the event recognition module with additional information (e.g., telephone
number). Moreover, this module can call the other phoneusing the given number.
The sensor monitor module dynamically monitors information, such as the acceleration sensors using the functionof the Android sensor API described in section 2.3. The
user’s state can be recognized from the information of eachsensor by registering the user’s behavior beforehand with
the SVM [3]. When a change is recognized in the registered state of behavior, this module sends the information
to the event recognition module. This module monitors
mainly the acceleration sensor and the brightness sensor,
as well as the state of the smart phone’s speaker (e.g., manner mode and silent mode).
The event recognition module recognizes informationfrom the telephone call management module and the sensor monitor module as an event, and sends information to
the user-support interface, according to the content of theevent, through the wireless communications module. This
module recognizes the state by which the smart phone receives or sends calls as the main event, and sends the stateof the user and the cellular phone as additional information.
The wireless communications module sends to and receives from the personal computer according to the procedure of section 3.1. This module regularly confirms therange of Bluetooth communication and confirms whether
the personal computer with which the user-support interface is operating is in useful range. When it is outside theeffective range, this module postpones the transmission of
information until entering a useful range.

• Step 5: Send and receive information by Bluetooth.

4.2

3.1

Information Sharing by Wireless Communications

Communication between a smart phone and a personal
computer is executed by the correspondence proceduresof wireless LAN and Bluetooth. To give priority to the useof wireless LAN for the reasons described in section 2, wehave established the following correspondence procedure.
• Step 0: Wait until entering the range of communication by Bluetooth.
• Step 1: To Step 2 if both terminals can use wireless
LAN and to Step 5 if otherwise.
• Step 2: Share IP address with wireless LAN according to Bluetooth communication.
• Step 3: Send and receive information by wirelessLAN.

The processing content of the modules in the user-support
interface on the personal computer is as follows.
The wireless communications module for the personal
computer sends to and receives from the Android phone
according to the procedure described in section 3.1.
The situation recognition module recognizes the receiving and sending situation of the phone through the wirelesscommunications module, and records this information in
the data base (DB). Next, this module transmits advice tothe user by collating new information with the vita information in the DB and sends the advice to the display module. For example, a smart phone receives a call; the smart
phone is in manner mode, and the brightness of the smart
phone is low (e.g., put away in the bag). This module considers the possibility that the user does not notice the calland sends advice (an alert) on the user-support interface.
The advice display module displays the advice information received from the situation recognition module on theuser-support interface. Several display methods of advice
exist. For example, this module generates a new supplementary window and displays important advice in large

In a range where Bluetooth can be used for communication, this system usually uses wireless LAN; Bluetoothcommunication is used only when wireless LAN is notpossible.

4

SYSTEM IMPLEMENTATION

In order to achieve the system depicted in Fig. 1, we
implemented each module on the smart phone (Android
phone) and a personal computer. Each module in the Android phone can be implemented using Android SDK [8]via a computer terminal. Since the programming language
is basically similar to Java, we used Java language for the
implementation of all modules.
4.1

Monitor Tool on the Android Phone

The content of the processing of the modules in the monitor tool on the Android phone is as follows.
The telephone call management module dynamicallymonitors the telephone call situation of the smart phoneusing the function of the Android telephone API described©ISAROB 2011

User Support Interface on a Personal Computer

747

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 20116

We designed a personal support system in telephone correspondence using smart phone. This system consists ofa phone-monitor tool that checks the use state on a smart
phone, and a personal support tool that gives the user advice on the personal computer. The phone-monitor tool
recognizes and manages the state of the smart phone, including outgoing and incoming calls, via the smart phone,and sends information via wireless communications forthe personal support tool, which is operating on a desktop or a laptop personal computer. These tools can selectthe suitable type of wireless communication, dependingon the situation. Our system determines the user’s stateusing the smart phone’s sensor system and achieves secretarial user support, including the advice of the response
confirmation for telephone correspondence by cooperation
between these two tools.

Figure 2: An alert display that presents a strong warningthat the user is receiving a callcharacters.
With these modules, the user can dynamically confirm
the receipt of a call even when the phone’s sound is off andthe phone is in a bag. Moreover, the user can confirm thehistory of the smart phone while the user is moving, after
the user arrives at the office or arrives home.

5

References[1] T. Brezmes, J. Cotrina, J. Gorricho, M. Rersa: User
activity monitoring using conventional cell phones
for IMCIC 2010 The International Multi-Conference
on Complexity, Informatics and Cybernetics: IMCIC
2010, 2010.

SYSTEM EXECUTION

When the user arrives at the office, it is possible to connect
the monitor tool on a smart phone with the user-support
interface on the office personal computer. As a precondition, the smart phone is assumed to connect with the
Wi-Fi router in the office and to acquire the IP address.
First, the monitor tool on the smart phone connects tothe user-support interface on the personal computer using
Bluetooth communication. The tool sends the IP addressof Wi-Fi of the smart phone to the user-support interface,
and the interface sends the IP address of the personal computer (private IP address in the office) to the tool. Next,
the smart phone’s monitor tool tries the socket connection
to the IP address of the personal computer that receives it.
If successful, information is sent and received via Wi-Fi
communication. If not, Bluetooth communication is used.
Supplementary communication is executed by automatically using Bluetooth when the communication fails with
Wi-Fi use. In addition, the monitor tool accumulates information and, after reconnection, retransmits the accumulated data when the Bluetooth communication fails.

[2] S. Fujino, G. Motoyoshi and H. Tani: Personal Network Accommodation System by a Mobile Network,
The transactions of the Institute of Electronics, Information and Communication Engineers. B, J92-B(10),pp. 1669-1676, 2009. [in Japanese][3] H. Nishiyama and F. Mizoguchi: Design and Implementation of Personal Service System based on Human’s Motion Recognition with Sensor Device, 27st
Conference Proceedings Japan Society for Software
Science and Technology, 2B-3, 2009. [in Japanese][4] K. Tajima, K. Ando, K. Ohshima and M. Terada: Automatic Information Display System: Design and Implementation, The transactions of the Institute of Electronics, Information and Communication Engineers.
B, J92-B(7), pp. 1084-1097, 2009. [in Japanese][5] K. Takasugi, M. Nakamura, S. Tanaka, M. Kubota and K. Koyanagi: Technology for Service Continuity Adapting to Dynamic Environment Seamlessly(Mobile Computing) , Transactions of Information Processing Society of Japan, 46(2), pp. 608-623,
2005. [in Japanese]

5.1 Processing When Receives a Call
When a smart phone receives a call with the phone put
away in a bag while the user is working in the office, thewarning advice (Fig. 2) is displayed on the user-support
interface. Figure 2 is an alert display with a strong warning that the user is receiving a call. This alert display assumes that the user does not notice the call because thesmart phone is in manner mode or silent mode. When thesmart phone is placed on a desk or the ring tone is the pronounced normal mode, the vita information is displayedon the GUI.

©ISAROB 2011

CONCLUSIONS[6] V. Vapnik: The Nature of Statistical Learning Theory,
Springer-Verlag, 1995.
[7] Mobile Content Forum: K-tai white paper 2010, Impress R&D, 2009. [in Japanese][8] Android API Guide, http://www.android.com/

748

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Advanced Networking and Robotics forSocietal Engagement and Support of Elders
Peter Sapaty
Institute of Mathematical Machines and Systems, National Academy of Sciences
Glushkova Ave 42, 03187 Kiev, Ukraine, sapaty@immsp.kiev.ua

Masanori Sugisaka
Department of Mechanical and Electrical Engineering, Nippon Bunri University
1727 Oaza Itiki, Oita, 870-0397, Japan, ms@alife-robotics.co.jp

Abstract
The percentage of older population is growing quickly,
especially in developed countries, where the retired elders
have to be paid by the expense of people who are working,often remaining, however, the most experienced,
knowledgeable, and skilful part of the society, also
accumulating considerable material wealth. Elders also havespecific requirements to be satisfied to ensure theirprolonged decent and independent life. The paper considers
the use of an advanced distributed knowledge processing
and control technology capable of solving broad andcomplex tasks in modern societies, including continuing
involvement of elders in their development.
The approach integrates distributed physical andnetworked virtual worlds, which can cover from smart home
to smart city to smart nation, being coordinated andprocessed altogether by a powerful spatial engine capable ofsolving any knowledge processing, control, andcoordination tasks in parallel and distributed manner. The
key element of the approach, Distributed Scenario
Language, is briefed, along with practical examples of
elderly population support in it. Having been prototyped indifferent countries and tested on various network-related
problems, the technology can be easily put on any softwareor hardware platform, with massive cooperative use inmobile phones and mobile robots.
Keywords: Ageing population, decent life, elderly support,
knowledge processing, distributed scenario language,
mobile phones, mobile robots.

1 Introduction
The world is facing a rapid growth of elderly population,
which causes serious social and economic problems. To
satisfy specific needs and ensure prolonged decent andindependent life of elders, advanced information andnetworking technologies are considered at the forefront ofpossible solutions.
European Union (EU) has created a special Ambient
Assisted Living (AAL) Joint Program and initiated anumber of big research projects to tackle such problems,with considerable funds being invested [1, 2, 3]. EU madethorough analyses of the potential for developing
Information and Communication Technologies (ICT) for theelderly and summarized them:• Europeans over 65 possess wealth and revenues of over3000 Billion € (Euro).

©ISAROB 2011749

• The market for smart home applications (age-related
assistancein
shopping,
dressing,
movingindependently) will triple between 2005 and 2020,
from 13 million people up to 37 million.
• 68 million people in 2005 had several forms of agerelated impairment. This will grow to 84 million in2020.
• Early patient discharge from hospital due to theintroduction of mobile health monitoring would save
€ 1.5 billion per year in Germany alone.
• EU research projects have developed technologies
for personalized route guidance; home care andremote health monitoring and advice; intelligent
alarms; natural interfaces for accessible ICT.
A related project in Japan [4, 5, 6] addresses
technological aspects of the AAL and employs
innovative methods and facilities for ambient
information control based on attaching spatialconnotation to surfaces that surround elderly people. In
this way, semantic surfaces for direct ambientinteractions can be created and invisible intuitive support
to the elders can be provided.
Semantic surfaces could make a difference in thelifestyle of the aging population and ensure the muchneeded self-confidence and dignity of the elders, also
continue keeping them as an integral and productive partof the society. These works are also in line with known
broader concepts of seamless interfaces “between people,
bits and atoms” [7].
The current work represents a further development ofthese and related projects in Europe and Japan towards
practical implementation in distributed networked
environments. The approach offered allows us to solve
various non-local problems for the elderly support,
especially those in need of involvement and joint use of
distributed resources (manned as well as unmanned) andsmart networking in dynamic situations. The paper also
inherits the semantic web concept [8], offering for thelatter its possible solution for a specific domain.

2 Using Spatial Grasp Technology
The developed Spatial Grasp Technology [9, 10, 11]
has been successfully prototyped in different countries
and studied for most diverse networked applications –from classical graph and network problems to distributed
knowledge bases to intelligent network management toThe Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

distributed interactive simulation of dynamic systems toroad traffic management to collective robotics to security
and defense [12-34]. Many of these applications (if not all)can be useful in some or other form for the alleviation ofdemographic problems linked with ageing population.
For the current application, the semantic surfaces,
assisting the elderly, can be represented altogether within aheterogeneous semantic network describing the wholedomain in an integral form, with certain nodes having
double digital-physical representation. This network can bearbitrarily and seamlessly distributed throughout any
territory (including smart home, smart city, “smart” country,
or even worldwide) using computers or other commonlyused electronic devices (like mobile phones, smart sensors,or mobile robots).
High-level operational scenarios in a special Distributed
Scenario Language (DSL) can start from any point (node),runtime covering this semantic world (with it, the physicalone too) in a spatial pattern-matching mode, bringing
operations to data, data to operations, and control to both,organizing overall goal-driven behavior in the way required.
Complex network-related problems can be expressed in
DSL in a compact and simple form (often up to a hundred
times shorter and proportionally simpler than, say, in Java).
And the language interpretation can be organized in aparallel and fully distributed mode, without central
resources, thus increasing robustness of the whole systemand capability of self-recovery from damages.
The use of SGT can be purely user-centric, when theinjected system scenario serves exclusive interests of asingle person (possibly, subsequently involving other people
and distributed resources for this). It is also possible tocreate any active distributed system and organize collective
behavior of many individuals in it (humans or robots)pursuing together a common goal, with global control ofthis distributed activity.
SGT also directly works within the same formalism with
distributed physical world, or any combination of thesemantic (virtual, digital) and physical worlds, with(parallel) operations in the former capable of guiding theones in the latter in a look-ahead distributed simulationmanner.

3 The Distributed Scenario Language (DSL)DSL is quite different from traditional programming
languages. Rather than describing data processing in acomputer memory, as usual, it allows us to directly movethrough, observe, and make any actions in fully distributed
environments (whether physical or virtual), seeing andtreating them as an organized whole rather than collection ofparts. DSL operates with:• Virtual World (VW), which is finite and discrete,consisting of nodes and semantic links between them.
• Physical World (PW), an infinite and continuous, where
each point can be identified with physical coordinates(with a certain precision).
• Virtual-Physical World (VPW), being finite anddiscrete similar to VW, but associating some or all
virtual nodes with PW coordinates.
DSL also has the following features:• A scenario expressed in it develops as a transition
between sets of progress points (or props) in the formof parallel waves.

©ISAROB 2011750

• Starting from a prop, an action may result in one ormore new props.
• Each prop has a resulting value (which can be
multiple) and resulting state, being one of the four:
thru (full success allowing us to proceed further
from this point), done (success with termination of
the activity in this point), fail (regular failure withlocal termination), and abort (emergency failure,terminating the whole distributed process, associatedwith other points too).
• Different actions may evolve independently orinterdependently from the same prop, contributing to(and forming altogether) the resultant set of props.
• Actions may also spatially succeed each other, withnew ones applied in parallel from props reached by
the preceding actions.
• Elementary operations can directly use local or
remote values of props obtained from other actions(the whole scenarios including), resulting in value(s)of prop(s) produced by these operations.
• These resultant values can be used as operands byother operations in an expression or by the next
operations in a sequence (the latter can be multiple,
if processes split). These values can also be directly
assigned to local or remote variables (for the latter
case, an access to these variables may invoke
scenarios of any complexity).
• Any prop can associate with a node in VW or aposition in PW, or both (when dealing with VPW); it
can also refer to both worlds separately andindependently.
• Any number of props can be simultaneously
associated with the same points of the worlds(physical, virtual, or combined).
• Staying with the world points, it is possible todirectly access and update local data in them.
• Moving in physical, virtual or combined worlds,
with their possible modification or even creationfrom scratch, are as routine operations as, say,arithmetic, logical, or control flow of traditional
programming languages.
• DSL can also be used as a universal programming
language (similar to C, Java or FORTRAN), thus
serving as a single language for expressing
knowledge processing, intelligent communication
protocols, and overall control in distributed systems.
DSL has recursive syntax, represented on top level asfollows (programs called grasps, reflecting their main
semantics as gasping and integrating distributed
resources into goal-driven systems).
grasp
phenomenon
constant
variableruleÆ phenomenon | rule ( { grasp , })
Æ constant | variable | special
Æ information | matter | combined
Æ heritable | frontal | environmental |nodal
Æ movement | creation | elimination |echoing | fusion | verification |assignment | advancing |branching | transference |timing | granting

The basic construct, rule, can represent any definition,action or decision, for example:• elementary arithmetic, string or logic operation;

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

• hop in a physical, virtual, or combined space;• hierarchical fusion and return of (remote) data;• distributed control, both sequential and parallel;• a variety of special contexts for navigation in space,influencing operations and decisions;• type or sense of a value, or its chosen usage, guiding
automatic interpretation.
There are different types of variables in DSL:
• Heritable variables – these are starting in a prop andserving all subsequent props, which can share them inboth read & write operations.
• Frontal variables – are an individual and exclusive
prop’s property (not shared with other props), beingtransferred between the consecutive props, andreplicated if from a single prop a number of props
emerge.
• Environmental variables – are accessing different
elements of physical and virtual words when navigating
them, also a variety of parameters of the internal worldof DSL interpreter.
• Nodal variables – allow us to attach an individual
temporary property to VW and VPW nodes, accessedand shared by props associated with these nodes.
These variables, especially when used together, allow us
to create efficient spatial algorithms not associated withparticular processing resources, working in between
components of distributed systems rather than in them.
These algorithms can also freely move in distributed
processing environment (partially or as a whole), always
preserving integrity and overall control.
DSL also permits the use of traditional operational
symbols and delimiters, to simplify and shorten programs, if
this proves useful.

4 Exemplary Solutions in DSL
We will consider here some elementary solutions in DSL
for the support of elderly population, which show
compactness and simplicity of the higher-level code. All
these can work in fully distributed environments, withcasual channels and casual computing devices appeared
available in different places and time (like individually kept
by elderly persons, embedded in wheelchairs, mobile robots,
smart sensors, or accessed via the internet).
4.1 Assisted Search for Forgotten or Lost Things
The part of a semantic network, oriented on guiding thesearch for regularly used items (like door keys, glasses, or
wallet, for example), setting up an optimized movement
between places in the house where such things may happen
to be, is shown in Fig. 1. Different DSL solutions based on
this network (from purely manual to fully robotic) may be
as follows.
Assisted manual solution: advised manual movement andmanual items recognition. The elderly person, informing thesystem about her current location by a physical contact withthe semantic surfaces, is then advised to go to the nearest
location where the item of interest can be found. This isfollowed by a further stepwise guidance to move to othersuch places via the optimum path between them (fixed inadvance by the semantic network of Fig. 1), unless the item
is found, about which the person informs the system. The
DSL solution may be as follows:©ISAROB 2011

751

Dustbin

Living
Room
Table

next
locatedlocated

x2y2

x1y1

next

x4y4

includeincludeKitchenDesk

Shopping
Bag

next

x3y3

next

next

x6y6

Bathroom next

include

Cloak
Room

includeCasual Placesx5y5

Neareststarting
node
Semantic
Surface
Contact
Current x0y0
Position

Figure 1. Networked search for a lost item.
heritable (Initial) = WHERE;
minimum destination (hop (direct, Casual Places);hop (include, all);distance (hop (located), Initial));repeat (output (‘go to’, NAME, ‘try finding’);
if (input == ‘ok’, done);hop (first come, + next))

Assisted automated wheelchair movement with manual
items recognition. This case differs from the previous
one in that the movement between Casual Places of Fig.
1 is performed automatically by a robotized wheelchairusing given physical coordinates of the places, withvisual search and recognition of the item by the elderly
person herself. The latter terminating the automatedmovement if she finds the item. The DSL solution:
heritable (Initial) = WHERE;
minimum destination (hop (direct, Casual Places);hop (include, all);distance (hop (located), Initial));repeat (WHERE = hop (located);
output (‘try finding’);
if (input == ‘ok’, done);hop (first come, + next))

Fully robotic solution: automatic movement and image
recognition. This may all be accomplished by an
intelligent robot, if employed in the house. It receives thename of the item to be found (supposedly having video
camera and automatic items recognition software, as wellas matching templates for different items), and starts
from the nearest Casual Place to its current physicalposition. In DSL this may be as:
heritable (Initial = WHERE;
Item = door keys);minimum destination (hop (direct, Casual Places);hop (include, all);distance (hop (located), Initial));repeat (WHERE = hop (located);if (discover (Item), done);hop (first come, + next))

In a similar way, more complex cases of navigationand movement throughout distributed territory by“dismounted” (albeit computer-assisted) individuals orwith the help of robotic wheelchairs (and in the ultimate
case—by single or multiple robots) can be organized in
DSL with distributed maps coded as semantic networks,as, for example, in [33].

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

4.2 Finding Shortest Paths
The shortest path found to a certain destination may be agood guide to move through a building, region, or city, say,
between nodes a and e as in the following DSL scenario(corresponding to Fig. 2). It is being found by navigating the
network of weighed links in parallel and fully distributedmode, without any central resources, where the digital
networked map can be arbitrarily distributed between any
electronic devices having processing capabilities.

f
Wave3

3

c

e

a, b, d, e
Shortest path foundStart 2

Figure 2. Finding shortest path between nodes a and e.
Many important problems of optimization and control canbe expressed as finding shortest paths in distributed spaces.
The solution shown (already tested on different distributed
network applications) may also be useful for tasks related tothe elderly and handicapped, like, for example, finding ways
to shops, hospitals, friends, even their own homes (in caseof being lost in a forgotten place). This may be possible infull, however, if advanced integration between distributed
physical and virtual worlds exists in the society, which may
be achieved with the technology discussed.
4.3 Tracking Physical Movement of Individuals by
Mobile Intelligence
As in recent known cases in Japan, the elderly people andtraces of them can be lost in the society. The DSL scenario
in this section shows how whereabouts of the elderly can be
regularly checked and traced by a mobile spatial intelligencepropagating in a virtual world while following the
movements in physical world, as in Fig. 3.
Current positions of certain individuals can be fixed bytheir physical contacts with semantic surfaces or by
networked video cameras (to which key pictures of thepersons can be delivered by mobile intelligenceaccompanying them, for automatic recognition). The
tracking intelligence can analyze and accumulate behaviorof the moving person, demand checking her current physicalcondition (e.g. heartbeat, blood pressure, body temperature,etc.). It can also alarm the nearest medical facilities in case
©ISAROB 2011

Place 13

Place 12

Place 6surface link

Place 10 Place 8
Place 11surface link
Place 9registeredregistered...

4.4 Collective Evacuation from a Disaster Zone

2

4

Tracking
mobileintelligence

frontal (ID = individual; History);
repeat (loop (belong (ID, hop (registered, all));
update (History, check condition);
if (problems (History),
alarm (nearest medical staff));
delay (delay));hop (surface link, all);belong (ID, hop (registered, all)))

1

Wave2
2

Tracking
mobilesurface link intelligence
Place 3Figure 3. Networked tracking of individuals.

d1

Place 4

Other registered individuals

Wave1
4

1

registered

Place 5surface link

Semantic
surfacecontact:
Person’s ID

2aPlace 7Start

The result gradually accumulated and returned to node awill be (a, b, d, e).
b

Place 2

Place 1

frontal (Remoteness, Path);sequence ((hop (direct, a); Distance = 0;
repeat (hop (all links);
Remoteness += LINK;or (Distance == nil,Distance > Remoteness);
Distance = Remoteness;
Predecessor = BACK)),(hop (direct, e);repeat (Path = NAME & Path;hop(Predecessor));output (Path)))Start 1of irregular situations. Many moving persons can be
simultaneously and individually checked and nursed by
the DSL scenario shown below.

752

In case of major disasters (earthquakes, hurricanes,
flooding, landslides, forest fires, etc.) the elderly andhandicapped may appear to be the least protected andmost vulnerable ones, thus needing particular help,
possibly, even to survive. A related DSL scenario may be
activated by any person caught in such an event, or by aspecial emergency organization setting up coordinatedmassive evacuation from the disaster zone, regularly
issuing instructions to individuals (say, via mobile
phones, hopefully still working) on where and how tomove.
A chained collective movement in this respect througha safe narrow passage in the disaster zone is shown by
the program below and in Fig. 4, where individuals movein a coordinated (with each other and with the waypoints
supplied externally) way. Only the first individual in thisemergency chain is a pure leader (directly following the
waypoints), and the last one is a pure follower, while theothers combine both functionalities (thus moving rightafter the previous person, and being directly followed by
the same next person).
cycle (if (exist (unmarked individuals),(N += 1; assign (next individual,
create (node (N))));(NAME == 1; Waypoints= (w1, w2, w3, …);loop (output (‘move to’,withdraw (Waypoints, 1));
wait (input == ‘ok’))),(NAME != 1; sling (Leader coordinates =(hop (direct, NAME - 1); WHERE);
output (‘move by’, direction(WHERE, Leader coordinates))))The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

LeaderFollower
4

5

Follower

3
LeaderFollowerx2y2

2

LeaderFollower

Start
Nodes:
1, 2, 3, 4, 5

inside (New, Polygon);and (empty (hop (New, Range)),non cleaned (New),
move_check_clean (New)))

Safe passage
through thedisaster zonex3y3

1 Waypoints
Leaderc1

Disaster zone
c5

Figure 4. Chained evacuation from a disaster zone.
4.5 Using Service Robots

Y1
X1

Finish

DY
X2

Start

Figure 5. Sequential cleaning by a single robot.
X1 =…; X2 =…; Y = Y1 =…; Y2 =…; DY =…;loop (move_clean (X1, Y); (Y += DY) < Y2;
move_clean (X2, Y); (Y += DY) < Y2)

Parallel cleaning. Parallel cleaning of a territory(assumed here to be of arbitrary shape) by a cooperative
robotic swarm can also be organized in different ways, forexample, randomly, as in Fig. 6. For this, each cleaning
device finds its next possible position independently by arandom choice, checking if the new destination is not
occupied by other robot to avoid collision, and also has notbeen visited (cleaned) before (using visual sensors).
Such parallel random navigation may have advantages
before any predetermined search as can eventually cover thewhole area with at least a single robot operable. Such asearch, where physical processes start from a number of
initial points (named c1 to c5), which can be chosenrandomly too, and keep a threshold distance from each other,
may be expressed by this program:

x5y5

x4y4
c2
RandomRandom

New
Range

c4
Random
c3

x1y1

x6y6

Figure 6. Parallel random cleaning by a robotic swarm.
Many such scenarios, due to high level and simplicity
of DSL, may be created by the users (elders) themselves,or may just be activated by names, being prepared inadvance and regularly updated by special social services,
as a library of recommended reactions on different
events (accessible from laptops or mobile phones).

5 Conclusions
The described approach, effectively integrating
physical and virtual worlds, with individuals comfortably
living in both under the guidance of a powerful spatial
engine, can allow us to effectively formulate and solve
complex non-local problems in human societies,
demographic ones including.
High level but fully formal societal scenarios in DSL
may contain a good deal of flexibility for theirimplementation: a) by younger people, who may befollowing them creatively, as general recommendations;b) by elders, with possibly weakened decision-making, tofollow them closely, and if needed, directly; and c) bymobile robots and their swarms—just executing themstrictly as formal programs.
The scenarios may provide any combination of andshifts between these options, with unified goal-driven
control allowing for gradual transition to fully unmannedsystems effectively solving urgent problems, including
special support of elderly and handicapped people andtheir further active engagement for the benefit of thewhole society.
The approach offered can be easily implemented on
any platform, say, as a special intelligent module inlaptops, mobile phones (the number of which isapproaching 3bn worldwide), smart sensors or mobile
robots, and this work can be effectively carried out byboth professionals and university students (as for theprevious versions of the technology [10, 11]).

References[1] European Commission, Ageing well in the Informationsociety, An i2010 Initiative, 2007, http://eurlex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:20
07:0332:FIN:EN:HTML.
[2] European Commission, Ageing well in the Informationsociety, An i2010 Initiative, 2007, http://eur-

move (c1, c2, c3, c4, c5);
Range = range;
Polygon = (x1y1, x2y2, x3y3, x4y4,
x5y5, x6y6); D = shift;loop (New = WHERE +twice (random (-D, D));©ISAROB 2011

Free
from
othersarea
Noncleaned
place
RandomThis example shows how home cleaning robots (on ademand of an elderly person) can process the area needed.
Let us consider DSL scenarios for a single cleaning robot,
and also for their swarm operating in parallel.
Sequential cleaning. The sequential coverage of aterritory (assumed here to be rectangular, for simplicity) bya single robot can be organized in a variety of ways,
including the one with minimum waypoints to pass, in a
zigzag fashion, as shown in Fig. 5. For this latter option, theDSL program may be as simple as follows (the movement
between adjacent waypoints is supposed to be accompaniedby the cleaning process).
Y2

Random753

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2007:
0332:FIN:EN:HTML.
[3] AAL, Ambient assisted living, Case studies, 2004.,
http://www.aal169.org/Published/casestud.pdf.
[4] Kanev, K., Mirenkov, N., Pervasive carpet encoding foractive knowledge semantic surfaces, Book chapter in
Ubiquitous multimedia computing, Qing Li and Timothy K.
Shih (Eds), Chapman & Hall/CRC Press, 2010.
[5] Kanev, K., Mirenkov, N., Brimkov, B., Dimitrov, K., 2009.
Semantic surfaces for business applications, In Proceedingsof the Int. Conf. on Software, Services and Semantic
Technologies, Sofia, Bulgaria.
[6] Kanev, K., Morishima, Y., Watanabe, K., Surface code
readers for image based human-computer interfaces, In
Proceedings of the Eleventh Int. Conf. on Humans andComp uters HC’08, Nagaoka, Japan, 2008.
[7] Ishii, H., Tangible bits: towards seamless interfaces
between people, bits and atoms, Proceedings of the 2nd Int.
Conf. on Tangible and Embedded Interaction, Bonn,
Germany, 2008.
[8] Berners-Lee, T., Hendler, J., and Lassila, O., The semantic
web, Scientific American Magazine, May 2001.
[9] Sapaty, P., Distributed technology for global control, Book
chapter, Lecture Notes in Electrical Engineering, Vol. 37,
Springer-Verlag, 2009.
[10] Sapaty, P., Ruling distributed dynamic worlds, John Wiley& Sons, New York, 2005.
[11] Sapaty, P., Mobile processing in distributed and open
environments, John Wiley & Sons, New York, 1999.
[12] P. Sapaty, and M. Sugisaka, “Distributed Artificial Brain
for Collectively Behaving Mobile Robots,” In Proc.
Symposium & Exhibition Unmanned Systems 2001, Jul.
31-Aug. 2, Baltimore, MD, 2001.
[13] P. Sapaty, and M. Sugisaka, “Towards the Distributed
Brain for Collectively Behaving Robots”, In Proc.
International Conference on Control, Automation and
Systems, ICCAS 2001, October 17-21, 2001, Cheju
National University, Jeju Island, Korea, 2001.
[14] P.S. Sapaty, M. Sugisaka, “A Language for Programming
Distributed Multi-Robot Systems”, Proc. of The SeventhInternational Symposium on Artificial Life and Robotics(AROB 7th ‘ 02), January 16-18, 2002, B-Com Plaza,
Beppu, Oita, Japan.
[15] P. Sapaty, M. Sugisaka, “Universal Distributed Brain for
Mobile Multi-robot Systems”, in the book “Distributed
Autonomous Robotic Systems”, H. Asama, T. Arai, T.
Fukuda, and T. Hasegava (Eds.), Sringer-Verlag Tokyo
2002, SPIN: 10869189. Also reported at the International
Symposium on Distributed Autonomous Robotic Systems
DARS’02, June 25-27, 2002, Fukuoka, Japan.
[16] P. Sapaty, M. Sugisaka, “Optimized Space Search byDistributed Robotic Teams”, Proc. International
Symposium on Artificial Life and Robotics (AROB 8th),
January 24-26, 2003, Beppu, Japan.
[17] P. Sapaty, M. Sugisaka, “Optimized Space Search byDistributed Robotic Teams”, Proc. World Symposium
Unmanned Systems 2003, Jul. 15-17, 2003, Baltimore
Convention Center, USA.
[18] M. Sugisaka, P. Sapaty , K. Imamura , K. Tokuda , N.
Masuda, R. Finkelstein, J. Albus, “Research on
Biologically Inspired Bipedal Dynamic Humanoid Robots”,
Proc. World Symposium Unmanned Systems 2003, Jul. 1517, 2003, Baltimore Convention Center, USA.
[19] P. Sapaty, M. Sugisaka, “Towards a Universal Distributed
Brain for Mobile Multi-Robot Systems”, Mathematical
Machines and Systems, ISSN: 1028-9763, No. 3-4, 2003.
[20] P. Sapaty, K. Kawamura, M. Sugisaka, R. Finkelstein,“Towards Fully Distributed Cognitive Systems”, Proc.
Ninth International Symposium on Artificial life andRobotics (AROB 9th), Beppu, Japan, January 2004.

©ISAROB 2011754

[21] P. Sapaty, K. Kawamura, M. Sugisaka, R. Finkelstein,“Towards Fully Distributed Cognitive Systems”,
Mathematical Machines and Systems, ISSN: 1028-9763,
No. 1, 2004.
[22] Sapaty, P., M. Sugisaka, “WAVE-WP (World
Processing) Technology”, Proc. First International
Conference on Informatics in Control, Automation andRobotics, Setubal, Portugal, Vol.1, August 25-28, 2004.
[23] Sapaty, P., N. Mirenkov, M. Sugisaka, and M. Osano,“Distributed Artificial Life Using World Processing
Technology”, Proc. of the Fifth Int. Conference on
Human and Computer (HC-2004), September 1-3, 2004,
The University of Aizu, Japan, 2004.
[24] P. Sapaty, M. Sugisaka, N. Mirenkov, M. Osano, R.
Finkelstein, “Grasping the Distributed Entirety”, Proc.
Tenth International Symposium on Artificial Life andRobotics (AROB 10th), Beppu, Japan, February 4-6,
2005.
[25] P. Sapaty, M. Sugisaka, R. Finkelstein, J. Delgado-Frias,
N. Mirenkov, “Emergent Societies: An Advanced IT
Support of Crisis Relief Missions”, Proc. EleventhInternational Symposium on Artificial Life andRobotics (AROB 11th’06), Beppu, Japan, Jan 23-26,
2006, ISBN 4-9902880-0-9.
[26] P. Sapaty, M. Sugisaka, R. Finkelstein, J. Delgado-Frias,
N. Mirenkov, “Advanced IT Support of Crisis Relief
Missions”, Journal of Emergency Management, Vol.4,
No.4, ISSN 1543-5865, July/August 2006.
[27] P. Sapaty, M. Sugisaka, R. Finkelstein, J. Delgado-Frias,
N. Mirenkov, “Emergent Societies: An Advanced IT
Support of Crisis Relief Missions”, Artificial Life andRobotics, Vol. 11, No. 1, 2007, ISSN 1433-5298.
[28] P. Sapaty, M. Sugisaka, J. Delgado-Frias, J. Filipe, N.
Mirenkov, “Intelligent Management of Distributed
Dynamic Sensor Networks”, Proc. Twelfth International
Symposium on Artificial Life and Robotics (AROB
12th’07), Beppu, Japan, Jan 25-27, 2007.
[29] P. Sapaty, M. Sugisaka, J. Filipe, “Making Sensor
Networks Intelligent”, Proc. of the 4th International
Conference on Informatics in Control, Automation andRobotics, ICINCO-2007, Angers, France, 9-12 May
2007.
[30] P. Sapaty, A. Morozov, R. Finkelstein, M. Sugisaka, D.
Lambert, “A new concept of flexible organization for
distributed robotized systems”, Artificial Life andRobotics, Volume 12, Numbers 1-2 / March, 2008,
ISSN: 1433-5298 (Print) 1614-7456 (Online), Springer
Japan.
[31] P. Sapaty, M. Sugisaka, J. Delgado-Frias, J. Filipe, N.
Mirenkov, “Intelligent management of distributed
dynamic sensor networks”, Artificial Life and Robotics,
Volume 12, Numbers 1-2 / March, 2008, ISSN: 14335298 (Print) 1614-7456 (Online), Springer Japan.
[32] P. Sapaty, K.-D. Kuhnert, M. Sugisaka, R. Finkelstein,“Developing High-Level Management Facilities forDistributed Unmanned Systems”, Proc. FourteenthInternational Symposium on Artificial Life andRobotics (AROB 14th’09), B-Con Plaza, Beppu, Oita,
Japan, Feb. 5-7, 2009.
[33] Sapaty, P., Sugisaka, M., Kuhnert, K.-D. “Spatial
scenarios for distributed unmanned systems”, Proc.
AUVSI’s Unmanned Systems North America 2009,
Washington, DC, USA, 2009.
[34] P. Sapaty, M. Sugisaka, “Countering AsymmetricSituations with Distributed Artificial Life and Robotics
Approach”, Proc. Fifteenth International Symposium on
Artificial Life and Robotics (AROB 15th’10), B-Con
Plaza, Beppu, Oita, Japan, Feb. 5-7, 2010.

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Construction of the Muscle Fatigue Evaluation Model
based on Accuracy of Power
Akihiro Suzuki, Norihiko Kato, Yoshihiko Nomura, Hirokazu Matsui
Mie University, Kurimamachiya1577, Tsu Mie Japan(Tel : 81-59-231-9473; Fax : 81-59-231-9663)(suzuki@robot.mach.mie-u.ac.jp)

Abstract: We develop a rehabilitation support system that assists the exercise training to reduce the physical load oftherapists. In this study, we propose a muscle fatigue evaluation method to decide appropriate momentum for patients.
Lots of studies of the muscle fatigue by EMG have been reported. However, muscle fatigue evaluation by EMG has
disadvantages of complexity of preparation and gives patients mental stress. To solve this problem, in this study, we
propose the muscle fatigue evaluation by the accuracy of muscle power. We verify proposed method on
abduction/adduction of shoulders. We investigate how the accuracy of the muscle power changes underabduction/adduction repeatedly maintaining constant power of tangential direction. From the experimental results, we
confirm correlation increase of amplitude of EMG and increase of power in direction of normal by power direction
shift from tangential direction that is direction of target as the muscle fatigue increase.
Keywords: Rehabilitation, Robotics, Shoulder, Muscle, Fatigue, Motor Unitof muscle power. We investigate correlation of themuscle fatigue evaluation by the EMG and accuracy ofthe muscle power, and propose muscle fatigue
evaluation model.

I. INTRODUCTION
Recent years, the load of therapists and caregivers
increases with the declining birthrate and a growing
proportion of elderly people of the society. We aim toassist and reduce the load of the physical therapists who
train rehabilitation, especially the exercise therapy
training (maintenance and increase training for joint's
range of motion, muscle-strengthening exercise,endurance training, and cooperated movement training,etc.), and then, we develop and the program of easy-touse robot arms which assists these training operationand evaluate them.
We propose the muscle fatigue evaluation method
to decide appropriate momentum for patients. Lots ofstudies of the muscle fatigue by EMG have beenreported. In the muscle fatigue evaluation by EMG, as
the muscle fatigue increases, amplitude of EMG
increases. Generally speaking, it is said that is becauseof the increase of the amplitude by the collection ofmuscle fibers to keep its power, shifting from large andfaster motor unit to a small and low one, and also
shifting of the frequency element to the low-frequency
area by the synchronizing ignition which ignites all
However, muscle
motor units at the same time.
fatigue evaluation by EMG has the disadvantage ofcomplexity of preparation and making the patientsmentally stressful. To solve this problem, in this study,
we propose the muscle fatigue evaluation by accuracy©ISAROB 2011

II. REHABILITATION SUPPORT SYSTEMAs shown in Figure 1, robot arm with 7 degrees of
freedom is used, and the force-torque sensor is installedin the tip of the arm. We control the motion by
operating robotic arm based on the information of
power and the torque measured by the sensor.

Fig.1. Rehabilitation Support System

III. MUSCLE FARIGUE EVALUATIONIn this study, we propose the muscle fatigue
evaluation based on the accuracy of the power. We

755

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

4.2. Result

verify the accuracy of the power according to themuscle fatigue from two viewpoints such as amount andthe direction. As shown in Fig.2, we divide the powerin Fr of subject into normal direction element Fn andtangential direction element Ft centers on the shoulder
joint, and evaluate them respectively. As shown with
Fg in Fig.2, it aims at the movement which only thevalue is specified only for a tangential direction works
effectively. We verify the accuracy of amount of
power by the increase and the decrease of the difference
from the targeted value |Fg-Ft|. And about, we verify
direction of power because of the increase and decreaseof the normal direction element of power |Fn| by power
direction shifted from tangential direction that is the
direction of target. We calculated BIN average of themeasuring data every one degree in the movement angle
and compared with the total values of every one roundtrip.

4.2.1. Change in Accuracy of Power of Each
Movement Frequency
We show the normal direction element |Fn| of eachmovement frequency by abduction/adduction in Fig.3(a) and the difference of the targeted value of thetangential direction element |Fg-Ft| in Fig.3 (b) by
average moving deviations of every round trip of five.
We confirmed that the normal direction element |Fn|
increased every time the movement frequency was
piled up. The similar tendency was not shown on thetangential direction element |Fg-Ft|.

(a) Normal Direction Element |Fn|
Fig.2. Evaluation of Muscle Fatigue

IV. EXPERIMENT
4.1. Method
The subjects are nine healthy persons. We hadthem move their arms within the angle of 30 to 90°.
Load was set to be 30% of maximum isometriccontraction for three seconds in 60°. We had themkeep working until they become less than set amountwith the constant movement angular speed (8deg/sec).
To understand the muscle activity under the movement,
we measured EMG of Deltoid (middle, anterior,posterior), Cowl muscle that relates to the abduction and
Pectoral major muscle relates to the adduction.
We had the subjects exercise to follow the targetedvalue of the power and the target orbit displayed in themonitor as much as possible.

©ISAROB 2011(b) Tangential Direction Element |Fg-Ft|
Fig.3. Accuracy of Power of each Movement Frequency
4.2.2. Amplitude of EMG of Each Movement
Frequency
In Fig. 4(a), a relative value which was taking at
the maximum isometric contraction of 100% in 6o° isshown. As the movement frequency is piled up, we
confirmed the increase in the level of each muscle
activity level.
We show the mean power frequency (MPF) of eachmovement in Fig.4 (b). We confirmed the muscle

756

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011of power |Fn|. We show the correlation coefficient of
the amplitude of EMG of each muscle and normal
direction element of power in Table 1. We confirmedthat there was a strong correlation between muscle
fatigue and the accuracy in direction of power.

fatigue from the increase of the amplitude of EMG andthe shift of the frequency element to the low frequency
region.

(a) Amplitude(a) Deltoid Middle(b) Mean Power Frequency
Fig.4. EMG
a: Deltoid Middle
b: Deltoid Anteriorc: Deltoid Posterior
d: Cowl Muscle
e: Pectoral Major Muscle(b) Deltoid Anterior

4.2.3. Correlation of Amplitude of EMG and
Accuracy of PowerWe show the correlation for the amplitude of EMG
in each muscle and normal direction element of power|Fn| in Fig.5. The horizontal axis in the graph shows is
the value in which measured amplitude of EMG
assuming the first mean value of three round trips to be
standard value of 0, and makes the last mean value of
three round trips a relative value as 100. The verticalaxis in the graph shows the normal direction element of
power in difference by the first mean value of threeround trips. As the amplitude of EMG increases, we
confirmed the increase of the normal direction element©ISAROB 2011(c) Deltoid Posterior

757

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

comparatively large. It is thought that the control ofpower roughens, and the decrease of accuracy of power
was caused by this. In addition, the smoothness ofmuscle contraction is ruined by the synchronization
phenomena of motor unit that all motor units ismobilized, and it becomes an explosive activity style inthe vicinity of the movement limit. Therefore, we thinkthat accuracy of power decreased more

VI. CONCLUSION
In this study, we proposed muscle fatigue
evaluation based on accuracy of power under a low load,and confirmed that there was a strong correlation in therelation between increase of normal direction element of
power caused by the decrease of accuracy in directionof power and muscle fatigue.
The comparative study of a load and the movement
speed that differs and depends is necessary as futuretasks. Moreover, we should consider the individual
variation depends on the subjects by constructing themodel type because there is a difference in the value of|Fn| when muscle fatigue reaches to the critical limit.
Also, we should verify whether our proposal
technique is useful for actual rehabilitation patients who
are waiting for.

(d) Cowl Muscle(e) Pectoral Major Muscle
Fig.4. Correlation of Amplitude of EMG and |Fn|
Table 1 Correlation Coefficient
Coefficient
Correlation
Deltoid Middle
0.964
Deltoid Anterior

0.958

Deltoid Posterior

0.971

Cowl Muscle

0.950

Pectoral Major Muscle

0.903

REFERENCES[1] Jyunichirou Aoki, Yuu Satou, Isao Muraoka(2005), sports physiology[2] Syuu Yatojima, Asahiro Kiduka, Hioroshi Yajima(2003) Influence that movement leg position and angle
variation when shoulder abduction on activity aspect ofsurroundings muscle of shoulder[3] Syuu Yatojima, Asahiro Kiduka, Naoki Mukai, Jin
Siragi, Yutaka Miyanaga, Muscle activity characteristic
when shoulder abduction on different movement
condition[4] Sinichi Huruya, Fatigue[5] Haruo Sugi (2003), The muscle is mysterious,
Shown mechanism that gives birth to power[6] Noriyuki Hayami, Eiishi Tanaka, Souta Yamamoto,
Hiorki Takeuchi, Modeling of muscle fatigue thatconsiders difference of fatigue tolerance of motor unit
type[7] Yasushi Itoh, Kumi Akataki, Katsumi Mita,
Watakabe Makoto, Kunihiko Itoh (2003), Time of
Mechanomyogram of continuation contraction withmuscle fatigue- Frequency analysis

V. CONSIDERATION
We had subject isotonic movement under low load,and confirmed that the normal direction element of
power increases by the decline of the accuracy indirection of power with increase of the muscle fatigue.
Immediately after the movement begins, the poweris maintained by the muscle which is comparatively
small (ST-MU) because of the load was set low. Whenthe tension of ST-MU decreases, the power is made up
by restarting of stopping muscle that motor unit is©ISAROB 2011758

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Anomalous Situations Detection Based on Confluence
Tatsuya Gibo†

Shigeki Aoki‡ Takao Miyamoto Motoi Iwata Akira Shiozaki
Osaka Prefecture University
1–1 Gakuen–cho, Naka–ku, Sakai–shi, Osaka, JAPAN, 599–8531
gibo t@ch.cs.osakafu-u.ac.jp†
aoki@las.osakafu-u.ac.jp‡

Abstract: Recently, surveillance cameras have been set up everywhere such as streets and public places in orderto detect anomalous situations. In the existing surveillance systems, since only a handful of surveillance agentswatch a lot of images from surveillance cameras, there may be some possibilities that they miss the importantscenes such as accidents or abnormal incidents. So, we propose a method of the sequential learning andrecognition of comprehensive behavioral patterns in the crowded places. Firstly, we extract the comprehensive
conﬂuence from an input image using optical ﬂows. Secondly, we extract the behavioral patterns based onchange-point detection of the conﬂuence. Finally, we recognize the behavioral pattern by comparing theprevious behavioral pattern in the database with a newly observed one. We carried out experiments to verify
the eﬀectiveness of our approach by placing a surveillance camera on our campus.
K eywords: Behavioral pattern recognition, Optical ﬂow, Dynamic programming.

1

Introduction

Recently, surveillance cameras have been set up
everywhere such as streets and public places in order to detect anomalous situations. In the existing
surveillance systems, since only a handful of surveillance agents watch a lot of images from surveillancecameras, there may be some possibilities that they
miss the important scenes such as accidents or abnormal incidents. In order to solve this problem, there
have been the considerable researches on the system
which automatically recognizes human behavior in asurveillance footage. It warns security guards when
it detects anomalous situations.
Several methods[1, 2, 3] for recognizing human behavior focus on a few people in a room, and it is diﬃcult to apply the methods to more people in crowded
places.
The methods for recognizing crowd behavior using a
range scanner or a 3D point measurement device[4, 5]
have been proposed. These researches, they canmake conﬂuence analysis of dense crowd possible.
However, it is not easy to apply these methods because most of the existing surveillance systems use
only a monocular surveillance camera.
So, we propose a method for detecting an anomalous situation in crowded places using a monocularsurveillance camera.

2

Figure 1: A mode values of optical ﬂows and a divided image into 20 × 20[pixels] segments. A mode
value was drawn on center of each segment.
keep track of people individually. Because there isan enormous types of motions of people, it is not easy
to recognize every behavioral pattern. So, in order tolearn the comprehensive behavioral patterns of multiple people and to recognize them, we regard the
motions of people as one comprehensive conﬂuencebased on the moving direction of people.

2.1

Learning and Recognition of Comprehensive Behavioral Patterns

In this subsection, we explain a technique for extracting a feature vector of motions.
Firstly, to extract a feature vector of the motions,
we compute an optical ﬂow of each pixel in an input
image. Secondly, a mode value vn = {du, dv}(n =1, 2, ..., N ) are computed by dividing an input image
into 20 × 20 [pixels] segments and in each segment,

In this section, we describe a method for recognizing the behavioral patterns of people in such places
as multiple people keep moving. In the public places,
occlusion occurs frequently, hence it is diﬃcult to©ISAROB 2011

Extraction of Feature Vector of Confluence

759

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Figure 3: The behavioral pattern extraction bychecking the existence of motion.
2.2.1

We regard the time when Vt = 0 were observed forthe fourth time in a row afterward Vt returned to
non 0 as division point for the start of the behavioral
pattern.

Figure 2: Example of change-point detection by degree of congestion and conﬂuence.

2.2.2
we compute a mode value of optical ﬂows. Where
N is the number of segments. Fig.1 shows an example of the mode values. Finally, a feature vector of
conﬂuence Vt = {v1 , v2 , ..., vN } is created by rasterscanning the mode value of the optical ﬂows vn ineach segment.

2.2

Division Point Setting Method byChecking Motion Existence

Division Point Setting Method by Detecting Congestion Changes

Firstly, we calculate the number of the non 0 mode
values vn (n = 1, 2, ..., N ) in the Vt . We deﬁned it
as ht . ht express a degree of congestion. Secondly,
when ht meets the following requirement, we consider the time t to be a division point on a sequenceof the feature vectors, where is a threshold value.

Extraction of Behavioral Patterns

1 ∑ht−m | >
5 m=1
5

|ht −

Generally, the following behavioral patterns areobserved in crowded places.

(1)

I. People appear in sight of a camera → people goout of sight of a camera

2.2.3

II. As shown Fig.2 (a) people walk on a pedestriancrossing → (b) when they are walking, vehicles
go toward the people

In this method, in order to survey the similarity between the conﬂuence at time t (Vt ) and the conﬂuence at time t − 1 (Vt−1 ), we use Wilk’s Λ and
M-BOX. Signiﬁcant level is 5%. The null hypothesisis ”distribution of Vt and Vt−1 is identical”, and thealternate hypothesis is ”distribution of Vt and Vt−1
is diﬀerent”. When the null hypothesis is rejected,
the alternate hypothesis is adopted. We set division
point when the alternate hypothesis is adopted.

III. As shown Fig.2 (c) people and vehicles gostraight → (d) they turn to the left
We regard the time when people appear in sight ofsurveillance camera, the time when degree of congestion changes (Fig. 2(b)) and the time when conﬂuence changes (Fig. 2(d)) as new behavioral pattern starts and we called it division point. We set
the division points to the sequence of feature vectors {..., V16 , V17 , ..., V21 , V22 , ...} to extract the behavioral patterns. We deﬁne the feature vectors of
conﬂuence between division point as a behavioral
pattern. We describe it the extracted p-th behavioral pattern Ip = {V1p , V2p , ..., VTpp }. Where Tp islength of the sequence Ip . Fig.3 shows, an exampleof Ip and Ip+1 extracted from a sequence of feature
vectors.

©ISAROB 2011

2.3

Division Point Setting Method by Detecting Confluence Changes

Learning and Recognition Method

The procedure of sequential learning and recognition of behavioral patterns and detection of anomalous situations was composed the three generalphases. Firstly, when behavioral pattern Ip is observed, we perform the comparison process (Fig. 4Comparison) to conﬁrm the existence of the same
patterns in the database. Secondly, we carry out therecognition (Fig. 4 Recognition) of Ip . When Ip is

760

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 1: Experimental results of sequential learning, recognizing and anomalous situation detection.
Date

Numberof patterns

Behabioral

Number of
Cumulative patterns

anomalous

2–8

12,993

recognition
11,920(94.9%)

misclassification
635(5.1%)

recognition420(95.9%)

misclassification
18(4.1%)

9 – 15

11,749

-

18

415

-

793
950

Total

24,742

-

658

835

-

-

2.3.1

START
Comparison

If Ip which was observed is daily behavior, the frequency of observation Ff of Rf is greater than thatof other reference pattern Rq (q = 1, 2, ..., Q) in the
database. If Ip is anomalous behavior, Ff is lower
than that of other Rq (q = 1, 2, ..., Q). So, we calculate the observation probability P r(f ) to detect
anomalous situation. If the value is lower than thethreshold γ, we detect Ip as an anomalous situation.

Calculating similarity cost q
of I p and R q (1 < q < Q )Recognitionf = argmax (| costq|)
1< q < Q

Add|cost f | > β

No

Q=Q+1

Ff − 1
Pr(f ) = ∑Q
q=1 FqYes
Output f as a recognition result

Method of Anomalous Situations Detection

Add I p to S as R Q

Ff = Ff + 1

3(2)

Experiments and Discussions

UpdateIn order to verify the eﬀectiveness of the proposed
method, we performed an experiment. We evaluatedthe recognition success rate and the misclassiﬁcation
rate of the behavioral patterns and the recognition
success rate and the misclassiﬁcation rate of the theanomalous situations. Data set were observed between Dec.2, 2008 and Dec.15, 2008 using a surveillance camera placed on Osaka-Prefecture University.
The image size is 360 × 240 [pixels]. The thresholds
are = 2.0, = 0.05 and γ = 0.01.

Update R f
Detection
Detection of anomalous situation

END

Figure 4: The procedure of sequential learning andrecognition behavioral patterns and constructing reference patterns and detecting anomalous situations.

3.1

The second column of Table 1 shows the numberof observed patterns in several days. The third andfourth columns show the number of recognition success and misclassiﬁcation of the behavioral patterns.
The recognition success of the behavioral patterns
is that the classiﬁer correctly recognizes behavioral
pattern as behavioral pattern. The misclassiﬁcationof behavioral patterns is that the classiﬁer falsely recognizes behavioral pattern as anomalous situation.
The ﬁfth and six columns show the number of recognition success and misclassiﬁcation of the anomalous situations. The recognition success of anomalous situations is that the classiﬁer correctly recognizes anomalous situation as anomalous situation.
The misclassiﬁcation of the anomalous situations isthat the classiﬁer falsely recognizes anomalous situation as behavioral pattern. The last column shows

extracted, we compare Ip with the fore-past behavioral patterns Rq in the database by computing thesimilarities |costq |(q = 1, 2, .., Q) using the dynamic
programming. After that, Rf that has the highest
similarity is chosen from the database. If the |costf |is higher than , we recognize Ip as Rf , and we perform updating process (Fig. 4 Update). Where is a
threshold value for recognition. If the |costq | is lower
than , we add (Fig. 4 Add) Ip to the database as
a new behavioral pattern IQ+1 . Finally, we perform
detection process (Fig. 4 Detection) of anomalous
situations.

©ISAROB 2011

Experimental Results

761

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

the number of patterns that were registered in the
database.
In terms of experiments during Dec.2–Dec.8, in order to verify recognition accuracy of our method, we
survey observed all patterns. In Table 1, 12,993 observed patterns during Dec.2–Dec.8 were recognizedby 793 diﬀerent reference patterns. We had 94.9%
recognition success rate of the behavioral patterns
and 95.9% recognition success rate of anomalous situations. In Dec.5, misclassiﬁcation of the behavioral
patterns occurs because it is rainy. In the case that
we omit the data of Dec.5 from the data sets, we had
99.8% recognition success rate of behavioral patterns
and 95.6% recognition success rate of anomalous situations, during Dec.2–Dec.8.
In terms of during Dec.9–Dec.11, in order to verify
detection accuracy of anomalous situations of our
method, we survey the objects which the classiﬁer
recognizes as anomalous situations. During Dec.9–
Dec.11, 749 diﬀerent reference patterns were registered in the database, and 11,749 observed patterns
were recognized by 950 diﬀerent reference patterns.
Therefore, we conﬁrmed a satisfaction of our
method.

3.2

learns many behavioral patterns on earlier stage, andthe classiﬁer learns the behavioral patterns which are
not existent in the database.
In order to detect anomalous situation, we use theobservation probability. Detected anomalous situations are throwing the ball back and forth, lingering
around. These behaviors were too observed. Therefore, we conﬁrmed that the proposed method canrecognize behavioral patterns and can detect anomalous situations.

4

We presented a novel method for detecting theanomalous situation in crowded places using a
monocular surveillance camera. To learn and to recognize behavior of the multiple people and to recognize them, we regard the motions of people as one
comprehensive conﬂuence with a focus on the moving direction of people. Besides, in order to detect
anomalous situations, we calculated the observation
probability based on the frequency of observation.
In the results, we conﬁrm the eﬀectiveness of the
proposed method in crowded places.

Discussions

References

In this paper, we have proposed a novel methodfor detecting the anomalous situations in crowded
places using a monocular surveillance camera. We
describe bellow outline of the proposed method.

[1] B. Schölkopf, J.C. Platt, J. Shawe-Taylor, A.J.
Smola, R.C. Williamson (2001), Estimating the
Support of a High-Dimensional distribution,
Neural Comput, Vol.13, No.7, pp.1443–1471

Extraction of feature vector of confluence
In this method, we divided an input image and wecompute the mode values of optical ﬂows to extract
comprehensive conﬂuence. We could extract conﬂuence in crowded places by this approach.
However, we could not extract the feature of quiescent people using optical ﬂows. To tackle theproblem, we will use optical ﬂows and backgroundsubtraction.

[2] H. Zhong, J. Shi, M. Visontai (2004), Detecting
Unusual Activity in Video, Proc. IEEE Conf. on
CVPR, Vol.2, pp.819–826[3] S. Aoki, Y. Iwai, M. Onishi, A. Kojima, K.
Fukunaga (2005), Learning and Recognizing Behavioral Patterns Using Position and Posture of
Human Body and Its Application to Detectionof Irregular States, System and Computers inJapan(SCJ), Vol.36, No.13, pp.45–56

Extraction of behavioral patterns
We extract of behavioral patterns by using the threerules combination. These are people motion existence, congestion changing points and conﬂuencechanging points. We could extract the behavioral
patterns correctly using the rules. If we use only theﬁrst rule, we do not extract behavioral patterns under the situation that constant ﬂow of conﬂuence isobserved always. However, we could extract behavioral pattern under such a situation by using second
and third rules.

[4] K. Nakamura, H. Zhao, R. Shibasaki, K.
Sakamoto, T. Ooga, N. Suzukawa (2005), Tracking Pedestrians Using Multiple Single-Row Laser
Range Scanners and Its Reliability Evaluation(in Japanese), IEICE Trans. D–II, Vol.J88–D–I
I, No.7, pp.1143–1152[5] M. Onishi, I. Yoda (2010), Comparison and Visualization Method of Pedestrian Trajectories in aComplex Oﬃce for a Long Period (in Japanese),
IEICE Trans. D, Vol.J93–D, No.4, pp.486–493

Learning, recognition and detection
From the results, The increase in the number of patterns in the database is reductive as day goes on.
From this reason, we could see that the classiﬁer©ISAROB 2011

Conclusion

762

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Automatic Segmentation of Liver Region from Non-contrast and Contrast
CT images Employing Tree-structural Image Transformations
Masafumi Komatsu1), Guangxu Li1), Hyoungseop Kim1),
Joo Kooi Tan1), Seiji Ishikawa1), Akiyoshi Yamamoto1), 2)
1) Kyushu Institute of Technology, 1-1, Sensui, Tobata, Kitakyushu, Japan2) Kyoaikai Tobata Kyoritsu Hospital, Japan(Tel: +81-93-884-3185; Fax: +81-93-861-1159)(komatsu@kimlab.cntl.kyutech.ac.jp)
Abstract: In the medical image processing field, segmentation from the CT image is one of the most important
problems for analyzing the abnormalities and diagnosis on visual screening. Many related segmentation techniques
have been developed for automatic extraction of ROI. It is however, there are still no fully automatic segmentation
methods that are generally applicable to ROI based on CT image set. In this paper, we present a technique for automatic
extraction of liver region on the MDCT images employing automatic construction of tree-structural imagetransformation (ACTIT). We propose a new technique for extraction of organs employing ACTIT with non-contrast andcontrast image set in order to introduce temporal change information. We apply the proposed technique to threeabdominal image set and satisfactory segmentation results are achieved.
Keywords: Segmentation, Liver region, MDCT, Tree-structural filter, ACTIT, Temporal change.
tracking. Shimizu et al. [4] proposed the simultaneous
methods that are generally applicable to regions ofinterest based on CT image sets. We also havedeveloped the segmentation technique of spleen andgallbladder region employing rib information andregion growing technique [5]. Many other techniques
are proposed for segmenting the organs. It is however,there are still no fully automatic and satisfactory
segmentation methods that are generally applicable to
ROI based on CT image set.
On the other hand, in the image analyzing fields,
automatic construction of tree-structural imagetransformations (ACTIT) is used for constructing thefiltering tree [6]. The ACTIT is the method that
automatically generates image transformation processfrom several given examples by use of genetic
algorithm. By use of the ACTIT, we can get the imagetransformation filtering tree as a combination of known
filters, which is able to be applied to unknown image.
In this paper, we present a technique for automatic
extraction of liver region on the MDCT images
employing ACTIT. In our study, we propose a newtechnique for extraction of organs employing the ACTIT
with non-contrast and contrast image set. In the past,
ACTIT used only one image as tree-nodes. In this paper,
we set the non-contrast image and the contrast one astree-nodes of filtering tree. By using this method, we
can introduce temporal change information between
non-contrast and contrast images.

I. INTRODUCTION
In recent years, various imaging equipments have been
introduced into medical fields. Especially, highresolution helical computed tomography (HRCT) is oneof the most useful diagnosis systems because it provides
a high resolution image to medical doctors as clear
image. Radiologist can easily detect abnormalities byuse of the image with high resolutions. Accordingly,
many related image processing techniques are proposed
into medical fields for extraction of abnormal area.
Automatic extraction of the region of the interest (ROI)
is an important theme of medical imaging and computer
vision technologies.
In the medical image processing field, segmentationfrom the CT image is one of the most important
problems for analyzing the abnormalities and diagnosis
on visual screening. Many related segmentation
techniques have been developed for automatic
extraction of ROI before analyzing the abnormalities.
Especially, in order to extract multi organs and tounderstand the structure of internal organ, several
approaches have been developed in the past. Ney et al.
[1] proposed a technique based on tracing the liverboundary section with a mouse. Gao et al. [2] applied amethod of automatic segmentation of the liver region
employing knowledge based and a model based
boundary finding scheme. Bae et al. [3] devised a
segmentation method of liver structures employing
Gaussian smoothing and eight-point connectivity©ISAROB 2011763

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Start

The organization of this article is as following. Firstly,
we present a method for segmentation of CT image byuse of ACTIT in section 2. Then we give someexperimental results in section 3. Finally, in section 4,
we give conclusions along with discussion of the
problems obtained from experimental results.

Initial genes for tree-structural filters
Calculating the fitness

Fitness is over 99% or
No change the fitness on 30 loops?

II. METHODS

Yes

Figure 1 shows the flow of the proposed method. In
the first step, we input non-contrast images, contrast
images and reference images which are made manually.
Non-contrast and contrast images are used as the nodesof tree-structural filter. In the second step, we apply
ACTIT in order to generate tree-structural filter. Finally,
we apply generated filter to unknown images.

No
Selection
Crossover
Mutation

1. ACTIT
ACTIT is the method which generates the structuralfilter as the combination of known filters. In order to
optimize the filters, the genetic algorithm is used. Fig. 2
shows the overall scheme of the ACTIT. First, we makea lot of tree-structural filters as initial genes. Next, we
calculate fitness by the following equation.
 Wx W y  wi ( x, y ) Oi ( x, y )  Ti ( x, y ) 
1 K  x 1 y 1fitness   1 Wx W yK i 1 wi ( x, y ) V max (1)x 1 y 1Here, Oi, Ti, and K shows output image, reference image,and the number of slices respectively. Wx, Wy, Vmax
shows image width, image height, and the maximum
value of image density respectively. wi shows the weight.
Furthermore, we optimize the tree filters by genetic
algorithm (elitist selection, crossover, and mutation(1%)). In the crossover step, we apply crossover
between branches.
If the fitness is over 99% or does not change for 30
loops, we stop the examination and get optimized filters.
Start

Stop
Fig.2 Overall scheme of ACTIT

2. ACTIT with temporal information
In general, ACTIT used one image as the nodes of tree.
However, when we applied the ACTIT to non-contrast
CT images, we could not achieve satisfactory results(Fig.3), because there are a lot of noises on the result
image.
In the abdominal CT images, soft tissues such as liver,kidney and spleen are shown as 50～60[H.U.]. And the
pixel value of muscles indicates 30～60[H.U.]. It is adifficult problem to separate between soft tissues andmuscles.
To overcome this problem, we propose ACTIT withtemporal subtraction information between the non-Input images
Applying ACTIT(a)(b)
Fig. 3 Experimental results of ACTIT with only one
image as nodes of tree filter. (a) shows original image,(b) shows the experimental result.

Applying generated filter to unknown images
Stop
Fig.1 The flow of procedure.

©ISAROB 2011764

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

contrast images and the contrast one. We can introduce
contrast medium information by exchanging a part ofnon-contrast image to the contrast one randomly.
Contrast medium changes image contrast and appears
around soft tissue or blood vessels with high CT values.
So we can get contrast medium information by
subtraction of images. Each features which is adopted
are shown in Table.2

Table 2 Filters for constructing tree-structural filter
FilterEffectAverage
3-neighbors average filter
Minimum
3-neighbors minimum filter
Maximum
3-neighbors maximum filter
InverseInversion of pixel value
Erosion
ErosionDilation
Dilation
Binarization
Binarization using(discriminant analysis) discriminant analysis
Binarization (average) Binarization using the
average value of image
LaplacianLaplacian filter
Laplacian + Vmax
Laplacian + the maximum
value of image
Sobel
Sobel filter
Segmentation of
Segmentation of maximum
Maximum area witharea with binarization using
binarization
discriminant analysis(discriminant analysis)
Segmentation of
Segmentation of Maximum
Maximum area witharea with binarization using
binarization (average) the average value of image
Elimination of
Elimination of symmetricalsymmetrical area
area
Elimination of nonElimination of nonsymmetrical area
symmetrical area
Subtraction
The absolute value ofsubtraction
Logical summation
The larger value between V1
and V2
Logical product
The smaller value between V1
and V2
Algebraic summation
V1+V2-(V1×V2)/Vmax
Algebraic product(V1×V2)/Vmax
Bounded summation
V1+V2
Bounded product
V1+V2-Vmax
Drastic summation
If V1 and V2 are not 0, it
returns Vmax. Else if only V1 is
0, it returns V2. Else if only
V2 is 0, it returns V1.
Drastic product
If V1 and V2 are not Vmax, it
returns 0. Else if only V1 is
Vmax, it returns V2. Else if
only V2 is Vmax, it returns V1.

III. RESULTS
1. Experimental environments
We apply the proposed technique to three real CT
cases in order to extract the liver region. The acquisition
parameters and experimental environments are shown in
Table 1. The images are 512×512 pixels in size with apixel spacing of 0.616 to 0.744 mm. And we use 5 slices
per case in order to construct tree filter and calculate
fitness. Table 2 shows the list of filters which we used
for constructing tree-structural filters. In Table 2, V1 andV2 show the pixel value of image 1 and image 2,
respectively. Furthermore Vmax shows the maximum
pixel value of the image.

2. Results
Figure 4 shows the experimental result for extractionof liver regions by use of proposed algorithm. In Fig. 4,(a) shows the non-contrast original image, (b) shows thecontrast image, (c) shows the reference image, (d)shows the experimental result which is achieved byapplying generated filter to (a) and (b). (e) and (f) show
other non-contrast image and the image obtained byapplying tree-constructed filter generated from (a) and(b) to (e), respectively.
Table 1 Experimental environments
OS
Microsoft Windows XP
Development environment
Microsoft Visual C++CPUIntel Pentium 4 3[GHz]Memory1[GB]CT scanner
Toshiba Aquilion 16
Image size
512×512[pixels]
Pixel spacing
0.616～0.744[mm]Image resolution
16[bits]
Slice thickness
2[mm]©ISAROB 2011765

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

(a)

(b)

(c)

(d)

achieved. We can check that the proposed technique is
useful for extraction of liver regions.
We proposed the new method for ACTIT in order tointroduce temporal change information by use of noncontrast and contrast images as nodes of tree-structuralfilter. We can check that it is useful for eliminating thenoises caused by similarity of pixel values between
muscle and other soft tissues.
The proposed technique is useful for extraction of theliver regions. But it still remains miss-segmentation
errors. We use the reference image including bloodvessels nearby liver regions, but when we apply theobtained tree filters to other CT cases, blood vessel
regions do not remain on the experimental result. It isbecause the fitness did not exceed 99%. We think thatthe combination between proposed technique as coarse
segmentation and level set method [7] as finesegmentation is useful in order to improve segmentation
accuracy. Furthermore improvement of algorithm is
now in progress. We believe this segmentation method
makes possible to improve the segmentation accuracy
for analyzing the internal structure of human body
before the operation.

REFERENCES(e)(f)
Fig. 4 Experimental results : (a) shows non-contrast
image, (b) shows the contrast image, (c) shows thereference image, (d) shows extracted region by applying
tree-constructed filter generated from (a) and (b). (e)and (f) show other non-contrast image and the result
obtained by applying filter used for generating (d) to (e)image.

[1] D. R. Ney, R. A. Drebin, E. K. Fishman, D. Magid,“Volumetric rendering of computed tomography
data : Principles and techniques”, IEEE Comput.
Graph. Appl., Vol.10, pp.24-32, 1990.
[2] L. Gao, D. G. Heath, B. S. Kuszyk, E. K. Fishman,“Automatic liver segmentation technique for threedimensional visualization of CT data”, Radiology,
Vol.201, No.2, pp.359-364, 1996.
[3] K. T. Bae, M. L. Giger, C. T. Chen, C. E. Kahn Jr.,
“Automatic segmentation of liver structure in CT
images”, Med. Phy., Vol.20, No.1, pp.71-78, 1999.
[4] A. Shimizu, R. Ohno, T. Ikegami et al.
“Simultaneous extraction of multiple rgans fromabdominal CT”, Proc. of CARS. P.1275, 2005.
[5] S. Toyota, H. Kim, S. Ishikawa et al., “Automatic
detection of spleen and gallbladder region employing
rib information”, IEICE, MI2006-97, pp.53-54, 2007.
[6] T. Nagao, S. Masunaga, “Automatic construction ofimage transformation processes using genetic
algorithm”, Proceedings of the ICIP’ 96, vol. 3, pp.
731-734, 1996.
[7] T. Hitosugi, A. Shimizu, M. Tamura et al.,“Development of a liver extraction method using alevel set method and its performance evaluation”,
Journal of Computer Aided Diagnosis of Medical
Images, Vol. 7, No. 4-2, pp.1-8, 2003.

Table 3 Fitness of experimental results
Reference
Case 1Case 2
Case 3
Case 1(97.95)[%]97.36[%]
97.56[%]
Case 297.42[%](97.86) [%]97.38[%]
Case 397.58[%]
97.25[%](97.59) [%]The fitness of experimental results by calculating
equation (1) is shown in Table 3.

VI. CONCLUSION
In this paper, we proposed a segmentation techniquefor liver region employing ACTIT. In the experimental
results, satisfactory segmentation performances are©ISAROB 2011766

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Obstacle Detection Using a Moving Camera
Shao Hua Qian1, Joo Kooi Tan1, Seiji Ishikawa1, Takashi Morie2
1

Department of Mechanical & Control Engineering, 2School of Brain Science & Engineering
Kyushu Institute of Technology
Sensuicho 1-1, Tobata, Kitakyushu, 804-8550, Japan
E-mail: {qian, etheltan, ishikawa}@ss10.cntl.kyutech.ac.jp, morie@brain.kyutech.ac.jp

Abstract: This paper proposes a method of detecting obstacles from a video taken by a moving camera mounted on avehicle by background subtraction. The background subtraction is often used to detect moving objects when camera is
static. But according to the characteristics of a road, we can also employ Gaussian mixture model to detect all objects(either static or moving objects) on the road in the case of moving camera. Then we use two consecutive image frames,
and warps the first image according to the geometrical relationship between these two images. The road area is thenextracted by comparing the warped image with the second image. Using this road area, we can delete all things which
are not obstacles. In the performed experiments, it is shown that the proposed method is able to detect obstacles such as
vehicles and pedestrians on a road.
Keywords: Obstacle detection, moving camera, monocular vision, Gaussian mixture model, road area detection.
In the first place, we employ a Gaussian mixture model[1,2] in reconstructing the background from a video
image sequence taken by a moving camera. According
to road characteristics, we can assume camera and roadare static, and then we can get an imaginary scene. In
this scene, the background is a road area; objects andpedestrians on the road, buildings, road marks and zebra
crossings are foreground objects. Because foregroundimage includes everything on the road, buildings,
shadows and road marks (such as zebra crossing, lanelines). Our goal is to extract obstacles on the road, so weneed to delete other things in the foreground image.

I. INTRODUCTION
Thousands of people die by car accidents year by
year. Many of those accidents could be avoided or
alleviated by vision-based driving assistance systems.
These systems cause drivers to respond more quickly inthe face of danger. In these systems, the ability to detectobstacles from a vehicle moving on a planar roadsurface is essential. In recent years, many obstacles
detection approaches have been developed. There aremainly three popular methods, based on a-priori
knowledge, based on optical flow, and based on stereo
vision. The method based on a-priori knowledge is often
used to detect specific objects or limited objects classes,
such as vehicles or pedestrians. We often call thismethod pattern recognition. Optical flow and stereo
vision methods can detect arbitrary objects which pose athreat to safe driving. But these two methods are
sensitive to vehicle motion, and when obstacles havesmall or null speed, optical flow techniques fail.
In this paper, we propose a method of estimating aroad area in general road environments. This method
uses two consecutive image frames, and warps the firstimage according to the geometrical relationshipbetween these two images. The road area is thenextracted by comparing the warped image with thesecond image.

Time t+1 image

Time t image

Gaussian mixture model
Camera motion
estimation
Background image
Warped image
Background subtraction
Road area detection
Foreground image

Output image

II. OUTLINE OF THE PROPOSED METHOD
Fig. 1. Overview of the proposed method.

The process flow of the proposed method is shownin Fig. 1.

©ISAROB 2011

767

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

In the second place, we estimate the camera motion
parameters from the correspondences of feature points
between two successive images [3]. The camera motion
parameters consist of 3D rotation and 3D translation
parameters. In this method, it is assumed that thecamera is calibrated, i.e. the internal parameters of thecamera are known. Two consecutive images are used at
any time, i.e. the image taken at time t and that taken attime t+1 are used at time t+1. In the third place, we
warp time t image using camera motion parameters[4,5,6]. Comparing the warped image with time t+1
image, we can get the road area at time t+1. Finally,
delete other things in the foreground image using thisroad area, and then we can get the obstacles on the road.

2. Fundamental matrix
The fundamental matrix F is the algebraic
representation of epipolar geometry. And F is the unique
3*3 rank 2 homogeneous matrix which satisfies

mt 1 Fmt  0
Tfor all corresponding points mt  mt 1 .

 f11Here, F   f 21
 f 31 xt 
mt   yt  1 

The values of a particular pixel over time are a timeseries of pixel values. At any time t (t=1,2,…,T), a
particular pixel (x0, y0) has Xt pixel values:{X 1 ,......, X T }  {I ( x 0 , y 0 , t ) : 1  t  T } (1)where I is the image sequence. GMM method models
each pixel by a mixture of K Gaussian distributions. The
probability to observe the current pixel value X is
K

(2)

k 1

where K is the number of distributions (currently, from
3 to 5 are used);  k , t is an estimate of the weight of
the i th Gaussian in the mixture at time t;  k ,t is themean value of the i th Gaussian in the mixture at timet;  k2,t is the covariance matrix of the i th Gaussian in
the mixture at time t; is a Gaussian probability density
function defined by

Step_2:Findingthefundamental

matrixˆ t  mˆ t 1 .
corresponding to the matches mF̂ˆTt . Matrix F
Step_3: Denormalization: Set F  Tt T1 F
is the fundamental matrix corresponding to the original

1
1 ( X t  k ,t )2 (3) ( X t ,  k , t , ) exp{
}2  k2,t
2 
2
k ,t

data mt  mt 1 .

IV. CAMERA MOTION ESTIMATION

3. Camera motion parameters
The relationship between the fundamental andessential matrices is

1. Feature point detection
In the image at time t, we detect feature points using
Harris corner detector [7], then, using Lucas-Kanade
method [8], we detect the corresponding points of theHarris feature points in the image at time t+1, and byusing RANSAC [9] we delete outliers. The camera
motion parameters are then estimated from thecorrespondence of the feature points.

©ISAROB 2011f13 
f 23 
f 33 
 xt 1 
mt 1   yt 1 
 1 

f12
f 22
f 32

In this paper, we use the 8-point algorithm [10] tocompute the fundamental matrix. The key to successwith the 8-point algorithm is proper careful
normalization of the input data before constructing theequations to solve. In the case of the 8-point algorithm,
the suggested normalization is a translation and scalingof each image so that the centroid of the reference
points is at the origin of the coordinates and the RMS
distance of the points from the origin is equal to 2 .
Algorithm:
Step_1: Normalization: Transform the image
coordinatesaccording tomˆ t  Tt mt andˆ t 1  Tt 1mt 1 , where Tt and Tt 1 aremnormalizing transformations consisting of a translationand scaling.

III. GAUSSIAN MIXTURE MODEL

P( X t )    k ,t   ( X t ,  k ,t ,  k2,t )(4)

E  K T FK

(5)
Where, K is a camera calibration matrix.
The essential matrix can be represented by motion
parameters of a camera between two images, i.e., therotation matrix R and the translation T.

E  T X R

768

(6)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Here,

 0
T X   t Z
 tY

 tZ
0
tX

tY 
 t X  is
0 

M

the

mt

corresponding skew-symmetric matrix of the translationT.
By applying the singular value decomposition to thematrix E, we have

E  UV T

mt 1

Time t+1

Time t(7)

Using the results of the singular value
decomposition (Eq.(7)), we calculate the rotation matrix
R and the translation T as follows;

R,T

Fig.2. The location of the camera at time t and t+1(8)
R  UWV T or R  UW TV T
T X  UWU T or T X  UW TU T (9) 0 1 0Here, W    1 0 0 0 0 1 xt1   r11s  yt1   r21
 1   r31

r12
r22
r32

r13
r23
r33

tX  X  (13)
tY   YZ
t Z   1 
 First, for each pixel in the image at time t, we
calculate the corresponding coordinate of 3-D point M
based on Eq. (12). Here, we assume all 3-D points M
are on the road plane, so Y is equal to the height of acamera above the ground.
Then we use X, Y and Z obtained from the above
procedure and we calculate xt1 and y t1 based on
Eq. (13). We create a new image, in which the pixel
value at (x’t+1, y’t+1) is the pixel value at (xt, yt) in theimage at time t. This new image is the warped image.

There are four possible choices of the camera
motion parameters, based on the two possible choices of
R and two possible choices of T.
The selection of the camera motion parameters isdone by using the four combinations to do the motion
compensation then check which combination’scompensation image is correct.

V. ROAD AREA ESTIMATION
2. Road area
In order to detect the road area at time t+1, we
calculate NCC (Normalized Cross-Correlation) between
the warped image and the image at time t+1. In theexperiment, we use a 7  7 window for calculating NCC.
The road area can be obtained by extracting those pixels
that have a NCC value below a specified threshold.

1. Motion compensation
The location of the camera at time t and t+1is shownin Fig. 2.Suppose the world coordinate system coincides
with the camera coordinate system at time t. Then the
projection equation at time t and time t+1 are given bysmt  K I 0M
(10)
(11)
smt 1  K R T M
where mt and mt 1 are the 2-D points on the image

VI. EXPERIMENTAL RESULTS
Experiments have been done on a video under the
existence of a person passing in front of a camera. The
video is taken by a camera mounted at the front seat of acar, and includes image sequences of frontal roadenvironments while the car is driving in the town. Fig. 3
shows the result of the detection of a pedestrian crossing
the road. (a) are the input images; (b) are the foregroundimages obtained from GMM; (c) are road area detection
results; and (d) are the results of obstacles detection.

plane at time t and time t+1; M is the 3-D point in theworld coordinate system; K is the camera calibration
matrix; R and T are the camera motion parameters; I is
the unit matrix; s is a scalar.
Eq.(10) and Eq.(11) are easily written in terms of the
known coordinates mt and mt 1 as follows; xt  1 0 0 0  X s  y t   0 1 0 0   YZ 1  0 0 1 0  1  ©ISAROB 2011

(12)

769

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011[3] R. Hartley and A. Zisserman(2004), Multiple View
Geometry in Computer vision. Cambridge University
Press, 2nd edition.
[4] K.Yamaguchi, A. Watanabe and T. Naito(2008),
Road region estimation using a sequence of monocular
images, Proc. the 20th International Conference on
Pattern Recognition.
[5] K.Yamaguchi, T. Kato and Y. Ninomiya(2006),
Vehicle ego-motion estimation and moving object
detection using a monocular camera. Proc. the 18th
International Conference on Pattern Recognition.
[6] K.Yamaguchi, T. Kato and Y. Ninomiya(2005),
Obstacle detection in road scene using monocularcamera. Proc. Information Processing Society of Japan,
69-76. (in Japanese)[7] C. Harris and M. Stephens(1988), A combined
corner and edge detector. Proc. Alvey Vision
Conference, 147-151.
[8] B. Lucas and T. Kanade(1981), An iterative image
registration technique with an application to stereo
vision. Proc. International Joint Conference on Artificial
Intelligence, 674-679.
[9] M. Fischler and R. Bolles(1981), Random sample
consensus: A paradigm for model fitting withapplications to image analysis and automatedcartography. Proc. Communications of the ACM, 24(6):381-395.
[10] R. Hartley(1977), In defence of the eight-point
algorithm. Proc. IEEE Transactions on Pattern Analysis
and Machine Intelligence, 19(6):580-493.

VII. DISCUSSION AND CONCLUSIONSThis paper proposes a method of detecting obstacles
on a video taken by a vehicle-mounted monocularcamera. As shown in the experimental results,
performance of the proposed method is reasonable. Thismethod has some advantages over other existing
methods. This method can detect arbitrary objects which
may pose a threat to safe driving on the road, not only aspecific object or limited object classes, but it can also
detect both static and moving objects. Moreover it canbe employed in both static and moving camera cases.

ACKNOWLEGDEMENT
This work was partly supported by a grant ofKnowledge Cluster Initiative implemented by MEXT.

REFERENCES[1] C. Stauffer and W.E.L. Grimson(1995), Adaptive
back-ground mixture models for real-time tracking.
Proc. Conference on Computer Vision and Pattern
Recognition, Vol.2, 246-252.
[2] P. KaewTraKulPong and R. Bowden(2001), An
improved adaptive background mixture model for realtime tracking with shadow detection. Proc. 2nd
European Workshop on Advanced Video Based
Surveillance Systems, Computer Vision and Distributed
Processing.

Fig.3.Experimental result: (a) Original video image frames; (b) foreground images; (c) road areas; (d) obstacle.

©ISAROB 2011

770

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Hardware implementation of a census-based stereo matching using FPGA
Jiho Chang, Seung Min Choi, Eul-Gyoon Lim, Jae-il Cho
Electronics and Telecommunication Research Institute, Korea(Tel : 82-42-860-6131; Fax : 82-42-860-6796){changjh, ccsmm, eglim, jicho}@etri.re.kr

Abstract: The real-time stereo vision is becoming important increasingly in the field of Robotics. It is very difficult toimplement the real-time stereo vision system, because it requires very powerful processors. For solving this problem,
we present the real-time hardware architecture of the census-based stereo matching IP (Intellectual Property)
comprising support-weight and trellis dynamic programming structure. We use census-based cost that is robust fordifference in brightness. Using Cost aggregation with support-weight window, our stereo vision system has morerobustness for real environment. The stereo matching processor is implemented on a FPGA with the pre-processing partfor the rectification and post-processing for reduction of noise. This stereo vision IP is implemented with HDL
language and performed up to 30 FPS.
Keywords: Stereo matching, Dynamic programming, Census transformdifficult to materialize using the PC or GPU. If there
were cases of materialization using high efficiency
embedded processors, but, the real time performance(30fps) cannot be guaranteed as well, since it is the useof the resources of main processor, other applications inembedded system cannot be performed.
In our previous work, census-based stereo matching
IP which is based on trellis dynamic programming[2].
For the case of previous work, since it uses census
transform, is robust for difference in brightness. But
there are some steak noises, because it is based on
dynamic programming and there are some calibration
errors.
In this paper, our stereo matching algorithm has
support-weighted cost aggregation and census-basedstereo matching IP. For the case of Trellis DP algorithm,
since it is strong for the occlusion by generating center
referenced disparity, it is very effective under thepractical circumference[4, 5]. And cost function of theStereo matching IP use census transform, so that isrobust for difference in brightness. Also we aggregate
the matching cost by adaptive Support weight that isused for decreasing steak noise.
The stereo matching IP has three parts. First, Preprocessing part is composed with the Rectification,
Calibration and controlling the brightness which affectto the result a lot in stereo matching. A second part ismain-processing which is consisted of census transform,
support weight and trellis DP algorithm. At last, postprocessing which reduces the noise that is being
generated in Dynamic programming.

I. INTRODUCTION
The stereo vision is becoming important
increasingly in the field of virtual interface, gesturerecognition, robot navigation, depth measurements andenvironment reconstruction as well as in many otheraspects of security, robotics, and entertainment[1, 7]. In
order to adapt 3D depth on these various applications,many different algorithms have been developed andmaterialized in various systems.
Especially in robots, the real time is a very
important issue since the circumstances are changing
continuously and many of studies have been progressedin various ways to indicate the stereo vision in real timebase[7]. However, due to the complexity of matchingalgorithm, it is very difficult to materialize in real timebase and this may require very powerful processors.
Normally, many real time systems use the local
method. Although it has low complexity, there are somelocal problems where it fails to match due to occlusion,
uniform texture, ambiguity of low texture etc. Also, inthe popular local matching method, the block matchingskill blurs disparity data in the object boundary[3, 8].
Through these results, though it is practically usable insimple applications, they bring many restrictions on theapplications which require more accurate data. Recently,
a few real-time global matching methods have been
implemented through GPU in the graphic card or MMX
of CPU[6, 9]. But, in the mobile robot which wascomposed with the small embedded system, since thesize and power usage is the important issue, it is©ISAROB 2011771

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

In this paper, we described for the total block
diagram and the performance and conclusion throughthe result gained from this system.

Figure 1. shows pre-processing block diagram.
There are n-lines buffer memory, rectification parameterregister and Calculation unit. This line buffer size
defends on user requirement and mechanical structureof stereo camera. Calculation unit computes using a
rectification formula descript on Luping An et al.’s
paper[12].

II. Stereo matching IP
In this paragraph, we described the stereo matching
IP which has three part - pre-processing, mainprocessing, post-processing.

2. Main processing partMain-processing part calculates the disparity fromthe rectified left and right input image which wasprocessed at pre-processing module and makes disparity
information as the intensity data. Following is thealgorithm for practical performance at stereo matchingmodule.
One of the deciding methods of the standard imageof the 3 dimensional images in stereo vision is to selectone of the left or right images and the other case is todefine a virtual one(center reference disparity) in
between those two. When express the depth map ofimage, there is a balance of these expressions if use thecenter referenced disparity while there are much ofimage loss for discontinuity of the other side if show the
information using one of the images only. Also, it has
many good points in interpreting the discontinuous
space and finding the solution.

1. Pre-processing partPre-processing part can improve the performance ofstereo matching by minimizing the distortion of camera
using the functions of individual brightness control andrectification of left and right images. The constraint of
one point in one side image in stereo image geometry
corresponds on epipolar line in other image. Therefore,
due to the constraint of this epipolar line, the more
epipolar line of the two left and right images matches
each other accurately, the more the accuracy is
increased. Since the stereo vision processing technology
in this paper composed the algorithm subject to this
epipolar line constraint, input after matching the left andright image accurately before it is input on the stereo
matching part affects on the performance a lot.
In order to match this epipolar line, extract therectification parameter in various ways. Thus save thepre computed parameters in the pre-processing register
files and rectify the images by paralyzing the epipolarline using the above parameters. Also, in order tocontrol the difference of physical characteristics of left
and right camera or brightness of left and right images
occurred by the strong light input in one camera only,
the brightness control function is applied. Abovefunctions are set to input the revised information
continuously matching to the characteristic andcircumstances through the host interface.
Pre-processing Module
l_cis_vsync
l_cis_href
l_cis_d[7:0]

RGB toIntensity

Line Buffer
Address generation

Brightness
ControlUnit

r_cis_vsync
r_cis_href
r_cis_d[7:0]Fig. 2. Center referenced coordinate systems.

rectification_imcl

RGB toIntensity

Rectification
Calculation

Figure 2. shows how the center referenced disparity
which became the base of the used algorithm in stereo
matching module can be formed. The coordinates ofcenter referenced disparity includes the territories thatcan be generated by both of the left and right images
and if it is composed with each of image N columns, it
is verified that the center disparity image has 2N + 1
indexes.
In Figure 2., white/black dots represent stereo
matching element that is consisted of cost computation

l_pre_vsync
l_pre_href
l_pre_d[7:0]r_pre_d[7:0]Line buffer

Sync generation
Register File

Host interface fromembedded system
For rectification andbrightness control parameterFig.1. Block diagram of pre-processing part.

©ISAROB 2011

772

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

and cost aggregation. Also, the trellis structure isconnected to those dots, has optimal path minimizedenergy. And we finds the optimized route in the space
using Dynamic Programming.
The block dots combines matching cost computationand cost aggregation. We use Census transform as
matching cost computation and fixed weight window
for cost aggregation. The Census Transform(CT) is astereo matching cost algorithm with high robustness toillumination variations and exposure difference. The CT
transforms the intensity data into feature data, before thematching costs of each disparity level are calculated.
For a higher accuracy of the algorithm, the calculationof the matching costs can be followed by a further
aggregation

this method is efficient for implementing hardware IP.
Even if a larger size of the window

Fig. 6.

Figure 6. are sample results by using Normal
Dynamic programming(left)
and SW Census
DP(right). This results represents that SW census costfunction make robust system in real world.

SM-processing Module
l_pre_vsync
l_pre_href
l_pre_d[7:0]r_pre_d[7:0]

CensusTransform
15x15

Buffer for
window cost
aggregation(L)Matching
cost
computation

CensusTransform
15x15

Buffer for
window cost
aggregation(R)BufferDepth estimation
Block_0
Block_1

Block_62
Block_63

Matching
sync
With depth
data

sm_vsync
sm_href
sm_d[7:0]

3. Post-processing partPost-processing part is designed to remove the noisefrom the disparity map which was acquired from Stereo
matching part and distinguish the objects and output
each of them. Especially, in the stereo matching using
Dynamic programming technology, the steak noise inhorizontal direction will be generated. For this kind ofnoise that is generated to outside of object cases, they
can be treated at backward application step. However,for the case it is generated to inside of the object, it
makes the object to be shown as disconnected. In orderto prevent these phenomena, it tried to reduce with 3
steps of filtering on the result from the Main-processing
part.

Depth
Reallocation

Register File

Host interface
For SM parameter setting

Fig. 3. Block diagram of main-processing method A.
Figure 3. is one method of stereo matchingprocessing module which calculates disparity. Mainprocessing part is consist of census transform module tocreate raw costs, cost aggregating for support weightwindow computation and depth estimation module withtrellis DP. But, this structure of block diagram use lotsof Flip-Flops and memory for saving census data.

l_pre_vsync
l_pre_href
l_pre_d[7:0]r_pre_d[7:0]l_pre_vsync
l_pre_href
l_pre_d[7:0]r_pre_d[7:0]

SW Census Module
CensusTransform
CensusTransform

TDP ModulezRegister_cost

Census
Cost
FunctionzProc_element_x2
Block_0

zFB_buffer_D

SupportWeightsm_vsync
sm_href
sm_d[7:0]

Block_1
Cost
Aggregation
ModuleSupportWeightResult images from main-processing part.

320x240
YY@6.75MHzzImage_b
uffer

Block_62

zPost_Proc_
block

Block_63

Fig. 7.

Results of post-processing part.

Fig. 4. Block diagram of main-processing method B.

III. Testing FPGA
Figure 4. is another method for SW census-based
trellis dynamic programming. This method is used morememory than method A for saving Census raw cost. But
this structure uses less Flip-Flops than method A, So©ISAROB 2011

Following picture shows the Test board set forverification. This test board set operates itself without
connection between any computer. Stereo vision system
is implemented with HDL using Xilinx virtex-5
XC5VLX330 FPGA. The performance of system is 30

773

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

system and this makes stereo matching result as
images(left, right, disparity map and others), they can beused in embedded system itself.

FPS(dependent on a CMOS camera) when size ofimages is 320x240(when census size is 15x15, support
weight window size is 5x5 and maximum disparity
range is 128), which is enough performance forautonomous robots and the other applications.

Acknoledgement
This work was supported by the R&D program of
the Korea Ministry of Knowledge and Economy (MKE)and the Korea Evaluation Institute of Industrial
Technology (KEIT). [Project KI001813, Developmentof HRI Solutions and Core Chipsets for u-Robot].

REFERENCES[1] Scharstein, Daniel and Richard Szeliski. A taxonomy
and evaluation of dense two-frame stereo
correspondence algorithms. International Journal ofComputer Vision. 2002[2] Jiho Chang, Sung-Min Choi, Eul-Gyoon Lim, Jaeil
Cho, Dae-Hwan Hwang. Hardware implementation of acensus-based stereo vision for real environment. The 7thInternational Conference on Ubiquitous Robots andAmbient Intelligence. 2010[3] Murray, Don and James J. Little, Using real-timestereo vision for mobile robot navigation. Autonomous
Robots. 2000
[4] Jeong, H. and S. Park. Generalized Trellis Stereo
Matching with Systolic Array. In Parallel andDistributed Processing and Applications, BerlinHeidelberg: Springer Verlag. 2004.
[5] Park, Sungchan and Hong Jeong. Real-time stereo
vision FPGA chip with low error rate. Proceedings ofInternational Conference on Multimedia and UbiquitousEngineering. 2007.
[6] Wang, Liang, Miao Liao, Minglun Gong, Ruigang
Yang, and David Nister. High-quality real-time stereo
using adaptive cost aggregation and dynamic
programming. Proceedings of Third International
Symposium on 3D Data Processing, Visualization, andTransmission . 2006.
[7] Nalpantidis Lazaros, Georgios Christou Sirakoulis,
and Antonios Gasteratos. REVIEW OF STEREOVISION ALGORITHMS: FROM SOFTWARE TO
HARDWARE.
International
Journalof
Optomechatronics. 2008
[8] Chris Murphy, Daniel Lindquist, Ann Marie
Rynning, Thomas Cecil, Sarah Leavitt, Mark L. Chang.
Low-cost stereo vision on an FPGA, International
Symposiumon
Field-Programmable
CustomComputing Machines. 2007
[9] Gong, Minglun and Yee-Hong Yang. Near real-time
reliable stereo matching using programmable graphics
hardware. Proceedings of IEEE Computer Society
Conference on Computer Vision and Pattern
Recognition. 2005

Figure 8: Example of the stereo vision system which
is presented in this paper
Slice Logic Utilization

Used

Available

Usage

Number of Slice Registers

52,804

207,360

25%

Number of Slice LUTs

171,836

207,360

82%

Number used as logic

152,121

207,360

73%

Number used as Memory

12,50754,72022%

51,84095%

Number of route-thrus

15,953

Slice Logic Distribution
Number of occupied Slices

49,571

Number of LUT Flip Flop pairs used

178,718

Number with an unused Flip Flop

125,914

178,71870%

Number with an unused LUT

6,882

178,718

3%

Number of fully used LUT-FF pairs

45,922

178,718

25%

Number of unique control sets

1,222
1,510

207,360

1%

281

288

97%

Number of slice register sites lost tocontrol set restrictions
Specific Feature Utilization
Number of Block RAM/FIFO

Table 1. Device utilization summary on XC5VLS330
Table 1. shows the results reported by Xilinx ISE
tool after synthesis and place-and-route.

IV. CONCLUSION
We developed the adaptive support-weighted
census-based stereo matching IP using trellis dynamic
programming structure and this includes the preprocessing part for the rectification and post-processing
for reduction of noise. Also, in order to use this Stereo
vision IP on various applications, we developed the test

©ISAROB 2011774

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Hand Detection using AdaBoost
Jae-chan Jeong1,2, Seung-min-Choi1, Ho-chul Shin1,2, Jae-il Cho1
Electronics and Telecommunication Research Institute1
University of Science and Technology2(Tel : 82-42-860-6116; Fax : 82-42-860-6796)(channij@ust.ac.kr)
Abstract: In this paper, we present hand database automatic generator and hand detection logic using AdaBoost. To
generate hand database, we acquires videos using webcam on restrict conditions: posture, background, number ofhuman and so on. After the video acquired, face is detected frame by frame on the videos. On the face region, color offace is analyzed to generate skin color model. Based on skin color model, hand candidate regions are detected. Only
one region may exist except face region because of restricted condition. The region of hand is clipped and is saved. By
this method, hand database is generated automatically. Once hand database is generated, none-hand database is also
generated by randomly clipping the region. Using hand and none-hand database, Fist Look Up Table (FLUT) is
generated by using AdaBoost training algorithm. Detecting hand is similar to the method of hand database generation.
Keywords: Hand Detection, AdaBoost, HRI

ì1 , if p j f j ( x) < p jq jh j ( x) = í
î0 , otherwise

I. INTRODUCTION
Interacting human and robot is one of the active
research topic. Keyboard, mouse, and joystickclassically are used for Human Robot Interaction(HRI)[1]. In these methods, users need to carry externalinterface devices to control robots. So nowadays, many
researchers try to do HRI using vision or voice[2]. In
case of vision, to recognize human gestures is mostimportant part. And most of gestures are made by handand arm. Therefore, detecting hand is necessary for HRI.
To detecting hand, there are two main methods: marker
based and marker-less. Without marker, skin detection
or pattern analysis are used. However, detecting hand isdifficult and its detection ratio low because of various
shapes of hand and illumination changes. We detect
only fist to avoid difficulty of hand detection. In thispaper, we propose fist detection system using AdaBoost.
First of all, we made fist database and trained it by
AdaBoost. FLUT is made by the result of training.
Using FLUT, we should rapidly detect hand with highdetection ratio.

In Table 1 shows the AdaBoost training procedure andisright side value of the final strong classifier 1
å a2
the threshold for detecting objects.
Table1. The AdaBoost Algorithm
T

t =1

t

IV. Hand DataBase

II. AdaBoost
We generate a hand database automatically. Before
generating the hand database, we record a video clip
with only one person who is forwarding a hand. And
then, we execute our automatic hand database algorithm
on this video clip. Fig1 shows a algorithm of automatic
hand database generation. We detect a face on the grayimage by MCT based face detector[4]. Skin color model
based on YCbCr is generated by analyzing pixel of theface location. Using the skin color model, we detect
hand candidate regions except face region. When we
make video, we restrict the posture of the human and

AdaBoost[3] is an aggressive mechanism forselecting a small set of good classification functions
which nevertheless have significant variety. Drawing ananalogy between weak classifiers and features,
AdaBoost is an effective procedure for searching out asmall number of good “features”. A weak classifier( h j (x) ) consists of a feature ( f j (x) ), a threshold ( q j ),and parity ( p j ) indicating the directions of theinequality sign:©ISAROB 2011775

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

background. Only one hand candidate region exists. To
find boundary of the hand candidate region, we use
corner detector. We extract hand image by boundary of
the hand. The extracted images are resized to 20x20.
The hands are stored to video clip. None-hand DB isgenerated randomly.
Fig.3 Hand and Non-Hand Data Base

Fig.1 Automatic Hand Database Generation
Fig.4 Hand detection Result

V. Hand Detection
VII. CONCLUSION
We presented hand detection by AdaBoost andgenerated hand database automatically. After that handdb was trained by AdaBoost with six haar features. The
train result is FLUT whis is used to detect hand. To
increase detection ratio, regions of hand candidates are
generated by face detector and skin color model. In the
future, we’ll collect more hand and non-hand images,
and apply the hand detection result to human computerinteraction or human robot interaction.

Fig.2 Hand Detection Method

Acknowledgement
Fig.2 describes hand detection method. Before
detecting hand, we find face location to make skin color
model. The skin color model is used to generate handcandidate regions. Among the hand candidate regions,face location is eliminated and hand is detected by
AdaBoost. To detect various size of hand, handcandidate region is resized repeatedly until the size ofimage become 40x40 which is image resolution of handdata base.

This work was supported by the R&D program of the
Korea Ministry of Knowledge and Economy (MKE)and the Korea Evaluation Institute of Industrial
Technology (KEIT). [Project KI001813, Developmentof HRI Solutions and Core Chipsets for u-Robot].

REFERENCES[1] Fong, T., Nourbakhsh, I., Dautenhahn, K., “A surveyof socially interactive robots" Robotics andAutonomous Systems, 42 (3-4), pp. 143-166, 2003
[2] Ude, A., Shibata, T., Atkeson, C.G., “Real-timevisual system for interaction with a humanoid robot”
Robotics and Autonomous Systems, 37 (2-3), pp. 115125,2001[3] Paul Viola and Michael J. Jones, “Robust real-time
face detection” In International Journal of Computer
Vision, pp. 137-154, 2004.
[4] Fröba, B., Ernst, A., “Face detection with themodified census transform”, 6th IEEE International
Conference on Automatic Face and Gesture Recognition,
pp. 91-96., 2004

VI. Experimental Result
To detect hand, we made hand train DB which is
capture with 2 people by Microsoft VX-500. The
training DB consists of 2427 hand image and 3450 nonhand. Fig.3 shows the train database. The size of image
is 20x20. After making database, we trained hand DB
by AdaBoost. Consequently, we generated FLUT. Fig.4
describes the result of hand detection.

©ISAROB 2011

776

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Improvement of Early Recognition of Gesture Patterns
based on Self-Organizing Map
Atsushi Shimada, Manabu Kawashima and Rin-ichiro Taniguchi
Department of Advanced Information Technology, Kyushu University
744 Motooka, Nishi-ku Fukuoka 819-0395 Japan(Tel : +81-92-802-3580; Fax : +81-92-802-3579)(Email: {atsushi, kawashima, rin}@limu.ait.kyushu-u.ac.jp)

Abstract: We propose an approach to achieve early recognition of gesture patterns. Early recognition is a method torecognize sequential patterns at their beginning parts. Therefore, in the case of gesture recognition, we can get arecognition result of human gestures before the gestures have finished. The most difficult problem of early recognitionis that when the system determines the recognition result. Most traditional approaches suffer from this problem since
the gestures comprehend ambiguity. Especially at the beginning part of them, it is very difficult to determinate therecognition result since enough input data has not been observed yet. Therefore, we have improved traditional approachby using Self-Organizing Map.
Keywords: Gesture Recognition, Early Recognition, Self-Organizing Map

postures, which are elements of all gestures. The
advantages of using SOM are 1) to reduce
dimensionality of gesture patterns, 2) to reduce someredundant postures, 3) to represent a gesture pattern by
a combination (set) of smaller number of neurons; wecall it “Sparse Code”. In the test phase, we introduce theHausdorff distance to measure the similarity to measure
the similarity between sparse code of unknown input
pattern and sparse code of training patterns. The
Hausdorff distance is more effective criterion forjudgment than Euclidean distance or so on, since it canmeasure the distance between not two elements but setsof pattern.

I. INTRODUCTION
A man-machine seamless interaction is an important
tool for various interactive systems such as virtual
reality systems, video game consoles, human-robot
communication, and so on[5,6]. To realize such aninteraction, the system has to estimate human gesturesin real-time. Generally, a gesture recognition result is
acquired after the gesture has finished. Therefore, if along gesture is observed, we have to wait for the
response until the recognition result is determined. Thisis a problem to realize a “real-time” man-machineinteraction.
Recent years, a new approach called “earlyrecognition” has been proposed for gesturerecognition[4,8,1]. The early recognition means that asystem outputs a recognition result before a gesture has
finished. It is a very useful technique to realize a realtime interaction. The most difficult problem of earlyrecognition is that when the system determines therecognition result. In other words, the system has toensure the recognition result before the observing
gesture has finished. Most traditional approaches sufferfrom this problem since the gestures comprehendambiguity. Especially at the beginning part of them, it isvery difficult to determinate the recognition result sinceenough input data has not been observed yet.
In this paper, we propose a new strategy to achieve
early recognition of gesture patterns. In the training
phase, Self-Organizing Map (SOM)[2,3] is used to learn©ISAROB 2011

II. DEFINITION OF
EARLY RECOGNITION
1. Typical Gesture Recognition
Let 𝑪𝑖 = *𝑐1𝑖 , … , 𝑐𝑛𝑖 + be a training gesture patternwhich belongs to gesture class 𝑖 ∈ 𝑳. The 𝑳 is a set ofclass labels. A gesture can be represented in a sequential
n-long posture patterns. Therefore, 𝑐𝑛𝑖 means the n-th
posture of the gesture. When an unknown gesture
𝑿 = *𝑥1 , … , 𝑥𝑙 + is observed, the typical gesturerecognition problem is to find the most similar gesturefrom training patterns by
𝑝 = argmin*𝑓(𝑿, 𝑪𝑖 )+𝑖where 𝑝 is the class label and 𝑓() is a distancefunction which evaluates the similarity between gesturepattern 𝑿 and 𝑪𝑖 .

777

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig. 1 Processing flow of training/recognition
to the map again. And then, we can get a “Sparse Code”
which represents a gesture pattern on the SOM.
In the test phase, the system observes person’s
gesture. Then, a parse code corresponding to theobserving gesture is generated/updated immediately
whenever a new observation is acquired. Finally, thesparse code is examined whether or not the system
outputs the recognition result. Actually, the examinationis achieved by measuring the distance between sparse
codes based on Hausdorff distance.

2. Early Recognition of Gesture Patterns
The key issue of early recognition is to output arecognition result before acquiring complete input
pattern. In the case of gesture recognition, it
corresponds to the following problem. When a part ofgesture pattern (unfinished gesture) 𝑿′ = *𝑥1 , … , 𝑥𝑘 +,(k < l) is observed, the recognition result is determinedby𝑝 = argmin*𝑓(𝑿′ , 𝑪𝑖 ) < 𝑇𝐻+𝑖

1. Sparse Coding
When a posture 𝑥𝑘 is input to the SOM, one neuron
will be selected as winner. When a set of postures which
consist of a gesture is sequentially input to the SOM,
some neurons will be activated. We regard such anactivation pattern as “Sparse Code”, which represents
an input gesture. Here, we define the notation of asparse code. In the training phase, all training gestures
are represented by using sparse code. For simplicity, we
use the same notation 𝑪𝑖 , which was used for
representation of training samples in section II, as thesparse code of them. Meanwhile, in the test phase, asparse code of observing gesture is represented by 𝑿′ .
Note that the sparse code described here has not an
ability to distinguish the gesture patterns whose
elements are the same but the sequences are different.
However, we can easily improve introduce temporal
information into the system by our previous study[7].

where 𝑇𝐻 is a threshold of distance which adjusts thetiming of recognition result. If the threshold is not
introduced, a recognition result will be output without
concrete proof. Therefore, we set a threshold to ensure
reliability for the recognition result.

III. STRATEGY OF
EARLY RECOGNITION
1. System Overview
First of all, we show the system overview in Fig. 1.
The process can be divided into two phases; training
phase and test phase. In the training phase, SelfOrganizing Map (SOM) is used to learn postures, which
are elements of all gestures. The advantages of using
SOM are 1) to reduce dimensionality of gesture patterns,
2) to reduce some redundant postures, 3) to represent a
gesture pattern by combination of smaller number ofneurons and so on. Due to space limitation, we skip thedetailed explanation about SOM and how to learn the
postures (refer to the literature [7]). After the training of
all postures, element postures of each gesture are input©ISAROB 2011

2. Similarity Evaluation with Hausdorff distance
The number of elements in sparse code is different
from each other since the number of activated neurons

778

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig. 2 10 kinds of gesture patterns

Fig. 3 Early Recognition Accuracy vs. Complete Ratilof Observing Gesturedepends on the gesture length and the gesture pattern.
Therefore, we introduce the Hausdorff distance tomeasure the similarity between two sets of sparse code.
Let 𝑿 and 𝒀 be two non-empty subsets of a
metric space. The Hausdorff distance 𝑓(𝑿, 𝒀), whichcorresponds to the distance function explained in
section II, is defined by𝑓(𝑿, 𝒀) = max*sup inf 𝑑(𝑥, 𝑦), inf sup 𝑑(𝑥, 𝑦)+𝑖𝑥∈𝑿 𝑦∈𝒀𝑦∈𝒀

other 20 patterns for test. We conducted the experimentthrough cross-validation among examinees.
2. Recognition Result
Fig. 3 shows the result of early recognition The
horizontal axis denotes the complete ratio of observing
gesture pattern, and the vertical axis denotes therecognition accuracy. The bold curve indicates theaverage ratio of accuracy. For example, the recognition
ratio exceeded 90% when more than 50% long gestures
had been observed on average.
Fig. shows the recognition accuracy of eachgesture class. And the detailed analysis results areshown in Table 1. Totally, we got good results for wholegesture classes. However, some cells in Table 1 havelower values. We investigated the matter why such
results came out. There are some similar gesturepatterns between gesture classes. For example, in thecase of gesture class 5 and 7, the person moves his/herright hand forward from in front of his/her body. The
difference between these gestures is the trajectory of thehand. However, the trajectory depended on examinee’s
habit. As the result, some trajectory was very similar𝑥∈𝑿

where 𝑑(𝑥, 𝑦) is the distance function. In our research,
we use L2-distance between the coordinates of activatedneurons.

IV. EXPERIMENTAL RESULT
1. Preparation
We demonstrate proposed early recognition of
gesture patterns using motion data prepared byourselves. Each gesture consists of a sequence of
postures, and each posture is represented by 5 measured
markers. Each marker is composed of data of (x, y, z)axis. We prepared 10 kinds of gesture patterns (𝑳 = 10,
see Fig. 2) from 7 examinees. Each person did eachgesture 40 times. We used 20 patterns for training and©ISAROB 2011

779

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 1 Recognition Accuracy of Each Gesture Class and Each Person
Person1
Person2
Person3
Person4
Person5
Person6
Person7

g0
0.84
0.85
0.50
0.84
1.00
0.86
0.74

g1
0.83
0.70
0.91
0.89
0.82
1.00
0.79

g2
1.00
1.00
0.85
0.72
1.00
1.00
0.79

g3
0.82
0.83
0.77
0.85
0.90
0.61
0.72

g4
0.53
1.00
0.99
0.63
0.67
0.75
0.61

g5
0.75
0.95
0.71
0.89
0.83
0.89
0.64

g6
0.85
0.75
0.63
0.83
0.75
0.74
1.00

g7
0.70
0.54
0.89
0.53
1.00
0.73
0.74

g8
0.81
1.00
0.65
0.68
0.62
0.53
0.55

g9
1.00
0.70
0.95
0.74
0.93
0.61
0.83

Average

0.81

0.85

0.91

0.79

0.74

0.81

0.79

0.73

0.69

0.82

Fig. 4 Recognition Accuracy of Each Gesture Class[2] T. Kohonen. “Self-Organization and Associative
Memory”. Springer-Verlag, 1989.
[3] T. Kohonen. “Self-Organizing Maps”. Springer
Series in Information Science, 1995.
[4] A. Mori, S. Uchida, R. Kurazume, R. Taniguchi, T.
Hasegawa, and H. Sakoe. Early recognition andprediction of gestures. In Proc. of International
Conference on Pattern Recognition, volume 3, pages
560–563, 2006.
[5] H. S. Park, D. J. Jung, and H. J. Kim. Vision-based
game interface using human gesture. In Advances in
Image and Video Technology, volume 4319, pages
662–671, 2006.
[6] J. Park and J. Yi. Gesture recognition based
interactive boxing game. In International Journal of
Information Technology, volume 12, pages 36–44, 2006.
[7] A. Shimada and R. Taniguchi. Gesture recognition
using sparse code of hierarchical som. In Proc. ofInternational Conference on Pattern Recognition, 2008.
[8] S. Uchida and K. Amamoto. Early recognition ofsequential patterns by classifier combination. In Proc. ofInternational Conference on Pattern Recognition, 2008.

with the trajectory of the other gesture class. This is oneof the factors which brought not good results.

V. CONCLUSION
We have proposed a new framework of earlyrecognition of human gestures. We have used SelfOrganizing Map (SOM) to learn human gestures. The
SOM outputs sparse codes for each gesture. We
estimated a human gesture based on Bayesianestimation using the sparse code. We got positive resultsof early recognition in the experiment. We are now
researching to tackle the problem of common postures
which are included in some gestures.

REFERENCES[1] M. Kawashima, A. Shimada, and R. Taniguchi.
Early recognition of gesture patterns using sparse codeof self-organizing map. In 7th International Workshop
On Self-Organizing Maps, pages 116–123, 06 2009.

©ISAROB 2011780

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Detection of a Bicycle and Its Driving Directions
Using HOG Feature
Heewook Jung, Joo Kooi Tan, Hyoungseop Kim, Seiji Ishikawa
Department of Mechanical & Control Engineering, Kyushu Institute of Technology, Kitakyushu, Japan(Tel: 81-93-884-3183; Fax: 81-93-884-3183){jung, etheltan, ishikawa}@ss10.cntl.kyutech.ac.jp, kim@cntl.kyutech.ac.jp

Abstract: Studies on car vision have currently been practiced around recognizing a human enthusiastically. The
Histograms of Oriented Gradients (HOG) feature has been proposed as useful feature for recognizing a human standingin various kinds of background. On the other hand, although a bicycle is important transportation vehicle in urban
environment, its automatic recognition or detection is not an easy task for a computer vision system, because bicycle’s
appearance can change dramatically according to viewpoints and a person riding on the bicycle is a non-rigid object.
Thus, automatic bicycle detection is an important research subject in an intelligent perception system using car vision.
In this paper, we propose a method of detecting a bicycle and its driving direction using the HOG feature and
RealAdaboost algorithm. Experimental results show satisfactory performance of the proposed method.
Keywords:

Bicycle detection, car vision, HOG feature, RealAdaboost.

I. INTRODUCTION

is increasing. Detection of a bicycle by a computer visionsystem is, however, not an easy task, because bicycle’s
appearance can change dramatically among viewpoints
and a person riding on a bicycle is a non-rigid object.
In this paper, we propose a method of detecting abicycle and its driving direction using improved HOG
feature and RealAdaboost algorithm.

An automated perception system with a car iscurrently very helpful in preventing traffic accidents andcontributes to decreasing traffic accidents. But it isdifficult to prevent traffic accidents perfectly. Factorsthat endanger traffic safety still remain around a movingcar. If the detection of the risk of traffic accidents (ahuman, a bicycle, a car, a bicyclist, etc.) is employed in
automated perception system, the number of traffic
accidents will further decrease.
The Histograms of Oriented Gradients (HOG) feature
has been proposed by Dalal and Triggs [1] as usefulfeature for recognizing a human standing in various
kinds of background. The HOG feature proposed bythem seems an effective feature for representing andrecognizing a human image. But, in their paper, it
includes an unnecessary part of the image such as the
background. It is not conceivable that the backgroundfeature contributes to high recognition rate of a humanimage. Zhu et al. [2] employ the HOG feature based on
variable block size, but they also include the backgroundin the feature. In this paper, we propose an object
detection method employing an improved HOG feature.
The research on car vision is currently practiced
around recognizing a human and not a bicycle which isimportant transportation in urban environment. Although
the number of whole traffic accidents is decreasing, therate of bicycles’ accidents to the whole traffic accidents©ISAROB 2011

II. LEARNING ALGORITHM
A learning algorithm is used to HOG feature and
RealAdaboost. The proposed method can detect bicycle’s
driving direction, because we make a bicycle modelusing HOG feature. The algorithm of learning is given in
Fig 1.

Input image

Extract HOG feature from images

Create a bicycle model and other models

Learning using RealAdabootEnd
Fig. 1 Overview of the learning algorithm.

781

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

1. HOG feature
The HOG feature [1] is a method that extracts featureof outlines in images. Because gradients of adjacentpixels receive histogramization and normalization ateach area, it is robust to illumination change and togeometric change.
The HOG feature is extracted in the following way.
Step1: Compute the magnitude m( x, y ) and orientationθ ( x, y ) of images using equation (1). Given coordinatesof images I ( x, y ) ,m ( x, y ) =2. A bicycle model
Many researchers calculate the HOG feature from anentire image irrespective of the foreground or the
background. We have proposed an improved HOG
feature calculation by the employment of a bicyclist
mask and a bicycle mask. This employment shows better
performance in the bicycle detection/recognition. This
improved HOG feature is employed in the proposed
method of this paper.
The main idea of the proposed method is to calculate
the HOG feature on a bicyclist and a bicycle mask
instead of calculating it on the whole image. A bicyclist
and a bicycle mask are created in the following way.
Step1: The intensity of gradient is calculated of all theimages in an image database.
Step2: The average values of the intensity of the gradientare calculated at every pixel on the images and anormalized average gradient intensity image is produced.
Step3: An edge image is produced from the average
gradient intensity image.
Step4: A silhouette image is created from the average
gradient intensity image and a skeleton image is madefrom the silhouette image.
Step5: Finally a bicyclist and a bicycle mask are yieldedby performing logical OR operation between the edge
image obtained at step 3 and the skeleton image obtained
at step 4.

f x ( x, y ) 2 + f y ( x, y ) 2

θ ( x, y ) = tan −1

f x ( x, y )f y ( x, y )

(1)

Here,

f x ( x, y ) = I ( x + 1, y ) − I ( x − 1, y )f y ( x, y ) = I ( x, y + 1) − I ( x, y − 1)
The result of the oriented gradients image is shown in
Fig. 2.
Step2: Derive the orientation histogram from theorientations and magnitudes.
Oriented gradients image has a large amount of
information. But if there is such an amount of information, it needs more time to classify and to learn in theprocedure of pattern recognition. Deriving the orientation
histogram from the orientations and magnitudes has aneffect of making the amount of information decrease.
Here each cell size is 5 × 5 pixels and the orientation
histogram has 9 bins with each cell.
Step3: Perform histogram’s normalization using an
overlapping block.
Here, each block size is 3 × 3 cells. Each blockcontains 9 × 9=81 features and each 30 × 60 sub-image
contains 4 × 10 × 81=3240 features.

3.RealAdaboost
The adaboost algorithm is a method of uniting weakclassifier of a simple hypothesis and generating a strongclassfier. A bicyclist and a bicycle model are defined andtrained via a RealAdaboost to detect bicycles undervarious circumstances. In training using RealAdaboost,
we use 1000 positive training samples and 2000 negative
samples and we train a two-component (a bicyclist and abicycle) model. Effectiveness of the proposed method isshown by experiment. The algorithm of RealAdaboost is
given in Table 1.
Table.1 RealAdaboost algorithm.

S = {( x1 , y1 ), ( x 2 , y 2 ), L , ( x N , y N )}as a sample space, where x ∈ X are feature vectors
and y ∈ {−1,+1} are labels.
1. Suppose

2. Dt is the initial distribution(a)(b)
Fig. 2 A processed bicycle image: (a) An input image,and (b) its oriented gradients image.

©ISAROB 2011

D1 (n) =782

1
NThe Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 20113.
for t=1 to T do
for m=1 to M do
Compute probability distribution Wt of

Input image

Extraction of HOG feature

weak classfier hi (x )n

W +j =∑ D (i)t
i , j∈J ∧ yi = +1

Detection of a bicyclenW −j =∑ D (i)t
i , j∈J ∧ yi = −1

Merging of detected windows

Compute estimation Z mDistinction of a bicycle’s driving direction

Z m = 2∑ W+jW−jj

Select the smallest Z m of the weak classifierEnd

ht = arg min Z t ,mFig. 3 Overview of the detection algorithm.

Renew sample weight Dt (i )1 W +j + εln
2 W −j + ε
Dt +1 (i ) = Dt (i ) exp[− y i hi ( xi )]The second step is shifting the mean to high density
employing a mean shift vector. A mean shift vector isgiven as follows;ht ( xi ) =2x − xixi k (∑
hi =1n

4. The final strong classifier is given by T
H ( x) = sign ∑ ht ( x)  .
 t =1m( x ) =III. DETECTION ALGORITHM

x − xi
k(
∑
h
i =1
n

)
−x

2

(3)

)

Here, the first term of the mean shift vector assumes
density and the second term shifted mean. When themean shift vector is 0, the density is the highest.

A detection algorithm is used to a bicycle model using
HOG feature. And we use five mask models for detecting
driving directions. The algorithm of detection is given in
Fig 3.

B. Nearest neighbor
Euclid distance d is computed from the point of thehighest density to each sample point.

1. Merge processing
When detection of a bicycle is performed using HOG
feature, many windows containing a single common
bicycle are obtained in an image. They are then merged
into a single window using mean shift clustering and thenearest neighbor algorithm.

d = ( x − xi ) 2 + ( y − y i ) 2(4)

Merging is executed, if d is smaller than a predefined
threshold. The result of the merging processing is shownin Fig. 4.

A. Mean shift clustering
The first step of mean shift clustering is assumption ofdensity using kernel function. Here we use a kernel
function represented byk ( x) =©ISAROB 2011c(1 − x )x <10otherwise(a)(b)
Fig. 4 Merge processing: (a) Before merging, and (b)after merging.

(2)

783

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

0

45

90
frame 10

frame 14135
180
Fig. 5 Bicycle’s five driving direction.

frame 34
frame 52
Fig. 7 Experimental results (180).

frame 10calculates the HOG feature on a bicyclist mask and on abicycle mask instead of calculating it on the whole image.
Therefore, the computation time of this method is fast.
Moreover, this method is applicable to the detection ofvarious objects. In future, we plan to enhance the performance of the detection and to detect various objects.

frame 14ACKNOWLEDGEMENT
frame 24
frame 29
Fig. 6 Experimental results (135).

This work was supported by JSPS Grant-in-Aid for
Scientific Research (C) 22510177

2. Detection of driving direction
We use five mask models (See Fig. 5) for detecting
driving directions. They are 0, 45, 90, 135 and 180
degrees.

REFERENCES[1] N. Dalal, B. Triggs, “Histograms of oriented
gradients for human detection”, Proc. IEEE Conf. on
Computer Vision & Pattern Recognition, 886-893, 2005.
[2] Q. Zhu, et al, “Fast human detection using a cascadeof histograms of oriented gradients”, Proc. IEEE Conf.
on Computer Vision & Pattern Recognition, 2006.
[3] K. Khattab, J. Dubois, and J. Miteran, “Cascadeboosting-based object detection from high-leveldescription to hardware implementation”, EURASIP
Journal on Embedded Systems, 2009.
[4] Y. Nakashima, J. K. Tan, S. Ishikawa, T. Morie, “On
the detection of a human and its body direction from athvideo”, Proc. of 15 Int. Symposium on Artificial Lifeand Robotics, 294-297, 2010.
[5] R. E. Schapire and Y. Singer, “Improved boostingalgorithms using confidence-rated predictions”, Machine
Learning, 37, 3, 97-336, 1999.

IV. EXPERIMENTAL RESULTS
Experiments have been done on a video containing abicyclist and a human passing in front of a camera. Fig. 6
and Fig. 7 show successful cases, but a human itself is
not detected. Because learning data of the positive classis a bicycle and a bicyclist.

V. CONCLUSIONSIn this paper, we propose a method of detecting abicycle and its driving direction using improved HOG
feature and RealAdaboost algorithm. This method has
several advantages over existing methods. This method©ISAROB 2011784

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Simple Analog-Digital Circuit for Motion Detection and Its
Application to Target Tracking System
Takuya Yamamoto and Kimihiro Nishio
Tsuyama National College of Technology, 624-1, Numa, Tsuyama, Okayama, 708-8509, Japan.
(Tel : 81-868-24-8266; Fax : 81-868-24-8219)(nishio@tsuyama-ct.ac.jp)

Abstract: We proposed in this study an analog-digital circuit for detecting the motion of the object. The proposed unit
circuit is simple structure since the digital part of the proposed circuit is only constructed with one NOT circuit and one
NOR circuit.
The proposed circuit was simulated with the simulation program with integrated circuitemphasis(SPICE). The simulation results showed that the proposed circuit can operate normally. The test circuit was
fabricated on the breadboard. The measured results of the fabricated test circuit showed that the proposed circuit can
output the pulsed voltage (motion signal) when the object moves on the circuit. We proposed in this study the system
for tracking the target. Since the array of the proposed analog-digital circuit was introduced at the first stage of thesystem, the circuit for tracking the target became simple structure. It was clarified from the measured results of thefabricated test system and the simulation results that the proposed system and the simulation results can track the target
and capture the target on the center of the input part.

Keywords: analog circuit, digital circuit, motion detection, target tracking, vision chip

The simulation results of the proposed circuit with thesimulation program with integrated circuit emphasis(SPICE) showed that the proposed circuit can operate
normally.
The test circuit is fabricated on thebreadboard. The measured results of the test circuit
showed that the proposed simple circuit can generate
the motion signal. The target tracking system was
proposed by applying the simple analog-digital circuits.
The measured results of the fabricated test system for
tracking the target and the simulation results showedthat the proposed system can operate normally.

I. INTRODUCTION
It is necessary for the monitoring system, robotics
vision and other systems to perform the high speed
image processing. However, it is difficult for typicalimage processing system using Neumann-type computer
to realize the high speed image processing. On theother hand, the biological vision system can perform thehigh speed processing since the nerve cells in the visionsystem perform information processing in parallel.
Analog complementary metal oxide semiconductor(CMOS) circuits for motion detection were proposed
based on the biological vision system [1]-[8]. These
circuits are characterized by the high speed processing,
low power consumption and simple structure.
Researchers tried to use these circuits as the input partof the target tracking system [6]-[8]. However, thesecircuits have a problem of incorrect operation by device
mismatches due to use the analog technology. Thus, it
is difficult to use these analog circuits to the application
systems such as the target tracking system.
Digital circuits for motion detection were proposed
based on the biological vision system [2]. These
circuits do not have a problem of incorrect operation.
However, the circuit by using digital technology has the
problem of the complex structure. If the simple digital
circuit for motion detection is proposed, it is able to use
it to various application systems.
In this study, we tried to propose the simple analogdigital circuits for motion detection by mimicking theinformation processing of the vertebrate retina.
Particularly, we tried to realize the simple digital circuit.

©ISAROB 2011

II. MOTHION DETECTION CIRCUITThe proposed unit circuit for motion detection basedon the vertebrate retina in this study is shown in Fig. 1.
By arranging the circuits in one- or two-dimensionally,
it is able to detect the motion velocity and direction of
the object. The circuit is constructed with the analog
part and the digital part. The analog circuit detects theobject (light) or the edge of the object. The digital
circuit generates the motion signal when the object
moves on the photodiode PD.
When the object is projected on PD, the current Ip
which is proportional to light intensity is generated.
The voltage VDD-Vp is proportional to Ip, where VDD is
the supply voltage. The constant voltage Vcon1 is set
to the circuit. The voltages Va and Vb are given by thefollowing equation.
Va  Vb 785

V DD  V p

V DD  V p   V con1 V DD

(1)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

One-dimensional array of
motion detection circuits

Edge detection circuit (Analog circuit)

VDD

Va

PD

Vout

Integrator

Vb

Ileft
Vcon1

Cm

Integrator
Vcent

CDMDL
Vleft

Fig. 1. Unit circuit for motion detection.
Fig. 2.

Table 1. A Truth table of proposed digital circuit.
Va

Vb

Vout

0
0
1
1

0
1
0
1

0
0
1
0

At first, the object is not projected on PD. In this case,
Va is equal to about 0 since VDD-Vp is smaller than Vcon1.
When the object is projected on PD, Va becomes about
VDD because VDD-Vp is larger than Vcon1. At the
moment that Va becomes VDD, Vb is 0. After time td, Vb
becomes VDD since the capacitor Cm is connected at theterminal of Vb.
A truth table of the proposed digital circuit is shownin table 1. When Va is 0 and Vb is VDD, Vout becomes
VDD. Therefore, the circuit outputs the pulsed voltage
when the object moves on PD. The width of the pulsed
voltage is td. The pulsed voltage is the motion signal
since the voltage is generated when the object moves on
PD.
The proposed digital circuit is constructed with a
NOT circuit and a NOR circuit. The NOT circuit
consists of two metal oxide semiconductor (MOS)transistors. The NOR circuit consists of four MOS
transistors. The proposed circuit is simple structuresince the circuit consists of only six MOS transistors.

Iright
CDMDR

MVright

The model for tracking the target.

Fig. 3. Circuit for detecting the motion direction(CDMDR).
right side. The integrator of the left side outputs thesignal Ileft. Ileft becomes large when the target moves tothe left side. The circuit in the center generates thesignal Vcent. Vcent is large when the target located on
the center of the array.
Ileft and Vcent are input to the circuit for detecting the
motion direction (CDMDL). Iright and Vcent are input to
CDMDR. The output signals Vleft and Vright become 0
when the target does not move on the input array. Vleft
becomes about VDD when the target moves toward theleft side. Vright becomes about VDD when the target
moves toward the right side.
The motor M rotates when Vright is VDD and Vleft is 0.
Then, the system can track the target toward the rightside. The motor rotates inversely when Vright is 0 and
Vleft is VDD. Then, the system can track the target
toward the left side.
After tracking the target, Vcent becomes large when
the target is captured on the center of the input array.
Then, Vleft and Vright become 0 (reset) by using Vcent.
When both Vleft and Vright are 0 or VDD, the motor is
stopped. Thus, using the model and system, it is able
to track the target and capture the target
Figure 3 shows the proposed CDMDR. The circuitis proposed based on the correlation model [9]. The
circuit is constructed with nine MOS transistors and 1

III. Target Tracking System
In this study, we tried to apply the proposed motion
detection circuits to the target tracking system. The
model for tracking the target is shown in Fig. 2.
At the first stage, the one-dimensional array of the
proposed circuits is introduced. When the target
moves on the array, the circuits output the motion
signals. The output signals input to the integrator.
The integrator of the right side outputs the signal
Iright. Iright becomes large when the target moves to the©ISAROB 2011

Unit

Circuit for generating
motion signal (Digital circuit)

Vp

Ip

Center786

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

capacitor.
structure.

Thus, the proposed circuit is simple

IV. EXPERIMENTAL RESULTS
1. Motion detection circuit
The proposed circuit for motion detection circuit
was simulated with SPICE. Input current Ip was set to
100 nA when the object was projected on PD. Cm was
set to 100 nF, The upply voltage VDD was set to 5 V.
Figure 4 shows the simulation result of the proposed
circuit. Ip was shown in Fig. 4(a). Figure 4(b) shows
the transient response of Vout. At the moment that thelight was projected on PD, Vout become 5 V. The
circuit output the pulsed voltage. Thus, it was clarified
from the simulation results that the proposed circuit cangenerate the motion signal.
The test circuit of the unit circuit was fabricated on
the breadboard. The analog part in Fig. 1 was
fabricated by using discrete MOS transistors(nMOS:2SK1398, pMOS:2SJ184). The digital part in
Fig. 1 was fabricated by the field programmable gatearray (FPGA) (Xilinx, Spartan III). VDD was set to 5 V.
Vcon1 was set to 1.6 V. Cm was set to 4.7 F. Thus, it
was clarified that the proposed circuit can generate thesignal for rotating the motor of the tracking system.
Figure 5 shows the transient response of Vout. At
the moment that the light was projected on PD, Voutbecame 5 V. The circuit output the pulsed voltage.
Thus, it was clarified from the measured results that the
fabricated test circuit can generate the motion signal.

Fig. 5. Measured results of the test circuit.
2. Target tracking system
The proposed circuit for tracking the target based on
the model in Fig. 2 was simulated with SPICE. Fiveunit circuits were utilized to each input part.
Cm was set to 300 nF. VDD was set to 5 V. Vcon1
was set to 3.5 V. Vcont2 was set to 0.5 V. Vth was set
to 1.58 V. Figure 6 shows the transient response ofVright obtained by SPICE. In this simulation, the object
moved toward the right side. Vright showed about 4 V
when the object moved.
The test system for tracking the target was
fabricated based on the model in Fig. 2.
Thephotograph of the fabricated tracking system is shownin Fig. 7. The current Ileft was generated by PDL. The
current Iright was generated by PDR. The voltage Vcent
was generated by PDC.
VDD was set to 5 V. Vcon2 was set to 1.27 V. CD was
set to 100 F. The light was provided as the object. The
motor was controlled by the motor driver (TA7257P,
TOSHIBA).
Figure 7 shows the measured results. The light
moved toward the right side from t=0 to t=1.2 s. At
t=1.2 s, the light was stopped. To t=2 s, the system
tracked the target. At t=2 s, the system captured thetarget on the center of the input array.

Fig. 6. Simulated results of the proposed circuit
transient response of output voltage Vright.

Fig. 4. Simulated results of the proposed circuit.
(a) Input current Ip. (b) Transient response of Vout.

©ISAROB 2011787

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

ACKNOWLEDGMENT
This research was partially supported by Artificial
Intelligence Research Promotion Foundation.

REFERENCES[1]C. A. Mead, Analog VLSI and Neural Systems,
Addison Wesley, Reading, MA, 1989.
[2]A. Moini, Vision Chips, Kluwer Academic, Norwell,MA, 1999.
[3]K. Nishio, K. Matsuzaka and H. Yonezu, “Simple
Analog Complementary Metal Oxide Semiconductor
Circuit for Generating Motion Signal”, Optical Review,
vol. 14, pp. 282-289, 2007.
[4]H. Yamada, T. Miyashita, M. Ohtani, K. Nishio, H.
Yonezu and Y. Furukawa, “Signal Formation of ImageEdge Motion Based on Biological Retinal Networks and
Implementation into an Analog Metal-Oxide-Silicon
Circuit”, Optical Review, vol. 8, pp. 336-342, 2001.
[5]W. N. Dalila, S. N. Ain and K. Nishio, “Analog
Motion Detection Circuits Using Simple Edge
Detection Circuits Based on the Vertebrate Retina”,
The 9th Int. IFAC Symp. on Robot Control (SYROCO
2009), pp. 159-164, 2009.
[6]K. Nishio and K. Matsuzaka, “Target Tracking
System Using Analog Circuit for Motion Detection”,
Proc. of 2008 IEEE Int. Conf. on Mechatronics andAutomation (ICMA 2008), TC3-5, 2008.
[7]K. Nishio and K. Matsuzaka, “Target Tracking
System Using Analog Motion Detection Circuit Basedon Biological Vision System”, Far East Journal of
Electronics and Communications, vol. 3, pp. 111-124,
2009.
[8]S. C. Liu and A. U. Viretta, “Fly-Like Visuomotor
Responses of a Robot Using a VLSI Motion-Sensitive
Chips”, Biological Cybernetics, vol. 85, pp. 449-457,
2001.
[9]W. Reichardt, Principles of Sensory Communication,
Wiley, New York, 1961.

Fig. 7. Measured results of the target tracking
system.

V. CONCLUSION
The simple analog-digital circuit for generating the
motion signal was proposed in this study. Particularly,
the digital part for motion detection is constructed witha NOT circuit and a NOR circuit. The test circuit isfabricated based on simulated results. The measured
results and the simulation results showed that the
proposed circuit can generate the motion signal. The
target tracking system was proposed by using the arrayof the proposed analog-digital circuit for motion
detection. The circuit for tracking the target is simple
structure. It was clear from the measured results andthe simulation results that the proposed system can track
the target and capture the target on the center of the
input array. The realization of the advanced image
processing system for the robotics vision, the security
system and other systems can be expected by applying
the proposed circuit and system.

©ISAROB 2011

788

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Analog Motion Detection Circuit Using CCD Camera Based on the
Biological Vision System and Its Application to Mobile Robot
Yasuyuki Kondo*, Takumi Yamasaki*, Kimihiro Nishio* and Toshinori Furukawa***Tsuyama National College of Technology, 624-1, Numa, Tsuyama, Okayama, 708-8509, Japan**Kurashiki University of Science and the Arts, 2640 Nishinoura, Tsurashima, Kurashiki, 712-8505, Japan(Tel : 81-868-24-8266; Fax : 81-868-24-8219)(nishio@tsuyama-ct.ac.jp)

Abstract: We proposed an analog motion detection circuit based on the biological vision system. The charge coupleddevice (CCD) camera was used as the input part which obtains the input image. By using the CCD camera, it isimmediately able to detect the motion of the high resolution images. The unit motion detection circuit is constructedwith 8 metal oxide semiconductor (MOS) transistors and 1 capacitor. The unit circuit becomes simple structure. The
test circuit was fabricated on the breadboard by using discrete MOS transistors. The measured results of the test
circuit showed that the proposed circuit can detect the motion direction. The fabricated test circuit was connectedwith the microcomputer introduced in the mobile robot. It was clarified from the measured results that the motion
detection circuit can control the mobile robot.
Keywords: analog circuit, motion detection, image processing, vision chip.

I. INTRODUCTION

II. MOTION DETECTION MODEL

It is necessary for robotics vision, monitoring
system and other systems to achieve the high speed
image processing. However, it is difficult to process
the image information for the typical image processing
system using Neumann-type computer because theinformation is processed in time sequential way. In the
biological vision system constructed with the retina andthe brain, it is able to accomplish the high speed image
processing. The processing is performed in massively
parallel nerve networks.
The integrated circuits (chip) were proposed basedon the biological vision system [1]-[6]. The circuit
can detect the edge and motion of the object in real time.
However, there is a problem of the low resolution in thecase to add many functions.
In this study, we proposed an analog motion
detection circuit based on the biological vision system.
The charge coupled device (CCD) camera was used as
the input part which obtains the input image. By using
the CCD camera, it is immediately able to detect the
motion of the high resolution images. The unit motion
detection circuit becomes simple structure.
Themeasured results of the test circuit showed that the
proposed circuit can detect the motion direction. The
fabricated test circuit was connected with the mobile
robot. The robot could operate by using the motion
signal.

Figure 1(a) shows the motion detection model basedon the biological vision system. We call the model thecorrelation model. The model is constructed with thephotoreceptors P, delay neurons D and correlators C.
The arrows show the flow of the signals. The solidline shows the model for detecting the right motion.
The dashed line shows the model for detecting the left
motion. In this paper, the model for detecting the rightmotion is described.
Figure 1(b) shows the signals of the motion©ISAROB 2011

Object
P1

P2

D1

D2

C2

Vright VD1 VP2 VP1

Vleft

C1
(a)

Time tVright(b)
Fig.1. Motion detection model. (a)Model. (b)Transient
response of each cell.

789

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

utilized as those of photoreceptors.

detection model when the object moves toward the rightside. Firstly, the object is projected on P1. P1generates the signal which is proportional to lightintensity. The signal of P1 is input to D1. The signalof D1 decreases. When the object is projected on P2,P2 generates the signal. The signal of P2 is input to C1
Since the output signal Vright of C1 is proportional to D1,
Vright becomes large when the object moves toward theright side. Then, the output signal Vleft of C2 is 0.
When the object moves toward the left side, Vleft
becomes large and Vright is 0. Thus, the model can
detect the motion direction.

2. Unit circuit
Figure 3 shows the unit motion detection circuit.
VP1 and VP2 are input voltages. VP1 corresponds to thesignal of P1 in Fig. 1. VP2 corresponds to the signal of
P2. Vright is the output voltage. In this circuit, theconstant currents Vth1, Vth2 and Vth3 are set.
The delay neuron D in Fig.1 is realized by 1 metaloxide semiconductor (MOS) transistor and 1 capacitor.
The correlator is constructed with 5 MOS transistors.
The unit circuit is constructed with 8 MOS transistors
and 1 capacitor. The circuit is simple structure.
In the case of the circuit for detecting the left motion,
the output voltage becomes Vleft. Thus, the motion
direction can be detected by using 2 unit circuits.

III. MOTION DETECTION CIRCUIT
1. Input part
Figure 2(a) shows the relationship between theobject and CCD camera. The object is projected on
the CCD through the lens.
Figure 2(b) shows the projected image. In thisstudy, it is necessary to generate the signals of P1 and P2,
as shown in Fig. 1. As shown in Fig. 2(b), the image issegmented. The signals of the segmented image are

IV. MEASURED RESULTS OF CIRCUITThe test circuit was fabricated with discrete MOS
transistors on the breadboard. C was set to 100 nF.
Vth1, Vth2 and Vth3 were set to 2.18 V, 1.2 V and 1.36 V,
respectively. The supply voltage VDD was set to 5 V.
Figure 4 shows the measured results of the test
circuit when the object moved toward the right side.
Vright became 5 V. Then, Vleft became 0.
Figure 5 shows the measured results of the test
circuit when the object moved toward the left side.
Vleft became 5 V. Then, Vright became 0.
Thus, the test circuit can detect the motion direction.

Fig.3. Unit motion detection circuit.

Fig.2. Relationship between the object and CCD camera.

©ISAROB 2011790

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Figure 7 shows the measured results of the test
system when the object moved toward the right side.
The robot handed up the right arm.
Figure 8 shows the measured results of the test
system when the object moved toward the left side.
The robot handed up the left arm.
Thus, it is able to control to the robot by using our
proposed circuit.

VI. CONCLUSION

Fig.4. Measured result of test circuit (right side).

An analog motion detection circuit was proposed
based on the biological vision system. By using the
CCD camera as the input part, it is able to detect the
motion of the high resolution images. The unit circuit
becomes simple structure since the unit circuit isconstructed with 8 MOS transistors and 1 capacitor.
The measured results of the test circuit showed that the
proposed circuit can detect the motion direction. The

Fig.5. Measured result of test circuit (left side).

V. APPLICATION TO MOBILE ROBOTWe tried to control the mobile robot by using our
motion detection circuits. The fabricated test system isshown in Fig. 6. In the robot (ROBONOVA-I, Hitec
Multiplex Japan), the microcomputer (MR-C3024) isintroduced. The output terminals of the test circuits,i.e., Vright and Vleft were connected with input terminalsof the microcomputer.
In this study, we programmed that the robot hands
up the right arm when the object moves toward the rightside, i.e., when Vright becomes VDD. We programmed
that the robot hands up the left arm when the object
moves toward the left side, i.e., when Vleft becomes VDD.

Fig.7. Measured result of test system (right side).

Fig.8. Measured result of test system (left side).

Fig.6. Photograph of the fabricated test system.

©ISAROB 2011

791

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

test circuit was connected with the mobile robot. It
was clarified from the measured results that our
proposed circuit can control the robot. In the future,
by applying the proposed motion detection circuit, it isable to use as the motion sensor such as the robotics
vision.

ACKNOWLEDGMENT
This research was partially supported by strategic
support project for university collaboration, science tryangle in Okayama sponsored by the education andscience ministry of Japan.

REFERENCES[1] C. A. Mead (1989), Analog VLSI and neuralsystems, Addison Wesley, Reading, MA[2] A. Moini (1999), Vision chips, Kluwer Academic,
Norwell, MA[3] K. Nishio and K. Matsuzaka (2009), Target tracking
system using analog motion detection circuit based on
biological vision system, Far East Journal of Electronics
and Communications 3 : 111-124[4] K. Nishio, H. Yonezu, A. B. Kariyawasam, Y.
Yoshikawa, S. Sawa and Y. Furukawa (2004), Analog
integrated circuit for motion detection against moving
background based on the insect visual system, Optical
Review, 11 : 24-33[5] K. Nishio, H. Yonezu, and Y. Furukawa (2006),
Analog integrated circuit for detection of an
approaching object with simple-shape recognition basedon lower animal vision, IEICE Transactions on
Fundamentals of Electronics, Communications andComputer Sciences E89-A : 416-427[6] K. Nishio, H. Yonezu and Y. Furukawa (2007),
Analog vision chip for motion detection of approaching
object against moving background based on insect
visual system”, Optical Review 14 : 111-119©ISAROB 2011

792

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Detecting Human Flows on a Road Different from Main Flows
Mison Park, Joo Kooi Tan, Yuuki Nakashima, Hyoungseop Kim and Seiji Ishikawa
Department of Mechanical and Control Engineering
Kyushu Institute of Technology
Sensuicho 1-1, Tobata, Kitakyushu 804-8550, Japan
E-maile: {park, etheltan, ishikawa}@ss10.cntl.kyutech.ac.jp, kim@cntl.kyutech.ac.jp

Abstract: Automatic detection of human flows on a road by a computer vision system is of great importance mainly insurveillance systems, where human flows are observed by a camera and a computer analyzes the videos that the camera
provides to detect a person having a different flow of movement, such as a person walking toward a certain direction
while most of the people walk in the opposite direction, or a person running in a group of walking people. This paper
describes a technique for finding a person having a different behavior or motion from others. The idea of the paper is toclassify motion flows (or optical flows) extracted from a video into respective groups having respective directions of
the motion by analyzing the motion flows. Experimental results show effectiveness of the proposed technique.
Keywords: Motion detection, abnormal motion, Harris corner detector, L-K tracker, clustering, feature space.

extract feature points from the initial image frame. Then,
track the feature points in the next frame to find the
motion vector. However, it is impossible to determine
whether the motion is normal or abnormal only by the
motion vector of the two successive frames.

I. INTRODUCTION
In recent years, along with the unceasing
intellectualization of video image monitoring
technology, the intelligent monitoring technology has
gained more and more domestic and foreign merchants’and scholars’ recognition and has conducted a series ofresearches. One of the most challenging tasks inintelligent monitoring technology is the analysis ofhuman motion in crowed scenes and detecting thepeople who have a deferent motion from others. The
detection of an abnormal motion can trigger video
transmission and recording, and can be used to attract
the attention of a human observer to a particular video
channel.
This paper describes a novel technique for finding aperson having a different motion from others The ideaof the paper is to classify motion flows (or opticalflows) extracted from a video into respective groups
having respective directions of the motion by analyzing
the motion flows. In order to realize this, the Harris
corner detector is applied to an initial image frame to
extract feature points on the image: The Lucas-Kanade
tracker is then applied to the successive frames to detectmotion flows on the video: Pyramidal search isconsidered, if necessary, to detect the motion flows
having different flow lengths indicating difference ofthe motion speed. The obtained motion flows are finallyclassified into some groups which have respective
motion directions or speed.

Start

Input initial
Frame

Extract
feature points

Remove thestatic points

Transform tofeature space

Next
frame

Track thefeature points

Calculate thedegree of similarity

Cluster thefeature points

Save thecoordinates
space

Detect theabnormal motion

N
Accumulated
frame = TEndYII. OVERVIEW OF THE ABNORMAL MOTION DETECTION SYSTEM
Fig.1. Overview of the proposed abnormal motion
detecting method

The process of detecting abnormal motion involves
two primary contents: 1) tracking feature points. First,©ISAROB 2011

793

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Therefore, we need to accumulate some image frames
from the first frame and store the location of the featurepoints that are tracked over the entire frames into thecoordinate space. 2) Clustering. Remove stationary
points from the coordinate space and convert thecoordinate space into a feature space. Finally, cluster thefeature points to detect abnormal motions. Fig. 1 shows
the overview of the abnormal motion detecting method.

III. METHOD
1. Extracting feature points
Generally, motion vectors are calculated by using
the pixel values in the local region around the attracted
point. We cannot obtain the right motion vector if
texture conditions are poor in this local region.
Therefore we need to find an adaptive feature point. In
this paper, Harris corner detector, a popular featurepoint detector, is applied to extract the feature points.
The Harris corner detector is based on the local
auto-correlation function of a signal, where the local
auto-correlation function measures the local changes ofthe signal with patches shifted by a small amount indifferent directions.
For a small shift [u, v], we have bilinear approximation as follows;u E (u, v)  u, v  M  v Fig.2. Feature points extraction

The L-K algorithm relies only on local informationthat is derived from some small windows surrounding
each of the points of interest. Based on the condition,
we can get the final expression in the form ofd  ( AT A) 1 AT b
where

 I x ( x1 , t )  I x ( x n , t ) A I y ( x1 , t )  I y ( x n , t ) (1)

IxI y I y2 

(2)

The Harris measure of a corner is defined by

R  det M  k  trace M 

2

(3)

where

det M  12
trace M  1  2

(4)

where 1 and  2 are the eigenvalues of the matrix M.
Find the points with large corner response function
R (R>threshold), and take the points of local maxima of
R. Fig. 2 shows the image of extracted feature points.
2. Tracking feature points
The Lucas-Kanade tracker, one of the most wellknown feature points tracking algorithms is employed in
the proposed method.

©ISAROB 2011

T

(6)

The disadvantage of using small local windows in
Lucas-Kanade algorithm is that large motions can movepoints outside of the local window and thus become
impossible for the algorithm to find them. This problem
led to the development of the pyramidal L-K algorithm,
which starts tracking from the highest level of an image
pyramid and working down to lower levels. Tracking
over image pyramids allows large motion to be caught
by local windows. Fig. 3 shows the image of motion
vectors.

where M is a 22 matrix of the following form
computed from image derivatives;

 I x2M   w( x, y ) x, y I x I y(5)

Fig.3. Motion vectors

794

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

3. Creating the feature space
Since it is difficult to detect an abnormal motion
only by the motion vectors between successive frames,
we need to accumulate some frames from the first frame.
The feature points are tracked over the entire frames
and their location information is stored into a coordinate
space. Suppose that a feature point n (n=0,1,2,…,N-1) istracked through T image frames and its position on theframe t (t=0,1,2,…,T-1) is denoted by (xt(n), yt(n)). We
then define a sequence of T coordinates of the featurepoint by the following form;X  x , y , x , y ,..., x, y  (7)Fig.4. Tracking result throughout T frames

X 0  x0( 0 ) , y0( 0 ) , x1( 0 ) , y1( 0 ) ,..., xT( 0) 1 , yT( 0) 1
1



(1 )

(1 )

(1 )

(1 )

(1 )

0

0

1

1

T 1

(1)

T 1



X N 1  x0( N 1) , y0( N 1) , x1( N 1) , y1( N 1) ,..., xT( N11) , yT( N11)

3. Detecting an abnormal motion
The flow chart of clustering is shown in Fig. 5,
where dij and Th are defined as follows;

dij  SSD ( s j , si )Fig. 4 shows the tracking result throughout the T frames.
However, the movement cannot be known only by
the position information of points in the coordinate
space. Therefore the coordinate space is converted tothree kinds of feature space such as velocity (8),velocity magnitude (9), and velocity orientation (10).

 V  v , v   x  x , yy Th   (max  min)  min    0.001





(1 )

(1 )

(1)

(1 )

(1)

x

y

T 1

0

T 1

 

(1)

0

VN  v x , v y  xT 1  x0 , yT 1  y0
(N)

(N)

(N)

(N)

(N)

(N)

Start

(8)
Calculate
all values ofdijThresholdprocessing

V0  ( xT( 0) 1  x0( 0 ) ) 2  ( yT( 0) 1  y0( 0 ) ) 2
V1  ( xT(1) 1  x0(1) ) 2  ( yT(1) 1  y0(1) ) 2(9)Changethe threshold

Clustering

VN  ( xT( N)1  x0( N ) ) 2  ( yT( N)1  y0( N ) ) 2

y (0)  y0( 0 ))xT( 0) 1  x0( 0 )All points are
clusteredor Th＞γ ?

 0  arctan( T 1y (1)  y0(1))
xT(1) 1  x0(1)1  arctan( T 1

N

Y

(10)
End


y ( N )  y0( N ))xT( N)1  x0( N ) N  arctan( T 1

©ISAROB 2011(12)

The similarity of the features with respect to thefeature points are calculated among them. This

V0  v x( 0 ) , v (y0 )  xT( 0)1  x0( 0 ) , yT( 0)1  y0( 0 )1(11)

Fig.5. Flow chart of clustering

795

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

similarity is clustered by threshold processing. At first,
the threshold is set strictly. The threshold is adjusted so
as to cluster all the points or larger than γ. Look-up
table is used in the clustering process. After clustering,count the number (n) of feature points clustered in eachclass. The class that satisfies the following condition,n  total number of points  R %, is defined as anabnormal motion.

created based on the tracked points coordinates. The
feature points are clustered by a Look-up table. Finallyabnormal motions are detected based on the clusteredpoints. The future work is to integrate the multiple
features in order to improve the accuracy.

REFERENCES[1] C. Harris, M. Stephens (1988), A combined cornerand edge detector, Proc. of the 4th Alvey Vision
Conference, pp.147-151.
[2] B. D. Lucas, T. Kanade (1981), An iterative image
registration technique with an application to stereo
vision”, Proc. of the 7th International Joint Conferenceon Artificial Intelligence, pp.674-679.
[3] M. A. Fischler, R. C. Bolles (1981), Random sample
consensus: a paradigm for model fitting withapplications to image analysis and automatedcartography, Communications of the ACM, vol.6, no.24,
pp.381-395.

IV. EXPERIMENTS AND CONCLUSIONS
The abnormal motion detection algorithm proposed
here was successfully tested on various outdoor scenes.
Fig. 6 shows the final result.
In this paper, we propose a novel technique ofabnormal motion detection from the images obtained
from a surveillance camera. The Harris corner detector
and the pyramidal Lucas-Kanane algorithm are applied
to feature tracking and calculation of motion vectors.
Then a coordinate space and a feature space are

(a)

(b)

(c)

(d)

Fig.6. Performance of abnormal motion detection from various scenes. The red line in the frames shows the abnormal
notion of a person having a different motion from others. Time elapses in the order of upper left, upper right, lower left
and lower right image.

©ISAROB 2011

796

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Study on the height measurement based on the image processing
techniqueJiwu Wang1

Weijie Gao1

Yisong Wang1

Department of Mechanical and Electrical Engineering, Beijing Jiaotong University,
Beijing, China, 1000441

Sugisaka Masanori2
Nippon Bunli University, Oita, Japan2

Abstract: It is difficult to realize the height measurement based on the image processing

technique. In this paper, we study how to carry on the height measurement with one CCD
camera. In order to reduce the influence of the lighting conditions in the environment, thenear-infrared laser and filter were used. We tested our method with various materials. The
experiments show we can get reliable results in our laboratory conditions. The further
research and some verifications of our algorithms will do later.
Keywords: Image processing, Heigh measurement, Near-infrared laser

INTRODUCTION

Height measurement problem has a long history,and
there are variety of measurement, some of thesemeasurement also use the technology of image
processing.It mostly use of mathematical computation
with extra scale and multifarious algorithm of image
processing,not only compute complexly but also is
influenced easily,so influence the precision of
measure.The following text will introduct how to carryon the height measurement with one CCD camera.

CCD camera

θ

l

β

filterthe plane of laser

II. The mechanism of measuremen
As illustrated,we make CCD camera and laser
fixed,then we can get the length between camera andlaser through precision measuring instrument.We also
can get the value of angle between the centre line ofcamera and horizontal plane by measuring instrument.In
the picture two black piece represent the reaction ofheight when the light emited by laser fired the measured
object.The angle of α and β can be got through image
processing,based on geometrical relationship we cancalculate the h as the next formula.

centre line

The picture depict the situation that CCD camera
centre line and measured pixel of luminous beam are in
the same plane.We call this plane for vertical center
plane.There is another situation which camera centre
line and measured pixel of luminous beam are not in thesame plane.We mark the angle that between centre line
and measured pixel as γ which we can get throughimage processing.We found a parameter L,make
L=l/cosγ,and then instead of h.Now we can Continue
using this formula.

h  l  tan(   )  l  tan(   )©ISAROB 2011

laser

α

I.

797

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

III. The principle of measurement
The light emitted by laser fired on the profile of aworkpiece,forming a bright curve that is the profile of
the workpiece.The bright points do not in a straight line
because of curve profile of workpiece.Through thecamera we can get the picture and then we process the
picture.
In image processing,we use the algorithm of model
conversion,edge detection,binaryzation and so on,above
all we use of filter.This can greatly reduce the difficultyof image processing,simple the algorithm,shorten thecomputation time,improve the detection efficiency,and
can obtain accurate and stable results.We already know
the wide angle of camera,through image processing,we
can get position relation of bright pixel and centr
pixel,then find out the angle of α and β.According toformula （1）,we can figure out height difference,andthen get the profile.

FilterAfter using filter,we will amazed:how a perfectpicture.However this is not the final result we want,wemust extract a line,a centre line of the luminous
beam.Then we can get the position of every pixel in theline.In order to get α,β and γ,we must calculate thedistance between measured pixel and centre line of thepicture,which is not difficult. We can use this equation
which is shown by matrix.We can achieve this equationin programming way,then we process these data that wehave get about height.With these data we can get the
profile easily.

IV. Experiment research
In the laboratory we build a simple platform,with ascrewdriver handle to be tested.First we show thepicture without filter;And then put the filter in front of
the lens,the picture we get is as 2; The third picture isafter processing.The effect of image processin as flows:

After processing

 h1 h 
 2   l / cos  l / cos 
1
2
  =

 

 
 hn Without filter©ISAROB 2011798

  l / cos  n The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011  tan(   1 )   tan(  1 )  

 

  tan(   2 )   tan(   2 )  

 









 


 





  tan(   )  tan(   ) 
n 
n 



V. CONCLUSION
Image processing method to measure the heightmainly includes two parts: hardware component andsoftware component.It will increase the costs andworking environment if we make hardware better
only.And it will increase the processing time if use
complicated algorithms.Now we use cheap equipment
to meet the requirements of precision, and greatly
reduce the equipment cost.However if we want toimprove measurement accuracy, better camera will be
used.

REFERENCES[1]Luping Xu.Digital Image Processing.Science
press,2007.
[2]Guan Xin, Luo Zhi sheng, Wu Jianle, Gao Xiaorong.
Locomotive wheel 3d reconstruction, IPPRIE,2010[3] Lin Yuping ， Medioni G ． Mutual information
computation and maximization using GPU[C]．In：
IEEE Compu~r Society Conference on Computer
Vision and Pattern Recognition Workshops．Alaska，USA，2008．1-6．

©ISAROB 2011

799

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Reinforcement learning with the mechanism of shortterm depression for learning rate
S. Kubota
Department of Biomedical Information Engineering, Yamagata University
4-3-16 Jonan, Yonezawa, Yamagata, 992-8510, Japan(Tel : 81-238-26-3356; Fax : 81-238-26-3299)(kubota@yz.yamagata-u.ac.jp)
Abstract: The temporal-difference (TD) reinforcement learning (RL), typically formulated in discrete state space, isfrequently applied to the control problems represented by continuous state variables. The use of a coarse space discretization to describe the RL algorithm may degrade the control performance, whereas a fine discretization requires a large
number of iteration steps to complete learning. In this study, I examine a novel RL algorithm by which the learning rateis dynamically and spatially modulated to improve the learning performance even when a relatively coarse space discretization is employed. This method is inspired by physiological phenomenon of short-term depression observed in biological synapses, and aims to produce a bias in the TD learning toward the states that have not been recently visited.
The proposed algorithm is incorporated with Sarsa-lambda and tested in a nonlinear control task of swinging up a pendulum using limited torque. The simulation results show that the proposed dynamic learning rate can robustly reduce
the number of trials required before accomplishing the task by facilitating efficient exploration in the RL process.
Keywords: Temporal difference, Reinforcement learning, Dynamic learning rate, Synapse.
in a discrete stochastic system. In this approach, a finediscretization of state space necessarily requires a large
number of memory storage as well as many iteration
steps to complete learning the value functions. In contrast, a coarse space discretization leads to a situation
where, when the state variables evolve slowly with time,
an update of value functions takes place many times
repeatedly at identical states; in such a case, it is likelythat wrong actions repetitively taken at the same state
are excessively reinforced, particularly during earlylearning phases.
Ideally, one would like to develop an algorithm that
can improve the RL performance even when a relatively
coarse space discretization is employed. In this study, I
propose a novel method, which incorporates spatiotemporal modulation of learning rate to enhance the learning efficiency in such coarse discretized space. Thismethod is inspired by physiological phenomenon ofshort-term depression (STD) observed in central synapses, i.e., a transient decrease in the strength of synaptic inputs following their repetitive activation. The
proposed algorithm is introduced in Sarsa (  ) [5] andapplied to a nonlinear control task of swinging up apendulum with limited torque. I show in simulations
that the task can be accomplished by employing the
proposed dynamic learning-rate modulation in a numberof trials less than by employing a conventional static
learning rate. Further, I demonstrate that the proposed

I. INTRODUCTION
The temporal-difference-(TD-)based reinforcement
learning (RL) not only provides an efficient approach tocontrol and decision problems, but also can be considered as an attractive model for studying the brain [1]. In
fact, a broad range of experimental evidence suggests
the involvement of neural activities occurring at a variety of brain areas in the mediation of reward processing
as well as the TD error signal [1-3]. A close connection
between the RL theory and the relevant physiological
data implies that the theoretical scheme of RL could
provide a quantitative framework for future studies in
the related area of neuroscience [2]. Furthermore, it also
appears possible to improve the RL algorithm by incorporating neurobiological mechanisms underlying adaptive behavior of animals, as this study aims to do.
An important application of RL technique is to control a strong nonlinear dynamical system, which would
be difficult to be solved by a conventional engineering
approach. In many interesting real-world control tasks,
the state variables evolve with time in continuous space,although the progress of RL theory has been mainly
restricted to the problems described by the Markov
decision process (MDP) formulated in discrete state
space [4]. Therefore, in the application studies of RL,
the most common approach is first to discretize the state
space and then to apply the learning algorithm described©ISAROB 2011

800

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

time is discretized by using Euler’s method, Eq. 4 canbe written, by defining parameters   1   v andmethod can decrease the sensitivity of the task performance to the scaling of learning rate, implying that thismethod can make the tuning of the learning parametereasier.

  t /  r ( 0   ,   1 ), as follows:  D(t )  [1  D(t )],D(t  1)  (following synaptic activation) (5) D(t )  [1  D(t )].
(otherwise)

II. METHODS
1. Sarsa (  )As a basic RL algorithm to which the proposed
method is to be incorporated, the author used Sarsa (  )[5]. In this algorithm, the action-value function Q( s, a ) ,for each state s and action a , is updated as follows:(1)
Q ( s, a )  Q ( s, a )   e( s, a ) .
Here,  ( >0 ) denotes the learning rate.  represents
the TD error described as(2)
  r   Q ( s(t  1), a (t  1))  Q ( s(t ), a (t )) ,where r is the reward obtained by the state transition
from s(t ) to s (t  1) through action a (t ) .  is a
discount rate satisfying 0    1 . Each action is assumed to be decided from the  -greedy policy withrespect to Q ( s, a ) . The eligibility trace e( s, a ) for all
the state-action pairs, s, a, is updated by the following
equation:

Assume that, based on the classical cell assembly
hypothesis [8], firing activity of a group of interconnected neurons encodes a specific information regarding
environment. Then, the repetition of the same state in
MDP could be represented in the brain by the repetitive
spiking of the same cell group, which will weaken input
activity for such group of neurons through STD [9].
Further, experimental evidence suggests that the resultant weakened activation of NMDA receptors(NMDARs), one of major receptor subtypes for glutamatergic synapses, may cause the suppression of longterm plasticity underlying learning in the brain [10].
This may correspond, in the terminology of RL, to thedecreased value of learning rate.
Therefore, to incorporate the mechanism resembling
STD to an RL algorithm, the author introduces a function d ( s ) ( 0  d ( s )  1 ) and considers that d ( s ) e( s, a )  1, ( s  s(t ) and a  a (t ))(3)e( s , a )  (otherwise) e( s, a ).

for all the states s are updated, similar to Eq. 5, as follows:  d ( s )  [1  d ( s )], (for s  s (t ))(6)d ( s)   d ( s )  [1  d ( s )]. (otherwise)
The value of  (Eq. 1) is multiplied by d ( s ) so that

2. STD mechanism for controlling learning rate
The author designed an algorithm for spatiotemporalcontrol of learning rate, which models short-term activity-dependent modification of synapses. In the central
nervous system, the synaptic inputs that are activatedwith higher frequency are rapidly and temporarily depressed so that the input-output gain for high-rate inputs
decreases [6]. This phenomenon, called STD, occurs
through the transmitter vesicle depletion at the presynaptic terminals, and can be modeled by the following
equation [6,7]:
dD (t ) / dt    v D (t )  (t  t j )  [1  D (t )] /  r , (4)the function Q is updated, instead of Eq. 1, as
Q ( s, a )  Q ( s, a )   d ( s )e( s, a ) .

According to Eq. 6, d ( s ) for a given state s will
decrease each time the state s is visited, whereas it
gradually recovers to 1 in the absence of visiting s.
Therefore, it can be expected that d ( s ) acts to decrease the change in the action-value function Q ( s, a )Table 1. The learning parameters
Parameter
Parameter to decide the level of STD 

jwhere D (t ) ( 0  D (t )  1 ) represents the synaptic
strength normalized by its maximum value, and t j isRate of recovery from STD 

the j th activation time of the synapse. Equation 4 indicates that the synaptic strength is weakened just follow-

Scale of learning rate 
Discount rate 

ing activation such that D (t j  ) = (1   v ) D (t j )Decay rate of eligibility trace Randomness of policy ( 0   v  1 ), whereas it recovers toward 1 with thetime constant  r in the absence of activation [7]. If©ISAROB 2011

(7)

801

Value
0.6
0.04
0.8
0.98
0.8
0.1

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

the Euler‘s method was used with the time step size t= 0.02. The reward given was set to be r = 1 for|  | <  / 4 , r = 75 / t for |  | > 4 , and r = 0
otherwise, where the negative reward for large  is toprevent the over rotation. The state space {(  ,  )| 4   ,   4 } was digitized into subspaces withthe length of  =  =  / 3 . A trial ended at t = 20 orwhen the pendulum became over-rotated ( |  | > 4 ). To
quantify the task performance, the total length of time atwhich the pendulum stands up ( |  | <  / 4 ) was definedas Tstand . The trial was considered to be successfulwhen Tstand  10 , and the learning speed was measured
by using the number of trials, N success , required before
achieving 10 successful trials.
Figure 1 shows the comparison of the time course of
Tstand obtained by using and not using the STD mechanism of learning rate (Figs. 1A and 1B, respectively).
Note that although the mean value of Tstand is largely

Fig.1. (A and B) The change in the average and standard
deviation (error bar) of Tstand as a function of the num-the same regardless of the inclusion of STD (Fig. 1C),
the temporal variation of Tstand becomes significantly

ber of trials in the presence (A;  = 0.6) and absence(B;  = 1) of the STD mechanism. The dashed lines

increased by the STD function (Fig. 1D). This would be
attributed to the fact that STD will contribute to assigning higher weight to the learning of ‘novel’ states thathave not been recently visited, as mentioned above. The
result here shows that the STD function will be effective
to facilitate the exploration in RL almost without changing the average task performance.
When similar simulations were performed by using
various values of  and  , the value of N success wasshow the level of Tstand above which a trial is considered successful ( Tstand  10 ). (C and D) The average (C)and standard deviation (D) of Tstand are plotted for boththe cases with (solid) and without (dashed) STD in thesame graph. The parameters for the system dynamics are
as follows: m = l = 1, g = 9.8, c = 0.01, andumax = 5 [4].

found to take a minimum at an intermediate value of for all  (Fig. 2A). Importantly, the minimum

for the states that have been visited in the near past.
This also implies that by the inclusion of the STD
mechanism, the TD learning tends to proceed with abias that assigns a higher weight to the states that havenot been recently visited. The learning parameters in
Table 1 are used unless otherwise stated.

 with  < 1 is considerably smaller than that for  = 1 (i.e., the case of no
N success value for eachSTD) (Fig. 2B), suggesting that the inclusion of STD
can robustly improve the learning performance. To
further explore the robustness of the outcome, the upperand lower limits of the  range, where the value ofN success is less than a threshold (= 250), were defined asIII. RESULTS
To test the proposed algorithm, the author performed
simulations for the control task of swinging up a pendulum with limited torque [4]. The dynamics of the pendulum are described as d / dt   and d  / dt =( c  mgl sin   u ) / ml2

U and  L , respectively, and the U /  L ratio wasplotted as a function of  (Fig. 2C). The figure showswith the external torquethat this ratio becomes greater than that obtained without STD when  is relatively large (0.7    0.9).

u = umax , implying that only its direction can be

This implies that in this range of  , STD can decrease

controlled. The physical parameters used are summarized in the Fig. 1 caption. For numerical integration,©ISAROB 2011

the sensitivity of the task performance to the scale of802

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

learning rate, which will make the tuning of the learningparameter easier. Note that around similar values of 
(0.5    0.9), the minimum N success becomesquite small (Fig. 2B), indicating that this range of 
will be near-optimal in that the task performance is bothrobustly and significantly enhanced.

IV. CONCLUSION
In this study, I have proposed an STD mechanism for
TD-based RL, where the learning rate is spatiotemporally modulated, and have shown that this method can
reduce the number of trials required before accomplishing the control task. This method is motivated by thephysiological experimental findings on synapses that
their activation are rapidly followed by depression [6].
The proposed learning rate modulation (represented by
the change in d ( s ) ) tends to assign a greater weight tothe learning of value functions for the states that havenot been recently visited. This appears to be similar tothe observed response of dopamine neurons, which canreflect the novelty of the presented stimuli [11]. Therefore, given that the TD error  closely resembles the
dopamine cell response representing the unpredictabilityof reward [11], the term d ( s ) (Eq. 7) may corre-

Fig.2. The comparison of task performance for various
values of  and  . (A) The change in the value ofN success , which is averaged over 32 cases, as a functionof  and  . (B and C) The minimum N success value(B) and the  U /  L ratio (C), which are obtained byusing various values of  , are plotted for each  . The

spond to the dopamine signal that encodes both thenovelty and unpredictability of the reward. The present
study may suggest a new framework of RL research inwhich the functional significance of a biologicalmechanism can be examined by constructing an RL
algorithm that incorporates the mechanism and testing it
through the application to control problems.

cases of  = 1 (denoted by the horizontal dashed lines)
correspond to those without the STD mechanism.
[6] Abbott L. F., Varela J. A., Sen K., et al. (1997)
Synaptic depression and cortical gain control. Science 275: 220-224.
[7] Wang X.J. (1999) Synaptic basis of cortical persistent activity: the importance of NMDA receptors toworking memory. Journal of Neuroscience 19:9587-9603.
[8] Lansner A. (2009) Associative memory models:from the cell-assembly theory to biophysically detailed cortex simulations. Trends in Neuroscience
32:178-186.
[9] Zucker R. S. (1999) Calcium- and activitydependent synaptic plasticity. Current Opinion in
Neurobiology 9:305-313.
[10] Bear M. F. (1996) A synaptic basis for memorystorage in the cerebral cortex. Proceedings of the
National Academy of Sciences of the United Statesof America, 93:13453-13459.
[11] Shultz W. (1998) Predictive reward signal of dopamine neurons. Journal of Neurophysiology, 80: 127.

REFERENCES[1] Doya K (2008), Modulators of decision making.
Nature Neuroscience 11:410-416.
[2] Schultz W, Dayan P, Montague R (1997), A neuralsubstrate of prediction and reward. Science
275:1593-1599.
[3] Schultz W, Apicella P, Ljungberg T (1993), Responses of monkey dopamine neurons to reward andconditioned stimuli during successive steps of learning a delayed response task. Journal of Neuroscience
13: 900-913.
[4] Doya K (2000), Reinforcement learning in continuous time and space. Neural Computation 12:219-245.
[5] Sutton R. S., Barto A. G. (1998) Reinforcement
learning: An introduction. The MIT Press, Cambridge.

©ISAROB 2011803

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A Method for Finding Multiple Subgoals
for Reinforcement Learning
Fuminori Ogihara and Junichi Murata
Kyushu University, 744, Motooka, Nishi-ku, Fukuoka, Fukuoka, Japan(Tel : 092-802-3675; Fax : 092-802-3692)(ogihara@cig.ees.kyushu-u.ac.jp)

Abstract: This paper proposes a new method for discovering multiple subgoals automatically to accelerate
reinforcement learning. There have been proposed several methods for discovery of subgoals. Some use state visiting
frequencies in the trajectories that reach the goal state. When a state visiting frequency is very high, this state isregarded as the subgoal. Because this kind of methods need that the goal state is reached many times to collecttrajectories, they take a long time for discovering subgoals. In addition, they cannot discover the potential subgoals thatwill become appropriate subgoals when the goal state changes. On the other hand, some methods identify subgoals bypartitioning local state transition graphs. But this kind of methods require large calculation amounts. We propose a new
method that solves the above drawbacks. The new method utilizes state visiting frequencies. But we collect trajectories
that go through particular non-goal states selected at random. For each particular state, trajectories are collected. Mostof the trajectories reach the particular state more easily that the goal state. Therefore, it is expected that we can discoversubgoals quickly and discover multiple subgoals together.
Keywords: reinforcement learning, subgoal discovery, the state visiting frequency, the particular state

number of positive trajectories. On the other hand, somemethods utilize the transitions between states [5]. If thenumber of transition paths between two states is small,the learning environment can be cut between thosestates and they are judged as subgoals. However, thiskind of methods must count the transitions for allneighboring pairs of states, and judge if each count oftransitions is lower than the threshold. So they requirelarge calculation amounts.
In this paper, we propose a new method for finding
multiple subgoals. The purpose of this paper isdiscovery of multiple subgoals. In addition, we solve
the drawbacks of the existing methods. The new method
utilizes the state visiting frequencies. However, wecollect not only positive trajectories but also negative
ones that go through particular non-goal states chosen at
random. These trajectories can be collected faster andthus the subgoals can be found more quickly.
The structure of this paper is as follows. In Section 2
we explain subgoals. In Section 3 we introduce a new
method for finding multiple subgoals. Experiment of theuse of the new method and the results are written in
Section 4. And Section 5 gives conclusions.

I. Introduction
Reinforcement Learning is the method that decides
proper actions at each state by trial and error. The trial
and error method is effective when the environment iscomplicated or unknown. But learning by trial and error
takes a long time. So it is important to accelerate thelearning. There have been proposed several methods
that accelerate reinforcement learning. SymmetricalActions [1] utilizes symmetry of the environment, and Macro-

Actions [2] and Options [3] divide the environment to
reduce the learning problem size.
Dividing environment by subgoals is one of themethods that speed up the learning, since the number of
selections of states or actions reduces. To use thismethod, it is necessary to discover subgoals.
There have been proposed several methods forfinding subgoals. These methods utilize the feature ofsubgoals [4]. Some utilize the feature that the
trajectories that reach the goal state always go throughsubgoals. In this kind of methods, we collect only
positive trajectories (the trajectories that reach the goalstate) and ignore negative ones (the trajectories that do
not reach the goal state). When the state visiting
frequency, which is the value given by dividing thenumber of positive trajectories into the visiting countsof the state, is very high, it is judged that the state is thesubgoal. However, it takes a long time to collect a©ISAROB 2011

II. Subgoal Overview
Subgoals are the states that the agent must go
through before the task goal is reached. In

804

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

we can collect three trajectories including two negative
ones. Therefore, we can obtain enough data toaccurately calculate the visiting frequencies and the
doorway between two rooms will be selected as thesubgoal more quickly. Besides for each particular state,
trajectories are collected. So it is expected that multiple
subgoals are found.

reinforcement learning, if the environment is large andmany states or actions are involved, learning tends to be
slow owing to trial and error. So it is very effective for
reduction of the number of states or actions taken to
divide the task environment at subgoals. For example,
when an agent goes from one room to another room by
way of the doorway, it is very difficult that the agent
goes to another room with random actions. But it iseasier that the agent goes to the doorway. In this case,the agent must go through the doorway to reach anotherroom. So the doorway becomes a subgoal, which
divides the whole task into two smaller tasks.
The methods of Macro-Actions and of Options also
divide the environment for speeding up learning. They
can be readily constructed once subgoals have been
discovered. So finding subgoals is very useful.

III. A new method for finding
Fig.1 The trajectories that go througha particular state.

multiple subgoals
The conventional methods that utilize the state
visiting frequency for finding subgoals have several
drawbacks [4]. The goal state must be reached manytimes to count the visiting frequencies, and finding
subgoals takes a long time in large or complicatedenvironments. Besides, if the goal state changes,
subgoals that were found are not effective for the newsituation. In addition, because the trajectories go
through the states near the start state very often, thevisiting frequencies of these states may be high andthese states are judged as subgoals by mistake.
We propose a new method for finding multiple
subgoals that solves the above drawbacks. The new
method utilizes the state visiting frequencies. But in thismethod, the definition of the state visiting frequency isdifferent from the past definition in terms of the twopoints.
First, the way of collecting trajectories is different.
The agent takes random actions for several episodes andthen selects several particular states among the statesthat have already been visited. We collect the
trajectories that go through the particular state
regardless of whether the task goal is reached or not.
The state visiting counts are counted up for the states
between the start state and the particular state. Figure 1
shows the way of collecting trajectories. Here is one
positive trajectory, so we can collect only one trajectory
for the existing method. But if we use the new method,©ISAROB 2011

Second, in this method, the visiting counts are
divided by the average state visiting frequency. The
average state visiting frequency is the expected
frequency that the trajectory goes through the state
when the agent selects an action with equal probability
at any state. Figure 2 shows the average state visiting
frequency of each state around the start state. The
average visiting frequencies of the states around thestart state tend to be high. On the other hand, as the state
becomes far from the start states, the average state
visiting frequency becomes low. So dividing theobserved visiting frequency by the average state visiting
frequency reduces the seemingly high visiting counts ofthe states around the start state, and prevents that thesestates from being judged as subgoals by mistake

Fig.2 The average state frequency of each state
in a gridworld environment.

805

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

The visiting frequency
particular stateis defined byof state

the trajectories that go through the particular non-goalstates for 300 episodes. The fixed threshold 36 is used.

for the

The Algorithm of the new method is sketchedbelow:
1.
2.

3.

4.
5.

At the first several episodes, the agent selects
random actions at any states.
The agent selects several particular non-goalstates at random among the states it has alreadyvisited. In addition, if the goal state has beenreached during these episodes, the states
belonging to a positive trajectory and the goalstate are selected as the particular states,
because this leads to easy finding of theeffective subgoals.
The agent collects trajectories that go throughthe particular state while it learns the policy
and selects actions with Q-learning.
For each particular state, the state visiting
frequencies are calculated.
For each particular state, if the state visiting
frequency of a state is the maximum over all
the states on the way to the particular state andhigher than the threshold, the state is judged as
the subgoal.

Fig.3 The environment.

Fig.4 State visiting frequencies
for the particular state (13,4).

IV. Experiment
We test the new method for finding multiple
subgoals. Figure3 shows the environment used in theexperiment. It consists of three rooms. The start state
and the goal state are indicated by S and G, respectively,
in Fig.3. Besides, the horizontal axis and the verticalaxis are indicated by x and y, respectively, and the stateis represented by (x,y). The location of the upper
doorway is (3,10) and the right doorway is (10,5). The
agent can select from four actions: going up, down,right, and left. It uses Q-learning with ε-greedy with εthat is 0.3 at the first episode and is reduced by 0.001 at
every episode down to 0.1. The learning rate α in Qlearning is 0.1, and the discount rate γ is 0.9.
The agent takes random actions for the first 25
episodes. Then the agent uses Q-learning, and collects©ISAROB 2011

Fig.5 State visiting frequencies
for the particular state (5,11).

Fig.6 State visiting frequencies
for the particular state (2,5).

806

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Figure 4 through Fig.6 show the state visiting
frequencies. The lighter the state’s color is, the higher
the state visiting frequency is. In Fig.4 the particularstate is located in the right room where the goal state
locates. This particular state is situated near the rightdoorway, so collecting the trajectories that go throughthe particular state is easier than collecting the
trajectories that reach the goal state. In fact, the numberof the trajectories that go through the state (13,4) is 24,
which is higher than the number of positive trajectories
that is 15. Figure 4 shows that the visiting frequenciesof the states around the right doorway are high. The
maximum visiting frequency is the visiting frequency ofthe state (11,5) that is 48 and higher than the threshold.
So the state (11,5) is selected as the subgoal.
In Fig.5 the particular state is located in the upper
room and near the upper doorway. For this particularstate, the visiting frequencies of the states around theupper doorway are high. It is not useful to select one ofthese states as the subgoal in this environment. But for
example, when the goal state moves to the upper room,
the upper doorway will become the good subgoal. In
other words, even if the goal state moves to the upperroom, the agent will not need to repeat the discovery ofsubgoals. The maximum visiting frequency is thevisiting frequency of the state (3,11) that is 40 andhigher than the threshold. So the state (3,11) is selected
as the subgoal. Again, the number of the trajectories thatgo through the state (5,11) is 20, which is higher than
the number of positive trajectories that is 15.
In Fig.6 the particular state is located in the start
room. Because the trajectories often go through thisparticular state before the agent gets out the start room,the visiting frequency of the doorway state is very low,
and the visiting frequency of one of the states in thestart room is the maximum. But the visiting frequencies
overall are generally low. The maximum visiting
frequency is the visiting frequency of the state (3,4) thatis about 6.8 and lower than the threshold. So no state is
selected as the subgoal from the trajectories passing
through this particular state.

method can collect the trajectories that go through theparticular state more than the positive trajectories. So, it
can find subgoals more quickly. Second, it can find thestate that may become an effective subgoal when thegoal state changes. Third, the use of the average state
visiting frequencies suppresses the erroneous selectionof the states around the start state as the subgoals.
But our method has a drawback. The particular nongoal states are selected at random. If all the particularstates happen to be located in the start room, any
doorways will not be selected as the subgoals. To solve
the drawback, there need some devises, for example,
increasing the number of the particular states, or
selecting each particular state to be distant from otherparticular states.

References[1]
M.A.S.Kamal
and
J.Murata(2004),
Reinforcement Learning for High Dimensional Problem
with Symmetrical Actions. Proc. of 2004 IEEE Int. on
Systems, Man, and Cybernetic: 6192-6197.
[2] T. Tateyama, S. Kawata, T. Oguchi (2002),
Automatic Generation of Macro-Actions using Genetic
Algorithm for Reinforcement Learning, SICE2002
AUG5-7, Osaka.
[3] Martin Stolle and Doina Precup (2002), Learning
Options in Reinforcement Learning, SARA 2002, LNAI
2371, 213-223.
[4] R. Mattew Kretchmar, Todd Feli, Rohit Bansal(2003), Improved Automatic Discovery of Subgoals for
Options in Hierarchical Reinforcement Learning, Jornalof Computer Science & Technology, Vol.3-No.2.
[5] Ӧzgür Şimşek, Alicia P. Wolfe, Andrew G. Barto(2005), Identifying Useful Subgoals in Reinforcement
Learning by Local Graph Partitioning, the Twentysecond International Conference on Machine Learning,
816-823.
[6] Shie Mannor, Ishai Menache, Amit Hoze, Uri
Klein (2004), Dynamic Abstraction in Reinforcement
Learning via Clustering, twenty-first International
Conference on Machine Learning, 71-77.

VI. Conclusions
In this paper we propose a new method for finding
multiple subgoals that solves the existing drawbacks.
We tested it in the three-room gridworld environment.
The experiment shows that our new method cansolve the conventional methods’ drawbacks. First, the©ISAROB 2011

807

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Temporal difference approach in linearly-solvable Markov decision
problems
Burdelis, M.A.P. and Ikeda, K.
Nara Institute of Science and Technology, Grad. School of Information Science,
A606, 8916-5 Takayama, Ikoma, NARA, 630-0192, Japan(Tel : 81-743-72-5984; Fax : 81-743-72-5989)(mauricio-b@is.naist.jp)

Abstract: Todorov has recently introduced a class of linearly-solvable MDPs (LSMDPs) which greatly simplifies
reinforcement learning. By attending some specific conditions, the problem of choosing optimal actions (sequential
decision making) can become linear, and then be solved in closed-form. A similar method to temporal difference
learning (TD learning) for this class of MDPs has also been introduced, and is called Z-learning. In this work we
present the results of simulations using Z-learning to solve a navigation problem of a virtual agent in a grid world, inwhich physical properties of the system (Newtonian mechanics) were introduced in the MDP model in the definition of
the passive dynamics probability distribution, which is crucial in the theory. Those MDPs were solved both in closedform and using Z-learning. In all experiments, the approximation errors of Z-learning consistently decreased with theincrease in the number of simulation steps.
Keywords: Linear Bellman Equation; Reinforcement Learning.

I. INTRODUCTION

II. LINEARLY-SOLVABLE MARKOV
DECISION PROCESSES (LSMDP)

The Reinforcement learning (RL) approach to
Machine Learning is a technique to learn how to makedecisions in order to achieve a desired goal. In RL, themodel does not include the presence of a supervisor, andthe agent must learn by trial and error, interacting withthe environment and observing a reward (or cost) signal[1]. Examples of possible applications of RL include:
playing board games like chess, checkers or “tic-tactoe”; and a nervous system generating muscle
activations to maximize movement performance [2].
RL problems are usually defined on a discrete-timeMarkov decision process (MDP) with stochastic
dynamics. Recent work [3] has presented a class of
linearly-solvable MDPs, which greatly simplify thesolution of reinforcement learning problems. Z-learning
is a temporal-difference approach to the RL problem,
which takes advantage of this class of MDPs, andpresents faster convergence than traditional RL methods(e.g. Q-learning) [3] [2]. The present work has the
motivation of including Newtonian mechanics effects(inertia and collisions) in the passive dynamics
probability distribution of the linearly-solvable MDP
used for Z-learning.

©ISAROB 2011

Let us denote the state of the environment in a
discrete time instant t as xt. If this state signal retains all
relevant information for the decision, it is said to havethe “Markov property”, andPr( xt +1 | xt ) = Pr( xt +1 | xt , xt −1...) .

(1)

This means that the history of past states has noinfluence on the probability of the next state. A
reinforcement learning task that satisfies the Markov
property is called a Markov decision process (MDP) [1][4].
A particular finite MDP is defined by: a set of
possible states X, a set of possible actions U, statetransition probabilities p(xt+1|xt, ut) (which mean theprobability that the next state is state xt+1 when thecurrent state is state xt and action ut is taken), andimmediate cost l(xt+1|xt, ut) for being at state xt, taking
action ut and having a transition to state xt+1.
The reinforcement learning agent’s sole objective is
to minimize the total accumulated cost it receives in thelong run [1] [2]. The “cost-to-go” function (or “value”
function) of a state (denoted v(x)) is defined as the
expected total cost the agent accumulates starting fromthat state, and following the optimal policy thereafter.

808

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

III. Z-LEARNING

The cost-to-go (or “value”) function is the only
solution to its Bellman equation [1] (refer to section III
for details). This solution can be obtained by using
Dynamic Programming (DP), but this can be timeconsuming due to the explosion of unknown variables.
Recent work [3] has introduced a class of MDPs
which greatly simplifies reinforcement learning. By
attending some specific conditions, the Bellman
equation of the MDP becomes linear, and its solution
can be obtained in closed-form. The conditions are as
follows [2]:In the original MDP framework, the agent specifies
discrete actions u ∈ U. The probabilities of statetransitions starting from the current state x ∈ X dependon the action u taken at that state. In other words, theprobability of transition from a state x ∈ X to a statex’ ∈ X is given by p(x’|x, u).
In LSMDPs, however, the agent can specify thetransition probabilities directly. In other words, there are
no discrete actions u nor the set of actions U. Instead,the agent can directly specify the probability oftransition from the current state x ∈ X to any possible
future state x’ ∈ X, without the existence of discrete
actions. These probabilities will be represented by using
the letter u and therefore p(x’|x, u) becomes u(x’|x) [2].
Another necessary condition is the definition of aprobability distribution called “passive dynamics”,
denoted pd(x’|x), which corresponds to the behavior of
the system in the absence of controls [2].
The total cost incurred in a state transition will bedefined as follows [2]:l ( x, u ) = q ( x) + KL(u (⋅ | x) || pd (⋅ | x) )When the passive dynamics distribution is known,
as well as all the states and their respective state costs,
then the problem of obtaining the cost-to-go functionv(x) can be solved using dynamic programming and the
Bellman equation. The Bellman equation expresses therelationship between the cost-to-go of state and the
expected cost-to-go of the next state:v( x) = min{l ( x, u ) + E x '~ p (⋅| x ,u ) [v( x' )]}where

E x '~ p (⋅| x ,u ) [v ( x' )] ≡ ∑ p ( x '| x, u )v ( x ' ) .

(5)

x'

In LSMDPs, the Bellman equation can be expressedas [2]:z ( x ) = e (− q ( x ) ) ∑ p ( x '| x ) z ( x' ) ,

(6)

x'

where

z ( x ) ≡ e (− v ( x ) )(7)

is called the “desirability function” of a state. This
Bellman equation (6) is linear in z.
Equation (6) can be written in vector notation, by
enumerating the states from 1 to n, representing z(x) andq(x) as column vectors z and q, and representing p(x’|x)
as a matrix P (where the row-index corresponds to x andthe column-index corresponds to x’) [2]. By partitioningz, q and P according to terminal and non-terminal states(using index N for non-terminal and T for terminal),
equation (6) becomes [2]:(2)Where q(x) is called “state-cost” (which depends
only on the state, therefore representing how
undesirable a state is) and(diag
diag (exp(q N )) − PNN )z N = PNT exp(−qT )u ( x '| x ) KL(u (⋅ | x) || p(⋅ | x) ) ≡ E x '~u (⋅|x ) logKL (3)p d ( x '| x ) (8)where “diag” transforms vectors into diagonal
matrices. By acknowledging that v(x) = q(x) at terminal
states, the unknown zN (vector of desirabilities at thenon-terminal states) can be calculated by using matrix
factorization or an iterative linear solver [2].
When the state costs and passive dynamics are notknown, then simulations must be made in order to solve
the problem of estimating the cost-to-go function byusing reinforcement learning.
Z-learning is a temporal-difference-like method
which takes advantage of the linear class of MDPs tois the Kullback–Leibler (KL) divergence between
the controlled state transition distribution and thepassive dynamics. This measures how “different” these
distributions are from one another, and is called “actioncost” [2].
The last condition is that u(x’|x)=0 when pd(x’|x)=0,
in order to keep the KL divergence well-defined andavoid impossible state transitions [2].

©ISAROB 2011

(4)

u

809

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

achieve
fasterconvergencethentraditional
reinforcement learning methods [2].
As in traditional TD learning methods, initial
estimates are constantly updated until convergence isobtained. The Z-learning update formula is as follows[2]:
zupd ( xt ) ← (1 − η t ) zcur ( xt ) + η t exp(−qt ) z cur ( xt +1 )(9)

Where zupd(xt) is the updated estimate of z(xt), zcur(xt)is the current estimate of z(xt), zcur(xt+1) is the current
estimate of z(xt+1), qt is the state cost of state xt, and ηt is
a learning rate which decreases over time.
Fig.1. A two-dimensional “grid world” (size 10x10)

IV. MODELING NEWTONIAN MECHANICSEFFECTSIn the present work two Newtonian mechanics
effects were modeled in the passive dynamics
probability distribution (which is usually defined simply
as “random walk”): inertia and collisions. These
Newtonian mechanics effects were modeled andsimulated in a 2-dimensional “grid-world” (10x10 size,with obstacles - Fig.1). The goal of the agent is to findthe goal position, while avoiding the obstacles on thepath.
Initially, a few difficulties have emerged frommodeling the state signal in the grid world as the
position of the learning agent. In order to model inertia,
it is necessary to know the current and the previous
position of the agent. If the state signal contains only the
information on the current position, then it is necessary
to know the current and the previous state in order tomodel the passive dynamics, and this would violate the“Markov property” (1).
Also, in order to model collisions with obstacles andwalls, the agent would originally need to be able to takediscrete actions of movement in the direction of the wall
or obstacle, and this would violate one of the necessary
conditions to have a LSMDP.
In order to overcome these two difficulties, the statesignal was modeled to include information of position
pairs: the current and the previous position.
A simple model of inertia was created by assigning a
pre-defined “highest probability” value (denoted hp) to
the passive dynamics probability of the next position in
the current trajectory of the agent. The remaining
adjacent non-obstacle positions equally share theprobability (1-hp) (Fig. 2).

©ISAROB 2011

Fig.2. Modeling inertia (with hp = 0.9)
Two types of walls and obstacles were considered inorder to model collisions: reflexive walls and obstacles(which reflect the normal components of impacts); andabsorptive walls and obstacles (which absorb the normalcomponents of impacts). The passive dynamics
probability distribution is updated based on the current
position and the previous position of the agent. Whenthe agent’s position is adjacent to a wall or obstacle, andthe most probable trajectory (driven by inertia) is in the
direction of the wall or obstacle, then the position thatreceives the highest probability value hp is the most
likely position after a collision with this wall or obstacle,
and the remaining possible positions share theremaining (1-hp), as shown in Fig.3

Fig.3. Collisions with a reflexive and an absorptive wall(with hp = 0.9)810

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

V. COMPUTATIONAL EXPERIMENTS

Absorptive walls and obstacles

The grid world shown in Fig.1 was solved
analytically (using equation (8) [2]) and also using Zlearning. The state cost of every state (except for thegoal state) is 1 (in order to penalize the agent for
producing long trajectories) and 0 only at the goal state.
Both reflexive and absorptive walls and obstacles wereconsidered. In Z-learning, three different policies wereused: policy equal to the passive dynamics; random
walk policy; and ε-greedy policy (with ε=0.2). Whenthe applied policy differs from the passive dynamics
distribution, importance sampling is required [2]: thelast term in equation (9) needs to be multiplied bypd(xt+1|xt)/û(xt+1|xt), where û is the applied policy – inthis case it is necessary to know the passive dynamics
distribution pd.
In order to measure the quality of the approximationof the cost-to-go function estimates produced by Zlearning, the approximation error (comparing theestimates with the correct values obtained in closedform) was calculated using the formula:N

Error = ∑ v( xi ) − v * ( xi )i =1

Fig.5. Z-learning approximation errors(absorptive walls and obstacles)
Reflexive walls and obstaclesN∑ v (x )
*

i

(10)

Fig.6. Z-learning approximation errors(reflexive walls and obstacles)

i =1

where xi represents the ith state; N is the total numberof states; v(x) is the current approximation of the costto-go function (obtained by Z-learning) and v*(x) is the
optimal cost-to-go function obtained analytically.
The results of the simulations can be seen in Fig.5
and Fig.6. In all experiments the learning rate ηt decays
obeying the formula ηt=c/(c+t) where t is the time stepand c is a constant which was different for each case.
The results for both models (the model with absorptive
walls and obstacles; and the model with reflexive walls
and obstacles) are very similar, indicating that theframework is considerably robust. When sampling fromthe passive dynamics, the error takes more steps toconverge, because the inclusion of inertia in the passive
dynamics makes this distribution less exploratory thanrandom walk. The other two policies (random walk andε-greedy) presented faster convergence, but their use
requires access to pd. In all cases the Z-learning
approximation errors consistently decrease as thenumber of simulation steps increases.

©ISAROB 2011

VI. CONCLUSION
The flexible framework of linearly-solvable MDPs
allowed for simple models of Newtonian mechanics
effects to be simulated in the passive dynamics
probability distribution. The cost-to-go function of theresulting model was obtained both in closed-form andusing Z-learning. The Z-learning algorithm was able toapproximate the correct cost-to-go function, presenting
errors which consistently decreased with the increase in
the number of simulation steps.

REFERENCES[1] Sutton R S, Barto A G (1998). Reinforcement
learning: An introduction. MIT Press.
[2] Todorov E. (2009) Efficient computation of optimal
actions. Proc Natl Acad Sci USA 106:11478–11483.
[3] Todorov E (2006) Linearly-solvable Markov
decision problems. Adv Neural Informatin Proc Syst
19:1369–1376.
[4] Iwata K, Ikeda K, Sakai H (2006). The asymptotic
equipartition property in reinforcement learning and its
relation to return maximization. Neural Networks 19(1),62–75.

811

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Mutual learning of multi consciousness agent including the ego forautonomous vehicle
Mori Atsunori, Matsui Hirokazu
Faculty of Engineering Mie Univ., Kurimamachiya1577,Z-city,Mie-pref,Japan(Tel : 81-59-231-9802)(mori@robot.mach.mie-u.ac.jp)

Abstract: In this report, we propose a multi agent learning system for an intelligent robot control, based on the model ofthe human consciousnesses with including the ego. We pay attention to an intelligent learning process of human beings.
We try to give high ability of learning to a robot by modeling roles of the human consciousnesses with including theego. In almost ordinary methods, the instructions for learning are given only from outside of system. In the proposed
method, the instructions are given not only from outside but also inside (from other agents in the system). Therefore, arobot can learn efficiently, since it uses more instructions than the ordinaries. The learning is more flexible, since agents
learn while a learning agent and instructing agents exchange the roles by changing of environment. We verify
experimentally that the proposed method is efficient by using an actual robot.
Keywords: Machine learning, Autonomous mobile robot, Multi-Agent system, Ego, Consciousness

thought that the ego is the subject of consciousness. We
model them for engineering based on the above
psychological definition.
There are ordinary researches for the model of the
consciousness. Minsky describes that a human
intelligent behavior has been achieved by the interactionof hierarchized mind agents. Maeno proposed
consciousness model that controlled by the ego for
episode memory.
However, there is no general model for theconsciousness including the ego. There are a few
researches about above-mentioned. In less cases, humanconsciousnesses were actually applied to robot control.
We define the consciousness including the ego as multiagent system, and try to apply to mechanical learning.
There are the following as a research of multi-agent
reinforcement learning. Wiering[1], Osada[2] improved
the learning efficiency by hierarchically arranging anagents and setting an sub-goal in each hierarchy. Ono[3],Fujita[4] learns efficiently by limiting the range of astate in Q-leaning used the partial state composed of anagent and another agent. In this report, we proposed amethod that is modeled as multi-agent system based on
the consciousness including the ego without thehierarchy, and verity that the proposed method isefficient for an actual robot by experiment.

I. Introduction
Autonomous robots are increasing such as cleaning
robots and guard robots. It is requested that robots has
more intelligence. Even if a human encounters acomplex situation, he or she can adjust to the situationby learning. We pay attention to an intelligent learning
process of human beings. Human beings has a featurethat he/she can learn an environment while evaluating
his/her decision by the ego. We try to give high abilityof learning to robots by modeling the feature.
We think about the consciousness and the ego. But,
none knows the mechanism of the consciousnesses andthe ego yet. Moreover, the words are defined variously
in various fields. For example, the words Consciousness
and Unconsciousness are defined in each field as
followings.
1. Psychology: Plural consciousness exists in a body.
Unconsciousness is consciousness without the ego.
2. Philosophy: Consciousness is an ability to recognize.
Unconsciousness is an ability to recognize oneself
objectively.
3. Medicine: Consciousness is state that can feelstimulation to senses by activate working of a brain.
Unconsciousness is state that cannot do them.
4. Upanishad philosophy: Consciousness are classified
into 4 states. One is Unconsciousness.
In another field, they are defined as another. We takethe idea of German psychologist Wundt. He analyzed
human consciousness by internal perception, and©ISAROB 2011

812

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

II. Proposed Model(8) When an agent learns, it uses recognition results ofother agent.

1. A definition of the consciousness and an ego
We define and model the human consciousness and theego as follows. The model is regarded by the internal
perception, and does not contradict the anatomy.
(1)Number of consciousness is more than two in abody.
(2)The human operation is caused by cooperating andcompeting of some consciousness.
(3)The ego is only one.
(4)The ego exists on the consciousness with thehighest priority.
(5)When the consciousness with the ego competes
against another in resources, it uses the resources bypriority, the another is stopped.
(6)Only the consciousness with the ego can learn.
(7)The consciousness with the ego learns with using
results of other consciousnesses recognition.

III. Experiment
We verify experimentally that the proposed method isefficient. We select an image sensing and a bodysensing for a mobile robot as agents that instruct andlearn each other. Each agent estimates the current poseof the robot from each sensing data.
We use an actual mobile robot (MieC) for theexperiment (Fig2).

2. Modeling of a consciousness and an ego
We treat the above-mentioned consciousnesses rules asmulti agent system (Fig.1).

Fig.2 Autonomous mobile robot MieC
The robot MieC has a camera as vision sensor, 4 touch
switches as touch sensor and two crawlers as movable device.
The robot MieC is on a horizontal plane enclosed byvertical planes. Borders are between the horizontal
plane and the vertical planes. Image sensing agent andbody sensing agent find the distance from the robot to aborder, respectively. Image sensing agent finds a borderon an image by an appropriate threshold, and estimates
3-D pose of a border in the environment bytransformation from image coordinate to robot
coordinate. Body sensing agent finds the border under avertical plane, if the robot MieC touches the plane.
Otherwise, body sensing agent estimates the 3-D poseof a border by dead-reckoning with appropriate
parameters.
In the future, each agent will be full built by the
proposed framework, but currently, the Image sensing
agent adjusts only the threshold, the body sensing agent
adjusts only the dead-reckoning parameters.
The less accurately each agent recognizes the pose of aborder, the higher priority it gets. For actual experiment,
moving agent is added to the other agents image sensing
and body sensing, but the agent doesn't learn nor
instruct.

Fig.1 Proposed model(1)Each consciousness is treated as agent.
(2) An agent is a data processer or a sensor with apriority and a condition for the running.
(3) If an environment satisfies a condition for therunning, an agent executes processing.
(4) An agent has higher priority, if the agent is satisfied
with the condition.
(5) An agent receives instructions and the priority fromother agents.
(6) An agent uses only the instruction with the highest
priority, when the instructions from other agentscompete with each other.
(7) Only an agent with high priority can learn.

©ISAROB 2011

1. Learning in a case of clear recognition for image
sensing agent
The image sensing agent could recognize a border in asituation (Fig.3 (b)). In other words, the image sensing

813

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

agent usually has lower priority than the body sensing
agent. In the initial state, moving agent has higher
priority than image sensing agent and body sensing
agent. The body sensing agent doesn't know the pose ofa border. The robot MieC might touch a vertical plane
according to moving agent (Fig.3 (c)). And then the
body sensing agent has higher priority than the others.
The moving agent is stopped, and the body sensing
agent is waked up and learns appropriate parametersfrom information of the image sensing agent (Fig.3 (d)).
Concretely, the body sensing agent adjusts the deadreckoning parameters while it is estimating the robot
MieC self pose from the border pose found by theimage sensing agent.

And then the image sensing agent has higher priority
than the others. The moving agent is stopped, and theimage sensing agent is waked up and learns appropriate
threshold from information of the body sensing agent
with already adjusted dead-reckoning parameters.
Concretely, the image sensing agent adjusts thethreshold so that it can find the border at the border pose
estimated by the body sensing agent (Fig.4 (g) (h)).

(e)

(a)
(f)

(g)

(b)

(c)
(h)
Fig.4 Learning of the image sensing agent
We experimented in the environment such as (Fig.5).
In the left side (Fig.5), the image sensing agent canrecognize the border easily, since the colors of thevertical and the horizontal planes are different clearly.
In the upper side (Fig.5), it can hardly recognize theborder, since the colors are similar. The robot MieC
learned in the sequence of two above-mentioned
environments.
By the above-mentions, we confirmed that two agents
could learn cooperatively by the environment, and the
proposed method was effective. (Fig.6) shows the poseof the robot MieC in 2-D coordinate system. (Fig.7)(d)
Fig.3 Learning of the body sensing agent
2. Learning in a case of hardly recognition for image
sensing agent
The image sensing agent could not recognize a borderin another situation (Fig.4 (e)). In other words, theimage sensing agent usually has higher priority than the
body sensing agent. The robot MieC might touch again
a vertical plane according to moving agent (Fig.4 (f)).

©ISAROB 2011

814

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011shows learning and instructing state of the body sensing
and the image sensing agents. We can confirm thefollowing from (Fig.6) and (Fig.7). In the beginning, thebody sensing agent was a learning agent, and learned
from the image sensing agent as instructing agent. And
then, the image sensing agent was a learning agent, andlearned from the body sensing agent as instructing agent.

developed in our lab.. Concretely, in the experiment, weselected two agents, the image sensing agent and the
body sensing agent, and confirmed that the agentsexchanged the roles of learner and instructor each otherto change the environment by MieC’s movement.
We expect that the proposed method can learn by lesstrial frequency than ordinary reinforcement learning,since the proposed method uses instruction not only
from the outside but also from the inside. But currently,
the proposed method has been programmed for only apart of environment. In the future, the proposed method
is applied to larger environment to increase agent types,
and to select an agent by environment, automatically.
Concretely, we prepare some templates of an agent.
When a robot cannot adjust an environment, an agent isgenerated by selecting a template automatically. In theinitial state, the agent’s parameter is given at random.
The agent adjusts parameter by learning. If it’s cannot
learning, a new agent is made by selecting anothertemplate. An agent is automatically generated byenvironment like the above-mentioned, and robot may
be able to adjust to a complex situation.

Fig.5 Environment of experiment

REFERENCES[1] M.Wiering and J.Schmidhuver (1997), HQ-learning.
Adaptive Behavior 6(2):219-246[2] H.Osada and S.Fujita (2005), CHQ:A Multi-Agent
Reinforcement Learning Scheme for PartiallyObservable Markov Decision Processes. IEICE Trans.
Inf. & Syst., Vol. e88-D No. 5:1004-1011[3] N.Ono O.Okeda (1996), Synthesis of Herding andSpecialized Behavior by Modular Q-learning Animats.
ALIFE V Poster Presentations:26-30[4] K.Fujita and H.Matsuno (2005), Multi-Agent
Reinforcement with the Partly High-Dimensional StateSpace (in Japanese). IEICE D-I Vol. j88-D-I No. 4:864872

Fig.6 Result of agent’s stateFig.7 Result of robot’s pose

IV. Conclusion
We proposed a learning model based on a mechanismof human learning that consists of consciousnesses
including the ego defined by the psychology. We
applied the proposed model to the actual robot MieC

©ISAROB 2011815

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Reinforced learning by using a learned results of a different form robot
Nobuo Shibata, Hirokazu Matsui
Kurimamachiya 1577 Tsu Mie(Tel : 81-59-231-9802)(shibata@eds.elec.mie-u.ac.jp)
Abstract: The reduction on the trial frequency is important for reinforcement learning under an actual environment. In
this report, we propose a method that can learn an unknown environment efficiently by applying an already learned Qtable to another new learning. Concretely, an agent (Target agent) learns efficiently by using Q-table that a different
from agent (Source agent) has learned at the same environment. The proposed method uses the two Q-tables of Source
agent and Target agent and an action-translation-table (AT-table). The Q-table of Source agent has already been learnedat the same environment. The AT-table is learned for the suitable mapping of actions between Source and Target agentsby information of the stored state transition probabilities when Source agent learned. Concretely, when Target agent
takes an action at a situation, the AT-table is learned so which action of Source agent is similar to the action of Target
agent. We think that Target agent can learn the suitable action efficiently by using the previous proposed method (forlearning efficiently by using dual Q-tables) with Target Q-table and the set of Source Q-table and the AT-table insteadof dual Q-tables. We verify that the proposed method is effective in a simulation.
Keywords: Reinforcement learningfor experience storing and for knowledge storing. We
thought experience and knowledge as followings. An
experience is a collection of instances in an environment, so we use the whole space of the environment for
the Q-table of experience. A knowledge is a compression of instances in an environment, so we use a compressed space (partial space) of the environment for the
Q-table of knowledge. The smaller Q-table (for theknowledge storing) makes the learning finish earlier.
The larger Q-table (for the experience storing) makes
the learning be in more detail.
The dual learning method consists of repeated steps,
the first step is that the action is selected by using the Qtable with lower information entropy, the second is thatthe two Q-tables are updated at the same time by theaction. This research aims at the method that learns anenvironment earlier and in more detail with using dual
Q-tables.

I. INTRODUCTION
The reduction on the trial frequency is important for
reinforcement learning. We propose an efficient learning
method for robots by using knowledge.
When human being walks, he stores them as knowledge, the relations between moving and change of viewing. When he drives a car, he learns efficiently the steering operation by using the stored knowledge in walking.
But, he cannot drive a car by only using the knowledge,he must learn new change of viewing, that never happens in walking. We apply an efficient learning methodof human being to the proposed learning method for
robots. In ordinary research, to reuse the knowledge, alearning task is separated into the smaller tasks[1], or
using multi-layered reinforcement learning system[2].
In this report, we propose an efficient learning methodthat reuses the knowledge learned by another robot
without transformation of learning modules. The proposed method is an extended method from our previous
proposed method "Reinforcement Learning with SelfInstruction by using dual Q-tables"[3]. We verify thatthe proposed method is effective in a simulation.

Fig.1. Q-tables for the partial space and the whole space

2. Materialization
Here, we materialize the above-mentioned concept
to the procedures, as shown in Fig.2.
(1) Init dual Q-tables
The agent initializes the two Q-tables for the partial
and for the whole Q-table.
(2) Select Q-table
The agent selects the Q-table with lower information
entropy H(s) , out of the partial and the whole Q-table,that is driven by Eq.1.

II. LEARNING WITH DUAL Q-TABLES
Here, we describe that the previous proposed
method(the dual learning method)[3] can learn the environment, earlier and in more detail by using the dual Qtables.
1. Algorithm
The dual learning method uses two Q-tables (Fig.1),©ISAROB 2011

816

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

H (s ) = ∑ p(a | s )log 2
a∈A1
p(a | s )(1)p(a | s ) is a probability of selecting action
a at state s , that is defined by Eq.2. We consider thatwhere

the Q-table to be so effective, that the information entropy of Q-table is lower.
(3) Select Action (for both Q-tables)
The agent selects an action by the Boltzmann selection used generally in Q-learning. The selection probability of the action a is shown by Eq.2.
 Q(s k , a) p(a | s k ) = exp T Q(s k , a' ) T∑ exp

(a)100% effective（2）

1. Algorithm
In the proposed method, Target agent has four tables,
two information tables and two learning tables. One
information table is the already stored Q-table forSource agent. The other is the already stored Environment table(Env-table), that is the probability table ofstate transition by the state-action of Source agent. One
learning table is an action-translation-table(AT-Table)
for associating the actions of Source agent with the actions of Target agent. The other is a new Q-table forstate and action of Target agent.
In the proposed method, the action of Target agent
by a state is selected in two ways. In one way, the action
for Source by the state is selected by using alreadystored Q-table for Source agent, and then the action for
Source is translated to the action for Target by using ATTable. By these steps, the action for Target agent by thestate is selected. In this way, the learning space cannot
be expressed perfectly for Target, but it is small. In theother way, the action by the state is selected directly byusing a Q-table only for Target agent.
We apply the dual learning method mathematicallyto the learning of the AT-table and Target Q-table assmaller Q-table and larger Q-table. We expect as similar
to the dual learning, that only by the first way Target
agent learn more roughly and earlier, only by the second
way it learn later and in more detail. But applying theprevious, Target agent can learn earlier in more detail.
In the proposed method, in order to learn more earlier,the agent can be evaluated by each action step, adding
an evaluation by the Env-table to the first way.

p(a | s k ) is probability of selecting action
a on state sk , k is times, T is temperature.
(4) Update Q-table (for both Q-tables)
Each Q-value is updated by Eq.3. The equation isused generally when updating Q-value.

)

Q(sk , a k ) ← Q(sk , ak ) + α (r + γ max Q(sk +1 , a ) − Q(sk , ak ) (3)ak is time, sk is state at time k .
Q( s, a ) is Q-value at state s and action a .
r (s, a ) is reward decided by each pair of s and a .
α is learning rate and γ is discount rate.
where,

3. Result of simulation
We show results of average of the 1000 trials simulation in Fig.3. Fig.3(a) is a result in the case that partial
Q-table is 100% effective, In other words, the statescomponent of partial Q-table can distinguish the neededstates by 100%. The case of Fig.3(b) and the (c) are
50% and 0% effective, respectively. The ordinary
method uses only the partial Q-table or only the wholeQ-table. As the results of (a)(b)(c) three cases, the dual
learning method is not less effective than the ordinaries
in any case.

III. PROPOSED METHOD
We propose a learning method that reuses the already stored Q-table and stored Environment table foranother agent with different structure by extending thedual learning method. We call the previous learning
Init Q-tables (1)Episode Loop
Select Q-table (2)WholePartial
Select Action by
Select Action byPartial Q-table (3p)
Whole Q-table (3w)

2. Materialization
Here, we materialize the concept in the abovementioned to the procedures of the proposed method, as
shown in Fig.5.
(a) Leaning of the Source Agent (the previous learning)
Source agent learns a suitable action at the environment and stores the state transition probabilities to Env-

Update Partial Q-table(4p)Update Whole Q-table (4w)
Until getting reward

Fig.2. NS chart of the previous proposed method©ISAROB 2011(c)0%effective

Fig.3. Results of computer simulation
agent, Source agent, and the current learning agent, Target agent. For the proposed method, we assume that theagent(Target) can access a result, that anotheragent(Source) with a different structure, learned in thesame environment for the same goal.

where

(

(b)50% effective

817

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

table at each combination of action, current state andnext state. The Env-table is shown in Fig.6(a).
(b) Learning of the Target Agent(the main learning)(1) Init Target Q-table and Load Source Q-table andEnv-table
Target agent initializes the Q-table of Target agent to
set each Q-value to an init value and load the AT-table
and the Env-table from the previous learned data. The
values of the AT-table and the Env-table are fixed(shown by the hatching areas in Fig.4(b)).
(2) Select way
Target agent selects the way with lower information
entropy H way1 or H way 2 , out of the two ways, that area t ∈ At1
H source − Q (s ) = ∑ p (as | s )log 2
p (as | s )a s ∈ Ass (selected by using Boltzmann selection with Source
Q-table).
(3) Select Action (for both Q-tables)
Target agent selects an action by the Boltzmann selection. The selection probability of action a is shown
by Eq.3.
(4) Action Translation by AT-table
Action a s selected by Source agent Q-table isLearning Q-table ofSource Agent (SA)translated to action a t by using the AT-table. Action

Suitable action for SA

a t corresponding to action a s is selected by the
Boltzmann selection with AT-table. The selection probability of action a t is driven by Eq.9.

Current State

way2
Learned Q-table ofSource Agent (SA)

Learning Q-table ofTarget Agent (TA)

Suitable action
for SA
Training Data(8)

table. We consider that the way to be so effective, thatthe information entropy of the way is lower. Function
as (s ) in Eq.7 is represented for that as is decided by(a) Leaning of the Source AgentStored
Env-Table

H way 2 (s ) = H t arg et − Q (s )p(a | s ) is probability of selecting action a at state s , that is defined by Eq.2.
p(at | as ) is probability of selecting action a t at
already selected action a s , that is stored in the AT-

(5)

way1

(7)

tion of Source agent,

Current State

Storing
Environment Table

H way1 (s ) = H source − Q (s ) + H AT − table (as (s ))where a t is an action of Target agent, a s is ac-(4)1
p (at | s )(6)at ∈ At

driven by Eq.4, Eq.5, Eq.6, Eq.7 and Eq.8.
H t arg et − Q (s ) = ∑ p (at | s )log 21p(at | a s )H AT −table (as ) = ∑ p(at | a s )log 2

 AT(a sk , a t ) 
p a t | a sk = exp

T AT



(

Suitable action
for TA

Associating
AT-table) AT(a sk , a t ) 


T AT



∑ exp

(9)where p(at | as ) is probability of selecting action

Suitable action
for TAk

Select way

a t at already selected action as , k is times, T AT

Suitable action for TA

is temperature.
(5) Update AT-table
The AT-table is learned for the associating of actions
between Source and Target agents by Env-table. ATtable is updated by Eq.10.

k(b) Learning of the Target Agent
Fig.4. Block diagram of the proposed method
Init Source Q-table (1)Episode Loop
Select Action by Source Q-table (2)

(

) (

(

)

(

AT as k , at k ← α AT AT as k , at k + γ AT p sk +1 | sk , as k

Update Source Q-table (3))) (10)Store Env-table (4)where p(sk+1 | sk ,as ) is state transition probability to

Until getting reward(a) Leaning of the Source Agent

k

state

Init Target Q-table and Load Source Q-table and Env-table(1)Episode Loop

sk +1 by pair of action as and state sk ,kα AT and γ AT are such that 0 ≤ α AT < 1 and 0 ≤ γ AT < 1 .
(6)Update Q-table
Each Q-value is updated by Eq.3.

Select way (2)
way1
way2
Select Action by
Select Action by
Source Q-table (3.1)
Target Q-table (3.2)
Action transformation by
AT-table (4)Update AT-table by
Env-table(5)Current
state

Update Target Q-table (6)ActionUntil getting reward

Action of
Source agent(a) Env-table(b) AT-table
Fig.6. Env-table and AT-table(b) Learning of the Target Agent
Fig.5. NS chart of the proposed method©ISAROB 2011

Action ofTarget agent

Next
state

818

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

IV. SIMULATIONThe reward r is set to 1.0, the learning rate α is set

In this chapter, we verify that the proposed method
is effective in a simulation.

to 0.4, and the discount rate γ is set to 0.9, the temperature T of Boltzmann selection is set to 0.05,
α AT is set to 0.9, γ AT is set to 0.35, T AT is set to 0.5.
In Table.1, high value is represented for correct
mapping between Source and Target action pair. For
example, “Forward Right” of Target agent allocated to“Pivot turn Right” of Source agent. So in Fig.9, themethod with Source agent Q-table and AT-table (way1)
can learn earlier than the method with Target agent Qtable (way2). But way1 cannot learn in detail since thepair of Q-table of Source agent and AT-table cannot
express all the corresponding action to Target agent.
Way2 can learn in detail but needs a lot of time. However, the proposed method (way1 and way2) can learn
the environment earlier and in more detail by using
Source and Target Q-table and AT-table.
Table 1. Result of AT-table

1. Environment
Experiment environment is shown in Fig.7. It consists of an agent, a goal and an object in afield(100[cm]x50[cm]). In the experiment, the agent
learns the task to carry the object to the goal area, given
reward at the case that the object carried into the goal.
We define an episode to be time until getting reward.
The initial pose of the agent, the object and the goal are
shown in Fig.7. An initial pose of agent is selected randomly from two pose shown in Fig.7 at each episode.
The pose of agent is changed 0.5[cm] and 0.1[deg] permove respectively. We define one step to be time until
the current state is changed to next state. The agent
keeps taking the same action until the current state ischanged.

SourceTargetForwardBackward
F Right
F LeftB Right
B Left2. Action set and State set
Action set: The actions of Source and Target agentsare shown in Fig.8. Source agent has 4 actions “go forward”, “go backward”, “pivot turn right” and “pivot
turn left” and Target agent has 6 actions “go forward”,“go backward”, “turn right (forward)”, “turn left (forward)”, “turn right (backward)” and “turn right (backward)”.
State set: A state set consists of the position states ofgoal and object. The state set has 160 states that areeach position and size of the object and the goal. The
object state has 6 states, combinations of positions(left,
center, or right) and sizes(large or small). The goal statehas 18 states, combination of positions(left, center, orright), sizes(large or small) and directions(left, front,
right). In addition to these 78 states, we add states that
only the object or only the goal view or lose.

0

500

1000

1500

2000

2500

3000

Steps until reward

EpisodeFig.9. Results of computer simulation

V. CONCLUSION
We verified that the proposed method can learn
more efficient than the MTQ and MSQAT in simulation.
In the future, we will apply the proposed method toan actual environment and verify that the proposed
method can learn efficiently in an actual environment.

REFERENCES[1] Akihiko Yamaguchi,Norikazu Sugimoto,Mitsuo
Kawato(2009):Reinforcement Learning with Reusing
Mechanism of Avoidance Actions and its Application to
Learning Whole-Body Motions of Multi-Link Robot(inJapanese), Journal of the Robotics Society of Japan
Vol.27 No.2,pp.209-220[2] Yasutake Takahashi,Minoru Asada(2003):StateAction Space Construction for Multi-Layered Learning
System(in Japanese),Journal of the Robotics Society ofJapan Vol.21 No.2,pp.164-171[3] Osamu NISHIMURA,Hirokazu MATSUI,Chieko
HIOKI,et al(2006):Reinforcement Learning with SelfInstruction by using dual Q-tables,AROB 11th

Goal

Fig.7. Experiment environment
Pivot Turn Right
Backward
Left

ForwardLeft

Backward

ForwardBackward
Right

ForwardRight

Pivot Turn Left(a) Source agent(b) Target agent
Fig.8. Actions of the Source and Target agents©ISAROB 2011

By way1 and way2

150

0

Agent (Selected either position randomly)

ForwardBy way12000yBackward

By way2250

50（100，25）（20，35）300

100（50，25）

50

0.172
1.761
0.405
0.339
0.645
0.711

Pivot turn
Left
0.150
0.405
0.002
1.292
1.189
0.003

350

x

Object

2.573
0.175
0.874
0.652
0.451
0.687

Pivot turn
Right
0.517
0.469
1.758
0.028
0.002
1.848

400

100
（20，15）

Backward450

3. Result
The results of the simulation are shown in Table.1
and Fig.9. Each graph is the average of 1000 trials. In
these results, all the values of Q-tables, AT-table andEnv-table are initialized to 0.0 and the parameters for
Q-learning and AT-table mapping are set to following.
0

Forward819

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

The acquisition of sociality by using Q-learning
in a multi-agent environment
Yasuo Nagayuki
Department of Social and Management Studies, Otemae University
2-2-2 Inano, Itami, Hyogo 664-0861, Japan
nagayuki@otemae.ac.jp

Abstract: In multi-agent environments, it is important that the agents have the "sociality". In this article, I propose areinforcement learning framework, which is based on Q-learning, that the agent is able to learn the "sociality" in a
multi-agent environment. In this framework, the agent learns to ignore the near goal, which is left for the other agent,and go toward the farther goal, if the agent judges that the decision is effective from the social viewpoint, but not fromthe agent's greedy viewpoint.
Keywords: multi-agent environment, reinforcement learning, acquisition of sociality, Q-learning

1. INTRODUCTION

2. LEARNING TASK

It is hard that the agent, which learns by using areinforcement learning [1] algorithm, learns to go
toward the farther goal rather than the near goal. If thevalues of the reward obtained by reaching each goal are
not different, it is all the more.
In the case that the environment the agent faces is asingle-agent environment, the agent should not ignore
the near goal because the agent should be greedy for theenvironment, that is, the near goal. If the agent learns to
go toward the farther goal rather than the near goal inthis case, the reinforcement learning algorithm musthave some defect.
In the case that the environment the agent faces is amulti-agent environment [2], on the other hand, there
are the situations that the agent should ignore the neargoal, which is left for the other agent, and go toward thefarther goal. If the agent's decision is not effective fromthe agent's greedy (selfish) viewpoint, but effectivefrom the social (global) viewpoint, it is called that theagent has the "sociality". In multi-agent environments, it
is important that the agents have the "sociality".
However, it is difficult that the agent applied thereinforcement learning algorithms proposed so far learn
the "sociality" in multi-agent environments, because thereinforcement learning algorithms are designed that theagent learns greedy policies for the environment.
In this article, I propose a reinforcement learning
framework, which is based on Q-learning [3], that theagents learn the "sociality" in a multi-agent environment.
In this framework, the agents learn to ignore the neargoal and go toward the farther goal, if the agent judgesthat the decision is effective from the social viewpoint.

In this study, I prepared a two-agents problem as atask. This task is designed that one agent should learn toignore the near goal and go toward the farther goal fromthe social viewpoint, and the other agent should learn to
go toward the near goal from the social and greedy
viewpoint.

©ISAROB 2011







In a 4×7 grid space, two agents exist and there
are two goal positions as shown in Fig.1.
At each episode, the initial positions of theagents are the positions shown in Fig.1.
At a discrete time step in an episode, the agentssynchronously execute one out of five actions:moving up, down, left, or right from the current
position, or staying in the current position. But,
the agents are not able to execute the actions
that the agents go out of the grid space. For
example, in Fig.1, the agent A1 is not able toexecute the two actions: moving down andright.
The agents are able to look over the wholegrid space.
G2

A2

G1

A1

Fig.1. Grid space of learning task: A1 andA2 are the agents, and G1 and G2
are the goal positions for the agents.

820

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A2

G2
9 time steps

2 time steps

A2

A1

A1

G1

(a) An example of shortest passes for theagents until achieving the goal situation1.

(a) Goal situation-1: the agent A1 reaches
the goal position G1 and the agent A2
reaches the goal position G2.
A1

G2
3 time steps

4 time steps

A2

A2







A1

(b) Goal situation-2: the agent A1 reaches
the goal position G2 and the agent A2
reaches the goal position G1.

(b) An example of shortest passes for theagents until achieving the goal situation2.

Fig.2. Goal situations of this learning task

Fig.3. Shortest passes until achieving eachgoal situation for each agent fromthe initial positions.

When an agent reaches a goal position (G1 or
G2), the agent obtains a positive reward. The
value of the reward is 1.0 regardless of G1 or
G2.
The reward of each goal position disappears
when an agent arrives at the goal position.
Therefore, in each goal position, the reward
is given for only one agent.
The global goal as this task is that each of
two agents reaches either of the two goal
positions. But, the goal positions two agents
reach must be different. That is, there are twogoal situations in this task as shown in Fig.2(a) or Fig.2 (b). When a goal situation isachieved, the episode ends.

Fig.2 (b), on the other hand, the agent A1 is able toreach the goal position G2 with 3 time steps, and theagent A2 is able to reach the goal position G1 with 4
time steps as shown in Fig.3 (b). Therefore, the numberof the shortest time step that the goal situation-2 isachieved is 4.
As above-mentioned, if each agent goes toward its
goal position with shortest pass, the goal situation-2 ismore effective than the goal situation-1 because thenumber of the time step until achieving the goal
situation is shorter. However, in order to achieve thegoal situation-2, the agent A1 must give up goingtoward the near goal position G1, which is left for theagent A2.

Here, I consider which goal situations are more
effective if each agent goes toward its goal position
from its initial position with the shortest pass in this task.
In the case of the goal situation-1 shown in Fig.2 (a), theagent A1 is able to reach the goal position G1 with 2time steps, and the agent A2 is able to reach the goal
position G2 with 9 time steps as shown in Fig.3 (a).
Therefore, the number of the shortest time step that thegoal situation-1 is achieved is 9. It is remarked that theagent A1 is waited until the agent A2 reaches the goal
position G2. In the case of the goal situation-2 shown in©ISAROB 2011

G1

3. MDP AND Q-LEARNING
The reinforcement learning framework proposed inthis article is based on Q-learning [3]. Q-learning was
originally defined to deal with Markov decision
processes (MDPs) [4].
3.1. MDP
An MDP is defined by a set of finite states of theenvironment, S, and a set of finite actions, A. At eachtime step, an agent observes a state s (∈S), executes anaction a ( ∈ A) and receives a reward r from the821

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

4. MULTI-AGENT REINFORCEMENTLEARNING

environment. The state transition of the environment isdefined by a transition probability function:

Ps a s ' = Pr( s ' | s, a )(1),

4.1. Point of problem
Here, I consider that each agent in the task shown insection 2 learns its policies by using reinforcement
learning. It is remarked that the goal situation-2 (shownin Fig.2 (b) or Fig.3 (b)) is more effective than the goal
situation-1 (shown in Fig.2 (a) or Fig.3 (a)) from theglobal (social) view point.
If the agents learn its policy by using the generalreinforcement learning algorithms proposed so far (likeQ-learning), it is unfortunately expected that the leaning
converges to the goal situation-1. The reason is that theagent applied the reinforcement learning algorithms
proposed so far is greedy for the nearer goal. Therefore,
it is expected that the agent A1 does not give up goingtoward the near goal position G1. Even if the agent A2
aims at reaching the nearer goal position G1 withshortest time step (4 time steps), the agent A1 reaches
the goal position G1 (with 2 time steps) earlier than theagent A2. Furthermore, because the positive reward of
the goal position G1 disappears after the agent A2
reaches the goal position G1, the agent A2 inevitably
goes toward another goal position G2. That is, it isdifficult that the agent applied the reinforcement
learning algorithms proposed so far learns the"sociality" in a multi-agent environment as shown insection 2.
In this article, I propose a reinforcement learning
framework, which is based on Q-learning [3], that theagents are able to learn the "sociality" in a multi-agent
environment as shown in section 2.

which is the probability that the environment changes toa new state s' (∈S) when the agent executes the action ain the state s. The reward r is a variable, possibly
probabilistic, which depends only upon the current states and action a.
3.2. Q-learning
Q-learning [3] is an incremental reinforcement
learning method. According to Q-learning, the agent
selects an action based on an action-value function(called the Q-function), Q( s, a) , which is a scalarfunction and represents the value that the agent executes
the action a at the state s. The Q-function is updated
using the agent's experiences. The learning flow is asfollows:
1.

For the current state s, the agent executes an action
ai (∈A) with the probability:

π(ai | s) =

1－ε
ε
+
h
| A|
ε
| A|

(ai = argmaxb Q(s, b))(2),(otherwise)

where π (a i | s ) represents the probability that theagent executes the action ai in the state s, and |A| isthe number of the element of the action set A,and ε(∈[0,1]) is a parameter for determining therandomness of the action selection, and h is thenumber of the action b(∈A), with which the valueof the Q-function Q(s,b) is max.
2.

The agent executes the action a selected in step 1.
The environment changes to a new state s'
according to Ps a s ' , and a reward r is given fromthe environment. The Q-function Q(s,a) is updated
as follows:Q( s, a) ← (1－α )Q( s, a)+ α (r + γ max a ' ∈ A Q( s ' , a' ))4.2. Learning framework
First, the grid space shown in Fig.1 is divided into
two virtual task spaces form each agent's view point. In
each virtual task space, there is only one goal position.
For example, the two virtual task spaces from the agent
A2's view point are as shown in Fig.4 (a) and (b).
Each agent has two Q-functions Q1(c1,a), Q2(c2,a)corresponding to the two virtual task spaces c1, c2,
respectively. Here, c1, c2 are the virtual task spaces
corresponding to the goal positions G1, G2, respectively.
Each agent does Q-learning with each virtual task space(that is, each Q-function), separately and simultaneously.
At the beginning of each learning episode, the agent
decides which of Q-function should be used by using
the value of wij. Here, wij, which is a variable, is updated(3),

where α (0 ≤ α < 1) is a parameter called thelearning rate, and γ (0 ≤γ ≤1) is a parameter
called the discount factor.
3.

If the new state s' satisfies a terminal condition,then the episode ends. Otherwise, let s ' → s andgo back to step 1.

©ISAROB 2011822

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

The environment changes to a new state s' (thevirtual task spaces c1', c2') and a reward r is given
from the environment. The two Q-function is
updated as follows:

G1

Q 1 (c1 , a ) ← (1－α )Q 1 (c1 , a)
+ α (r + γ max a ' ∈ A Q (c1 ' , a' ))A2Q 2 (c 2 , a ) ← (1－α )Q 2 (c 2 , a)(a) Task space-1.

+ α (r + γ max a ' ∈ A Q(c 2 ' , a' ))4.

Fig.4. Virtual task spaces from the agent
A2's viewpoint.

when either of the goal situations is satisfied. The
update rule is as follows:(4),iwhere θ(∈[0,1]) is a parameter, and y j is the numberof the time steps from the initial state (as shown in
Fig.1) until a goal situation, in which the agent Ai
reaches the goal position Gj, is satisfied. The measure ofchoosing the Q-function, which is used when the agent
selects the action during the episode, is as follows. If
wi1>wi2, then Q2(c2,a) is chosen with probabilityω (0 ≤ω ≤1) , or Q1(c1,a) is chosen with probability
1－ω . If wi1 ≦ wi2, then Q1(c1,a) is chosen withprobability ω (0 ≤ω ≤1) , or Q2(c2,a) is chosen withprobability 1－ω .
The learning flow is as follows:
1.

At the beginning of each episode, the agent
decides which of Q-function should be used during
the episode based on the value of wij as above
mentioned. Here, it assumes that Q1(c1,a) waschosen.

2.

For the current state s (the virtual task spaces c1,c2), the agent executes an action ai (∈A) with theprobability:

π(ai | s) =

3.

1－ε
ε
+
h
| A|
ε
| A|

6. CONCLUSION
In this article, I propose a reinforcement learning
framework, which is based on Q-learning, that the agent
is able to learn "sociality" in a multi-agent environment.
In this framework, the agent learned to ignore the neargoal, which is left for the other agent, and go toward thefarther goal.

REFERENCES[1] Sutton RS, and Barto AG (1998), Reinforcement
Learning: An Introduction, MIT Press[2] Wooldridge M (2009), An Introduction to Multi
Agent Systems, John Wiley & Sons
[3] Watkin CJCH, and Dayan P (1992), Technical note
Q-learning. Machine Learning 8(3):279-292[4] Puterman ML(1994),Markov Decision Processes.
Wiley Interscience(ai = arg maxb Q1(c1, b))(5).
(otherwise)

The agent executes the action a selected in step 2.

©ISAROB 2011

If the new state s' satisfies a terminal condition,then the episode ends. Otherwise, let s ' → s andgo back to step 2.

Here, I did two experiments for the task shown insection 2. One is the experiment that the usual Qlearning (as shown in subsection 3.2) is applied to theagents, the other is the experiment that proposed
reinforcement learning framework (as shown insubsection 4.2) is applied to the agents. The parameters
are: α =0.3, γ =0.9, ε =0.3× 0.9977num_ep, θ =0.3 ×
0.9977num_ep, ω =0.5 × 0.9977num_ep, where num_ep isthe number of the learning episodes.
In the case that the usual Q-learning is applied, thelearning converges to the goal situation-1 at the 1000
learning episodes. In the case that the proposed learning
framework is applied, on the other hand, the learning
converges to the goal situation-2 at the 1000 learning
episodes.

(b) Task space-2.

w ij ← θ w ij + (1－θ ) y ij

(7).

5. EXPERIMENT AND RESULTS

A2

G2

(6),

823

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Effect of interaction between rules on rule dynamics
in a multi-group minority gameTakashi Sato
Okinawa National College of Technology
905 Henoko, Nago, Okinawa 905-2192, JAPAN(Tel : +81-980-55-4179; Fax : +81-980-55-4012)(stakashi@okinawa-ct.ac.jp)

Abstract: Rules, such as laws, institutions, and norms, are shaped by interaction among social members, which are affected by therules. Since they have such mutually regulating relationship, the rules can be changed dynamically. This paper discusses how
interaction between rules shaped in different groups has an influence on the rule dynamics. In order to investigate it, we propose amulti-group minority game model in which each agent plays it in different groups independently with contact to other group’s
members through the game. Our simulation showed that the frequency of synchronization in the rule dynamics is more than a model
without interaction. Further, we found that, in a model with interaction by exchanging agents in different groups, the frequency is
increased till the exchange number of agents between different groups is increased to half of the members, but this effect is lost by
decreasing the interaction opportunity and increasing the exchange number from more than half of the members.
Keywords: Rule Dynamics, Multi-Group Minority Game, Multi-Agent System, Internal Dynamics, Micro-Macro Loop, Simple
Recurrent Network with Self-Influential Connection

action depends, refers to as the internal dynamics. As theresults in our previous work, we found that internal dynamics
and micro-macro loop are necessary to form and maintain an
endogenous dynamics of social structure. Further, oursimulation showed that the macro structure shaped by all
agents’ actions is internalized in the agents through thelearning process.
In the previous simulation system, only one group wasused, and coexistence and competition of many macro
structures could not be observed simultaneously in the one
group. However, there are many groups, many rules, andinteraction among them in actual world. Moreover, there are
hardly researches which take internal dynamics, micro-macro
loop and interaction between different rules into consideration.
The purpose of this study is to investigate how interaction
between rules shaped in different groups has an influence on
the rule dynamics. In order to discuss it, we propose a multigroup minority game in which many agents in different groups
play the minority game independently and interact with othermembers.

I. INTRODUCTION
In our society, there are many rules such as laws,
institutions, and norms. Such rules shown in a macro level are
shaped by interaction among social members at a micro levelfrom bottom up. The rules have an influence on the members.
The members affected by the rules might change their ways ofthinking and understanding about the world. Consequently,
the rules might also be changed, and totally-new rules might
emerge. This mutually regulating relationship between a micro
and a macro levels can be referred to as the micro-macro loop[1]. In this paper, we describe such dynamical change of rule
as rule dynamics.
For example, the institutions have been discussed among
many economists. Veblen [2] defines an institution as “settled
habits of thought common to the generality of men.”
According to North [3], institutions can be defined as “theformal rules (common law, etc), the informal constraints(norms, etc), and the enforcement characteristics of each.”
Both of Veblen and North emphasize an importance ofinteraction between institutions and individuals (ororganizations consisted of individuals) in their discussions on
the institutional change.
Based on these arguments, in our previous works [4-5], we
investigated about relationship among the micro-macro loop,
internal dynamics, and changes of macro structures like therules by using our proposed multi-agent system. This system
is consisted of many agents with the internal dynamics. An
autonomous change of internal state, on which the agent’s©ISAROB 2011

II. MULTI-GROUP MINORITY GAMEThe minority game (MG) [6] is a simple game which canbe regarded as a simple stock market model. In this game,n(odds)-players must select one out of two hands (e.g., -1 or 1)
independently. And then, those who are in the minority side
win. Here, the MG is employed as social interaction. The

824

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011previous work [4] as illustrated in Fig.3. The SRN-SIC is an
Elman-type network [7] modified by adding recurrent
connections between the output and the input layers so that theagent can determine his/her own action based on its pastaction.

schematic view of the basic MG in our simulation is illustrated
in Fig.1.

Fig.1. The schematic view of the basic MG in our simulation.
(a) Each agent receives past minority hand shaped by all
agents’ actions. (b) All agents learn a time series of theminority hands for past m steps.

Fig.3. The SRN-SIC proposed in our previous work as anarchitecture of agent with internal dynamics. This is aparticular Elman-type network with additional recurrent
connections between the output and the input layers. The
symbol L indicates a differentiable nonlinear function to
output a real number between -1.0 and 1.0. Not allconnections are shown.

We propose a multi-group minority game (MGMG),
where there are many groups playing the MG independently,
and at the same time interacting with other members ofdifferent groups. There are three types of interacting ways as
follows:
1.

All agents in each group play the MGMG
independently. That is to say, they have nointeraction (Fig.2a).

2.

Each agent receives not only own past minority handbut also other group’s past minority hand at eachstep (Fig.2b).

3.

The agents in different groups play together withother group’s members by periodically exchanging
agents between groups (Fig.2c).

(a)

(c)

In our previous work, we observed various orderedpatterns such as fixed point and periodic motions at macrolevel. These ordered patterns can be generated by some rulesof dynamical systems. Therefore, when an ordered pattern isshown in a time series of minority hands, we can interpret thata rule emerges through interaction among the agents. Besides,
we confirmed that the agents can internalize the rules in their
SRN-SIC and act based on the internalized rule. That is to say,by using the MGMG and the SRN-SIC, we can observe
dynamics of interaction between certain rules shaped indifferent groups.

III. SIMULATION RESULTS
The basic parameters of our simulation are set up as
follows:
The population size of a group is set to 101. The number ofgroup is set to 2. The settings of the SRN-SIC are omitted forspace constraint (For more details refer to [4]). The simulationis conducted for 1,000 turns, where 10,000 steps are referred
to as 1 turn, namely, for 10,000,000 steps. The learning ofeach agent is processed by using the error-backpropagation
every 1 turn.

(b)

Fig.2. Three types of interactions in our proposed MGMG. (a)No interaction. (b) Each agent receives not only their group’s
past minority hand but also other group’s past minority handat each step. (c) Each agent interacts with other group’s
members by exchanging agents at fixed intervals.

1. Interaction by receiving another group’s minority
hand on rule dynamics
First, we compare the results of the simulation without
interaction to that of the simulation with interaction in which

We adopt a model of dynamical cognitive agent withinternal dynamics represented by a simple recurrent networkwith self-influential connection (SRN-SIC) proposed in our©ISAROB 2011825

803050803100Step

80315080320010.5
0803200803250803300803350803400

Step
1
0.5
0
803400803450803500803550803600Real num of Mside

0803000Real num of Mside

1

0.5

Real num of Mside

Real num of Mside
Real num of Mside

Real num of Mside

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 201110.5
0803000803050803100Step

80315080320080325080330080335080340080355080360080375080380010.5
0803200

Step
1
0.5
0
803400803450(a)

No interaction

Receiving minority
hand shaped byother group

Real num of Mside

804000693200Realnumnumof MsideRealof Mside

0.5
080360080365010.5
0.5
08038000693000803850693050803900693100
Step
Step

803700Step

1803950693150

1
1
0.5
0.5
0
6932000693250693300693350693400

43000

43050

Step
43100
Step

43150

43200

1

1
0.5

0.5
0
6934000693450693500693550693600

43200

43250

Step
43300

43350

43400

Step

1

1
0.5

0.5
0
6936000693650693700693750693800

43400

43450

Step
43500
Step

43550

43600

1

1
0.5

0.5
0
6938000693850693900693950694000

43600

43650

Step
43700
Step

43750

43800

1

Real num of Mside

Real num of Mside

Receiving minority
hand shaped byother group

Next, we confirm whether the patterns of the ruledynamics shaped by different groups in the simulation withinteraction are completely the same or not. Analysis reveals
that most patterns of rule dynamics except for period-2
dynamics are different, although the period number of the
dynamics is the same. Even if the patterns between the ruledynamics are the same, most of them are out of phase. It israrely seen the cases that the dynamics shaped in different
groups are completely the same. These rule dynamics are
exemplified in Fig.51.

The graphs in Fig.5 are drawn after converted the timeseries of the minority hands to the real number between
0.0 and 1.0.

©ISAROB 2011

803750

1
1
0.5
0.5
0
8038000693000803850693050803900693100Step
Step8039506931508040006932000.5
06932000693250693300693350693400

43000

43050

Step
43100
Step

43150

43200

0.5
0
6934000693450693500693550693600

43200

43250

Step
43300

43350

43400

1
1
0.5

1

1
0.5

Step

1

1
0.5

0.5
0
6936000693650693700693750693800

43400

43450

Step
43500
Step

43550

43600

0.5
0
6938000693850693900693950694000

43600

43650

Step
43700
Step

43750

43800

1

1
0.5

1

2. Interaction by exchanging the agents between
different groups
As the other simulation with interaction, we conduct the
MGMG in which some agents exchange between different
groups in order to investigate what kind of effect it has on therule dynamics and what type of change it produces in the one.
Figure 6 shows how the equivalence of minority hands
dynamics is affected by changing the number of exchanging
the agents between different groups and its interval.
As the number of agents exchanged between different
groups is increased, it increases the number of times that eachperiod number of the rule dynamics shaped in different groups
is the same at the identical turn is increased. When theexchange number is 50, the number of times peaks and ismore than the one shown in the simulation with the interaction
receiving another group’s minority hand, as can be seen fromcomparison between Fig. 4a and 6.
As the exchange number is increased from greater thanhalf of the members, and as the exchange interval is extended
from 10,000 to 1,000,000 steps, however, when the number oftimes that the rule dynamics shaped in different groups aresynchronized, the number of times is decreased. On thesurface, it also seems that extending the exchange interval has
an effect of decreasing the synchronization of the ruledynamics. But in fact, by changing the length of steps andmaking the number of interacting opportunity the same, we
found there is not a dime's worth of difference between theequivalence observed in our simulations using different
exchange intervals. Therefore, it suggests that learning the ruledynamics at different group emphasizes synchronization of therule dynamics.

Fig.4. Comparison between the results of our simulation
without interaction and with interaction in number of the ruledynamics with the same period between different groups. (a)The difference of the number of all rule dynamics that theperiod number is the same at the identical turn. (b) The
difference of the number of times that both of different
groups’ rule dynamics in each simulation become only
aperiodic at the same turn.

1

8037001(c)
Fig.5. Three examples of the rule dynamics with the same
period numbers. (a) Both period numbers are 610 but thepatterns are absolutely different. (b) Both period numbers are
12 with the same patterns but out of phase. (c) This is a rarecase that the dynamics are exactly the same. Each period
number in this case is 5.

(b)

No interaction

803650of Mside RealRealof Mside RealRealof Mside RealRealof Mside
RealReal
numnumof Mside
numnumof Mside
numnumof Mside
numnumof Mside(b)8038000.5
0803600803500
Step

1

Step
Real
of MsideRealnumnumof Mside(a)

RealReal
numnumof Mside
numnumof Mside
numnumof Mside
numnumof Msideof Mside RealRealof Mside RealRealof Mside RealRealof Mside

all agents receive not only their minority hand but also the one
shaped by another group.
Figure 4a shows the difference of the number of all ruledynamics that the period number is the same at the identical
turn. However, all rule dynamics with more than 3,500 periods
are regarded as the same aperiodic dynamics, since last 7,000
steps of 1 turn are used to analyze the period number of the
dynamics. As can be seen from Fig.4a, the number of the ruledynamics with the same period in the simulation having
interaction is more than that of the dynamics in the simulation
without interaction.
Figure 4b depicts the difference of the number of times
that both of different groups’ rule dynamics in each simulation
become only aperiodic at the same turn. As shown in Fig.4a,there is a major difference between the result of the simulation
without interaction and the simulation with interaction.

Real num of Mside

Step

826

0.5

0
43800

43850

43900
Step

43950

44000

0.5

0
43800

43850

43900
Step

43950

44000

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig.6. Effect of exchange number of agents and exchangeinterval on equivalence of minority hands dynamics. As theexchange number is increased, the equivalence of minority
hands dynamics is also increased til the exchange number is
50. Howerver, this effect is lost by increasing the exchange
number from more than half the members.

interaction, with interaction by receiving other’s minority
hand, and by exchanging the agents in different groups.
The simulation results showed that the frequency ofsynchronization of the rule dynamics shaped in different
groups is increased by interaction that the agents receive not
only their own minority hand but also other’s minority hand.
Further, we have found that, as the exchange number of theagents is increased to about half of the group member, theequivalence of rule dynamics is also improved.
Our simulation results suggested that internalizing andcommoditizing several different rules in the individuals
through direct learning of the rules shaped by others play animportant role to emerge stable societies and rule dynamics.

IV. DISCUSSIONACKNOWLEDGEMENT

Let us consider the difference causes of results shown inthree types of simulations.
In the simulation in which agents receive other group’s
minority hand, the agents can know not only the ruledynamics formed by themselves but also the one shaped bydifferent group. Therefore, both of rules are internalized intheir SRN-SIC through the learning process, andsynchronization between the rule dynamics is induced by all
agents’ behaviors based on their internalized rules. However,
the agents cannot learn the other’s rule dynamics appropriately,
because teacher’s signal using the learning is only their own
minority hand. Thus, unstable states expressed as complex
aperiodic dynamics in different groups would emerge.
Why is the equivalence shown in the simulation withinteraction exchanging the agents in different groups higher
than the one observed in the simulation with interaction
receiving the another group’s minority hand? In the simulation
having the replacement, the agents can learn both of different
rule dynamics shaped in each group by alternating between
the groups and form such dynamics as the members of thesame group temporarily. That is to say, it would appear thatboth rules are internalized and commoditized in the agents at
the mutual groups. Therefore, it is important that theexchanged agents learn the opponent’s rule dynamics to
improve the equivalence of rule dynamics shaped in different
groups.

The author thanks Takashi Hashimoto and Zacharie
Mbaitiga for their important discussions. This work was
supported by the Grant-in-Aid for Young Scientists (B)No.22700242 from the Japan Society for Promotion of
Science (JSPS).

REFERENCES[1] Shiozawa Y (1997), Consequences of Complexity(in Japanese), NTT Shuppan[2] Veblen T (1919), The Place of Science in Modern
Civilization and Other Essays. Huebsch, New York
[3] North D (1994), Economic performance throughtime, American Economic Review, 84(3):359-368[4] Sato T, Hashimoto T (2007), Dynamic social
simulation with multi-agents having internal dynamics.
In: Sakurai A (ed), New Frontiers in Artificial
Intelligence: Joint Proceeding of the 17th and 18th
Annual Conferences of the Japanese Society for
Artificial Intelligence. Springer, Berlin, pp. 237-251[5] Sato T, Hashimoto T (2005), Effect of internal
dynamics and micro-macro loop on dynamics of social
structures (in Japanese), Transactions on Mathematical
Modeling and its Applications (TOM), 46:81-92[6] Challet D, Zhang YC (1997), Emergence ofcooperation and organization in an evolutionary game,Physica A, 246: 407-418

V. CONCLUSION
In order to investigate how interaction between rules
shaped in different groups has an influence on rule dynamics,
we have proposed a multi-group minority game (MGMG) andconducted three types of the MGMG simulation without©ISAROB 2011[7] Elman JL (1990), Finding structure in time,
Cognitive Science, 14(2):179-211

827

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Geometrical analysis and design of motion transmitting element
for mobile vehicles
Masaharu Komori, Fumi Takeoka, Jungchul Kang and Yukihiko Kimura
Department of Mechanical Engineering and Science, Kyoto University
Yoshidahonmachi Sakyo Kyoto 606-8501 Japan(Tel : 81-75-753-5858; Fax : 81-75-753-5858)(komorim@me.kyoto-u.ac.jp)
Abstract: The range of speed or torque of motors is limited to some degree. Industrial vehicles working in factories are
required to realize both outputting a large force when they carry loads and moving with high velocity when they moveto the destination. However, it is impossible to realize those requirements if the conventional reduction devices is used.
Velocity variation devices using gears are widely used to change the velocity ratio between input and output shafts.
However, the motion transmission from the input shaft to the output shaft is interrupted during the velocity ratio
variation process. In order to solve this problem, velocity variation method that can transmit motion precisely is
proposed, in which a motion transmitting element is used. In this report, the geometrical analysis method of the motion
transmitting element is proposed, and the expressing method of the curve using dispersed points is proposed and thecalculation method of the inclination and the distance concerning the curve is developed. Based on the proposed
method, the geometrical form of the transmitting element is analyzed.
Keywords: ratio variation
points is proposed and the calculation method of theinclination and the distance concerning the curve isdeveloped. Base on the proposed method, thegeometrical form of the transmitting element isanalyzed.

I. INTRODUCTION
The range of speed or torque of motors is limited tosome degree considering the size, weight and cost of the
motors. Industrial vehicles are required to realize both
outputting a large force when they carry loads andmoving with high velocity when they move to thedestination in order to shorten the moving period. If thevelocity ratio between input and output shafts ischanged, a wide range of speed or torque can berealized even if the ability of the motor is limited in
speed or torque. Velocity variation devices using gears
are widely used in the industrial fields. In a gearedtransmission, it is needed to change the working gear
pairs to vary the velocity of the transmission. However,
the motion transmission from the input shaft to the
output shaft is interrupted during changing process.
Therefore it is impossible to change the velocity whiletransmitting rotation between the input and output shafts.
In order to solve this problem, velocity variation methodthat can transmit motion precisely is proposed in our
research, in which a motion transmitting element is used.
In this report, the geometrical analysis method of the
motion transmitting element is proposed. A variety of
motion characteristics of the motion transmitting
element are possible but it is necessary to obtain the
inclination of each point on the curve and the distance
between the points on the curve in order to calculate thegeometrical form of the motion transmitting element.
The expressing method of the curve using dispersed©ISAROB 2011

II. VELOCITY VARIATION METHOD
In the proposed velocity variation method, as shownin Fig.1, there are two shafts, i.e. input shaft and output
shaft, and three gear pairs A, B and C, and threeclutches TA, TB and TC. When the clutches TA is
engaged, the gear pair A works and the velocity ratio ofgear pair A is realized. When the clutches TB is engaged,
the gear pair B works and the velocity ratio of gear pair
B is realized. In the change of the working gear pairfrom gear pair A to gear pair B, the clutches TC is
engaged, and gear pair C works.

Fig.1. Structure of the Velocity Ratio Variation Device

828

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

In the pitch curve of noncircular gear pair C, there are
four sections. The pitch curve of the noncircular gear
pair C is partly same as that of gear pair A, and partly
same as that of gear pair B. Those parts of pitch curve
are smoothly connected. Therefore, the velocity ratio
can be changed from that of gear pair A to that of gear
pair B while gear pair C works. Therefore, the rotationalmotion is precisely transmitted from the input shaft tothe output shaft.

Length ofcurve LPn-1
Sum ofdistances isapproximated L

Discretepoint onpitch curve

Pitch curve

Pn
Approximatednormaldirection ofpitch curve

Pn+1(a) Approximation of(b) Approximation oflength of pitch curve
normal direction
Fig. 2 Approximation in the proposed method

III. ANALYZING METHOD OFGEOMETRICAL PROPERTYThe noncircular gear has such a complex profile
form that it is difficult to analyze its geometrical form.
An analyzing method of geometrical property of
noncircular gear is developed. The profile of a gear is
generated by the relative motion between the tool andthe work material along the pitch curve of the gear.
Therefore, the profile of the gear can be calculated byinvestigating the locus of the profile of the tool. If thepitch curve of the gear is expressed by a mathematical
function, the length of the pitch curve of the gear L andthe angle φ of the normal direction of the pitch curve
can be obtained, which are essential to calculate thelocus of the profile of the tool. However, it is impossible
to express any pitch curve by the mathematical function.
Therefore, to deal with various forms of noncircular
gears, a method to express the pitch curve by the set ofthe discrete points is proposed. In this method, the pitch
curve of a gear is composed of the set of the discrete
points, and corresponding profile is calculated based on
the points. The length L and the angle φ are calculatedusing approximation. Figure 2 shows the concept of theapproximation. The length L is approximated by thesum of the distances between the discrete points on thepitch curve as shown in Fig. 2(a). The angle φ of thenormal direction on point Pn is approximated by that of
the bisector of the angle between the line passing
through points Pn-1 and Pn, and that through points Pn
and Pn+1. Suppose that the coordinate for the discrete
points on the pitch curve are (xi, yi) (i=0, 1, 2, ...). The
length L and the angle φ corresponding to the pitch
point (xn, yn) are approximated as follows.

Fig. 3 Curvature radius of profile of gear C
Based on the above discrete point method, thenumerical analysis program to calculate the profile ofthe noncircular gear is developed. Figure 3 shows
curvature radius of two different teeth of a noncircular
gear. The profile of the noncircular gear is calculated
without any problem by using the developed program.
Figure 3 shows that there is a different of form between
two teeth.

VI. CONCLUSION
In this research, the velocity ratio variation device
that can realize precise transmission of the rotation fromthe input shaft to output shaft is treated. The expressing
method of the curve of the motion transmitting element
using dispersed points is proposed and the calculation
method of the inclination and the distance concerning
the curve is developed.

ACKNOWLEDGEMENT
This study was supported by Industrial Technology
Research Grant Program in 2009 from New Energy andIndustrial Technology Development Organization(NEDO) of Japan.

nL=∑ (x − x ) + ( y − y )
2

i

i −1

i

2

i −1

(1)

i =1

tan −1φ=©ISAROB 2011

y n+1 − y ny − y n −1
+ tan −1 nxn+1 − xn
xn − xn−1 π
−
2
2

(2)

829

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Underwater unknown acoustic source localization based on Sound
Propagation Loss : theory analysis and simulation results
Han Zhang, Bin Fu , Chunhu Liu,Lian Lian
Shanghai Jiaotong University* Email: bfu@sjtu.edu.cn; Fax: 021-54361833; TEL: 021-34202134

Abstract: A new method based on SPL to estimate the position of unknown acoustic source was put up forward. And
theoretical analysis and computer simulation has been used to analyze the accuracy of underwater sound source
localization wh ich is interfered by water background noise and multipath reflection. Using simulated data of two
different real marine sound sources, we show that when the sound source is far fro m hydrophone array(greater than
200 meters), the accuracy of this method is high, up to 1 meter. However, with the distance decreasing, the intensereflection of offing brings multipath reflection interference and a correction based on decline analysis ofmu lti-centre frequency of sound source was proposed which can increase localization accuracy by 10%-30%.
Therefore, for short-range localization, use of a co mbination of passive sonar and this method can achieve a good
accuracy. The research results verify the effectiveness of the function relationship between acoustic attenuation andthe propagation distance in underwater unknown sound source localization . Especially it is advantageous on the
remote positioning and can be combined with high accuracy passive positioning to realize g reat-range andhigh-accuracy location search.
Keywords: Unknown Sound Source； Sound Propagation Loss(SPL)；Sound Pressure；Characteristic Frequency

achieved.
In this paper, we put up forward a new method basedon SPL to estimate the position of unknown acoustic
source, and we carry on the research, using theoreticalanalysis and computer simulat ion, to analyze the acousticpressure of underwater sound signal which is interferedby water background noise and multipath reflection.
According to the results, we got functional relat ion
between acoustic pressure attenuation of the unknown
source and the propagation distance. Then we have done
preliminary calculat ion and analysis in the accuracy of
the unknown acoustic source localizat ion, wh ichprovides theoretical basis for large scale and low cost
acoustic positioning technology and offers a new
approach for tracking the movement of co mplicatedunderwater source, such as the unknown sailing noise,
underwater construction noise, biological noise, natural
disasters noise.
The waveform and frequency spectrum of echo signalof each node are analyzed and when we use the single
center frequency of acoustic pressure information in
short range, the accuracy of localization is not high. To
solve this problem, we ext ract mult iple center frequencyof acoustic pressure information, optimize theinformat ion and eliminate the incorrect acoustic source

1. INTRODUCTION
Acoustic positioning technology concerns the safety
and efficiency of underwater operation. Previous
acoustic positioning cannot satisfy the needs in scientificresearch, marine explo ration, underwater operation andbiological targets monitoring. The primary and new
demand is how to locate the unknown acoustic source.
The aim o f sound source localization is to estimate theCartesian coordinates of the source or the speed of themoving source. Many algorith ms have been put up
forward for sound source localizat ion. Approaches using
distributed microphone network are presented. In
addition, approaches based on energy Measurements
have also been studied in recent years. But in the ocean,
there is still no appropriate sound source localizat ion
algorith m. Now the time -difference-of-arrival estimat ion
methods, such as Super Short Base Line system (SSBL)
and Lone Base Line system (LBL) are widely accepted,
but these methods have large errors in noisy andmu ltipath reflections condition. In recent years, wirelesssensor networks and Sound Propagation Loss (SPL) has
been studied. However, due to the complicated dataprocessing and the limitations of localization algorith m,
the accurate position of unknown source cannot be©ISAROB 2011830

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

( x  x1 ) 2  ( y  y1 ) 2  ( z  z1 ) 2  r1 2

2
2
2
2
( x  x2 )  ( y  y 2 )  ( z  z 2 )  r2

2
2
2
2
( x  x3 )  ( y  y3 )  ( z  z 3 )  r3

2
2
2
2
( x  x4 )  ( y  y 4 )  ( z  z 4 )  r4

informat ion. Then we can obtain the position of
unknown acoustic source with the SPL algorith m. The
computer simulat ion results also show that extracting
single center frequency of acoustic pressure informat ion
has enough precision in the long distance. But when the
distance between acoustic source and hydrophone array
becomes short, owing to the powerful interference ofreflection wave and the phase difference between
directed wave and reflected wave, ext racting one center
frequency of acoustic pressure informat ion cannot meet
the accuracy needs. Then we ext ract mult iple center
frequency and optimize the results, which great ly
reduces the errors.

2.

（r1）
20 log 10(Tv / Rv1 )  Rx  Rg  20log10
20 log 10(Tv / Rv )  Rx  Rg  20log10
（r2）

2
(6)

（r3）
20 log 10(Tv / Rv3 )  Rx  Rg  20log10（r4）
20 log 10(Tv / Rv 4 )  Rx  Rg  20log10
3. NUMERCIAL SIMULATIONPRINICIPLE

3.1. Simul ati on Conditions
3.1.1 Sound source
Use two different signals : submarine p ropeller noise andmarine life sound.

2.1. Sound Propagati on Loss
SPL mainly includes two sections: Extended Lossand Absorption Loss, show as (1).

PL  20 log 10(r )  r  Extended Loss:

PL = 2 0 l o gr

3.2 Simulati on Contents
3.2.1 Layout of hydrophone array.
Four hydrophones can be set up in Figure 1. We canestablish OXY coordinate system, as shown in Figure 1.
Then nodes coordinate: Node1 （ -30,0,0 ） 、
Node2(-10,0,10)、Node3(10,0,0)、Node(30,0,0).

(1)
(2)

Absorption Loss: based on experimental fo rmula ,determined by  .
When sound transmits with electrical signal,
following equation (2) can be obtained.

PL  20 log 10(Tv / Rv)  Rx  Rg (3)Where, Tv [V] is Transmission voltage, Rv [V] is
Reception voltage, Rx [dB]: Reception sensitivity,
Rg [dB] is Reception gain. Rv is measured data andother 4 are all constants.
By solving the equations (1) and (3) simu ltaneously,(4) can be obtained.

20 log10(Tv / Rv)  Rx  Rg  20log10（r）（4）
We got the relations between propagation distance r
and Tv , Rv . By measuring Rv at different distances andthrough the following mathematical model, source
coordinates was got.
Figure 1 Layout of hydrophone array and simu lation
contents

2.2 Mathematical Model
Set the surface nodes as Ni（xi , y i , zi ）(i=1,2,3,4),and
sound source as P(x, y, z), build the equation (5) and（6）,By measuring Rv at Ni and through equations (5)and (6), source coordinates was got.

©ISAROB 2011

(5)

831

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

3.2.2 Contents(1) Sound source moves fro m far and near in XOY plane,
the vertical distance of 10 meters to the X axis, along
negative direction on X axis, verify ing the influence of
directivity of hydrophone on positioning accuracy.
(2) Sound source moves fro m far and near in XOY plane,
along Y axis, verifying the influence of d istance between
sound source and array on positioning accuracy.
(3) Sound source moves fro m far and near in YOZ plane,
the vertical distance of 10 meters to the Z axis, along
negative direction on Z axis, verifying the influence ofreflected wave on positioning accuracy.

(a) Submarine propeller noise

Figure 2 x-coordinate positioning error with distance
fro m sound source to array

3.3 Simulati on results(1) Simu lation content (1) results are shown in Figure 2,
when the distance between hydrophone array and soundsource is greater than 200 meters, error in X axis is lessthan 1 meter; Ho wever, when the distance is less than
200 meter, the directiv ity of hydrophone array changes ,which results in error in X axis increases sharply, up to
20 meters, about one-tenth of slope distance. Erro r in Y
and Z axis is so small that related error figures is not
given.
(2) Simu lation content (2) results are shown in figure 3,
when the distance between hydrophone array and soundsource is greater than 200 meters, error in Y axis is lessthan 1 meter.；Ho wever, when the distance is less than
200 meter, error increases gradually, up to 8.5 meter.
Error in X and Z axis is so small that related error figures
is not given.
(3) Simu lation content (3) results are shown in Figure 4,
when the distance between hydrophone array and soundsource increases , the phase difference of direct wave andreflected wave fluctuates among “greater than one times
wave length----- half-wavelength----less than one times
wave length”, wh ich results in error in Z axis fluctuates
among “large ---s mall---large”; when the distance isgreater than 200 meters, the phase difference goes
towards zero, erro r goes towards 10 meters. Error in X
and Y axis is so small that related error figures is not
given.

©ISAROB 2011(b) Marine life sound.

(a) Submarine propeller noise(b) Marine life sound.

Figure 3 y-coordinate positioning error with distance
fro m sound source to array(a) Submarine propeller noise (b)Marine life soundFigure 4 z-coordinate positioning error with distance
fro m sound source to array

4. ERROE CORRECTION
4.1 Correcti on Method
when the distance between hydrophone array and soundsource is less than 200 meters, by using positioning
method based on single characteristic frequency ,significant erro r may occur, even cannot solve position
equation .Due to the interefernce in close enviroment,t hephase difference of direct wave and reflected wave
fluctuates among “greater than one times wave
length----- half-wavelength----less than one times wave
length” that results in Rv cannot be calculated easily. To
solve this problem, we selected more characteristicfrequency, and analyzed sound pressure loss in different
characteristic frequency. As shown in Figure 5, In thereceived signal spectrum, mo re characteristic frequency

832

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

has been selected and sound pressure can be calculated,then according to Section 2.2 mathemat ical model, thesound source coordinate can be got, at last weighted
average.

different types of measured sound, By means ofcomputer simulat ion, analy zed the positioning accuracy
influenced by background noise and mult ipath reflection.
The simu lation results show that when the distance
between the hydrophone array and sound source is solong, the accuracy of the method based on SPL is high,However, when the distance is short, the accuracy is very
poor. On this basis, localizat ion model based on mult iple
characteristic frequency was put forward andunanswered questions of positioning equation can beeffectively solved, moreover this method has a certain
correction effect to locating at a close range.
In future work, we will further discuss correction
method to reduce the influence by phase difference andimprove positioning accuracy. Topological structure ofhydrophone array has an important effect on imp roving
positioning accuracy and we prepare some relevant
experiments to verify it.
The author would like to exp ress their sincere
gratitude to Bin Fu, School of Naval Architecture andOcean Civil Eng ineering, Shanghai Jiaotong University,
The work of this study was Granted supported by the
National Science Foundation for Young Scientists of
China

Rvs2

R vs1

R vs3

R vs4
R vs5
Rvs0

Figure 5

Collection of mult i-centre frequency

4.2 Corrected Results
Corrected results of simu lation content (2) are shown in
Figure 6,co mpare using single characteristic frequency(Rv 0 ), three characteristic frequency (Rv 0 ,Rv 1 ,Rv 3 ) andfive characteristic frequency (Rvs0, Rvs 1 , Rvs 2 , Rvs3 ,Rvs 4 , Rvs 5 .)The results show that using morecharacteristic frequency can effectively eliminate
impractical position equation, the positioning accuracy atclose ranges has been improved.

REFERENCE[1]Milne, P.H. M ilne, Underwater Acoustic Positioning
Systems, Gulf Pub. Co., Huston ,1983.
[2] I.F. Aky ild iz, W. Su, Y. San karasubramaniam and E.
Cayirci, A survey on sensor networks, IEEE
Co mmunicat ions Magazine 40 (8) (2002), pp. 104– 112.
[3] 2010 B. Fu, F. Zhang, M. Ito, Y. Watanabe, T. Aoki,
L. Lian, Sea trials of new underwater positioning system
Based on Sensor networks I. Experiments of underwater
ranging method of Sound Propagation Loss, Proceedingof Oceans 2010.
[4] ChristosI.Papadopoulos, 2003, Develop ment of an
optimized, standard compliant procedure to calculate
sound transmission loss: numerical measurements ,Applied Acoustic(64),1069-1085.
[5] X. Chen, U. Tureli, Underwater source localization
based on energy measurement with randomly d istributed
sensor array, in: E.M. Carapezza, Unattended Ground,
Sea, and Air Sensor Technologies and Applications IX,
Proceedings of the SPIE, vol. 6562, 2007.

Figure 6 Corrected results of mult i-centre frequency

5. CONCLUS ION
In a previous work basis of underwater positioning
method based on wireless sensor networks, A
mathematical model of Underwater unknown acoustic
source localization was put forward. And by using two©ISAROB 2011

833

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Position Recognition System of Autonomous Vehicle via Kalman Filtering
Dong Jin Kim*, Jong Il Bae**, Kil Soo Lee*, Dong Seok Lee*, Yun Ja Lee***, and Man Hyung Lee**

School of Mechanical Engineering, Pusan National University, Busan, 609-735, Korea.
(Tel : 82-51-510-1456; Fax : 82-51-512-9835)(dong7@pusan.ac.kr, melgibsoo@pusan.ac.kr, dslee@pusan.ac.kr, mahlee@pusan.ac.kr )**Department of Electrical Engineering, Pukyong National University, Busan, 608-739, Korea(Tel : 82-51-629-6314)(jibae@pknu.ac.kr)***
Department. of Dance, Pusan National University, Busan, 609-735, Korea.
(Tel : 82-51-510-2949)(E-mail: leeyj@pusan.ac.kr)

Abstract: This paper proposes a position recognition system for autonomous vehicle via Kalman filtering. Absolute
positioning is required to obtain successful operation of an autonomous vehicle’s process. To get reliable positions,
there are two ways, either using a GPS or dead reckoning from velocity and steering angle of the vehicle. Error
elements exist in both the uses of GPS and dead reckoning. Stable position data of the autonomous vehicle is necessary
for a successful operation even with the error elements. Kalman filter is suggested between the GPS and deadreckoning to get stable position data. Correction is performed during the localization of the processing position byusing Kalman filter between GPS and dead reckoning.
Keywords: Navigation System, Autonomous Vehicle, GPS, Dead Reckoning, Kalman Filter, Position Location

positioning of the vehicle. It calculates the vehicle's
position from the velocity and steering angle. However,this method accumulates errors [2, 3]. Since GPS does
not operate well in places such as cities withskyscrapers, tunnels, and forests, the method using only
the GPS does not guarantee successful positioning
performance. Also the method relying only on the deadreckoning is not reliable due to the accumulation of theerrors with wheel sliding, mechanical errors, andsurface roughness.
In order to improve the positioning performance,
this paper proposes a position recognition system for anautonomous vehicle via Kalman filtering. The Kalmanfilter algorithm is designed for the integration of GPS
and dead reckoning. The positioning performances ofthe proposed Kalman filtering algorithms are verified
and evaluated by experiments.

I. INTRODUCTION
With the continued development of the auto industry,vehicle parts have changed from the mechanical linkage
device to the electronic device to the computing system.
Also, the vehicle has been changed from a simple
transport device to one of the most advanced technology
integrations. This development also has the potential tosave lives and make lives more convenient based on the
ITS(Intelligent Transport System).
The ITS is intended to solve real-world problems,such as reducing the number of motor vehicle accidents.
An autonomous system which studies the intelligent
system for the vehicle to assist operators or control
whole system of the vehicle by itself is part of the ITS.
Its aim is to prevent motor vehicle accidents[1]. The
autonomous system is the most important part in the
future auto industry. In Korea as well as advanced
countries, research and development of an autonomousvehicle is actively being conducted.
For successful operation of an autonomous vehicle,
stable positioning is required. GPS(Global Positioning
System) is widely used for absolute positioning but it
requires at least 4 satellites to get the position data. The
obtained position data is determined with about 1merrors by GPS receiver. There is no accumulation oferror. Dead reckoning is another method to get the©ISAROB 2011

II. DEAD RECKONING FOR VEHICLE
When considering a car-like vehicle, the mobile
frame is chosen with its origin P attached to thecenter of the rear axle. The x-axis is aligned with thelongitudinal axis of the car. At time tk , the vehicle
position is represented by the ( xk , yk ) Cartesian834

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

coordinates of in a world frame. The heading angle isdenoted k .

The dead reckoning state equation of the vehicle canbe described as follows :xk 1  f (rk , uk , k )yky0

 xk  P cos(k   / 2)  1,k   yk  P sin(k   / 2)  2,k k    3,k



xk 1

I

 / 2



(4)

xk

R

Here

Pk 1

Pk

state

vectorx   xD yD D Tiscomposed of the position measured at the deadreckoning. The system noise  is regarded as theWhite Gaussian noise.

k

y

the

P

x
O

x0

III. POSITION RECOGNITION SYSTME VIA
KALMAN FILTERING

Fig. 1 Elementary Displacement between Two
Samples
Let Pk and Pk 1 be two successive positions.

In the proposed position recognition system, theGPS and the dead reckoning calculate the position
independently and then Kalman Filter estimates the
position using the position data.
The measurement equation of the GPS can be
described as follows:

Supposing the road is perfectly planar and horizontal, as
the motion is locally circular as shown in Fig.1.

P  R (1)

Here P is the length of the circular arc followed
by P ,  , R (the radius of curvature), I (theinstantaneous center of rotation).
Supposing the car is moving forward, the variationon the position is expressed as:x  Pk Pk 1  cos k  k / 2y  Pk Pk 1  sin k  k / 2

1 0 0
Hk  0 1 0
0 0 1(6)

Here the measurement vector z   xG yG G T is
composed of the position measured with the GPS. The
measurement noise v is regarded as the White Gaussian
noise.
In order to estimate the position of the vehicle, theproposed Kalman Filtering was applied from state
equation (4) and measurement equation (5):approximate P  Pk Pk 1 . The integration process is
then:



(3)

xk 1  f (xk , uk , 0)
Pk 1 k Pk Tk  Q
Kk  Pk  H k T ( H k Pk  H k T  R )1

In Fig.1, the distance traveled P , and the angle
changed  , resulting from the movement Pk 1 form
x k  x k  K k (z k  H k x k )Pk can be calculated in terms of the incremental

Pk  ( I  Kk H k ) Pk changes of the dead reckoning measurements of theright and left wheel motions[4,5].

©ISAROB 2011

(5)

(2)

In general, the sampling rate of state is very small
compared to their rate of change, so we canxk 1  xk  P  cos(k   / 2)
yk 1  yk  P  sin(k   / 2)
K 1  k  

zk  Hk xk  k

835

(7)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Here P is the error covariance matrix, K is the
Kalman gain, and the Jacobian matrix k is given as
follows:

Prior estimate x 0 and its error covariance P0 
Compute Kalman gain

K k  Pk  H k T ( H k Pk  H k T  R )1

Update estimate with measurements

Project aheadx k 1  f (xˆ k , u k , 0)

k 1

P

xˆ k  xˆ k   K k (z k  H k xˆ k  )  k Pk   Q
TkFig. 3 Autonomous Vehicle.

Compute error covariancePk  ( I  K k H k ) Pk 

Fig. 2 Kalman filter loop.

1 0 P sin(k   / 2) k  0 1 P cos(k   / 2)
0 0

1

(8)

Fig. 4 PCI 7841.
Experiment was done with path set on the ground as
10km/h.
Fig. 5 shows the test area. The result of position
location does not follow the reference path because
velocity data consists of constant value. So its reliability
is less than other external sensors. However,autonomous driving was done without additional
external sensors.
GPS also does not follow the reference path. Since
the experiment was done in an open area, the error
element is bias error.

Equation (7) comprises the Kalman filter recursive
equations. It should be clear that once the loop is
entered, it can be continued infinitum. The pertinentequations and the sequence of computational steps areshown pictorially in Fig. 2[6].

IV. EXPERIMANT
Fig.4 is a vehicle (MOHAVE of KIA motors) used
for experiments. A control PC acquires velocity andsteering angle data from vehicle through the CAN
Communication. Fig. 4 is PCI 7841 by ADLINK whichtransmit CAN data from the Local CAN Network to acontrol PC. For autonomous driving, it requires current
vehicle position and heading angle. In this paper,
ProPak-V3 of Novatel's GPS receiver is used as the
localization system. And two antennas, GPS-701-GG
are installed on the autonomous vehicle's roof in a row
for getting heading angle.

Fig. 5 Test Area.

©ISAROB 2011

836

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011[3] J. Y. Park, D. S. Kang, and J. H Lee, “Kernel RLS
Aided Kalman Filtering for Integrated GPS/INS,”KIIS Spring Conference 2009. Vol. 19, No. 1, 2009.
[4] K. S. Lee, M. K. Kim, and M. H. Lee, “Control of
Unmanned
Container
Transporters
andManagement System Development,” Korea
Association of Automatic Control 2009.
[5] C. S. Kim, J. H. Park, and M. H. Lee, “Improving
odometry accuracy for car-like vehicles by using tireradii measurements,” Industrial Electronics Society,2004. IECON 2004. 30th Annual Conference ofIEEE Vol. 3 pp.2546–2551, 2004.
[6] Robert G. B and Patric Y. C. Hwang, Introduction to
Random Signals and Applied Kalman Filtering: with
MATLAB exercises and solutions, Wiley & Sons,
1997.

Fig. 6 Experiment Result.

V. CONCLUSION
In this paper, we suggest an autonomous vehicle's
position location for navigation system. To get
successful operation, absolute position and heading
angle are required. GPS and dead reckoning used
separately to get the current position of the vehicle does
not warranty successful operation. GPS and deadreckoning both have error elements. Dead reckoning hasconstant velocity data and accumulation of the error
element. So it needs to be compensated with algorithm.
For successful operation of navigation system, theKalman filter is suggested in this paper. The trace of
wheels is stable with Kalman filter than using GPS anddead reckoning separately. GPS can compensate theerror accumulation of dead reckoning. Results indicate
good performance with GPS and the internal system ofdead reckoning through the Kalman filter.

REFERENCES[1] Keiichi Yamada, Tomoaki Nakano, and ShinYamamoto, “A Vision Sensor Having an Expanded
Dynamic Range for Autonomous Vehicles,” IEEE
transactions on vehicular technology, Vol. 47, No 1,
pp. 332-341, 2007.
[2] Philip G Mattos, “Integrated GPS and Dead
Reckoning for Low-cost Vehicle Navigation andtracking,” Vehicle Navigation and Information
Systems Conference Proceedings of IEEE. pp. 569574, 1994.

©ISAROB 2011

837

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Motion Analysis of Towed Vehicle on Survey System for Deep Sea

Kohei Oshima, Etsuro Shimizu, Masanori Ito
Tokyo University of Marine Science and Technology
Abstract: The aim of this paper is to develop a survey system for deep sea with high accuracy andspeed. The proposed survey system consists of tow ship and towed underwater vehicle equipped withsensors. A towed underwater vehicle is thought of as an effective method to survey the bottom of theseabed at large depths.In order to develop the system, motion analysis of towed vehicle for survey
system is carried out in this paper.

Keywords: Underwater vehicle, BELUGADEEP
there is no screw-noise.

I. Introduction

In the use of the towed vehicle, keeping positionand stable movement are required. The shape of

It is necessary to survey an exact depth when land

towed vehicle is important for keeping position

reclamation works are carried out in the ocean.

and stable movement.

The sonar equipped on the ship is used in ordinary

In this study, fluid analysis of the towed vehicle issurvey. However, reclamation area has been

carried out. The effect of the change of body

expanded and deeply in recent years. The

shapes and the effect of the change of attachment

ordinary survey method cannot provide theposition are analyzed. Solidworks and Flowworks

required accuracy. Therefore, a new survey

including Solidworks are used for analysis.

method that using an underwater robot equipped

Fig. 1 shows survey method using underwaterwith sensor is proposed. Underwater robots are

towed vehicle.

divided into Autonomous Underwater Vehicles(AUVs), Remotely Operated Vehicles (ROVs)
and Towed Vehicles. When underwater towed
vehicle compared to other underwater robots canobtain advantages such as:(I) Vehicle position can be obtained easily.
(II) Vehicle can operate for a long time, because
power can be transmitted through the cable.
(III) The data can be transmitted in real time.
(IV) High accurate data can be obtained because©ISAROB 2011

Fig. 1 Survey method

838

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

II. Analysis model
1.

2. Rear attachment
Fig. 4 shows a rear attachment for vehicle.

Body

The model created in Solidworks is shown in Fig.

Attachment is set up in rear of BELUGA DEEP.

2. The origin of the analyzed model is BELUGA

Rear attachment can change each angle in a

DEEP shown in Fig. 3, which is developed byperpendicular plane or horizontal plane. The

TOA-CORPOLATION. Table. 1 shows summary

effect of attachment is to stabilize posture ofof the vehicle.

vehicle and the effect like rudder. And them, theattachment changes each angle, a flexible action

Table. 1 Summary of the BELUGADEEP

can be done in a changeable real sea area. And,

1.45[m]*0.74[m]*0.84[m]when the ship turns, it is possible to follow

Size

quickly.

(L*W*H)
Mass

140[kg]Depth rating

150[m]In this time, the angle changes in only aperpendicular plane. The role like rudder is given
to attachment.

Towed Speed

3[knot]Multibeam

Seabat 8101(RESON)

Echosounder
1～400[m],0.01[m]Attitude and

AHRS440(Crossbow)

Heading±0.1[deg]
Micro P(Applied
Depth

Microsystems)

Navigation

0.05% of depth

Fig. 3 analyzed model

6000[m]Fig. 4 Vehicle and attachment
Fig. 2 BELUGADEEP©ISAROB 2011

839

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Ⅲ. Fluid analysis

rudder. Because, the pressure that hangs to the

1. Analysis conditionattachment is larger than the pressure that hangs

The model is analyzed on the condition that Table.

to the tow body. This is the same also in the

2. Condition other than Table. 2 are defaults

simulation by 2[knot] and 5[knot].

condition. When analyzing it, A, B, C in Fig. 4 isIf the angle of 20[deg] attaches to the flow in thechanged, and each result is considered.

speed of 2[knot], forecasting the behavior isdifficult because the pressure in front of the stick
distributes like spots. If the angle of 30[deg]Table. 2 Analysis condition

The fluid liquid

Water

attaches to the flow, because the differential
pressure on the front side and the back of the stick,

Temperature of

280[K]

liquidstability will be ruined.

Water pressure1114.575[kPa]

When the angle of 10[deg] attaches to the flow.

Type of analysis

External flow

There is no large difference in the pressure

The velocity ofa moving fluid
Resolution levelof the result
The degree of
A,B,C in Fig .4

distribution that hangs to the stick. From this, the2[knot],5[knot]

attachment will be not likely to ruin stability
though keeps the function of the rudder.

level 3 of 8 level

It is preferred that the applied angle of theattachment is about between from 10[deg] toeach angle is20[deg] to the flow.

10[deg] or 20[deg] or

For the simulation by 5[knot], it is roughly

30[deg]consideration similar to the case of simulating itby 2[knot]. However, the influence with the

2. Simulation result

attachment grows because the difference of the

The angle and the speed are changed, 24 patterns

pressure distribution is large for simulating it by

in total are simulated. Distribution of the pressure2[knot].

and path of particle are calculated.
The sub title (a), (b) of Fig. 5 and Fig. 6 meanB.

“velocity [degree of A/ degree of B/ degree of

The path of particle around the vehicle andC]”.

attachment is displayed in Fig. 6.

Path of particle

There is no difference in the shape of the path of©ISAROB 2011

A. Distribution of the pressure

particle when 2[knot] and 5[knot] are compared.

The distribution of the pressure that hangs to the

It is thought that tracks of the fluid do not change

vehicle and the attachment is displayed in Fig. 5.

depending on the speed. The path of particle

From the pressure that hangs to the tow body andchanges greatly by angle about the attachment

the pressure that hangs to the attachment, it islarge to the flow. Stability will be ruined by thepossible that the attachment functions enough as alarge change of the path of particle. Therefore, it

840

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011is necessary not to change the angle of theattachment too much greatly.
Moreover, when the angle of A of fig. 4 changes.
The fluid flows into the joining section of the tow
body and the attachment. The whirlpool occurs if
the fluid flows in the small place. The existenceof the whirlpool will ruin the stability of the tow
body. Therefore, A of Fig. 4 should be 0[deg]. It isnecessary to get the effect as rudder by changing
B and C.

(a)(b)
Fig. 6 Path of particle((a)5knot[-30/0/0],(b)5knot[-10/0/0])

Ⅳ. Conclusion
In this paper, the survey system for deep sea withhigh accuracy and speed was developed. The(a)2knot[-10/-10/-10]

proposed survey system consists of tow ship andtowed underwater vehicle.
The following results were obtained.
The attachment is effective, and carries out thefunction as the rudder. However, ruining thestability of the vehicle is predictable under aspecific situation.

Reference(b)5knot[-10/-10/-10][1] 3DCAD system practical guide Solidworks(In Japanese)

Fig. 5 Distribution of the pressure[2] Solidworks simulation Tutorial (in Japanese)[3] http://www.toa-const.co.jp/©ISAROB 2011841

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Path planning of an autonomous mobile robot
considering region with velocity constraint in real environment
Tae Hyon Kim∗ , Kiyohiro Goto† , Hiroki Igarashi∗ , Kazuyuki Kon∗ ,Noritaka Sato∗ and Fumitoshi Matsuno∗∗ Kyoto University, Yoshida Honmachi, Sakyo-ku, Kyoto, 606-8501, Japan†The University of Electro-Communications, 1-5-1 Chofugaoka, Chofu-shi, Tokyo,182-8585, Japan.
(Tel : +81-75-753-5235; Fax : +81-97-594-5235)(t.kim@ky2.ecs.kyoto-u.ac.jp)
Abstract: Recently, various autonomous mobile robots are tried to drive in the real world in many researches
or competitions. In this case, it is very important for the robot to identify the self-position and orientationin real time. Therefore, we apply a localization method based on particle filter to the autonomous robot.
Moreover, in order to improve the safety of autonomous locomotion, we improve the algorithm for path
planning and trajectory generation so that it can consider the regions with the limitations of maximum
velocity. In order to demonstrate the validity of the proposed methods, we will participate in the Real World
Robot Challenge 2010.
Keywords: Autonomous robot, trajectory generation, localization

1

Introduction

locations. Therefore the localization method fully depends on GPS is not suitable for such environment.
In this study, we improve the trajectory generation
method so that it can consider not only the registeredobstacles but also the unknown or dynamic obstacles,
while considering the RVCs as in [7, 8]. Moreover,
we improve the accuracy of the localization by combining gyroodometry and particle filter based localization method. We validate the improved system in
RWRC2010. The contents of this paper are as follows. In Section 2, we improve the trajectory generation method, and in Section 3, describe localization
method used in this paper. In Section 4, we show thedeveloped robot system and show some experimental
results. We conclude this paper in Section 5.

Recently, many researches related to the autonomous mobile robot have been reported. Various
competitions for autonomous robots have also been
held to validate the autonomous locomotion functionsin the public environment (see e.g [1, 2, 3, 4]).
We have participated in RWRC since 2007 (see[5, 6, 7]), to validate our autonomous robot systemin the real environment. In RWRC, the robot has tomove within the general pedestrian paths including
general people, unicycle and etc. Therefore, unlike theother competitions, the robot is required not only theautonomous locomotion functions but also the safetyof the autonomous locomotion. In [7, 8], we proposeda trajectory generation method considering “The Region with Velocity Constraint” (RVC) to improve thesafety of the autonomous locomotion. The RVCs are
the region where the velocity of the robot is restricted
to the predefined velocity, and introduced to aroundthe hazardous regions by the operator. The trajectory
generation method in [7, 8] generates the trajectory
avoiding the hazardous regions or reducing the risk in
the hazardous region by constraining the velocity according to RVCs. However, there is a limitation thatthe method cannot consider the unknown or dynamic
obstacles (e.g. humans and unicycles), because it generates the trajectory based on the predefined map.

2

As mentioned before, the trajectory generation
method[8] considers RVCs. The RVCs are the region where the velocity of the robot is restricted tothe predefined velocity, and introduced to around thehazardous regions by the operator in advance. For
example, the hazard areas where the robot is not allowed to enter (e.g steps, neighborhood of the pond)
are registered as the regions with velocity constraint
vmax = 0, and the area where the robot should reduce the velocity (e.g rough road, narrow street) areregistered as the regions with the velocity constraint
vmax < 1.0 (see Fig. 1). Since the robot plans the
trajectory according to the registered information, it
is expected that the predicted hazard can be avoided.
(e.g. reduction of the shock if the robot collides an
obstacle, passing smoothly at the complicated narrowroad)
In [8], the gird map including the RVCs, the known
obstacles and waypoints information is given in advance, and is used for trajectory generation. There-

Moreover, in [6], we estimate the position of therobot from the odometry data and GPS data by using
the extended kalman filter. In general, if the satellitecondition is good, we can obtain the accurate position
data from GPS by using diﬀerential GPS or Real time
kinematic GPS. However, since there are many obstacle (e.g. trees, buildings) within the course of RWRC,
we could not obtain the GPS data with enough accuracy for correcting the position of the robot at many

©ISAROB 2011

Trajectory generation with RVCs

842

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig.1 The concept of RVCs

Fig.3 Example of raw odometry and gyroodometry

3·1

The gyroodometry is a method estimating a poseof the robot from the odometry and gyro sensor data.
In this study, we use the 1-axis optic fiber gyro, andgyro sensor data is used for yaw rate.
Fig. 3 shows one of our experimental results. In
this example, the robot is operated by manually on
the campus road at Kyoto University from start position to goal position according to the direction ofarrows(about 500[m]) as shown in Fig. 3(a). Fig. 3(b)shows the trajectory of the robot estimated by raw
odometry and gyroodometry. Note that, the start
position and goal position on the map were set at(x, y) = (0, 0). From Fig. 3(b), it can be see thatthe gyroodometry shows much better result comparedwith the raw odometry. In fact, the error between
start and goal position is bounded less than 2[m].

Fig.2 Example of generated trajectoryfore, the generated trajectory avoids only the registered obstacles. However, since there are unregisteredobstacles (e.g. humans and bicycles) in the real world,
the robot also has to avoid such dynamic obstacles.
To this end, we modify the grid map used in trajectory
generation so that it can take into account unknown
and dynamic obstacles. More precisely, we generatea new map in each sampling time by combining thepriori grid map and current obstacle information obtained from sensor. Then the trajectory is generated
based on this map in each sampling period.
Fig. 2 shows an example of the trajectory generated
by two methods. Fig. 2(a) shows the experimental
settings. In Fig. 2(b), a red line shows a trajectory
generated by the past work and a blue line shows one
by the method described in this section. From thisfigure, we can see that the trajectory, generated bypast work, collides with the unknown obstacle (i.e.
an unregistered obstacle). Therefore, since we haveto use external obstacle avoidance function to apply
this trajectory to the real robot, the optimality ofthe trajectory might be reduced. On the other hand,the proposed method generates the optimal trajectory
while avoiding every obstacles.

3

3·2

Pose correction by using particle filter

In order to correct the gyroodometry data, we use
a Monte Carlo Localization(MCL) method[9]. MCL
is a method for estimating the pose of the robot basedon Particle Filter(PF). In this study, we use a priori
2D grid map and range data from a laser range scanner for MCL. The outline of our localization method
is as follows: First, the pose of the robot in each particle are calculated based on gyroodometry. Next,
the range data obtained from LRF is converted into
global map based on the pose of the robot in each particle. Then the weight of each particles are calculated
by comparing the priori grid map and the converted
obstacle map in each particle. Finally, resample theparticles according to their weight. By iterating above
procedure in each sampling period, the pose of therobot is estimated.
The priori map used in MCL is generated according to the following procedure. First, we obtain 3D
range data from the 2D laser range finder mounted
on a pitch rotating platform. In RWRC, since many
dynamic obstacles(e.g. humans, unicycles) are exist,
much noise is included in the lower parts of 3D range
data. In order to reduce aﬀection by noise, we use
only the 3D range data in the part higher that 2[m]height. The partial 3D range data is converted into
global map according to gyroodometry and converted

Localization method

The localization method used in this study can bedivided into two part: gyroodometry and particle filter based localization. First the accuracy of the odometry is improved by using gyroodometry[10] based on
the fiber optic gyro (Sec. 3·1). Second, the gyroodometry data is corrected by the particle filter based localization method using the priori map and current obstacle information obtained from a laser range scanner(Sec. 3·2).

©ISAROB 2011

Gyroodometory using optic fiber gyro

843

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig.4 Example of the priori map

Fig.6 The full course of the RWRC2010

Fig.5 System configuration of the developed robot

into 2D map with orthogonal projection to groundplane(e.g. Fig. 4(a)). Then, the map is saved as thegrid map to reduce the memory (e.g. Fig. 4(b)). In
the case of Fig. 4, Fig. 4(b) is used as a priori map in
PF based localization.
Note that, as mentioned in Sec. 2, this priori map
is used for trajectory generation as well. For trajectory generation, waypoints, RVCs and current obstacle information are included to this grid map in each
iteration,

4

Fig.7 The estimated trajectory in Experiment 1

240[m]) and full course (about 1100[m]). Experiment
1 is done in the full course, and Experiment 2 and 3
are done in the trial course, because of the restrictionof time in RWRC2010.
4·1

Experiments

The trajectory generation method in Sec. 2 and the
localization method in Sec. 3 are applied to the developed robot as shown in Fig. 5. The developed robot
equips a GPS, a fiber optic gyro, a 3D laser scanner,encoders, an emergency stop buttons and a 2D laser
scanner to detect obstacles. Note that the software is
implemented by using RT-Middleware[11].
In order to verify the developed system, we executethe following three experiments in RWRC2010.
Experiment 1 Verification of the performance withthe fiber optic gyroodometry described in Sec. 3·1

4·2

Experiment 2 Validation of trajectory generation
method described in Sec. 2

Experiment 2

In Experiment 2, the RVCs are set as shown in
Fig. 8(b). The robot moves autonomously according to the trajectory generation method described inSec. 2. The gyroodometry data is used for localization(i.e. PF is not used in this case).
Fig. 9 shows the translational velocity of the robot

Experiment 3 Verification of particle filter based localization method described in Sec. 3·2
In RWRC2010, as shown in Fig. 6, the course isdivided into to two parts: the trail course (about©ISAROB 2011

Experiment 1

Fig. 7 shows the experimental result of Experiment
1. In this experiment, the robot is operated manually.
From Fig. 7, it can be seen that the gyroodometry
reduces the estimation error compared with the raw
odometry data. However, compared with the course
in Fig. 6, the estimated trajectory diﬀers from theglobal map as the robot moves long distance. Especially, we can see that the trajectory curves slightly
even in the straight line course. This is mainly because the drift of the gyro sensor. Therefore, although the gyroodometry is enough accurate for ashort course, the correction by PF is necessary to run
the long course.

844

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig.8 RVCs for the trial course

Fig.10 The estimated trajectory in Experiment 3

the validity of our system. One of the future works
is to predict motion of the dynamic obstacles, andimprove the safety of the autonomous locomotion byconsidering their information.

Acknowledgements
This work was supported by commissioned businessof New Energy and Industrial Technology Development Organization (NEDO).

Fig.9 Translational velocity profile in Experiment 2

Referencesin Experiment 2. In Fig. 9, a red line shows a admissible maximum velocity of the robot according to theregistered RVCs information. From Fig. 9, we can seethat the robot reduces the velocity before moving into
the each RVC enough, and satisfies the velocity constraint of each RVC. Moreover, the robot could reachgoal position of the trial course without colliding any
obstacle, even though there were many unknown anddynamic obstacles. Therefore, it can be said that the
improvement of the method works correctly.
4·3[1] Urban Challenge,
http://www.darpa.mil/grandchallenge/index.asp[2] Intelligent Ground Vehicle Competition,
http://www.igvc.org/[3] Thailand Intelligent Vehicle Challenge,
http://www.ise.ait.ac.th/TIVChallenge/index.htm[4] Real world robot challenge,
http://www.ntf.or.jp/challenge/challenge10/index.html[5] K. Miyazawa, et al., Development of an autonomous
mobile robot in outdoor environment based on Sensor Fusion For RWRC , Proceeding of SICE System
Integration , pp.965-966, 2007 (in Japanese)[6] K. Kon, et al., Development of RTCs for mobile
robots with autonomy and operability Report 7, Proceeding of SICE System Integration , pp.381-382,
2008 (in Japanese)[7] K. Goto, et al., Development of RTCs for mobile
robots with autonomy and operability Report 13,
Proceeding of SICE System Integration , 2B1-1, 2009
(in Japanese)[8] K. Goto, K. Kon and F. Matsuno, Motion planningof an autonomous mobile robot considering regionswith velocity constraint, IEEE, Int. Conf. on Intelligent Robot and Systems, pp.3269-3274, 2010[9] S. Thrun, W. Burgard and D. Fox, Probabilistic
Robotics”, The MIT Press, 2006[10] J. Borenstein and L. Feng, Gyroodometry: A new
method for combining data from gyros and odometry
in mobile robots, Proc. of IEEE Int. Conf. on robotics
and automation, pp.423-428, 1996[11] RT-Middleware, http://www.is.aist.go.jp/rt/

Experiment 3

Fig. 10 shows the experimental result of the Eexperiment 3. In Fig. 10, a green line shows a trajectorywhen the priori grid map is generated, a red line showsa trajectory estimated by the gyroodometry, and ablue line is a trajectory estimated by the particle filter. In other word, a green line is a ”true” trajectoryin this case. From Fig. 10, we can see that the trajectory estimated by gyroodometry diﬀers from theone in the priori map (i.e. a green line). On the other
hand, the trajectory estimated by PF agrees well withthe green line. Therefore our particle filter based localization method estimates the position correctly.

5

Conclusions

In this study, we improved the trajectory generation
method proposed in [8] and the localization method
for an autonomous robot in the real environment. We
participated in RWRC2010 to validate the developed
system, and some experimental results showed that

©ISAROB 2011845

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A Study of SVM using the Combination with Online Learning Method andMidpoint-Validation Method
Shingo Yamashita, Takeshi Yoshimatsu, Hiroki Tamura and Koichi Tanno
Dept. of Electrical & Electronic Engineering, University of Miyazaki, 1-1 Gakuen-kibanadai-nishi, Miyazaki-city,
Miyazaki, 889-2192 Japan(Tel : +81-985-58-7409; Fax : +81-985-58-7409)(htamura@cc.miyazaki-u.ac.jp)

Abstract: The support vector machine (SVM) is known as one of the most influential and powerful tools for solving
classification and regression problems, but the original SVM does not have an online learning technique. Therefore,
many researchers have introduced online learning techniques to the SVM. In our former paper, we proposed an
unsupervised online learning method using the technique of the self-organized map for SVM. In other paper, we
proposed the midpoint-validation method for the improvement of SVM. Therefore, we test the performance of SVM
using the combining of two techniques in this paper. Moreover, we compare its performance with the original hardmargin SVM, soft-margin SVM and k-NN method, and also experiment our proposal method on surface
electromyogram recognition problems with changes in the electrode position. From these experiments, our proposal
method has the best performance in the technique of other SVM and corresponds to the changing data.
Keywords: Support vector machine, Online learning, Midpoint-validation, Pattern classification problem, Surfaceelectromyogram.

two techniques in this paper. Moreover, we compare its
performance with the original hard-margin SVM, softmargin SVM and k-NN method, and also experiment
our proposal method on s-EMG recognition problems
with changes in the electrode position.

I. INTRODUCTION
The SVM proposed by Cortes and Vapnik [1] is one of
the most influential and powerful tools for solving
classification problems [2][3][4][5].
We are studying the surface-electromyogram (s-EMG)
recognition of using SVM. The purpose is developmentof the human interface of using s-EMG. In this study, we
pay attention to problems such as the change of the sEMG pattern by the muscle fatigue and the position gapof the sensor to measure. And in our former paper [6],we proposed the online unsupervised learning method
using a technique of self-organized map for a SVM.
Furthermore, the proposed method has a technique forthe reconstruction of a SVM.
In addition, we are studying SVM which is not limited
to recognition of s-EMG. In this study, we pay attention
to a problem of the deflection of the separating
hyperplane in the input space of non-linear SVM andproposed the improvement method [7][8]. We call thismethod Midpoint-Validation Method. This Method
assumes a Midpoint between the classes of training data
an index of the deflection and moves the separating
hyperplane according to the index. This method also has
a technique for the reconstruction of a SVM.
These two studies achieve good result each. However,
we have not combined two methods so far. Therefore,
we test the performance of SVM using the combining of©ISAROB 2011

II. PROPOSED METHOD
In this section, we introduce SVM, our Online Learning
Method and Midpoint-Validation Method. And we
propose the method that combined Online Learning
Method with Midpoint-Validation Method.
1. SVM
The SVM is a mechanical learning system that uses ahypothetical space of linear functions in a highdimensional feature space.
Nonlinear SVM is expressed by the three equations(Eq. (1), Eq. (2) and Eq. (3)). Here, we used theGaussian kernel given in Eq. (1) as the kernel function,
while the SVM decision function g(x) and the output of
the SVM are as given in Eqs. (2) and (3).

  || x  xi || 2 
K (x, xi )  exp 
2 2



(1)

N

g (x)   wi K (x, xi )  b
i 1

846

(2)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011O  sign( g (x))(3)

2. Online Learning Method [6]In this subsection, we introduce unsupervised online
learning method using SOM algorithm for SVM andrestructure technique.
Let the input space be denoted by xin∈R. xin ( in∉{i=1,...,N}) is the input vector without the label. The
training vectors are included in kernel function, xi with i= 1,…, N , belongs to either of the two classes. Thus
these are given a label yi ∈{-1,1}. Each training vector
has the same dimension of input space.
Next, the flows of our online learning method areshown in Fig.1.
Fig.2. The flow of Online Learning Method
Step 1: To find the smallest distance of input vector
xin with the training vectors xi , the Euclidean
distance between xin and each xi is compute(Fig.2.a).
Step 2: The following processing (Step 3-4) are notdone to xin when the label of xwin is not thesame as the label of the output result of
SVM of xin.
Step 3: To find the smallest distance of the xwin withthe training vectors xj, the Euclidean distancexwin and each xj is computed (Fig.2.b).
Step 4: If dw is condition of rule of Eq. (4), xwin is
update according to the learning rule ofEq. (5) (Fig.2.c and Fig.2.d).
Step 5: Step1-4 are done to all input vector.
Step 6: Proposed method repeats Num cycles theseprocessing (Step1-5) using same inputs vectorxin.

d w  dO

(4)

old
old
x new
win  x win   (x in  x win )(5)

Fig.3 Adjustment value and improvement rate
(Pima-indians-diabetes data)

Step 1: SVM is created by the known training data.
Step 2: Create the midpoint data.
Step 3: The output value of SVM of the midpoint
data and training data computed.
Step 4: The value of B is computed according to
Eq. (8).
Step 5: The output of the proposed SVM is also
computed according to Eq.(6) and Eq.(7).

Fig.1 Flow of the Online Learning Method
Parameter ηis update parameter. The idea of thisrule is an idea near the adaptive resonance theory-like.
If SV changed after the update, SVM is restructured
using the updated training vectors. Even if training
vectors changes using the Step 1-5, maximizing themargin of SVM is kept from this restructuring
processing.

h(x)  g (x)  B
O  sign(h(x))
1 M
B    g (x m )M m1

3. Midpoint-Validation Method [7][8]In this subsection, we introduce Midpoint-Validation
Method. Midpoint-Validation Method adjusts the SVM
output with Midpoint data. Here, Midpoint is a point tobe located on midway between two classes on input
space. In Fig.3, we showed a change of the
improvement rate when Midpoint-Validation Method©ISAROB 2011

(6)
(7)
(8)

Fig.4 Flow of the Midpoint-Validation Method
added adjustment value B to the SVM output. We
showed a procedure of Midpoint-Validation Method in
Fig.4.

847

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

The experiment method, first acquires the training datafrom s-EMG concerning the movement of forearm.
Next, SVM and C-SVC learn the relation between sEMG and motion from the training data (the training
vectors). And, each motion is identified 60 times. Next,
the object moves the electrode position (sensor 1) by
2mm. And, additional unsupervised learning data (theinput vector: each motion is 40 times) is obtained fromeach motion. Afterwards, test data for recognition rate
calculation is identified 20 times of each motions. The
experiments tested the measurement four times in total
by moving the electrode position of 2mm, 5mm, 7mm
and 10mm (Fig.6).
The base of proposed method is hard-margin SVM
using Eq. (1). Gaussian kernel parameters of SVM were
decided from the evaluation that used training data.
Subject T.Y was 0.7, K.Y was 2.0, S.Y was 0.9, and T.M
was 0.3. In these experiments, the value of parameter η
was 0.1. And the value of Num was15.

By past study, we know that this method improvesuccess rate with high probability when satisfies
condition |B|≧0.50. And we know that the classifier is ahigh tendency when the value of |B| is small [8].
4. Combination with Online Learning Method
and Midpoint-Validation Method
In this subsection, we propose combination method
using above-mentioned 2 methods. The procedure atfirst, we perform Online Learning Method. Next, wecreate SVM by the updated training data. Finally, we
perform Midpoint-Validation Method. By this procedure,
we can complete the proposed method.

III. EXPERIMENTS
In this section, the system configuration for recognition
experiments of forearm motions using s-EMG is
explained. Next, the result of computer simulations is
described.

3. Experimental Result
We performed with each method and the simulation
results are table 1. In table 1, we showed the average ofsuccess rate. Here, “k-NN with Online Learning” is kNN method using training data updated by Online
Learning Method. From the table 1, proposed methods
are the highest value and the second highest value.
In Fig.7, we showed the success rate by 3 methods,
Proposed Method (|B|≧0.25), SVM, k-NN with Online
Learning. The horizontal axis expresses the sensor
position. Proposed method has improved SVM at 14

1. Experimental Condition
S-EMG of each movement pattern is measured withelectrode sensors, and the feature quantity is extracted
from the s-EMG. The feature quantity is given to therecognition machine as an input and each movement
pattern that generates s-EMG is presumed. The feature
quantity uses minimum-maximum (abbr. min-max)
values and integration values [9]. Paper [9] showed thattechnique of min-max values and integration values are
more easy and superior to FFT processing. The
sampling frequency of the measurement data is 1 KHz.
And the band is from 0 Hz to 500 Hz.
2. Experiments of Forearm Muscles
We experimented on the effectiveness of the proposed
method by the s-EMG recognition problem that thefeature quantity changes by the electrode position. We
compared proposed method performance with Online
Learning Method, Midpoint-Validation Method, theoriginal hard-margin SVM, soft-margin SVM (C-SVC)
and k-NN method. Proposed method and MidpointValidation Method have two results.
The experimental subjects are 4 healthy men (T.Y,
K.F, S.Y, T.M). The subjects sit on a chair. The
recognition experiment of the 6 motions pattern isconducted by using s-EMG obtained from four sensors
set in the arm of the right hand (Fig.5). Moreover, theinput given to the identification machine is eight inputs.
The experiments are conducted for one day.

©ISAROB 2011

Fig.5. Image figure of forearm motion

Fig.6. S-EMG recognition problems with changes in theelectrode position (2mm, 5mm, 7mm and 10mm)

848

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 1. Success rate average [%]Proposed
Method(|B|≧0.25)

Proposed
Method(|B|≧0.50)

Midpoint
Validation(|B|≧0.25)

Midpoint
Validation(|B|≧0.50)

SVM with
OnlineLearning

SVM

C-SVC

k-NN with
OnlineLearning

k-NN

82.8

82.7

78.8

79.9

80.7

77.8

76.1

80.8

79.8

Fig.7 Experimental results of four subjects[3] Joachims T (1999) Making large-scale support vector
machine learning practical. In: Shoelkopf B, Burges C,
Smola A (eds) Advances in kernel methods: support
vector learning. pp 169-184[4] Mangasarian OL, Musicant DR (2000) Active
support vector machines. Technical Report 00-04, Data
Mining Institute, Computer Sciences Department,
University of Wisconsin, Madison, WI, April[5] Mangasarian OL, Musicant DR (2001) Lagrange
support vector machines. J Mach Learn Res 1:161-177[6] Tamura H, Yoshimatu T, Tanno K (2010) Support
Vector Machines with Online Unsupervised Learning
Method and its Application to s-EMG Recognition
Problems. NOLTA 2010[7] Tamura H, Tanno K (2008) Midpoint-validation
method for support vector machine classification. IEICE
Trans Inform Syst, vol E91-D, No. 7, pp 2095–2098[8] Yamashita S, Tamura H, Toyama T, et al (2010) The
Effectiveness of Midpoint-Validation Method for
Support Vector Machines (in Japanese). Proceedings ofElectronics, Information and Systems Conference ; IEEJ,
CD:GS12-5[9] Tamura H, Gotoh T, Okumura D, et al (2009) A
Study on the s-EMG Pattern Recognition using Neural
Network. International Journal of Innovative Computing,Information and Control, Vol.5, No.12(B) pp.4877-4884

items among 16 items. In comparison with k-NN with
Online Learning, Proposed method gets high success rate
generally. From these simulation results, it is shown thatthe combination of proposed method is effective.

VI. CONCLUSION
In this paper, we proposed the improvement method for
SVM that is the combination with Online Learning
Method and Midpoint-Validation Method. The
experiment results showed that the proposed method
was effective to s-EMG recognition problem withchanges in the electrode position. SVM had improved byusing our proposed method. In future work, we will
experiment on the effectiveness of the proposed methodby the other recognition problems.

REFERENCES[1] Cotes C, Vapnik V.N (1995) Support Vector
Networks. Machine Learning, 20(3), pp. 273-297[2] Burges CJC (1998) A tutorial on support vector
machines for pattern recognition. Data Mining andKnowledge Discovery, vol 2, no 2

©ISAROB 2011

849

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Video Object Segmentation Using Color-Component-Selectable
Learning for Self-Organizing Maps
Shin-ya Umata, Naotake Kamiura, Ayumu Saitoh, Teijiro Isokawa and Nobuyuki Matsui
Division of Computer Engineering, Graduate School of Engineering,
University of Hyogo, 2167 Shosha, Himeji, 671-2280, Japan
Abstract: In this paper, self-organizing-map-based video object segmentation is proposed, assuming that either
Y-quantification or HSV-quantification can be systematically selected. Given a video sequence, the value of
probability density function is calculated for each component value according to kernel estimation at the first fame.
Some areas randomly chosen from the background are then examined, using each component value, whether it is
misjudged that they include the target object. The quantification is determined so that occurrence frequency of theabove false extraction can be reduced. The data presented to maps are generated, based on the selected quantification.
Experimental results show that the proposed method well recognizes the target object.
Keywords: self-organizing maps, block-matching-based learning, video object segmentation

with it is relevant to the target object. Some rectangles
are next randomly clipped from the background. They
are examined, using labeled component values, whether
units in them are accurately classified as part of the
background. The quantification selection depends on thenumber of units accurately classified in such manner.
The training data are generated according to the selected
quantification, and a map for segmentation is constructedby T-BMSOM learning using them. Experimental
results reveal that the quantification selection works wellfor the segmentation without excessive overextraction.

I. INTRODUCTION
A number of video object segmentation algorithms[1]-[4] have been developed, based on soft computing.
In [3] and [4], fast block-matching-based self-organizing
maps (SOM’s) referred to as T-BMSOM’s are employed.
In them, a rectangular area covering a target object in aframe of a given video sequence is spilt into units withsome pixels. The area is referred to as a window. A
unit has a vector with element values associated with Y
color component. The color attribute of the window istherefore quantified by the above vectors of all units in it.
The map for segmentation is constructed by using suchvectors as training data. The vectors corresponding tounits in the subsequent frame under segmentation are
also presented to the map. The map then judges
whether the unit corresponding to the presented vector
belongs to the target object.
The adequacy inquantifying color attributes is thus strongly related tosegmentation capability of the map. It is clear that theabove Y-quantification does not always fit to arbitrary
video sequences.
This paper proposes map-based video object
segmentation, assuming that the quantification is
selectable. The HSV-quantification is prepared inaddition to Y-quantification.
The quantification
selection is made with the first frame in a given video
sequence. The value of probability density function is
calculated for each component value according to kernel
estimation[5], using the area with the target object. The
function value associated with some component value
determines the label to be assigned to that component
value. The label specifies whether the component value©ISAROB 2011

II. PRELIMINARIESIn this paper, T-BMSOM learning[3] is adopted. A
block is defined as shown in Fig. 1. The average ofneuron reference vectors in it is given as its reference
vector.
The Euclidean distance between a blockreference vector and the presented data is calculated tofind a winner. For the N×N-sized map, the maximum (orminimum) block size is (N−1) × (N−1) (or 2×2).
Straightforwardly adopting the concept of blocks
brings about increase in computational time complexity
for learning. In [3], the decision-tree-like search of
winner and batch learning process are employed toovercome this issue.
The former chooses (N−2)
candidates for winner, each time a member of the
training data set is presented to the N×N-sized map.
The first candidate is one of the four (N−1)×(N−1)-sizedblocks. The (N+1−s)-th candidate is chosen out of thefour (s−1)×(s−1)-sized blocks included in the sames×s-sized block, which has been most recently
determined as the (N−s)-th candidate. The candidate

850

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

n pixels

n pixels

Unit

Fig. 1. An N×N-sized map and a block itwith the shortest distance to the presented data isformally determined as winner block for it. The vectors
to be updated are those of neurons in the winner.
The batch learning process is summarized as follows.
The values associated with modifications on each neuron
reference vector are stored for every member in the
training data set. Once an epoch is complete, all of theneuron reference vectors are simultaneously updated.
The learning termination condition is specified by the
number of epochs.
Detailed steps of T-BMSOM
learning are described in [3].

first flamelengH

2lengH

Fig. 2. Window with a target object

lengB
2lengB

Center of gravity of extracted object.

Fig. 3. Window with center of gravity of extracted object
as its center

III. MAP-BASED VIDEO OBJECTSEGMENTATION(or lengH) denote the base (or height) length associatedwith the minimum rectangle. Fig. 3 depicts a schematic
representation of the window. The window has thecenter of gravity of the extracted object as its center.
In [3], the unit has a vector with two element values.
Y values of YUV color components of all n2 pixels in theunit are averaged, and the resultant value is given as one
element value. The other equals the standard deviation
for the Y component associated with pixels in the unit.
The training data associated with the above unit
corresponds to the above vector with average andstandard deviation. Y-quantification hereinafter denotes
the above approach quantifying color attributes based on
Y component colors.
If some quantification scheme is unsuitable for agiven video sequence, it is highly likely that the target
object is lost at some point in time when object
segmentation is in progress. To overcome this issue,
data are generated on condition that HSV-quantification is also available. When HSV-quantification isadopted, the average and the standard deviation arecalculated for each component as well as when
Y-quantification is adopted. The number of elements is
six for each of the data generated from the units.
A method for determining the quantification scheme
is described as follows. It is solely applied to the first
frame. The value of probability density function is

The proposed extraction quantifies the first frame in agiven video sequence by vectors employed as membersof a training data set. T-BMSOM learning is thenapplied. In the resultant map, the following two
clusters are formed: the cluster with neurons mainly
firing for the data generated from the moving object andthat mainly firing for the data generated from thebackground. The moving object in each subsequent
frame is extracted using the constructed map, and an
extraction result appears on the computer screen in theform of a window-like area including the target object.
A. Generation of training data
In the first frame, a rectangular window including atarget object is defined as shown in Fig. 2. T-BMSOM
learning uses only the data generated from this window
for map training. The first window (i.e., the window in
the first frame) is formed as follows. In the following,
an n×n-sized set of pixels is considered to be a unit.
The background difference is applied to the first frame,
and units belonging to the target object aresystematically extracted. The minimum rectangle withwhich all units of the extracted target object can becovered is then specified, provided that its center equals
the center of gravity of the extracted object. Let lengB©ISAROB 2011

Window

851

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

B. Construction of maps and object extraction
processing
Once T-BMSOM learning is complete, neurons arelabeled as follows. All of the members in the training
data set are presented again to the just trained map.
Each of the members has either ‘target object (TO)’ or‘background (BG)’ as its label. Let NLiTO (or NLiBG )denote the firing frequency for the i-th neuron while allof the members with label TO’s (or BG’s) are presented.
Let BN i denote the label to be assigned to the i-th
neuron. It is given as follows:

calculated for every component value, according to thekernel estimation[5]. Let the l-th window denote thewindow with the target object in the l-th frame. The
calculation is executed by using units belonging to thetarget object in the first window. Let t uq denote theaverage of q-component values in the s-th unit belonging
to the target object, where q∈{Y, H, S, V}, 1≤u≤W and
W is the number of units in the target object. The
density function is given as follows:p q (t ) = ( 1 Whq ) ∑ K ((t − tuq ) hq ) ,
W

∧

(1)

s =1

where K (t ) and hq are kernel function[5] andbandwidth, respectively. They are as follows:

K (t ) = e (− t 2 )2π ,h = 0.9σ

15

2

q

q

W

 TO
LBN i = 
 BG

(2)

where σ =min(SD, IQR/1.349), SD is the sample standard
deviation and IQR is the inter-quartile range. A label isnext assigned to each q-component value. The label
implies whether the q-component value is relevant to thetarget object. Let LB tq denote the label assigned tosome q-component value, tq. It is determined according
to the following equation:∧
TO if p(t q ) > th q ,q
LBt = (4)∧qq
BG if p(t ) ≤ th ,where th q equals the average of values of p q (t )calculated for the first window. If LBtq = TO (or BG ),it is considered that the value tq strongly characterizes thecolor attribute of the target object (or the background).
Next, WN rectangles are randomly clipped from thebackground area excluding the first window, provided
that the size of each rectangle is equal to the size of thefirst window. The average of the q-component values
associated with pixels is calculated for any unit in eachchosen rectangle, and the label that Eq. (4) assigned to
the value equal to the average is checked. If the label is
TO (or BG), the unit with this average has highprobability of being incorrectly (or correctly) judged as
the target-object part (or the background part). This
scheme is similarly applied to any unit in the clipped WN
qrectangles. Let N BG
denote the number units withhigh probability of being correctly judged as theHbackground part, where q∈{Y, H, S, V}. For N BG,
SVN BG and N BG , if at least two values are larger thanY
N BG
, the data based on HSV-quantification are employed
for training a map; otherwise, the data based on
Y-quantification are employed. For example, HSVYHYS
< N BG
and N BG
< N BG.
quantification is chosen if N BG∧

©ISAROB 2011

NL iTO > NL iBG ,NL iTO ≤ NL iBG ,(5)

The map is unavailable to extract the target object until
all of the neurons are labeled in the above way.
Let us briefly explain object extraction processing.
Once the segmentation is complete for the (l−1)-th frame,
the data to be presented is generated from each of theunits in the l-th window, on condition that coordinates offour vertexes of the l-th window are equal to those of the(l−1)-th window. This l-th window is said to benonconclusive. The data is generated according to thequantification determined in the first frame. If a winner
for the data associated with some unit is a neuron labeled
TO (or BG), it is judged that the corresponding unit
belongs to the target object (or background). After
arbitrary units are similarly examined, the nonconclusive
l-th window area is updated using the extracted object asshown in Fig. 3. The updated area is fixed as theconclusive l-th window. Extracting the target object issimilarly conducted for each of the subsequent frames.

(3)

,

if
if

IV. EXPERIMENTAL RESULTS
T-BMSOM learning produces well-trained maps [3],[4] on condition that training data sets are updated during
learning, compared to conventional SOM learning. The
method based on T-BMSOM learning is therefore
applicable as follows.
1) The l-th map is constructed, using the training data
set generated form the l-th window.
2) The l-th map classifies units in the (l+1)-th window.
3) The training data set is updated according to theclassification result associated with the (l+1)-thwindow. The (l+1)-th map is constructed, using
the updated training data set.
If the map constructed with the training data set
generated from some window has enough capability of
appropriately extracting the units in several of its

852

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

subsequent windows, incremental learning conducted in
the above manner can be skipped for such windows.
This capability is clearly useful in reducing the loads
imposed on the computer. The proposed segmentationis therefore evaluated on condition that only the training
data set generated from the first window is available.
Experimental conditions are as follows: the frame with
640×480 pixels, the unit with 8×8 pixels, 5×5-sized map
and learning termination condition is 20 epochs. These
are similarly employed in [4]. WN is set to 100. The
proposed segmentation is compared with thesegmentation in [4].
They are implemented oncomputer (CPU: Athlon 2.0GHz, Memory: 1.0GB).
The proposed segmentation and the segmentation in[4] are applied to sequences referred to as Video 1, Video
2 and Video 3. Figure 4 depicts some frame in eachsequence and extraction results. Recall that the method
in [4] is solely based on Y-quantification, whereas the
proposed scheme uses HSV-quantification in addition toY-quantification. The proposed method generates data
based on Y-quantification for Video 1, and data based on
HSV-quantification for Video 2 and Video 3. Note thatthe extraction results obtained by the method in [4]correspond to the cases of data generation solely basedon Y-quantification.
In Figs.4 (a)-(c), units, each of which is judged aspart of the target object (pedestrian), are marked by dots.
For the results achieved by the proposed method, thepedestrian appropriately appears in the rectangular
window. On the other hand, the method in [4] judges anumber of units to be actually included in the
background as those of the pedestrian, as shown in Figs.4(b) and (c). It is thus revealed that the quantification
selection works well for generating data presented tomaps.

(a) Video 1

Proposed method

Method in [4](b) Video 2

Proposed method

Method in [4](c) Video 3

Fig. 4. Object segmentation results for eachquantification
selection is useful in generating data tolerant to theoverextraction that occurs in the background.
In future, the proposed method will be modified sothat extraction accuracy can be improved.

REFERENCES[1] H.Mochamad, H.C.Loy and T.Aoki, “SemiAutomatic Video object Segmentation Using LVQ with
Color and Spatial Features,” IEICE Trans. INF. & SYST.,
vol. E88-D, no. 7, pp. 1553-1560, 2005.
[2] H.Grabner and H.Bischof, “On-line Boosting andVision,” In Proc. IEEE Conf. on Computer Vision and
Pattern Recognition, vol. 1, pp. 260-267, 2006.
[3] T.Isokawa, K.Iwatani and A.Ohtsuka et al., “On
Self-Organizing Maps Learning with High Adaptability
under Non-Stationary Environments,” Proceeding of
SICE-ICASE International Joint Conference 2006,
pp.4575-4580, 2006.
[4] N.Kamiura, Y.Ohki and A.Saitoh et al., “On Video
Object Segmentation Using Fast Block-Matching-Based
Self-Organizing Maps,” Proc. Of IEEE TENCON 2008,
Hyderabad (CDROM), 2008.
[5] B.W.Silverman, “Density Estimation for Statistics
and Data Analysis,” London: Chapman and Hall, 1986.

V. CONCLUSIONSThis paper proposed video object segmentation, using
a map constructed on condition that training data weregenerated in accordance either with Y-quantification orwith HSV-quantification. The quantification selection
is made with the first frame in a given video
segmentation. Each component value is labeled eitherq
as TO or as BG, and N BG
’s are calculated, using labeled
component values, as measures correlating withprobability that units in the background are rightly
classified. The quantification selection depends onq
N BG’s. Experimental results established that the©ISAROB 2011

Proposed method

Method in [4]853The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Properties of Localized Oscillatory Excitationon the Non-Linear Oscillatory Field
Ryota Miyata∗ and Koji Kurata∗∗∗

Graduate School of Engineering and Science, University of the Ryukyus∗∗
Faculty of Engineering, University of the Ryukyus∗,∗∗
1 Senbaru, Nishihara, Okinawa JAPAN 903-0213
miyata@mibai.tec.u-ryukyu.ac.jp

Abstract: We analyze the dynamics of the non-linear oscillatory field composed of Radial Isochron Clocks(RICs) or Stuart-Landau (SL) oscillators, which are the simplest dynamical systems that have one stable
limit cycle around one unstable equilibrium. According to our computer simulation, the non-linear oscillatory
field with two kinds of Mexican-hat-type connection had the function of several peak ditection of an external
input by the localized oscillatory excitation areas. Moreover, this non-linear oscillatory field could also realize
in-phase phase-locking within each localized oscillatory excitation area, but maximize the phase difference
between different localized oscillatory excitation areas. As Amari (1977) model of the nerve field provided
mathematical base for the self-organizing map (SOM) algorithm, the non-linear oscillatory field is expected
to provides theoritical base for the oscillatory SOM algorithm.
Keyword: non-linear oscillatory field, localized oscillatory excitation, RIC, SL oscillator, phaselocking, Mexican-hat-type connection

1

Introduction

a hypothesis that the property binding should be
represented by phase-locking among neuronal oscilMany 1970’s experimental results suggested thatlatory firing, called synchronous firing hypothesis.
feature-extracting cells are self-organized in visual
Eckhorn et al. (1988) and Gray et al. (1989) discortex through postnatal sensory experience [1, 2].
covered synchronous periodic firing of neurons in
Hirsch and Spinelli (1970), for example, observed
the visual cortex of the monkey and that of the catsuch self-organization in the visual cortical cells of[8, 9].
young kitten [1].
Regarding periodic neuronal firing as oscillation,
In response to this, a lot of neural network modand synchronous firing as in-phase phase-locked osels were proposed at the time to describe this phecillation, we extend Amari model of the nerve field
nomenon. Malsburg (1973) proposed the earliest
to the oscillatory field. Kuramoto (1982) first promodel which explained the mechanism of self-organization posed the oscillatory field [10]. The difference ofof the visual cortex [3]. Amari and Takeuchi (1978)
our model from the preceding models such as Kuformulated the Malsburg model in the mathmatiramoto model is that oscillation occurs in local area
cally simple form [4]. In this model, they adopted
in the oscillatory field. We call this excitation patAmari (1977) model of the nerve field, which hadtern localized oscillatory excitation [13]. In thisbeen shown to allow stable localized excitation arpaper, we study the mechanism of formation of loeas [5]. This stable excitation pattern in the nerve
calized oscillatory excitation areas in the oscillatory
field provides base for neighborhood learning usedfield. To this end, we consider a non-linear osin Malsburg model and Kohonen’s Self-Organizing
cillatory field composed of Radial Isochron Clocks
Map (SOM) algorithm [6]. SOM is one of the sim(RIC) or Stuart-Landau (SL) oscillators, which areplest learning model of the cerebral cortex. Amari
the simplest dynamical systems that have one stable
expressed the property of peak detection of the funclimit cycle around one unstable equilibrium [11, 12].
tion through the dynamics of the nerve field with the
Our computer simulation showed that the nonMexican-hat-type connection pattern, while Koholinear oscillatory field with two kinds of Mexicannen built that of peak detection to the algorithm
hat-type connection had the function of several peakdirectly in the SOM.
ditection of an external input by the localized oscilMeanwhile, recent studies found that the signlatory excitation areas, and could also realize inand magnitude of synaptic plasticity depend critiphase phase-locking within each localized oscillacally on the precise timing of pre- and postsynaptic
tory excitation area, but maximize the phase diffiring. This phenomenon is called spike timing deference between different localized oscillatory excipendent plasticity (STDP) [7]. Moreover, there istation areas.

©ISAROB 2011854

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

This non-linear oscillatory field is expected toprovide theoritical base for the oscillatory SOM algorithm, as Amari (1977) model of the nerve fieldprovided mathematical base for the SOM algorithm
[14].

1
r
θ
-1

1

-1

2

Model
Fig. 1: The structure of a RIC (SL oscillator).

2.1

RIC (SL Oscillator)

RICs (SL oscillators) are known to be one of thesimplest dynamical systems which have one stable
limit cycle around one unstable equilibrium [11, 12].
The dynamics of RICs are written in polar form as
below.
dr= r(1 − r2 )dt

(1)Fig. 2: Our model of the non-linear oscillatory field.

dθ
=1
dt

(2)

As shown in Fig.1, any orbit beginning with r > 0
approaches the stable limit cycle (r = 1) as t increases. According to the equation (2), the behaviorof the phase θ does not depend on the amplitude r.

2.2

are arranged in one dimension. Moreover, only theconnections from the oscillator in the center to theothers is discribed in Fig.2, but actually, the otheroscillators also have similar connection.
The oscillator located at place x has informationof the amplitude r(x) and the phase θ(x). the positive value of the amplitude r(x) > 0 represents
neuronal firing, and r(x) = 0 represents that theneuron is not firing. The small positive constant δdeviate the amplitude r(x) from an unstable equilibrium (r(x) = 0).
The input I1 (x) effects on the amplitude r(x)of the oscillator. One localized oscillatory excitation area exists over a range of the positive value ofI1 (x) > 0. On the other hand, the input I2 (x) effects the phase θ(x). The positive value of I2 (x) > 0
increases the phase velocity.
As shown in Fig.3, two kinds of Mexican-hat-typeconnection w1 (ξ − x) and w2 (ξ − x) depend on the
distance from one oscillator at place x to anotheroscillator at place ξ. Although both w1 and w2
are Mexican-hat-type connection, these two typesof effect on the distant unit are different. While
w1 has no effect on the distant unit as w1 (∞) = 0,
w2 has the negative effect on the distant unit asw2 (∞) = −C, in order to maximize the phase difference between the distant units. For in-phase phaselocking within one localized oscillatory excitationarea, the range of positive effect of w2 (ξ − x) > 0
must be wider than that of the localized oscillatory
excitation area, I1 (x) > 0, namely, σ1 < σ2 .
This non-linear oscillatory field allows one or several localized oscillatory excitation areas, as describedin the next chapter.

Non-Linear Oscillatory Field

Let us consider a non-linear oscillatory field consisting of RICs (SL oscillators) as shown in the following equations.
dr(x)= r(x)(I1 (x) − r(x)2 ) + δ
dt

(3)

dθ(x)
= 1 + I2 (x)dt(4)

I1 (x) =X

w1 (ξ − x)r(ξ)(5)ξ

I2 (x) =X

w2 (ξ − x)r(ξ) sin(θ(ξ) − θ(x)) (6)ξ

w1 (x) = (1 − 2(x 2x) ) exp{−( )2 }
σ1
σ1

(7)

x 2
) }−C
σ2

(8)

w2 (x) = exp{−(

Fig.2 illustrates our model of the non-linear oscillatory field. One osillator is connected with all theother oscillators through two kinds of Mexican-hattype connection, w1 and w2 . To facilitate visualization, we assume in this section that the oscillators©ISAROB 2011855

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Iex(x)
1

0.5

0

0
10

x

20
30
40 0

Fig. 3: Two kinds of Mexican-hat-type connection.

3yTable 1: Parameters on the computer simulation

Settings

δ0.001

In this chapter, we assume two-dimensional field
x = (x, y), x0 = (ξ, η). Hense, the mathematical expression x0 − x represents kx0 − xk. The non-linear
oscillatory field consists of 40 × 50 oscillators, andthe distance between one oscillator and the neighboring oscillator is defined as 0.05.
The winner of the ordinary dot-product SOM has
the weight vectors which maximize the dot-productbetween the input [6]. In other words, the ordinary
SOM algorithm has the ability of the peak detection. Amari (1977) model justifies the ability of the
SOM algorithm by the dynamics of the nerve field[5].
Recently, oscillatory SOMs, the extended versionof the SOMs by the oscillators, have been proposed[14]. The oscillatory SOMs allow the several winners in the output layer. In other words, the oscillatory SOMs detect the several maximal points ofthe function, or the dot-product between the input
vectors and the weight vectors. The ability of theoscillatory SOMs to detect the several peaks of function is also required to be justified by the dynamics
of the oscillatory field.
Therefore, we carried out the computer simulation of peak detection of function, using our modelof the non-linear oscillatory field.
As shown in Fig.4, the constant extranal input
Iex (x) which had three maximal points was added
to I1 (x) in the following equation (9). The external
input was given by the equation (10), and the threemaximal points were located at place x1 = (7, 20),x2 = (20, 40), x3 = (30, 10).
X
I1 (x) =w1 (x0 − x)r(x0 ) + Iex (x)
(9)

σ1
0.25

σ2
0.5

C
0.5

kx − x1 k2)
σ2
kx − x2 k2)+ exp(−
σ22
kx − x3 k)+ exp(−σ2σ
0.5

Iex (x) = exp(−(10)

The simulation was carried out by using the RungeKutta method of which the size of time step was
0.005. Each oscillator was set the initial value ofthe amplitude to 0, and set that of the phase to theuniform pseudorandom number from 0 to 1. Eachparameter was given as Table 1.

3.2

Results

Fig.5 illustrates the non-linear oscillatory field
when the external input which has the three local
maximal points is given. The circle represents theactivated oscillator r(x) > 0. The direction of theradus represents the phase θ(x) of the oscillator,or the timig of neuronal firing. Three framed rectangles denote the maximal points of the external
input.
As shown in Fig.5, the non-linear oscillatory field
kept stably three localized oscillatory excitation areas around the same number of maximal points ofthe external input.Therefore, the non-linear oscillatory field had the ability of detection of several

x0©ISAROB 201150

Fig. 4: The external input Iex (x) on the computer
simulation.

Computer Simulation

3.1

20

10

40

30

856

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011and vertically oriented receptive fields in cats.”
Science, Vol.168, pp.869-871, 1970.
[2] C. Blackmore and G. F. Cooper, ”Developmentof the brain depends on the visual environment”,
N ature, Vol.228, pp.477-478, 1977.
[3] C. von der Malsburg, ”Self-organization of orientation sensitive cells in the striate cortex”,
Kybernetik, Vol.14, pp.85-100, 1973.
[4] S. Amari and A. Takeuchi, ”Mathematical theory on formation of category detecting nerve
cells”, Biological Cybernetics, Vol. 29, PP.127136, 1978.
[5] S. Amari, ”Dynamics of pattern formation in lateral-inhibition type neural fields”,
Biological Cybernetics, Vol. 27, PP.77-87,
1977.

Fig. 5: The non-linear oscillatory field given the external input which had three maximal points. Threelocalized oscillatory excitation areas were kept stably around the same number of maximal points.

[6] T. Kohonen, ”Self-Organizing Maps” SpringerVerlag, 1995.

peaks from the external input by the localized oscillatory areas.
In this case, the oscillatory field entrained inphase within each localized oscillatory excitationarea, and entrained the phase difference 2π/3 between the different localized oscillatory excitation
areas.

[7] L. F. Abbott and S. B. Nelson, ”Synaptic
plasticity: taming the beast”, N ature, Vol.3,
pp.1178-1183, 2000.

4[9] C. M. Gray, P. Konig, A. K. Engel, and W.
Singer, ”Oscillatory responses in cat visual cortex exhibit intercolumnar synchronisation whichreflects global stimulus properties”, Nature, 388,
pp.334-337, 1989.

[8] R. Eckhorn, R. Bauer, W. Jordan, M. Brosch,
W. Kruse, M. Munk, and H. J. Reiboeck, ”Coherent oscillations: a mechanism of feature linking in the visual cortex?”, Biological Cybernetics, Vol. 60, pp.121-130, 1988.

Conclusion

We analyzed the dynamics of the non-linear oscillatory field consisting of RICs, or SL oscillators.
Our computer simulation showed that the nonlinear oscillatory field with two kinds of Mexicanhat-type connection could keep three localized oscillatory excitation areas stably around the same
number of maximal points of the external input.
Thus, our model has the ability of peak detectionof the function, which Amari model also has. Moreover, the non-linear oscillatori field could also realize in-phase phase-locking within each localized
oscillatory excitation area, but maximize the phasedifference between different localized oscillatory excitation areas.
This non-linear oscillatory field provides theoritical base for the oscillatory SOM algorithm, as
Amari (1977) model of the nerve field provided mathematical base for the SOM algorithm [14].

[10] Y. Kuramoto, ”Chemical oscillations, waves,and turbulence”, Springer V erlag, 1982.
[11] F.C. Hoppensteadt, and J.P. Keener, ”Phaselocking of biological clocks”, Journal of Mathmatical Biology, Vol.15, No.3, pp.339-349, 1982.
[12] T. Aonishi, ”Macroscopic asymptotic property
from Stuart-Landau oscillator to phase oscillator -A case of the third order Stuart-Landau
equation- (in Japanese)”, IEICE Technical Report, Vol.101, No.238, pp.17-24, 2001.
[13] R. Miyata, A. Date and K. Kurata, ”Phaselocking in localized oscillatory excitation on neural oscillatory fields (in Japanese)”, SOFT, inpress, 2010.
[14] R. Miyata, and K. Kurata, ”Solving the binding problem with separated extraction of information by Oscillatory self-organizing maps”,
SCIS & ISIS, in press, 2010.

References[1] H. V. B. Hirsch and D. N. Spinelli, ”Visualexperience modifies distribution of horizontally©ISAROB 2011857

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Multiple Granger causality tests for network
structure estimation from time-series data
Hikaru Harima1, Shigeyuki Oba1,2, and Shin Ishii1
Graduate School of Informatics, Kyoto University
2
PRESTO, Japan Science and Technology Agency(harima-h@sys.i.kyoto-u.ac.jp, oba@i.kyoto-u.ac.jp, ishii@i.kyoto-u.ac.jp)
1

Abstract: To identify network structures is a key for elucidating functions of various kinds of networks such as cortical
local circuits. Granger causality (GC) test has been used for estimating directed network structure from time-course of
neuronal activities. Although GC statistic for a pair of nodes can be substantially influenced by other nodes, ignoringsuch influence can degrade detection performance of multiple GC tests. To improve the multiple GC tests, and hence
the estimation of large network structures, therefore, we propose an extension of GC by introducing optimal discovery
procedure (ODP) that shows the best detection power in general multiple testing problems. Applying our proposed
method to a benchmark dataset, we show the performance of estimating the network structure is improved over thoseby the exiting methods.
Keywords: Granger causality, network structure, time-series analysis, optimal discovery procedure

becomes large, the number of possible links grows huge,
which makes the estimation of false positives and false
negatives in the results obtained by GC tests quitedifficult. A solid treatment of such multiplicity waspreviously proposed by Storey [4,5]; his optimal
discovery procedure (ODP) is known to be the mostpowerful test in multiple simultaneous statistical
hypothesis testing (MSHT), like the detection problemof causality links from huge number of possible links inneuronal networks.
In this study, we propose a new method to determine
directed network structure based on application of ODP
to GC tests. We demonstrate the performance of causal
link detection of the proposed method is better thanthose by the existing method when applied to abenchmark datasets.

I. INTRODUCTION
To identify network structures is a key for
elucidating functions of various kinds of networks; forexample, a cortical local circuit would have its ownfunction in the brain information processing, which isbasically defined by the connectivity between
constituent neurons. Since it is often difficult to directly
observe anatomical structures especially in vivo, there
have been numerous attempts to estimate network
structures based on electrophysiological activities ofneurons. For examples, multiple-electrode arrays [1]have been widely used, and recently, functional multineuron calcium imaging (fMCI) [2] comes to beavailable for observing neural activities with both
temporally and spatially high resolutions.
Causal structure of a directed network can be
uniquely represented by the set of directed causal links
between pairs of network nodes; then testing whether adirected link exists or not between a single pair of nodes
is the basis for estimating the whole network’s causal
structure. Granger causality (GC) [3] has become
popularly used for estimating existence of each link; adirected causal influence from a node A to a node B isdetected if time-series prediction accuracy for node B
by means of an autoregressive (AR) model issignificantly improved by incorporating time-series
observation for node A into its explanatory variable.
Although GC provides a useful tool for estimating
network structure, there still remains the problem of‘multiplicity’. As the number of elements (e.g., neurons)©ISAROB 2011

II. METHODS
1. Granger Causality
A GC test tries to identify causal relationshipbetween two nodes by assuming an autoregressive (AR)model between them. Consider a test of a causal linkfrom an input node Ii to an output node Oi based on( Ii )( Oi )
theirobservations x
and x
,
where
x* = ( x1* ,..., xT* ) denotes a time-course observed at anode *. The null hypothesis assumes the output node
admits an individual AR process: then, if the null
( Oi )
hypothesis holds, xt
is predicted by

858

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011setting any threshold on the ODP statistic becomes themost powerful test. More concretely, in MSHT, we
consider m hypothesis tests at the same time, in each ofwhich it is tested whether the null or alternative
hypothesis is plausible after given observations of asingle variable: X i , i = 1,..., m , based on the null andpxˆt( Oi ) (x( Oi ) ) = ∑ w(jOi:null ) xt(−Oij ) ,j =1

where p is the order of the AR process. On the other
hand, the alternative hypothesis assumes causal( Oi )
isinfluence from Ii to Oi . In this case, xt
predicted by a vector autoregressive (VAR) model
which employs not only the past observations of Oi
but also those of Ii as its explanatory variable:p∑ (w( Oi )xˆ=(x( Oi ) , x( Ii ) )t( Oi:null )j =1( Oi:alta ) ( Oi )
j
t− j

x

+w

x ),

situation, the true ODP statistic (TODP) is defined by

S TODP ( X i ) =(1)( Ii:alta ) ( Ii )jt− j

( Oi:alta )likelihood functions: f i ( X i ) and gi ( X i ) . In thisTfi (x(Oi ) , x( Ii ) ) = ∏ N ( xt(Oi ) ; xt(Oi ) (x(Oi ) ), µ , σ i2:null ) ,S EODP ( X i ) =t= p +1Tgi (x(Oi ) , x( Ii ) ) = ∏ N ( xt(Oi ) ; xt(Oi ) (x(Oi ) , x ( Ii ) ), µ , σ i2:alta ) ,denotes Gaussian probabilityσ2
i:alta=ΣTt= p +1( Oi )− x t(x(Oi ) )) 2T−pj

 ij ≡ f ( X ) and G
 ij ≡ g ( X ) ; weobservations. Let Fjjiicall these terms self-likelihood terms if i = j andmutual-likelihood terms otherwise. The ODP statistic
involves mutual-likelihood terms, and then, thedetection power is improved over ordinary likelihood
ratio tests which employ only the self-likelihood terms.
3. ODP for GC
Here, we apply the idea of ODP to multiple GC tests.
When a directed network consists of N nodes, thenumber of its possible connections is as many as=
M N ( N − 1) . Although the conventional application of
GC tests to network structure estimation has been
ignored this multiplicity, introduction of the idea of
ODP would improve the detection power, because of
possible correlations between constituent nodes in the
network. In multiple GC test situations, the i-th test
would be performed based on a set of observations of
the corresponding pair of nodes. Then, the mutuallikelihood functions of null and alternative hypotheses
in ODP employ observations of the two nodes as:.

After obtaining the null and alternative likelihood
functions, we have the log-likelihood ratio as σ i2:null 
T−p
LLRi =log  2  .
 σ i:alta 2Since we have assumed the two likelihood functions
both obey Gaussian, we can apply the standard F-test to
the likelihood ratio; thus, the GC test can be used for
identifying the causal relationship.

 ji = f (x(Oi ) , x( Ii ) ) ,Fj ji = g (x(Oi ) , x( Ii ) ) .
G

2. Optimal Discovery Procedure
The ODP defines the most powerful test in MSHT
situations [4]. According to the theory of ODP, expected
number of true positives (ETP) and expected number offalse positives (EFP) in MSHT are considered, and astatistical test is said most powerful if any other testwith smaller EFP cannot have larger ETP. Storey
showed that such an ideal statistic, ODP, exists that

©ISAROB 2011

i

,

T−p
( Oi )( x (Oi ) − x t (x(Oi ) , x( Ii ) )) 2tji

by maximum likelihood (ML) estimation, based on theobservations. The variances in the null and alternative
likelihood functions can also be determined by ML
estimations, as:( Oi )

ilikelihoods, respectively, estimated based on availablef (x( Oi ) , x( Ii ) ) and g (x( Oi ) , x( Ii ) ) , can be determined
i
i

T

j

 is estimated by a conventional statistical test,
where wi
 ( X ) are null and alternativeand f ( X ) and gdensity function of mean µ and variance σ 2 . The
regression parameters in these likelihood functions,2Σ(xσ i:null = t= p +1 t,

 j ) g ( X )Σ mj =1 (1 − w
i
j
,
m  
Σ wj f (X )j =1

t= p +1

N ( x; µ , σ 2 )Σ mj=1w j f j ( X i )where wi = 1 (or wi = 0 ) holds if the null (oralternative) hypothesis is true for the i-th test.
Since fi ( Xi ) , gi ( Xi ) and wi are unknown in actual
situations, however, the ODP statistic should be
estimated based on observed data. Then, instead, we use
estimated ODP (EODP):( Ii:alta ), wj, and w jarewhere w j
regression parameters. By simply assuming theprediction errors obey Gaussian distribution, the nulland alternative likelihoods are given bywhereΣ mj=1 (1 − w j ) g j ( X i )j

To obtain the estimate EODP statistic, the weightparameters wi in equation (1) is necessary; in this case, = 1 (or w = 0 ) if the null (or alternative)
we set wiihypothesis is accepted in the individual i-th GC test. It
should be noted that this individual test is just for

859

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

defining EODP statistic, and actual MSHT is performed
based on the EODP statistic, not by the individual GC
test. Consequently, we obtain ODPGC (EODPGC)
statistic as:

EODPGCi (x ( Oi ) , x ( Ii ) ) = j )GΣ j (1 − w
ji
.
jF

Σ w
j

ji

III. SIMULATION
1. Setting
Computer simulation was performed to examine theproposal method. By assuming that true network
structure consists of 5 nodes and 5 directed causal links(Fig. 1), simulated time-series were obtained by meansof a VAR model of the true order being p = 3; thisbenchmark setting was also used in a previous study [6].

Fig.2. ROC curves of EODPGC (black), TODPGC(light gray) and GC test (gray): ( p, T , M ) = (3,50,50)
In this case, ODPGC with the true weight values( w ) (TODPGC) achieved almost 100% accuracy withappropriate thresholds, though the true weightparameter is not available in actual applications.
EODPGC, which employs the estimated weight values
performed better than the GC tests; this is also
confirmed by AUC scores (EODPGC: ODPGC with the ), GC: Granger causality).
estimated weight values ( wNext, the performances were compared by varying
the order of AR model p. The number of nodes M was
set at 50. Both GC and EODPGC performed the bestwhen p = 3. This result is reasonable because we used a
VAR-model of the order of 3 to generate the simulation
time-series data.
Next, we examined how the AUC behaves as the
network size increased: M = 5,10,..., 70 (Fig. 3). The
order of AR model p was fixed at 3.

Fig.1. A five node network
In addition, we prepared larger but redundant networks
which were constructed by copying the 5-node graph(Fig. 1) m = 1, 2,...,14 times; then we have in total 14
networks with M = 5,10,..., 70 nodes. For example,
the network with 15 nodes has three identical subnetworks of five nodes and no causal link between the
sub-networks. For each of the 14 networks, time-seriesof length T = 50 was for every node generated. This
length T = 50 was set to examine the applicability ofstatistical tests to short time-series.
Detection accuracy of causal links was examined forthe proposed method (EODPGC), ODP with true
weights (TODPGC), and simple individual GC tests.
The general performance of statistics was examined interms of ROC curves. Since an ROC curve represents
the behavior of the statistic within a two-dimensional
plane of the false positive rate (horizontal axis) and thetrue positive rate (vertical axis), for each thresholdsetting. Because detection accuracy of the statistic
depends on this threshold setting, we also examinedarea under the ROC curve (AUC) as a general criterion;a larger AUC means higher detection accuracy
regardless of the threshold setting.

Fig.3.AUC for a various numbers of network nodes,
where ( p, T ) = (3,50)

2. Result
Figure 2 shows ROC curves for the three methods inthe case of p = 3 and M = 50.

©ISAROB 2011

In this figure, the mean AUC value and half of standard
deviation over 20 trials are shown for M = 5,..., 40 ,860

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Journal of the American Statistical Association, 77:
304-313.

and a single AUC value for M = 45,..., 70 . As themultiplicity became large due to the increase in thenumber of network nodes, such as M > 40 , theperformance of EODPGC was likely better than that of
GC. Although individual GC tests do not consider thecorrelation between the tests, the correlation does exist
effectively, because the connectivity between a pair ofnodes is not independent of that of another pair, due tothe eventual effective correlation between the nodes in
the network. The statistics based on ODP show good
performance especially when such effective correlation
exists between the multiple tests to be performed at thesame time.

IV. CONCLUSION
We proposed a powerful method to identify a large
causal network structure. This method is an applicationof the ODP theory to multiple GC tests. Comparing our
method with the standard GC tests, we showed a good
performance for identifying the network structure
especially when the network size is large. There would
be some remaining issues; comparison with recent
methods, for example, conditional granger causality(CGC) [7] and partial granger causality (PGC) [6]should be done in the near future. Application toelectrophysiological data from real neuronal networks isalso an important future work.

REFERENCES[1] R. Q. Quiroga, S. Panzeri (2009), “Extracting
information from neuronal populations: information
theory and decoding approaches”, Nature Reviews
Neuroscience, 10:173-185.
[2] T. Sasaki, N. Matsuki, Y. Ikegaya (2007),“Metastability of active CA3 networks”, The Journal of
Neuroscience, 27:517-528.
[3] C. Granger (1969), “Investigating causal relations by
econometric models and cross-spectral methods”,
Econometrica, 37:424-438.
[4] J. D. Storey (2007), “The optimal discovery
procedure: a new approach to simultaneous significancetesting”, Journal of the Royal Statistical Society. Series
B, Statistical methodology, 69: 347-368.
[5] J. D. Storey (2006), “The optimal discovery
procedure for large-scale significance testing, withapplications to comparative microarray experiments”,
Biostatistics (Oxford, England), 8: 414-439.
[6] S. Guo (2008), A. K. Seth, K. M. Kendrick, C. Zhou,
J. Feng, “Partial Granger causality–Eliminating
exogenous inputs and latent variables”, Journal of
Neuroscience Methods, 172: 79-93.
[7] J. Geweke (1982), “Measurement of linear
dependence and feedback between multiple time series”,©ISAROB 2011861

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A Simulation Study of Visual Perceptual Learningwith Attentional Signals
Satoshi Naito1, Naoto Yukinawa1 and Shin Ishii1,2
1Graduate School of Informatics, Kyoto University
Gokasho, Uji, Kyoto, Japan
Tel : 0774-81-3938; Fax :0774-38-3941
2
RIKEN, Computational Science Research Programnaito-s@sys.i.kyoto-u.ac.jp, naoto-yu@sys.i.kyoto-u.ac.jp, ishii@i.kyoto-u.ac.jp
Abstract: Repeated exposure to a specific stimulus can enhance animal’s sensitivity to it so that the perceptual
capability is improved. This experience-induced perceptual improvement is referred to as perceptual learning. However,the neural system has some robustness and is not necessarily modified by its any input. In the case of visual perceptual
learning (VPL), perceptual performance for a task-relevant stimulus can be selectively improved without any
sensitivity change to task-irrelevant stimuli which are presented even simultaneously with the task-relevant one. In thisstudy, we propose a feed-forward spiking neural network model consisting of a primary visual cortex (V1) layer and ahigher visual area (V4) layer; their inter-layer feed-forward connections are modified by synaptic learning in aparticular interest in how VPL can be affected by neural activities in the higher area due to attentional signals. Throughsimulations, we show attentional inputs are needed to facilitate inter-layer synaptic learning which yields improved
sensitivity to the task-relevant stimulus, and thus to increase the task performance.
Keywords: Perceptual learning, Visual attention, Spiking neural network, Spike-timing dependent plasticity
hypothesis. In terms of Fisher information, sharpeningof a tuning curve leads to increase in information,suggesting the improvement of representation of
sensory information encoded by neuronal population [3].
In addition, it has been suggested that active andpersistent attention to a feature to be learned is alsoneeded for VPL [4]; one of the previous studies reported
that when two different stimuli were presented
simultaneously but the subject was required to pay
attention to one of them, no VPL occurred for the
stimulus to which the subject did not pay attention [5].
Another existing study reported that responses of V4
neurons to a particular stimulus with paying attention
were significantly higher than those without paying
attention [6], which also suggested the existence ofattentional control on VPL.
Nevertheless, there is still a missing link; noprevious study has clarified the direct relationshipbetween sharpening of tuning curves of V4 neurons andconnection changes between V1 and V2 which isaffected by the attentional control during VPL. For
unified explanation of attentional effects on V4 neurons
and sharpening of their tuning curves induced by VPL,
we propose in this study a feed-forward spiking neuralnetwork model which includes a V1 layer and a V4
layer; neurons in the V4 layer receive inputs fromneurons in the V1 layer and also top-down attentional
excitatory signals. Performing a learning simulation ofthis network, we show that the task performance is

I. INTRODUCTION
Animal’s brain continues to modify the structures of
its neural networks to adapt to environmental changes
even after its maturation. Repeated exposure to a
specific stimulus can enhance animal’s sensitivity to it
so that the perceptual capability is consequently
improved.
This
experience-induced
perceptual
improvement is referred to as perceptual learning.
However, the neural system has some robustness and is
not necessarily modified by its any input. In the case ofvisual perceptual learning (VPL), perceptual
performance for a task-relevant stimulus can beselectively improved without any sensitivity change totask-irrelevant stimuli which are presented evensimultaneously with the task-relevant one. Theseobservations raise the following questions: what is thecomputational basis of VPL and how does the brain
accomplish VPL in such a noisy environment withappropriately detecting relevant information?
There is a hypothesis that VPL is due to changes insynaptic connections between neurons of the primaryvisual cortex which acts as feature extractors of visualelements and those of the higher visual cortical areas
which are involved in decision making functions [1]. A
preceding study reported that the bell-shaped tuning
curve for the task-relevant stimulus gets sharpened in
V4 neurons which receive inputs from V1 neuronal
population, but not in V1 neurons [2], supporting this©ISAROB 2011862

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

I attention

improved due to changes in neuronal connections, andis more facilitated by V4’s activities enhanced by theattentional signals.

vi

m

w

II. MODELIn this study, we used a grating task [7] for
simulating a VPL situation. In a single trial of the real
grating task, two grating images are sequentiallypresented to a subject; the second grating image isrotated from the first one. Then a subject is required toanswer whether the tilt orientation of the second image
has changed from that of the first one in a clockwisemanner or in a counterclockwise manner. A previous
experimental study observed that repeated exposure tosuch grating stimuli induces changes in tuning curves of
V4 neurons [2]. According to the hypothesis that VPL isdue to connectional changes between neurons of the
primary visual cortex and those of the higher visual
cortical areas [1], it is thought that changes in V4
responses are mainly due to connectional changes
between V4 and V1 neurons; the former corresponds tothe higher visual cortical area and is projected by the
foveal region of V1 directly [8], and the latter codes tiltorientation of presented grating stimuli by means of its
orientation selectivity. Based on these assumptions, inthis study, we propose a hierarchical and feed-forward
neural network model which consists of two neuronallayers of V1 and V4 where connections between thelayers can be modified by a spike-timing dependent
plasticity (STDP) rule (Fig. 1). In the actual
experimental setting, a subject would be exposed not
only to the target stimuli but also to many distracters;
however, in this study we assume that the modelneurons respond only to grating image stimuli forsimplicity. We present the details of the model in thefollowing sections.

 input
Fig.1. The architecture of network model
where a ,  r and R are constants; we set a  0.02 , r  0.5 and R  0.002 in the simulation. Each V1
neuron has own orientation selectivity to gratings (  r )and the preferred tilt orientation (  PD ). The V1 layer
consists of 8 groups of 10 neurons which have different
orientation preference (Fig. 1); the preferredorientations of the 10 neurons in each group do not vary
so much around its average  PD , but they havedifferent means between the 8 groups.
2. V4 layer
The V4 layer consists of 100 neurons each of which
was implemented as a leaky-integrate and fire neuronmodel:dv
 v i  (v  Vrest )  I i  I attention ,(2)
dtvi  Vrest if vi  Vthreshold ,where  v is a membrane time constant. v i , Vrest ,and Vthreshold are membrane potential, resting potential,
and firing threshold, respectively. In this study, we set v  1 , Vrest  64 and Vthreshold  30 . I i andI attention represent input current and attentional input
current, respectively. A V4 neuron i receives synaptic
currents I i from V1 and V4 neurons:
t


v4



I i   f (t  t ) wiju v1
j (t )   mik uk (t ) dt ,
0
k
 j

t 1 tf (t )  e  u ,u1. V1 layer
The V1 layer consists of 80 model neurons each ofwhich was implemented as a Poisson spike generator
with frequency r (input ) ; the frequency was given by aGaussian-like neuronal response function with apreferred orientation  PD , to the tilt orientation of an
input grating stimulus  input :r ( input )  a exp(  input   PD

 PD   PD   PD ,2 r2

where u v1
and u v4
represent indicator functionsj (t )j (t )of spike occurrences at V1 neuron j and V4 neuron i ,respectively. When a spike occurs in V1 (V4) neuron
and otherwise u v1(v4). wij
j (i) at t , u v1(v4)j ( i ) (t )  1
j ( i ) (t )  0
is synaptic strength of feed-forward connection from V1
neuron j to V4 neuron i , and mik is synaptic strengthof recurrent connection from V4 neuron k to V4
neuron i . In this study, we set mik  0.1i  k  or
 0.04i  k. f (t ) is a spike response function withthe time constant of  u ( 1.5) .

)  R,(1)

2
 PD  n  8 (n  0,..., 7),  PD ~ N (0, PD
),



©ISAROB 2011

(3)

863

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

3. Spike-timing dependent plasticity
During VPL, synaptic strength w ij between a V1
neuron j and a V4 neuron i was updated by an additive
STDP rule [9] as follows:
w ij  w ij  w,
  t   exp  if t  0,   w  
 t   if t  0,  exp   t  t i  t j ,(4)where t i and t j are spike timing of neuron i (post)
and j (pre), respectively.   (  )  0 and   (  )define the maximum amount of synaptic change and thetime constant of long-term potentiation (depression).
 We
used    0.0103 ,    0.0051,    13.3 and   34.5 in this study. On the other hand, we assumed
that the connections within the V4 layer did not change,
so they were fixed during VPL.

Fig.2. Orientation selectivity of V4 neurons before
VPL. Each tuning curve was calculated based ontuning curves of V1 cells weighted by the connection
values from the V1 cells to the corresponding V4 cell.
The horizontal axis represents tilt of input grating
stimuli and the vertical axis represents the normalized
response. It is assumed that the V4 network can decode
the orientation of input grating by integrating thepatterns of V4 responses.

III. SIMULATION EXPERIMENTS

We then estimated “sharpness” of each V4 neuron tomeasure the degree of sensitivity to input stimuli byfitting its normalized tuning curve to a Gaussianfunction and then obtaining its variance. Finally, wetested how the sharpness of the neuronal population
changed between before and after the VPL phase basedon one-sided t-test [2]. Since we are also interested in
the dependence of VPL on the top-down attentionalsignals, we conducted the simulation above by changing
the level of attentional inputs.

1. Simulation procedures
To examine the effects of top-down attentional
inputs on VPL, we conducted a network simulation
assuming the grating task. First, we initialized thesynaptic connections between V1 and V4 neurons w ij bypresenting 20,000 uniformly random stimuli input to themodel network. This initial phase encouraged each V4
neuron to form a sparse receptive field for input stimuli,sparse coding
which is similar to the result by thescheme [10]. An example of the resultant tuning curves
which correspond to responses of V4 neurons toorientation of input grating stimuli are presented in
Fig.2.
We then trained our model network by simulating
the grating task. In a single simulation trial of thegrating task, two grating stimuli were successively
presented to the model network for 300 ms each. Tilt of
the first grating was fixed to the base orientation
input  o , and that of the second one wasinput  o  d in each trial. In this simulation, we set o  24 and  d was randomly drawn from a
uniform distribution of U(-10, 10).

IV. RESULTS
After simulating the model network in the gratingtask by setting various levels of attentional siginals,
I attention  {0,..., 30} , we obtained tuning curves after
learning (Fig. 3). This figure shows that the turning
curves of some V4 neurons got sharpened significantly
after VPL with a relatively large level of attentionalsignals ( I attention  22 ), which is consistent with theexisting experimental result in which tuning curves of
V4 neurons became sharpened after VPL [2]. Fig. 4
shows histograms of sharpness of tuning curves of V4
neurons, before learning (upper panel) and after
learning with no attentional inputs (middle panel) andrelatively large attentional inputs (lower panel). The
sharpening of tuning curves after VPL occurred in the
population-wide manner in the V4 layer regardless of

2. Evaluation of attentional effects
To evaluate how the VPL proceeds in the model
network, we calculated normalized tuning curves of V4
neurons, each of which was the connection-weighted
summation of tuning curves of V1 neurons andnormalized to make its maximum response a fixed value.

©ISAROB 2011864

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

the level of attentional inputs, whereas the stronger
attentional signals likely induced sharper tuning curves( I attention  0, p  0.065 and I attention  22, p  0.0011 ).
In addition, the minimum value of the sharpness with
I attention  22 ( 18.8 ) was smaller than that with
I attention  0 ( 23.1 ). On the other hand, too large
attentional inputs induced broader tuning curves; indeed,mean sharpness after VPL with I attention  30 was
39.9  0.00 (the initial mean sharpness was
30.0  8.7 ). These results imply that appropriate
level of attentional control to V4 neurons would workfor promoting perceptional sensitivity.

Fig.3. Tuning curves of V4 neurons before (dashedline) and after (solid line) 5,000 gating task trials withattentional input of I attention  22 . Each tuning curve
was rescaled and shifted to adjust the peak to thecenter of the horizontal axis.

V. DISCUSSION AND CONCLUSION
In this study, we simulated network learning inwhich synaptic connections between V1 and V4 neurons
were modified by STDP during the grating task, andshowed that our network model well reproduced
sharpening of tuning curves of the V4 neurons. We also
found that moderate attentional inputs yield sharpen
tuning curves of the V4 neurons. These results impy thatappropriate level of attention contributes to facilitating
VPL while VPL occurs even without paying attention.
In our network model, reductions in connectional
strength between V1 and V4 neurons were often
observed in V4 neurons which acquired sharpened
tuning curves (result not shown). These synaptic
reductions can depress responses of V4 neurons to input
stimuli, while a previous study reported increase in
spike frequency after VPL [2]. This inconsistency may
be resolved by introducing backward attentional control
from V4 to V1, as suggested in a previous study [11].
Backward attentional pathway would increase spikefrequency of V1 neurons such to facilitate activities ofthe projected V4 neurons. In future work, therefore, wewill examine attentional effects in VPL by corticalmodel networks including mutually-connected V1 andV4 layers.

Fig.4. Histogram of sharpness of tuning curves of
V4 neurons. Upper: before learning. Middle: after
learning with no attentional inputs (Iattention = 0).
Lower: after learning with a relatively strong
attentional inputs (Iattention = 22).
[4] Ahissar M and Hochstein S (1993), Attentional
Control of Early Perceptual-Learning. Proc. Natl. A
cad. Sci. U.S.A. 90(12): 5718-5722.
[5] Schoups A, Vogels R et al. (2001), Practising orientation identification improves orientation codingin V1 neurons. Nature 412(6846): 549-553.
[6] Luck SJ, Chelazzi L et al. (1997), Neural mechanisms of spatial selective attention in areas V1, V
2, and V4 of macaque visual cortex. J Neurophysiol
77(1): 24-42.
[7] Orban GA, Dupont P et al. (1997), Human brain
activity related to orientation discrimination tasks. Eur J
Neurosci 9(2): 246-259.
[8] Zeki SM (1978), The cortical projections of fov
eal striate cortex in the rhesus monkey. J Physiol 2
77:227-244.
[9] Bi GQ and Poo MM (1998), Synaptic modificat
ions in cultured hippocampal neurons: Dependence
on spike timing, synaptic strength, and postsynaptic
cell type. J Neurosci 18(24): 10464-10472.
[10] Olshausen BA and Field DJ (1996), Natural image statistics and efficient coding. Network 7(2): 3
33-339.
[11] Buffalo EA et al. (2010), A backward progression of attentional effects in the ventral stream. Proc.
Natl. Acad. Sci. U.S.A. 107(1): 361-365.

REFERENCES[1] Dosher BA and Lu ZL (1998), Perceptual learni
ng reflects external noise filtering and internal noise
reduction through channel reweighting. Proc. Natl.
Acad. Sci. U.S.A. 95(23): 13988-13993.
[2] Yang TM and Maunsell JHR (2004), The effectof perceptual learning on neuronal responses in m
onkey visual area V4. J Neurosci. 24(7): 1617-1626.
[3] Seung HS and Sompolinsky H (1993), SimpleModels for Reading Neuronal Population Codes. Proc.
Natl. Acad. Sci. U.S.A. 90(22): 10749-10753.

©ISAROB 2011865

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Medical Image Diagnosis of Lung Cancer by Revised GMDH-type
Neural Network using Various Kinds of Neurons
1),2)

Tadashi Kondo1) and Junji Ueno2)
School of Health Sciences, The University of Tokushima, 3-l8-15 Kuramoto-cho Tokushima 770-8509, Japan
1)
kondo@medsci.tokushima-u.ac.jp

Abstract: A revised Group Method of Data Handling (GMDH)-type neural network algorithm using various kindsof neurons is applied to the medical image diagnosis of lung cancer. The optimum neural network architecture for the
medical image diagnosis is automatically organized using revised GMDH-type neural network algorithm and theregions of lung cancer are recognized and extracted accurately.
Keywords: Neural network, GMDH, Medical image diagnosis

I. INTRODUCTION

All combinations of r input variables are generated
in each layer. The number of combinations isp!
. Here, p is the number of input variables and( p  r )! r!
the number of r is usually set to two.
3) Selecting the optimum neuron architectures
For each combination, the optimum neuron
architectures which describe the partial characteristicsof the nonlinear system can be calculated by applying
the regression analysis [8] to the training data. The
output variables yk of the optimum neurons are calledintermediate variables. In the GMDH-type neuralnetwork, the optimum neurons are selected fromdifferent neuron architectures.
4) Selecting the intermediate variables.
The L intermediate variables which give the Lsmallest test errors calculated using the test data are
selected from the generated intermediate variables yk.
Selected L intermediate variables are set to the input
variables of the next layer and calculations fromprocedure 2 to 4 are iterated.
5) Stopping the multilayered iterative computation.
When the errors for the test data in each layer stop
decreasing, the iterative computation is terminated. The
complete neural network which describes thecharacteristics of the nonlinear system can beconstructed using the optimum neurons which are
generated in each layer.

The conventional GMDH-type neural networks[1]-[3] are automatically organized using the heuristic
self-organization method [4],[5] and the structuralparameters such as the number of layers, the number ofneurons in each layer, useful input variables andoptimum neuron architectures are automaticallydetermined so as to minimize the prediction error
criterion Akaike’s Information Criterion (AIC) [6] orPrediction Sum of Squares (PSS) [7]. In this paper, arevised GMDH-type neural network algorithm isdeveloped. In this algorithm, the polynomial type andthe radial basis function (RBF) type neurons are used
for organizing the neural network architecture. A lot ofcomplex nonlinear combinations of the input variables
fitting the complexity of the nonlinear system are
generated using the polynomial type neurons and only
useful combinations of the input variables are selected
for organizing the neural network architecture. In the
output layer, the RBF type neuron is used for organizing
the neural network and the output value of the neuralnetwork becomes between zero and one. The revised
GMDH-type neural network is applied to the medical
image diagnosis of lung cancer and it is shown that therevised GMDH-type neural network is accurate anduseful method for the medical image diagnosis of lungcancer.

III. REVISED GMDH-TYPE NEURALNETWORK USING VARIOUS KINDS OF
NEURONS

II. HEURISTIC SELF-ORGANIZATIONThe GMDH-type neural network algorithm can
automatically develop the optimum neural networkarchitectures by the heuristic self-organization. The
heuristic self-organization in the GMDH-type neuralnetwork is implemented through the following fiveprocedures:
1) Separating the original data into training and testsets.
The original data are separated into training and test
sets. The training data are used for the estimation of theweights of the neural network. The test data are used fororganizing neural network architectures.
2) Generating the combinations of the input
variables in each layer.

©ISAROB 2011

The revised GMDH-type neural network architectureis shown in Fig.1. Here, nonlinear function gi is describedby the following Kolmogorov-Gabor polynomial:gi (x1,x2,,xp)= a0+aixi+aijxixj+
(1)
i

i j

This nonlinear function is automatically organized byusing the polynomial neurons. The architecture of therevised GMDH-type neural network is automaticallyorganized using the heuristic self-organization and is
produced as follows:
In the revised GMDH-type neural network, theoriginal data are not separated into training and test sets

866

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

because PSS can be used as the test errors.

X1
X2
X3
．
．
．

．
．
．

Xp

Σ f
．
．
．

．
．
．

In the output layer, the output values of the neuralnetwork are calculated from zk as follows:(6)
yk= exp ( - zk2)
So, in the output layer, the neuron architecture becomes
as follows:: (Nonlinear function)zk =  wi gi (x1,x2,,xp)(7)
f : (Nonlinear function)
 =exp ( - zk2)
(8)
At last, the complete neural network architecture is
produced by selected neurons in each layer.
Figure 2 shows the flowchart of the revised GMDHtype neural network.

φ

．
．
．

gi (x1,x2,,xp) : (Nonlinear function)zk =  wi gi (x1,x2,,xp)

Start
Set the layer number (h) to 1

f : (Nonlinear function)
 =exp ( - zk2)
Fig.1 Architecture of revised GMDH-type neural network

Generate the combinations of
the input variables

1. The first layer(j=1,2,,p)
(2)
uj=xj
where xj (j=1,2,,p) are the input variables of thesystem, and p is the number of input variables. In thefirst layer, input variables are set to the output variables.
2. The second layer
Many combinations of two variables (ui,uj) are
generated. For each combination, the neuron
architecture is described by the following equations:: (Nonlinear function)zk=w1ui+w2uj+w3uiuj+w4ui2+w5uj2+w6ui3+w7ui2uj+w8uiuj2+w9uj3 –w01(3)
f : (Linear function)(4)
yk= zkwhere 1 =1 and wi ( i=0,1,2,,9) are weights between
the first and second layer. The weights wi (i=0,1,2,,9)are estimated by using the multiple regression analysis.
This procedure is as follows:First, the values of zk are calculated using thefollowing equation:(5)zk=(-loge ’)1/2
where ’ is the normalized output variable. Then theweights wi (i=0,1,2,,9) are estimated by using the
stepwise regression analysis which selects useful input
variables using the PSS. Therefore, only usefulvariables in (3) are selected and neuron architecture canbe organized by these selected useful variables.
From these generated neurons, L neurons which
minimize PSS values are selected. The output values(yk) of L selected neurons are set to the input values of
the neurons in the third layer.
3. The third and succeeding layers
In the third and succeeding layers, the same
computation of the second layer is continued until PSS
values of L neurons are not decreased. When theiterative computation is terminated, the following
calculation of the output layer is carried out.
4. The output layer©ISAROB 2011

Select the optimum neuron architecture
for the each combination
Calculate PSS values
Select L intermediatevariables

Set h to h+1

Set L intermediate variables to
input variables of next layer
Yes

Are PSS values decreased?
No
Construct thecompleteneural network

End

Fig. 2 Flowchart of the revised GMDH-type neuralnetwork

IV. APPLICATION TO THE MEDICALIMAGE DIAGNOSIS OF LUNG CANCER
In this study, the regions of lung cancer wererecognized and extracted automatically by using therevised GMDH-type neural network. Multi-detector
row CT (MDCT) images of the lungs are used in thisstudy. In the recognition procedure, the revised
GMDH-type neural network is organized to recognize
the lung regions and then the regions of lung cancer
are extracted.

867

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

1. Extraction of the candidate image regions of lungcancerA lung image shown in Fig. 3 was used fororganizing the revised GMDH-type neural network. The
statistics of the image densities and x and y coordinatesin the neighboring regions, the NN pixel regions, wereused as the image features. Only five parameters namely,
mean, standard deviation, variance and x and ycoordinates were selected as the useful input variables.
The output value of the neural network was zero or one.
When NN pixel region was contained in the lungregions, the neural network set the pixel value at thecenter of the NN pixel region to one and this pixel was
shown as the white point. The neural networks wereorganized when the values of N were from 3 to 15. It
was determined that when N was equal to 4, the neuralnetwork architecture had the smallest recognition error.
Five useful neurons were selected in each hidden layer.
Figure 4 shows the variation of PSS values in the layers.
The calculation of the revised GMDH-type neuralnetwork was terminated in the sixth layer. The PSS
value in the second layer was not small but the PSS
value was decreased gradually through the layers andthe small PSS vale was obtained in the sixth layer. The
revised GMDH-type neural network outputs the lung
image (Fig.5) and the first post-processing analysis of
the lung image was carried out. In the firstpost-processing of the output image, image processing
such as closing, opening and so on were carried out andthe small isolated regions were eliminated and theoutlines of the lung regions were expanded outside by
N/2 pixels. Figure 6 shows the output image after the
first post-processing and the lung regions that contains
the abnormal regions were extracted. The output image
after the first post-processing was overlapped to theoriginal image (Fig.3) in order to check the accuracy of
the image recognition as shown in Fig.7. The
recognized lung regions are accurate. The lung regions
were extracted from the original image using the output
image. Figure 8 shows the extracted image of the lungs.
The second post-processing such as closing, opening
and so on was carried out and the lung regions which
did not contain the abnormal regions was obtained asshown in Fig.9. Figure 10 shows the extracted image ofthe lungs. The candidate image region of lung cancer
were extracted from Fig.8 using Fig.10 and shown in
Fig.11. The recognition results were compared withthose of the conventional sigmoid function neuralnetwork trained using the back propagation method.

10

o
o
o

PSS
5

o

ooo
oo

0
1

2

3

ooo o
o

oo
ooo

ooooo

4

5

6

Layer
Fig.4 Variation of PSS in the revised GMDH-type
neural network

Fig.5 Output image ofthe neural network

Fig.6 Output image after
the first post-processing

Fig.7 Overlapped image

Fig.8 Extracted image (1)Fig.9 Output image after
the second post-processing

Fig.10 Extracted image (2)Fig.11 The candidate image region of lung cancer

Fig.3 Original image©ISAROB 2011o868

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

2. Recognition results of the conventional neuralnetwork trained using the back propagation
algorithm
A conventional neural network trained using theback propagation algorithm was applied to the same
diagnosis problem and the results were compared withthe results obtained using the revised GMDH-type
algorithm. The conventional neural network had a threelayered architecture, which was constructed using the
input, hidden and output layers, and the same five input
variables, which were mean, standard deviation,
variance, x and y cordinates, were used in the input
layer. Weights of the neural network were estimatedusing the back propagation algorithm and initial valuesof the weights were set to random values. The learning
calculations of the weights were iterated changing
structural parameters such as the number of neurons in
the hidden layer and the initial values of the weights.
The output images, when the numbers of neurons in thehidden layer (m) are 3, 5 and 7, are shown in Fig.12.
These images contain more regions which are not partof the lungs and the outlines of the lungs are notextracted with required clarity compared with the output
images obtained using the GMDH-type neural network
algorithm, which is shown in Fig.5. Note that, in case of
the conventional neural network, we obtain manydifferent output images for various structural parametersof the neural network and many iterative calculations of
the back propagation are needed for various structuralparameters in order to find more accurate neuralnetwork architecture. In case of the revised GMDH-type
neural network, the optimum neural networkarchitecture is automatically organized so as tominimize prediction error criterion PSS using heuristic
self-organization method [4],[5] and many iterative
calculations for various structural parameters are notneeded because all structural parameters areautomatically determined.

useful method for the medical image diagnosis of lungcancer because the neural network architecture is
automatically organized so as to minimize theprediction error criterion defined as PSS.

(a) m=3(c) m=7
Fig.12 Output images of the conventional sigmoid
function neural network
Acknowledgment This work was supported by (JSPS)
KAKENHI 22560403.

REFERENCES[1] Kondo T. and J.Ueno (2008), Multi-layered
GMDH-type neural network self-selecting optimum
neural network architecture and its application to
3-dimensional medical image recognition of bloodvessels, International Journal of innovative computing,information and control, Vo.4,No.1, pp.175-187.
[2]. Kondo T.(1998), GMDH neural network algorithm
using the heuristic self-organization method and its
application to the pattern identification problem, Proc.
of the 37th SICE Annual Conference, pp.1143-1148.
[3] Tamura H. and T.Kondo (1980), Heuristics freegroup method of data handling algorithm of generating
optimum partial polynomials with application to air
pollution prediction, Int. J. System Sci., Vol.11, No.9,
pp.1095-1111.
[4] Farlow S.J. ed. (1984), Self-organizing methods inmodeling, GMDH-type algorithm, Marcel Dekker, Inc.,New York.
[5] Ivakhnenko A.G. (1970), Heuristic self-organizationin problems of engineering cybernetics, Automatica,
Vol.6, No.2, pp.207-219.
[6] H. Akaike (1974), A new look at the statisticalmodel identification, IEEE Trans. Automatic Control,
vol.AC-19, no.6, pp.716-723.
[7] H. Tamura, T. Kondo (1980), Heuristics free groupmethod of data handling algorithm of generating
optimum partial polynomials with application to air
pollution prediction, Int. J. System Sci., vol.11, no.9,
pp.1095-1111.
[8] Draper N.R. and Smith H. (1981), Applied
Regression Analysis, John Wiley and Sons, New York.

V. CONCLUSION
In this paper, the revised GMDH-type neural network
algorithm was applied to the medical image diagnosis oflung cancer and the results of the revised GMDH-type
neural network were compared with those of theconventional sigmoid function neural network trainedusing the back propagation algorithm. Structuralparameters such as the number of layers, the number ofneurons in hidden layers and useful input variables areautomatically selected to minimize prediction error
criterion defined as PSS. In the case of the conventional
neural network, we obtain many different output images
for various structural parameters of the neural networkand many iterative calculations of the back propagation
are needed for various structural parameters in order tofind more accurate neural network architecture.
In this paper, the revised GMDH-type neuralnetwork algorithm was applied to the medical image
diagnosis of lung cancer and it was shown that therevised GMDH-type neural network algorithm was a©ISAROB 2011(b) m=5869

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Neural network application using GPGPU
Y. Tsuchida and M. Yoshioka
Osaka Pref. Univ., Sakai, Osaka 599-8531 Japantsuchida@sig.cs.osakafu-u.ac.jp

Abstract

the neural network structure was discussed to adapt
to parallel processing.

In this paper, we have proposed the speed up
method of neural network’s running especially inlearning time using GPU resource. Implementationof the General-Purpose computing on GPU- GPGPU
became easier by the integrated development environment, CUDA distributed by NVIDIA. GPU has dozens
or a hundred arithmetic circuit, whose allocations arecontrolled by CUDA. We propose the neural network
implementation using GPGPU. Local minimum, oneof the limitation of neural network, is conquered byusing some networks with diﬀerent initial weight coeﬃcient in parallel. On the other hand, the neuralnetwork structure was discussed to adapt to parallel
processing. One of this implementation is applied to
the remote sensing. As a result, this implementationis 20 times faster than CPU.
keywords GPGPU ,CUDA, Nural Network

1

2

CUDA, Compute Uniﬁed Device Architecture, is a
software development kit distributed free by NVIDIA.
By employing a supported graphics board, the parallel computing architecture is used easily. In computing using CUDA, the threads are stored in hierarchy streucture. The grid exist as highest hierarchyin GPU, in the grid there are 65,535 x 65,532 blocks,
and the threads are managed by three dimensions in
the block. Some types of memories are implemented
in CUDA, and, among them, the shared memory exists in each blocks. This memory is implemented as
on-chip memory, therefore the access to the memory
is faster than any other memories. However this memory is used in threads only. This memory is “shared”
among threads existing in a block.

Introduction

Recently, graphic boards have higher performancewith development of 3DCG and movie processing thanCPU, and widely used with progress of computer entertainment. In this paper, we have proposed that theneural network’s running time become speed up especially in learning time using GPU resource. Implementation of the General-Purpose computing on GPUGPGPU became easier by the integrated development
environment, CUDA distributed by NVIDIA. GPU
has dozens or a hundred arithmetic circuit, whose allocations are controlled by CUDA. We propose theneural network implementation using GPGPU. Local
minimum, one of the limitation of neural network, isconquered by using some networks with diﬀerent initial weight coeﬃcient in parallel. The performance depends on the allocation of threads, because the CUDA
has hierarchical framework to treat many threads.
Second, CUDA has many types of memories too, especially, how to treat the shared memory, one of the
on-chip memory, inﬂuences performances. Therefore©ISAROB 2011

CUDA

3

The parallelization for the whole
neural network

At ﬁrst, one neural network is implemented as one
computational thread of parallel computing, and thesethreads have each diﬀerent initial weight coeﬃcients.
In the CUDA, it must be conﬁgured which hierarchy the threads are assigned to. Therefore, when N
networks are executed, the number of thread in blockis expressed by m and the number of blocks in grid is
N/m, which m and N are integer. The relationshipbetween m and processing time are evaluated ﬁrstly.
Table 1 shows the speciﬁcation of GeForce GTX480
which is used in this research. Test network has 2 neurons in input layer, 3 neurons in hidden layer, and the
1 neuron in output layer. There are 4 lerning pattarns, and the learning is repeated by 2200 steps ateach pattern. This routine is employed as one thread,and the time of processing for N threads are measured

870

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 1: GeForce GTX480
Number of cores
480
Global Memory
1.5GB
Constant Memory 64KB
Shared Memory
48KB/block
Clock rate
1.40GHz
Maximum threads 1024/blockd1

y0

SupervisordkdnThreshold Valuez1

Input Signals

xi

yj

zk

zn

xl

Row

e1

‑

ek

‑

‑

ym

en

Parallelizey1x1

Output Signals

x0‑
Nonexistent Nodeby changing N and m. Figure refmap shows the measuring result, where m = N represents all threads are
assigned in one block. Because of the limited numberof arithmetic units, the process time is proportional to
N upward N = 40 in m = 1, and because of the timeof memory access, the performance is down in m = N
too.

400

needed until the output from the previous layer become complete. The block signals are employed in
front of each layers, which show ’stop’ until the all
neurons have been calculated completely. It is necessary to calulate one network, The number of threads
which is necessary to calculate one network is maximum numbers of neurons per layers, nmax . Note thatthe processing waits for the other when it comes in thelayer where the neuron is nonexistent.
The phase of foreword propagation is independent
in calculating other patterns too, so p patterns loop
can be calculated in parallel. In the phase of the backpropagation, more ﬂags are needed to calculate in parallel, these are hoisted to protect the one neuron fromother propagation.
The sizes of grids and blocks are expressed by three
dimensions, where z-direction of grid is 1. In the experiment, the grid allocate (N/m, p, 1) blocks, and theblock allocate (m, nmax , 1), and the processing time
was measured. The sample network is same as previous section, therefore the p = 4, nmax = 3. Figure 3
shows the result.

200
100

0

20

40
60
80
The number of network N

100

120

Figure 1: Processing time for whole network thread

One of the goal of the parallelization, the calculation time became ﬂat independent of N, is achieved
rid of m = 1. From this result, the time is depend onm, however the inﬂuence is small.
The problem of m is inﬂuenced to the shared memory on implementation, because this memory size uses
m times, and this is allocated ﬁxed size in each block.
Because shared memory is less than grobal memory,
m must be decided by the scale of the network.

600

processing time(msec)

4

Parallelization of the inner structureof network

In previous section, one entire network is implemented as one thread, therefore we propose that the
network structure is broken up and scattered on GPU.
Figure 2 shows the model of the neural network. In the
network, some neurons exist in a layer, these neurons
can be evaluated in same time.
However, the inputs of the neurons are previous layer’s results, therefore the momentary pause is©ISAROB 2011

ColumnFigure 2: The neural network model

300

0

Hidden Layer Output Layer

blocking signal

m=1
m=5
m=20
m=N
m:allocate in block

500
Processing Time(msec)

Input Layer

500
400

m=1
m=4
m=8
m=20
m=N

300
200
100
0

20

40
60
80
100
The number of network N

120

Figure 3: Processing time for scatterd network

871

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 2: CPU(host machine speciﬁcation)CPU
AMD Phenom 9600
Frequency
2.31GHz
Memory3.8GByte
Operating System Windows XP sp2

The number ofhidden layer

Table 4: The ratio of CPU to GPU (CPU/GPU)As a result,the time is proportional to N in earlystage when m = 1. The tendency of the total plot
resembles a previous section, then the inﬂuence of an
calculation cost by m on hardware is small. The result
was speed-up only 2.4 times in comparison with theforegoing chapter. So, the further improvement of theblock signal and ﬂags is necessary.

5

6This suggestion was compared with a CPU. Table
2 shows the speciﬁcation of a comparison CPU.
The experiment is evaluated in processing time bychanging the number of hidden layer and pattern.
Test network of this section has 3 neurons in input
layer and the 3 neuron in output layer. The learning isrepeated by 2200 steps at each pattern. The numberof network N = 40 and the network in block m = 4 isassigned. The number of hidden layer is selected from3, 6, 9 and the pattern is chosen from 5, 10, 15, 20.
Table 3 shows the processing time of the GPU, andtable 4 show the ratio of CPU to GPU. The columns
is the number of pattern, and the row is the numberof hidden layer.

The number ofhidden layer

Table 3: Processing time for patterns and hidden layers by GPU

The number of pattern
5
10
15
20
65.34 107.92 161.19 246.22
69.47 110.72 159.07 249.67
111.75 218.38 326.04 436.61
112.53 214.67 318.54 436.96

Compared with the CPU, 22.4 times in average isfaster. When the number of hidden layer is 3 and 6 ,the tiem of the same number of pattern is equivalent.
And when the number of hidden layer is 9 and 12, thetime is equivalent too.
This result shows that the usage of the block inﬂuences the performance.

©ISAROB 2011

Conclusion

We proposed the speed-up method of neural network learning by using GPGPU. The GPGPU has thehierarchical structure to store the threads, the performance is inﬂuenced by assigning hierarchy. At ﬁrst,
the learning time of some networks in parallel wasmeasured to show the eﬀect of the hierarchy.
The neurons of each layer in network are processedin parallel, and patterns are processed in parallel too.
This method shows the 2.4 times faster than the ﬁrst
method, and compared with CPU, the processing time
is 22.4 times faster.
This method will be applied for home use. The
method is applied with the graphics or motion pattern recognition easily, because GPU is the hardware
for graphic processing. This goal of this method application is usign for the speech recognition.
In the other hand, this method is used in remotesensing ﬁeld. The goal in the future is that the newlearning algorithm adopted on GPGPU is created.

Comparison with the CPU

3
6
9
12

3
6
9
12

The number of pattern
5
10
15
20
15.33 17.53 17.16 14.79
21.57 26.19 27.73 23.39
18.06 18.08 18.01 18.14
22.37 23.11 23.20 22.43

872

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Evaluation of an optimal design method for multilayer perceptron byusing the Design of Experiments
E. Inohira and H. Yokoi
Kyushu Institute of Technology, 2-4 Hibikino, Kitakyushu, Fukuoka, Japan(Tel: 81-93-695-6050; Fax: 81-93-695-6050)({inohira,yokoi}@life.kyutech.ac.jp)

Abstract: We evaluate performance of an optimal design method for multilayer perceptron (MLP) by using the Designof Experiments (DOE). In our previous work, we have proposed the optimal design method for MLPs in order todetermine optimal values of such parameters as number of neurons in hidden layers and learning rates. In this paper, weevaluate performance of the proposed design method through a comparison with a genetic algorithm (GA) based
design method. We target at optimal design of MLPs with six layers. Moreover, we evaluate the proposed designed
method in terms of calculation amount of optimization. Through the above-mentioned evaluation and analysis, we aim
at improving of the proposed design method in order to obtain the optimal MLP with less effort.
Keywords: Multilayer perceptron, Neural network, Optimal design, Design of Experiments, Genetic algorithm

method. However, we have a problem of evaluation of
the proposed design method. The problem isquantitative comparison between the proposed design
method and other design methods. It is clear that the
proposed designed method is better than trail-and-error,
brute-force approaches. We focused on a genetic
algorithm based design method, which is a nonlinear
optimization technique and expected to be as efficientmethod as the proposed method.
In this paper, we evaluate performance of the
proposed design method through a comparison with agenetic algorithm based design method. We target at
optimal design of MLPs with six layers. When we deal
with few design parameters, a difference between the
proposed design method and other method is small. Our
previous work implied that accuracy of MLPs withmore layers would become high for the same training
data. Therefore, we should focus on MLPs with six
layers. Moreover, we evaluate the proposed designed
method in terms of calculation amount of optimization.
We use various types of training data because
performance of a trained MLP depends on training data.
We refer to UCI machine learning repository [6] for theevaluation. Through the above-mentioned evaluation
and analysis, we aim at improving of the proposed
design method in order to obtain the optimal MLP withless effort.

I. INTRODUCTION
A multilayer perceptron (MLP) can approximate anarbitrary nonlinear mapping at an arbitrary accuracy [1].
Accuracy of a trained MLP depends on two factors. The
first factor is learning algorithm. Connection weights
and biases of the neurons are adjusted according to thelearning algorithm. Typical learning algorithm is error
back-propagation (EBP) algorithm [2]. There are many
learning algorithms other than EBP algorithm. It is clearthat accuracy of a trained MLP depends on it learningalgorithm. However, the learning algorithm is theunique factor. Another factor is design of the MLP.
Before training, number of layers, numbers of neurons
in hidden layers and training conditions such as learning
rates are determined. Trial-and-error, brute-force
approaches, network construction and pruning are used
as conventional design methods. It is difficult to apply
these methods to MLPs with many layers because theirdesign parameter space becomes huge. Another problem
is that the approximation accuracy of MLPs with thesame design parameters has variation due to use of
random number to initial values of connection weights.
We need a design method with statistical analysis for
MLP.
In our previous work [3,4], we proposed design
method using the Design of Experiments (DOE) [5],which features efficient experiments with an orthogonalarray and quantitative analysis with analysis of variance(ANOVA). We demonstrated that optimal design offive-layer MLPs could be obtained using our design©ISAROB 2011873

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig.1. A flow chart of DOE-based optimal design
methodFig.2. A flow chart of GA-based optimal design
methodII. OPTIMAL DESIGN OF MLP

We implemented our optimal design process on
MATLAB programs except training of MLP. In [4], Step
8 needed manual operation. On the other hand, in [5]and this paper, Step 8 is programmed with predefinedrules.

1. DOE-based optimal design method
We have proposed an optimal design method using
DOE in our previous work [3,4]. Our basic idea is that
DOE is applied to an optimal design problem of MLP.
We show a flow chart of our proposed method using
DOE in Fig.1. Detailed explanation has been describedin [4]. Training of MLPs corresponds to experiments in
DOE. For example, we used the number of hidden
nodes, learning rates, momentum coefficients and range
parameters of initial connection weights as the design
parameters in [4]. And we used squared sum of training
errors after specific learning cycles as the performanceindex. The used training dataset is common in all
experiments.

©ISAROB 2011

2. GA-based optimal design method
We have proposed the optimal design method using
DOE and evaluated it on three-layer MLPs and fivelayer MLPs. We have demonstrated our method and notcompared with other methods. We focused on genetic
algorithm (GA). GA-based approaches [7,8] havealready been proposed. However, our proposed method
and GA-based approaches cannot be compared directly
because selected design parameters are different.

874

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

In order to compare our proposed method and a GAbased approach, we prepared GA-based optimization
programs using Global Optimization Toolbox on
MATLAB. We show a flow chart of optimal design
method using GA in Fig.2. We programmed GA-based
optimization programs as similar to our proposed
method using DOE as possible. Step 1 to specify design
parameters and Step 5 to train MLPs are common inboth optimization programs. The remaining part of the
optimization depends on the optimization algorithms,
that is, DOE or GA. Our GA-based optimization
program can deal with the same design parameters andtraining data set as the DOE-based program.
We used a simple GA-based optimization algorithm.
An individual consists of design parameter values inreal numbers. A value of the used fitness function is a
training error of a MLP. The default crossover functionof the Global Optimization Toolbox is used.

Fig.3. A comparison between DOE-based method
and GA-based method in median of
performance indicesIII. Experiments
1. Training dataset
We used concrete compressive strength data [9,10]
in UCI machine learning repository as the training
dataset. The training dataset has 8 input variables and 1
output variable. Therefore, input and output of a MLP
are determined by specifying the training dataset. The
number of instances is 1030. Ranges of the variables arelargely different in this dataset. We normalized eachvariable to be range -1 from +1 for input and -0.99 and+0.99 for output. We used a sigmoid function ranging
from -1 to +1. When an input of the sigmoid function is
infinity, its output becomes +1. This takes very longlearning time. To avoid this situation, we set range of
output variable to be from -0.99 to +0.99.
Fig.4. A comparison between the two methods indistribution of performance indices at thetenth cycle

2. Targeted MLP and design parameters
We used six-layer MLPs for the target system. Their
design parameters are as follows: the number of nodes
in hidden layers (4 parameters) and learning rate in eachlayer (5 parameters). Range of the number of nodes inhidden layers is from 1 to 50 in order to limit theparameter space. Range of the learning rate in eachlayer is from 0 to 1. On the optimization program, thedesign parameters are normalized in range from 0 to 1.
When MLPs is set up before training, the design
parameter values are restored. In DOE, we used anorthogonal array with 27 combinations and 3 levels.

©ISAROB 2011

Therefore, 27N MLPs are trained each optimization
cycle. Here N denotes trial iteration counts and isadjusted automatically in the optimization. In GA,
number of individuals is set to 27 or the maximum
number of 27N because of comparison with the DOEbased optimization. This means that calculation amountis the same in both methods. When number ofindividuals is 27, training of MLPs is performed N
times under different initial connection weights. In thiscase, value of fitness function is average of those results.

875

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

3. Training algorithm and performance index
We used a back-propagation algorithm to training
MLPs. In this paper, the learning rates are prepared
every layer. This approach has an effect to reduce thetraining error as shown in our previous work [4]. We
used common logarithm of squared sum of error for all
instances as the performance index. Small performanceindex is better because small error is good. We also
evaluated distribution of performance indices at specificoptimization cycles instead of best performance index.
The reason is that the best performance index has large
variance. We used median of the performance indices asrepresentative value.

In this paper, a kind of MLP and training dataset wasused. It is necessary to apply the optimal design
methods to many types of problems in order to clarify
their limitation

VI. CONCLUSION
We evaluated performance of an optimal design
method for MLP by using DOE through a comparison
with a GA-based design method. We demonstrated
optimal design of MLP with six layers by using DOEbase method is better than GA-based method under thesame calculation amount.
In the future work, our proposed optimal design
method will be applied to other types of problem suchas classification and control in order to investigate
applicable scope of our method.

IV. EXPERIMENTAL RESULTS
We show experimental results in Figs. 3 and 4. Fig.3
shows a comparison between DOE-based optimization
method and GA-based method in median of
performance indices at each optimization cycle. In bothmethods, median of performance indices decreases
when optimization proceeds. After the sixth cycle, theDOE-base method is better than the other. Fig.4 shows
distributions of performance indices of the two methods
at the tenth cycle. The distribution in the DOE-based
method is on the left of the distribution in the GA-based
method. This means a set of combinations of parametervalues in the DOE-based method is better than the GAbased method. In other words, probability of getting thebest performance index in the DOE-based methods ishigher than the GA-based method.

REFERENCES[1] Tikk D, Koczy T and Gedeon TD (2003), A sur
vey on universal approximation and its limits in sof
t computing techniques, Intl J of Approximate Reas
oning 185-202[2] Rumelhart D, Hinton G and Williams R (1986),
Parallel Distributed Processing, MIT Press.
[3] Inohira E and Yokoi H (2007), An optimal design
method for artificial neural networks by using thedesign of experiments, J of Advanced Computational
Intelligence and Informatics. 11(6):593-599.
[4] Inohira E and Yokoi H (2010), Development of an
optimal design method for multilayer perceptrons byusing the design of experiments (in Japanese), Proc of
the 2010 IEICE general conference. D-2-1.
[5] Dean A and Voss D (1999), Design and analysisof experiments.
[6] Frank A and Asuncion A (2010), UCI Machine
Learning Repository [http://archive.ics.uci.edu/ml],
Irvine, CA: University of California, School ofInformation and Computer Science.
[7] Castullo PA, Merelo JJ, Prieto A, Rivas V and
Romero G (2000), G-Prop: global optimization ofmultilayer perceptrons using Gas, Neurocomputing,
35:149-163[8] Leung FHF, Lam HK, Ling SH and Tam PKS(2003), Tuning of the structure and parameters of aneural network using an improved genetic algorithm,
IEEE Trans Neural Networks 4(1):79-88[9] Yeh IC (2007), Modeling slump flow of concrete
using second-order regressions and artificial neuralnetworks, Cement and Concrete Composites 29(6):474480.
[10] Yeh IC (1998), Modeling of strength of highperformance concrete using artificial neural networks,
Cement and Concrete Research 28(12):1797-1808

V. DISCUSSION
We showed that the DOE-based method is better
than the simple GA-based method in optimal design of
MLP. The problem of the simple GA-based method is toignore variance of performance indices. In Fig.4, theresult of GA 27, which means number of individuals is
27, is worst. The reason is that mean of N times results
is used in GA 27. Calculation amount is wasted by using
mean because variance is ignored. GA 162 is worse thanthe DOE-based method under the same calculation
amount. The reason is that the GA-based method has no
statistical analysis. In the GA-based method, children ofa good individual are not always better because
performance indices have variance. On the other hand,
the DOE-based method can adjust parameter values
properly without misleading due to ANOVA.

©ISAROB 2011876

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A Learning Method for Dynamic Bayesian Network Structures
Using a Multi-Objective Particle Swarm Optimizer
Kousuke Shibata, Hidehiro nakano, Arata,Miyauti
Tokyo City University
1-28-1, Tamadutsumi, Setagay–ku, Tokyo, 158-8557 Japan(Shibata@ic.cs.tcu.ac.jp)
Abstract: In this paper, we present a multi-objective Discrete Particle Optimizer (DPSO) for the learning of Dynamic
Bayesian Network (DBN) structures. The proposed method introduces a hierarchical structure consisting of DPSOs anda Multi-Objective Genetic Algorithm (MOGA). Groups of DPSOs find effective DBN sub-network structures and agroup of MOGA finds whole of the DBN network structure. Through numerical simulations, the proposed method canfind more effective DBN structures and can obtain them faster than the conventional method.
Keywords: Dynamic Bayesian Networks, Structure Learning, Multi-Objective Optimization,
Discrete Particle Swarm Optimization
overcome this problem. However, in these algorithms,
the learning for large scale DBN structure requires
significantly long computation time.
In this paper, we present a multi-objective Discrete
Particle Optimizer (DPSO) for the learning of DBN
structures. The proposed method introduces ahierarchical structure. First, the objective DBN isdivided into plural sub-networks depending on observed
state variables. The lower layer consists of groups of
DPSO. The DPSO is well-known as one of the fastsolvers for various optimization problems [5]. In the
proposed method, each particle has binary statescorresponding to causal relations between statevariables in each sub-network. Each group of particles
in the lower layer finds each effective sub-network
structure. The higher layer consists of MOGA [4]. A
group of individuals in the higher layer finds the wholeof the DBN structure sharing information from thelower layer by migration. Evaluation values of eachparticle or individual are given as Pareto solutions forlikelihood and complexity to obtained DBN structures.
The hierarchical structure in the proposed method can
reduce computation time for the learning of larger scale
DBN structures. We evaluate both likelihood andcomplexity to obtained DBN structures, and comparewith the conventional learning method based on MOGA[3]. Through numerical simulations, the proposed
method can find more effective DBN structures and canobtain them faster than the conventional method.
Generally, there exist trade-off relations between
likelihood and complexity to DBN structures in various
actual applications. The proposed method can also
provide many candidate structures of DBNs.

I. INTRODUCTION
The Dynamic Bayesian Network (DBN) describes
causal relations in various systems by using stochastic
network structures, and represents them as directed
acyclic graphs and conditional probabilities fortransitions of each observed state variable [1]-[3]. The
DBN is an extended model of the Bayesian Network(BN). As compared with the BN, the DBN can describe
temporal causal relations of state variables. The DBN
has been applied to speech recognition, genetic
networks and so on. When the structural topology of a
DBN is unknown, the learning of the DBN structure isneeded [2][3]. In the learning, two trade-offcharacteristics must be considered. One is acharacteristic which indicates how well the network fits
the observed data. Another is structural complexity of
the network. A fully-connected network can represent
all relationships between each state variable. However,such a redundant network cannot adapt target model
well, and the network size becomes extremely large.
Therefore, it is preferred that networks should beconstructed as simple as possible. That is, the degree ofconnectivity in the networks must be controlled inlearning methods. Many learning methods introduce
criteria which evaluate networks in terms of bothprobabilistic likelihood and structural complexity.
However, they generally use a single evaluation
function to combine these factors by a weight parameter.
Since appropriate weight parameters depend on target
models, it is difficult to determine a unique parametervalue. The learning methods based on Multi-Objective
GA (MOGA) [4] or Immune Algorithm (IA) can©ISAROB 2011

877

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

where parent() is the number of parent nodes.
In case of using BIC, we must determine an optimum
weight parameter between two terms. If setting of thisparameter is improper, the resulting graph is toocomplex or sparse. However, it is difficult to determine
a unique optimum weight beforehand, because it
depends on target models and training data sets. To
solve this problem, the learning method by using a
Multi-Objective Genetic Algorithm (MOGA) has been
proposed [3]. The method applies a Pareto ranking
scheme to the MOGA [4]. In the Pareto ranking scheme,
each vector is evaluated by the number of the othervectors which have better values about all objectives.
Using this method, a variety of solutions about
likelihood and complexity can be obtained.
However, multi-objective methods tend to need muchiteration to convergence, compared with singleobjective methods. Since these methods acquire
solutions for a variety of likelihood and complexity, thesolutions are dispersed in search space. Typically,evaluation of the network requires enormous
computation time which is proportional to complexity
and amount of data. Therefore, it is desirable to reduce
computation time to the learning convergence.

II. BACK GROUNDA Dynamic Bayesian Network (DBN) is a kind of
probabilistic networks which represent temporal
relationships between observed state variables. A DBN
has three elements represented by (i) Node: observed
state variables, (ii) Edge: dependencies between eachnode, (iii) Conditional Probability Table (CPT): degreeof dependencies. An end point node and an origin nodeof an edge are a child node and a parent node,
respectively. Let X 1[t ],, X N [t ] be N discrete statevariables at time t . A DBN consists of (i) a prior
network DBN0 that specifies prior probabilities
Pr(X [0]) and (ii) a transition network DBNT thatspecifies transition probabilities Pr(X [t  1] | [ X [t ]) .
The Joint probabilities over all the variables for timet  0,1,2,T is
T 1

DBN0 ( X [0])   DBNT ( X [t  1] | X [t ]) (1)t 0For simplicity, this paper considers on the transition
network only.
If a DBN structure is unknown, some learning methods
by using search algorithms and evaluation criteria arenecessary. Evolutionary computation and BayesianInformation Criterion (BIC) [3] have been widely used
to the learning of DBN structures. BIC has two terms
that indicate likelihood and complexity of networks.
BIC is represented by the following formula.

III. PROPOSED METHOD
In this paper, we use Discrete Particle Swarm
Optimizers (DPSOs) [5] as a search algorithm to reduce
computation cost to structure learning of DBNs. The
DPSO is an optimization method that is a kind of swarm
intelligence. In DPSO, particles efficiently searchsolutions in target problems, by updating their positionsand velocities based on personal best solutions which
each particle has and a global best solution which all theparticles have. The DPSO is known as simple and fastalgorithm. In the proposed method, a structure of a
DBN is represented by binary variables in the DPSO.
Particles have binary variables which denote existenceof connections in the DBN. If a connection exists
between a child node and a parent node, the value of thebinary variable is 1. Otherwise, it is 0 (see Fig 1). If N
variables can be observed, N  N strings are required.

BIC  Likelihood   Complexity (2)where  is the weight parameter which balances
between Likelihood and Complexity. The Likelihoodterm signifies the plausibility of the network. It is
calculated by counting occurrences within sequences.
The likelihood is then defined as: N i , j ,k 
i i
 (3)
Likelihood  N i, ji ,ki  log


N
i
ji ki
 ki i, ji ,ki 
where Ni, j ,k denotes the number of occurrences ini iobservation data sequences such that a child node i
has a value ki and its parent node has a value ji .
The Complexity term signifies the structural complexityof the network which is defined as:

Complexity  parent( X i )

(4)

i

Fig.1

©ISAROB 2011

878

A DBN structure and binary variables in DPSO.

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Learning area

In order to solve multi-objective problems by the
DPSO, some schemes are needed. In this paper, we use
the existing method. We apply a multi-objective
optimization scheme by using the Archive scheme [6].
The Archive has two features that are the Archive
controller and the Grid. The Archive controller
determines whether to save solutions into Pareto
solutions. If the Archive has no solution with better
evaluation values about all objectives compared with aselected solution, the Archive controller saves theselected solution into the Archive. The Grid produceswell-distributed Pareto front. The function-space isdivided into grids. If particles are dense in a grid, one ofsolutions in the grid is eliminated. With these features, avariety of solutions can be obtained.
In addition, we apply a hierarchical structure to
DPSOs in order to reduce computation cost. In formulas(3) and (4), BIC can be divided into computations foreach child node i .This means that the learning of
DBN structures can be split for each child node. In thispaper, we propose a learning method by two layers. In
the lower layer, each group of Multi-objective DPSOs
finds each sub-network structure divided for each childnode. That is, in the lower layer, the number of groups
corresponds to the number of child nodes. In the higher
layer, a group of MOGA finds the whole of the DBN
structure. For every iteration, the lower and higher
layers exchange respective solutions by migration. The
overview of this hierarchical structure is shown in Fig 2.
The proposed algorithm is described by thefollowing steps:(step1) In the each layer, initialize particles andindividuals.
(step2) Evaluate particles and individuals.
(step3) Update the positions of each particle in thelower layer by equation (5).

if (rand()  sig(vid )) then xid  1else xid  0
1
sig(vid ) 
1  e vdi

 c2  r2  (Gd  xidk )©ISAROB 2011

t 1

X

X

Learning area

lower layer
Higher layer

X
MDPSO

Archive

t

t 1

X

X

Y

Y

Z

Z

X ,Y , Z

Y

Y

MDPSO

Z

MDPSO

Z

Fig.2

Archive

MOGA

Archive

Overview of a hierarchical learning method.

(step6) Manipulate individuals in the higher layer.
(step7) Migrate between each layer.
(step8) Repeat from (step2) to (step7).
In migration steps, a particle which is chosen randomly
overwrites to a part of individual whose rank is not 1.

VI. SIMULATION RESULTThe proposed method is applied to several
benchmarks which have a variety of complexities andperformances. In order to compare the proposed methodwith the conventional method, we use some benchmarks
which were tested by the conventional method. Target
networks (a) and (b) are shown in Fig 3. Table 1 shows
training data sets. These nodes in the networks generate
discrete probabilistic values according to dependencies.
If nodes have no parents, they generate random values.
Parameters used for the simulation are shown in Table 2.
J
J

K

H

A

K

G

G

A

D

E
B

F

C

F

T
R

D

B

E

S

F
(a)

P

H

(5)

Table 2 Simulation parameters.
Parameter
Conventional
Methoditeration
50
population
200,500
particles
GA operation
tournament

(6)

879

I

C

N

O
L

Q

M

(b)

Fig.3 Target networks.
Table 1 Training data sets.
Network
Variables
Links
Examples
(a)
10
20
100
(b)
20
40
200

(step4) Update the each Archive.
(step5) Update the velocities of each particle in thelower layer by equation (6).

vidk 1  w  vidk  c1  r1  (Pid  xidk )tParent Links
0-3
0-4

Proposed
Method50
100,250
100,250
tournament

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Figs. 4 and 5 show the simulation results for the
networks (a) and (b). It can be found that the proposed
method obtains a variety of solutions which have better
values with less iteration than the conventional method.
A structure which has high complexity and goodlikelihood becomes complex and has extra edges. On
the other hand, a structure which has low complexity
and bad likelihood has many missing edges. Generally,
there exist trade-off relations between likelihood andcomplexity to DBN structures in various actual
applications. The proposed method can also provide
many candidate structures of DBNs.

Future problems include (1) application to otherbenchmarks, (2) evaluation for learning speed, and (3)consideration of appropriate group size of particles orindividuals.

REFERRENCES[1] K.Murphy, "Dynamic Bayesian Networks:
Representation, Inference and Learning," PhD thesis,
Computer Science Division, UC Berkeley, 2002
[2] W.Guo, X.Gao & Q.Xiao, "Bayesian Optimization
Algorithm for Learning
Structure of Dynamic
Bayesian Networks from Incomplete Data," Proc.
CCDC 2008, pp. 2088-2093, 2008.
[3] B.J.Ross & E.Zuvria, "Evolving dynamic Bayesian
networks with Multi-objective genetic algorithms,"
Applied Intelligence, vol. 26, no. 1, pp. 13-23, 2007.
[4] R.E.Steuer, Multiple Criteria Optimization: Theory,
Computations, and Application. New York: John Wiley& Sons, Inc., ISBN 047188846X, 1986.
[5] J.Kennedy & R.Eberhart, "A discrete binary versionof the particle swarm optimization algorithm," Proc.
SMC '97, pp.4104-4109, 1997.
[6] C.A.C.Coello, G.T.Pulido & M.S.Lechuga,"Handling Multiple Objectives with Particle Swarm
Optimization," IEEE Transactions on Evolutionary
Computation, vol. 8, no. 3, pp. 256-279, 2004.

V. CONCLUSION
In this paper, we have proposed a hierarchical multiobjective DPSO for the structure learning of DBNs.
Dividing computations, the proposed method requires
less iteration to learning convergence than theconventional. In addition, diversity and accuracy ofsolutions are equal or higher than the conventional
method. Since the hierarchical structure can reduce thecomputation cost for the learning, the proposed method
is effective in high-dimensional problems.

Fig.5 Pareto solutions for the network (b).
Upper: iteration=10. Lower: iteration=50.

Fig.4 Pareto solutions for the network (a).
Upper: iteration=5. Lower: iteration=10.

©ISAROB 2011880

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Remarks on Folding Behavior of Mapping Capability of
Neural Network Direct Controller for Many-to-one Plant
Takayuki Yamada
Department of Computer and Information Sciences
Faculty of Engineering
Ibaraki University
4-12-1 Nakanarusawa, Hitachi, Ibaraki, 316-8511, Japan
Abstract: We believe that a neural network learns an inverse dynamic of a target plant in a servo level neural network
controller application. However, a practical plant generally has a nonlinear dynamics and there is no inverse dynamics ofit usually. This paper studied this problem through the use of the neural network direct controller. Simulation results
confirmed that the neural network learned the branch of the inverse characteristics (; a part of the inverse characteristics
and it can be expressed as a one-to-one function mathematically) and if the neural network learned only one branch, theplant output matched with the desired value in whole region. Through the simulation, the whole inverse characteristics ofthe plant seems to be folded into one branch when the neural network learns this branch. This behavior is called the neuralnetwork folding behavior in this paper.
Keywords: Neural network, Controller, Learning, Adaptive
I. INTRODUCTION
Many studies have been undertaken in order toapply both the flexibility and the learning capability ofneural networks to control systems. We are believingthat a neural network learns an inverse dynamic of a target
plant in a servo level neural network controller
application.[1][2]This is because the target plant
dynamics can be cancelled by this learned inverse
dynamics if the neural network can learn it. That is, thiscancellation means that a plant output completely matcheswith a desired value and we can realize an ideal control
system. On the other hand, a practical plant generally has
a nonlinear dynamics and it is mathematically expressed a
many-to-one function whose more than one input values
which correspond to one output value. It is well known
that there is no inverse function of such many-to-one
function.
However, many practical neural network
controller applications have been successfully reported.
Does the neural network learn the inverse dynamics of theplant on the practical applications? For this question, we
studied that the neural network obtains the branch of theinverse characteristics of the plant (; a part of the inverse
characteristics and it can be expressed as a one-to-one
function mathematically) if the learned region is restricted
to the one-to-one region.[3] If the learned region is not
restricted, what happen? When we apply the neuralnetwork to the practical applications, the plantcharacteristics are unknown usually. Such restriction is
impossible. For this question, we studied that the neuralnetwork learned the mean characteristics of the several
plant inverse branches for off-line learning neural network©ISAROB 2011881

controller.[3] However we did not study the detail of theon-line neural network controller for this problem yet.
Thus, this paper studies the above question throughthe use of a neural network direct controller with online
learning. The reason to use this type controller is that it
is simplest among the servo level neural network
controllers. The sine function is selected as a target plant.
This plant is static, but it is suitable for a basic study.
Simulation results confirm folding behavior of the neuralnetwork mapping capability. This behavior is that theneural network learns only one branch of the inverse
characteristics of the target plant in order to obtain wholeplant output. This fact means that we can realize an ideal
control system if the neural network can learn only one
inverse branch of the plant. This is because the plant
output can match with the desired value. When theneural network realizes such input-output mapping, thewhole inverse characteristics of the target plant seem to befolded into one branch of the inverse characteristics.
II. NEURAL NETWORK STRUCTURE FORTEST OF FOLDING BEHAVIORThis section explains a neural network structure forthe test of folding behavior. For this test, a following
sine function is selected as a target plant.

Y(k) = sin(U(k))(1)

where Y is the plant output, U is the plant input and k is
the sampling number. The reason of this selection is thatthe sine function has smooth feature and it is easily

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

mapped by a neural network. The reason of the discrete
time system selection is to examine the discrete time
control system as a future study. Since this paper selects
the direct controller, the plant input U is composed of thefollowing equation.
n

U(k) = ! [!i(p) f{Wi(p)Yd(k) + Ti(k)}]$!J(p)= – ! {( "(k) #i(p)f '(Wi(p)Yd(k) + Ti(p)) cos( Uk)}
!Ti(p)
k=0(11)
where f’ is the derivative of the sigmoid function and η isthe parameter to determine the neural network learning
speed.

(2)

i=1

III. SIMULATION
where ωi is the i th element of the weight vector between
the hidden layer and the output layer, Wi is the i th elementof the weight vector between the input layer and thehidden layer, Ti is the i th element of the offset values
added to the hidden layer neuron, n is the neuron numberof the hidden layer, p is the learning number and f is the
sigmoid function expressed by the following equation.

X {1-exp(-4x/Xg )}f(x)= g2{1+exp(-4x/Xg )}This paper selects the following sine wave as adesired value for control.

Yd(k) = sin(X t(k))

Yd

(3)

Neural network

Input

(4)

"

Weight!
Vector(5)
Fig.2 Structure of neural network.

(6)

"J(p)
Wi(p) = Wi(p+1) – !
"Wi(p)(7)"J(p)Ti(p) = Ti(p+1) – !
"Ti(p)

(8)

$
!J(p)
= – ! {(#(k) f(Wi(p)Ij(k-1) + Ti(p))cos(Uk) }! "i(p)
k=0

(9)

$
!J(p)
= – ! {( "(k) #i(p)f'(Wi(p)Yd(k) + Ti(p))Yd(k) cos(Uk)}!Wi(p)
k=0

(10)

882

1

Plant output#J(p)# !i(p)©ISAROB 2011

Plant

OutputWeightVector W

where Yd is the desired value for the control system and ρ
is the sampling number within one learning period. The
learning rule of this neural network controller is designed
so as to minimize the cost function J. When we apply theδ rule to this learning rule, it is expressed as!i(p) = !i(p+1) – "YOffset value T

#

1
2
J(p)=
2 k = 1 ! (k)U

Fig.1 Scheme of neural network direct controller.

where x is the input of the sigmoid function and Xg is theparameter which defines the sigmoid function shape. The
scheme of the neural network direct controller and thestructure of above neural network are shown in Fig.1 andFig.2 respectively.
The output error ε and the cost function J are defined
as follows:!(k)=Yd(k)-Y(k)(12)

Xg = 0.8! = 0.02
0Plant output-1
0

Desired value
25
Sampling number
Fig.3 Learning result of example 1.

50

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

©ISAROB 2011883

Neural network output

2!

Neural network output
Ideal value!
Xg = 0.8" = 0.02
0025
Sampling number

50

Fig.4 Neural network output of example 1.

Neural network input

1
Xg = 0.8" = 0.02
0
Neural network
input
Ideal value
-1

!

0

Neural network output

2!

Fig.5 Neural network input-output relation of example 1.

1

Plant output

where Xt is the signal for the generation of the desired
value. The sampling number within one learning period
ρ =50 is also selected.
Figure 3 shows the learning result of the example 1.
The solid line in this figure is the plant output and thebroken line is the desired value. As shown here, theoutput error remains between the plant output and thedesired value, but it is small and the plant output wellmatch with the desired value. This fact means that theneural network learning well performs and our neuralnetwork can map the sine wave. Figure 4 shows theneural network output (equals the plant input). The solidline shows the neural network output. As shown in thisfigure, it is restricted from π/2 to 3π/2 although the plant
output matches with the sine wave in one cycle as shownin fig.3. If the neural network can map the inverse
characteristic of the sine wave from 0 to 2π, the neuralnetwork output should be 0 to 2π as shown in the broken
line in fig.4. Fig.5 shows the neural network input-output
relation of the example 1. The solid line is the learned
neural network input-output relation. The broken line is
the inverse characteristics of the plant from 0 to 2π. As
shown in this figure, the neural network learns a part of thewhole inverse characteristics and the learned part isrestricted from π/2 to 3π/2. In this restricted region, theinverse characteristics is expressed as the one-to-one
function. Such region is called the branch of the inverse
characteristics in this paper. In other word, if the neuralnetwork learns only one branch, the plant output matcheswith the desired value in whole region. This feature of
the neural network is called the folding behavior in thispaper. This is because the whole inverse characteristics
seems to be fold into one branch.
Figure 6 shows the learning result of the example 2.
The solid line and the broken line are the plant output andthe desired value respectively. As shown here, the neuralnetwork learning also performs well. Figure 7 shows theneural network output. The solid line is the neuralnetwork output and the broken line is the ideal
characteristics if the neural network can map the whole
inverse characteristics of the plant. As shown here, theneural network output is restricted from 3π/2 to 5π/2
although the plant output is similar to that of the example 1.
Figure 8 shows the neural network input-output relation of
the example 2. The solid line is the neural network
input-output relation and the broken line is the ideal
characteristics if the neural network can map the whole
inverse characteristics of the plant. As shown here, theneural network input-output relation is restricted from 3π/2
to 5π/2. This fact means that the neural network learns thedifferent branch from that of fig.5 of the example 1. The
difference between the examples 1 and 2 is only initialneural network weight. That is, the initial neural network
weight determines which branch is learned by the neuralnetwork and if the neural network learns any branches, theplant output can be matched with the desired value.

Xg = 0.8! = 0.02
0Plant output
Desired value
-1

0

25
Sampling number
Fig.6 Learning result of example 2.

50

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Control of Dynamics Systems Using Neural Networks”,
IEEE Transactions on Neural Networks, 1-1,
pp.4-27(1990)[2]M.Kawato, “Computational Schemes and Neural
Network Models for Formation and Control of Multijoint
Arm Trajectory”, Neural Networks for Control(W.T.Miller, R.S.Sutton, P.J.Werbos ed.), MIT Press,
pp.197-228(1990)[3]T.Yamada, “Remarks on Neural Network Controller for
Inverse Dynamics of Many-to-One Plant”, IAS’95 (1995International IEEE/IAS Conference on Industrial
Automation and Control; Emerging Technologies), Taipei,
Taiwan, R.O.C., May 22-27,(1995)

3!
Neural network output

With regard to other remarkable feature of the neuralnetwork, the branches monotonously increase or decrease.
This fact means that the derivative of the plantcharacteristics learned by the neural network also
monotonously increases or decreases. As shown ineqs.(9)-(11), the leaning rules in this paper use thederivative of the plant characteristics. However, it may
not be necessary. The reason of the requirement of theplant derivative is that we believe the neural network
should converge to the whole plant inverse characteristics
and its derivative changes in whole region usually. The
value of this derivative is the petty problem. This isbecause it relates to the convergence speed and we cancompensate through the use of the parameter η tuning in
the learning rules. The sign of its derivative is the serious
problem. This is because this relates to which the neuralnetwork converges or not.
However, the above
simulation results confirm that the sign of the slope of theplant inverse characteristics learned by the neural network
is constant. This is because the signs of the branch
derivatives are constant. That is, if the parameter η ineqs.(6)-(8) has the constant sign (plus or minus) and thesmall value, we can expect that the neural network
converge to either branch although the derivatives of theplant inverse characteristics are removed in the learning
rule. As shown in above simulation results, if the neuralnetwork converges to either branch, the plant output
matches with the desired value and we can obtain the ideal
control characteristics in whole region. This expectation
will be shown in my future work.

2!
Xg = 0.8" = 0.02!
0025
Sampling number

50

Fig.7 Neural network output of example 2.

IV. CONCLUSIONACKNOWLEDGMENT
The author wishes to express his thanks to Mr. Junpei
Oota, graduated student, Ibaraki University, for his
programming and simulation.
REFERENCES[1]K.S.Narendra and K.Parthictsarathy, “Identification and884

Neural network input
Ideal value

1
Neural netwiork inputThis paper studied what characteristics did theneural network learn through the use of the neural network
direct controller. This is because we expect that theneural network learns the inverse dynamics of the object
plant, but such inverse dynamics does not exist for theusual nonlinear plant. Simulation results confirmed thatthe neural network learned the branch of the inverse
characteristics (; a part of the inverse characteristics and it
can be expressed as a one-to-one function mathematically)
and if the neural network learned only one branch, theplant output matched with the desired value. Through thesimulation, the whole inverse characteristics of the plant
seems to be folded into one branch when the neuralnetwork learns this branch. This behavior is called theneural network folding behavior in this paper.

©ISAROB 2011

Neural network output
Ideal value

Xg = 0.8" = 0.02
0

-1

0

2!
Neural network output!

3!

Fig.8 Neural network input-output relation of example 2

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Parts Layout Decision of Cell Production Assembly Line using Genetic
Algorithm and Virtual System
Hidehiko Yamamoto1, Tomokazu Watanabe2, Takayoshi Yamada3, Masahiro Nakamura4, Raafat H. Elshaer5
1,3Department of Human and Information Systems, Gifu University, Japan1(yam-h@gifu-u.ac.jp), 3(yamat@gifu-u.ac.jp)
2
Graduate School of Engineering, Gifu University, Japan, (o3128038@edu.gifu-u.ac.jp)
4Lexer Research Inc., Japan
5
Zagazig University, Egypt, (r_h_elshaer@zu.edu.eg）
Abstract: One of the problems of cell production system is how to decide the parts layout locations. Traditionally, theproblem is solved using trial and error method which takes a lot of efforts and time. In this paper, we propose a Virtual
Assembly Cell-production system (VACS) for the cell production assembly line. The VACS use a genetic algorithm(GA) system to find a reasonable solution and a virtual production (VP) simulator for giving us a visibility of thatsolution in the production system. The validation and the efficiency of the proposed VACS system are tested on ten
varieties of a product. The simulator results show that the VACS system is capable of getting good solution in areasonable computational time when compared to the traditional one.
Keywords: Genetic algorithm, Parts layout locations, Cell production, Virtual production.
(GA) system whose crossover method is the original
called TTC.
The paper is further organized as follows. Section 2
introduces the VACS system and its two functions, PLD
and VP systems. In section 3, the test problem and thecomputational results are presented. Finally, section 4
concludes the paper.

I. INTRODUCTION
Recently, the production system has changed frommass production of a limited variety of products to low
volume production of a wide variety of products, masscustomization[1,2]. It is because the consumers’
individual needs are diversified. Therefore, it isimportant to produce products of a wide variety
efficiently. One of the needs for modern production
methods is the cell production system[3]. A cell
production system is a production system in which asingle worker or small team of production workers
perform multiple production jobs in short segment lines.
The cell design places a wide range of tools andequipment in close proximity to workers, enabling themnot only to perform a wide range of production tasks,
but to customize the products as well. One of the
problems of cell production system is the parts layout
locations. The current state of this problem is to besolved empirically using a trial and error method whichtakes a lot of effort and time. In addition, theimprovement in production efficiency is going up
gradually during the assembling process where there is
no opportunity to make the production efficiency a peakfrom the beginning.
In this research, we propose a Virtual Assembly
Cell-production System (VACS), a cell production
simulation system, for solving the problem. The VACS
integrates a Parts Layout Decision system (PLD system)
and a Virtual Production simulator (VP simulator). To
carry out the PLD system, we adopt a genetic algorithm©ISAROB 2011

II. VACSThe VACS system consists of two collaborating
functions, the PLD System including GA system and the
VP simulator, as shown in Figure 1. The steps of the
VACS system are described in Table 1.

Parts Layout
Decision
System(GA system)

VirtualProduction
Simulator

Fig.1 VACS system
Table 1. Steps of the VACS system.
Step 1. VP simulator draws the workshop floor, andsends the coordinated data of the locations, such as
shelves and worktables, to the PLD system.
Step 2. In the PLD system, the parts layout locations
are decided using the GA system and are sent backto the VP simulator.
Step 3. VP simulator draws the parts layout andanimately visualizes the working environment.

885

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011transformed into the shape that can be cross over, and it
is reversely transformed after crossover to its former
shape. One point crossover is applied with probability
CP. The TTC is as shown in Table 2. As an example,
from Figure 2, consider Ch1<CEDFAB> andCh2<AFBCDE> be two selected chromosomes forcrossover. The steps of TTC for generating two
offsprings, Off1 and Off2, from the two chromosomes,
Ch1 and Ch2, are explained in Table 3.
Each gene in the chromosome may be mutated withprobability MP. Mutation method is to swap the selectedgene with randomly selected one.

1.

VP simulator
The VP simulator arranges the parts inside the
shelves in workshop floor according to the received
information from PLD system showing that in three
dimensions. In addition, it visualizes the working
environment in which an animation of the assemblingprocess is shown (see section III.3).
2.

PLD system
The PLD system uses a GA system to decide thebetter parts layout locations in terms of minimum total
moving distances. In the GA system [4], the informationof the parts layout locations encodes into feasible
chromosome. In the searching process for finding abetter parts layout locations, genetic operators, such as
crossover and mutation …etc., are repeated, until apredefined stop criteria is verified. More details about
the proposed GA in the following paragraphs.

Table 2: Twice transformation crossover method
1
In non-decreasing order the parts are arranged
in list L
2
Take the first gene, i, in the current
chromosome
3
While L is non-empty, Do:
3.1
Replace gene i with its order in list L.
3.2
Remove part i and update list L
3.3
i equals next gene
4
Apply Steps from 1 to 3 to transform two
selected chromosomes.
5
Apply one point crossover to generate two
offsprings
6
Use the reverse method of steps 1 to 3 totransform the two offsprings into the original
shape

1.

Chromosome representation
To treat the information of the parts layout locations,
we use a direct representation in which a part number isexpressed as a gene, and the part position in thechromosome is expressed as the part location in thelayout. For instance, Figure 2 is expressed with thefollowing chromosome: <DFABEC>.

Table 3. An example for the TTC method
First Chromosome:Ch1< CEDFAB >L={A, B, C, D, E, F}, and i = C
Ch1<3EDFAB> , Update L
L={A, B, D, E, F}, and i = E
Ch1<34DFAB>, Update L
L={A, B, D, F}, and i = D
Ch1<343FAB>, Update L…
Ch1<343311>
By the same method Ch2<151111>
Assume that a one point crossover is randomly
chosen between position 3 and 4 to generate Off1
and Off2.
Off1<151311> and Off2<343111>
Decode Off1 to original shape:
L= {A, B, C, D, E, F}
Off1<A51311>, Update L
L= {B, C, D, E, F}
Off1<AF1311>, Update L…
Off1<AFBECD>
By the same method Off2<CEDABF>

1~6: Location name
A~F: Part nameFig.2. An example of chromosome representation
2.

Initial population and fitness function
Initial population is randomly generated and thefitness function is expressed as the reciprocal of thetotal moved distances to achieve a certain amount of
production.
3.

Selection, crossover and mutation methods
In selection for crossover [4], in this paper, Roulettewheel selection method is used. Applying conventionalcrossover methods in our proposed chromosome may
generate a lethal chromosome. To solve this problem,
we develop what we call Twice Transformation
Crossover (TTC). By this method, the chromosome is©ISAROB 2011886

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

best total moving distances of the both methods. The
percentage improvement of the GA and thecomputational CPU time are shown in columns 4 and 5
respectively. Using T-test at 99% confidence level, we
find that our developed algorithm makes a statisticallysignificant improvement over the random one.

4.

Stop criteria and GA parameters
The control parameter values and terminating
condition used in our GA was selected based on several
preliminary runs with alternate control parameters andterminating conditions on different instances of the
problem. These values were then used for the test
problem reported in the computational results. The final
parameter values are summarized in Table 4.
Table 4. Parameter values for the proposed GA
Description
ValueszPop. Sizez
Crossover ratez
Mutation ratez
% of solutions replaced by new gen.
z
Stop criteria*Stop after 100 generations without improvement

100
0.9
0.05
0.95
100*

Fig.4. workshop floor shown the worktable and shelves

III. COMPUTATIONAL RESULTS
1.

Test problem
The developed VACS system is tested on a cellproduction assembly line of a personal computer. The
workplace design and the required parts are as shown in
Figure 3. The workshop shelves layout and the final
product are as shown in Figures 4 and 5 using VP
simulator. There are ten varieties of the product inwhich each one contains at most 18 parts. The assembly
process sequence of each product type is known. The
parts of the same type are arranged in one shelf. The
workplace contains one worker, and the movement
between the shelf and the worktable is in a straight line.

Fig.5. 3D configuration of the final product
Table 5. Comparison between PLD with GA andwithout GA (units in meter)
Simulation
No.
1
2
3
4
5
6
7
8
9
10

GA
Method
5003
5053
5053
5011
5092
5027
5025
4975
5190
5021

% of
Imp.
1.65
3.53
3.77
4.11
1.68
2.39
2.56
2.42
0.39
3.95

GA’s CPU
time (msec)
718
795
780
920
468
843
921
1104
717
655

A comparison between the fitness curve and the
distance reciprocal curve of the random method forgetting a best layout in the first simulation is as shownin Figure 6. It is clear from the figure that the random
method makes a dramatically improvement in the
beginning after that it continues without gaining any
improvement. In the other side, the fitness curve isgetting better. This difference between the two methods
is because the GA exploits the historical information to
make improvement, but the random method is not.
Figure 7 shows the best parts layout location obtainusing the GA from the first simulation.

Fig.3. Test problem information
2.

Results
Table 5 shows the computational results of 10 runs
for the PLD system with the proposed genetic algorithm
and with the random method. Columns 2 and 3 show the©ISAROB 2011

RandomMethod
5087
5238
5251
5226
5179
5150
5157
5099
5210
5227

887

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

From the computational results, we found that the
PLD system with the proposed GA was statisticallysignificant impact on the results than the PLD without
the GA. Moreover, the VP simulator can be used for
educational purposes where the steps of assemblingprocess are visualized with animation.

2.05E-04
2.00E-04
Fitness

1.95E-04
1.90E-04
1.85E-04
1.80E-04

Fitness
Random method

1.75E-04
0

100

200

Generation

Fig.6. Best fitness curve(a)Fig.7. Best parts layout using GA from first simulation(b)
Fig.8. Virtual production for the layout shown in Fig.7.

3.

Visualization by VP simulator
VP simulator receives the output results from PLD
system and visualizes the working environment. Figure8(a and b) shows snapshots from the virtual productionof our test problem for the best parts layout locations
shown in Figure 7.

REFERENCES[1] Juan Diego Frutos, Denis Borenstein(2004), A
framework to support customer-company interaction inmass customization environments, Computers in
Industry 54 (2):115-135[2] Suresh Kotha (1996), From mass production to masscustomization: The case of the National Industrial
Bicycle Company of Japan, European Management
Journal 14(5):442-450[3] Katsuhide Isa, Tsuyoshi Tsuru (2002), Cell
production and workplace innovation in Japan: Toward
a new model for Japanese manufacturing?, Industrial
Relations: A Journal of Economy and Society
41(4):548-578.
[4] D.E. Goldberg, Genetic algorithms in search (1989),
optimization, and machine learning, Addison-Wesley

VI. CONCLUSIONSIn this paper, we developed VACS system that
integrated PLD system and VP simulator. The PLD
system used our developed GA system whose crossover
method used the original TTC. The PLD system
obtained a good layout for the parts locations in areasonable computational time. The VP received thelayout from the PLD system and visualized the working
environment.

©ISAROB 2011888

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Parallelizing Fuzzy Rule Generation using GPGPU
T. Uenishi, T. Nakashima and N. Fujimoto
Osaka Prefecture University, 1-1 Gakuen-cho, Nakaku, Sakai, Osaka(Tel : 81-72-254-9351; Fax : 81-72-254-9915)(nakashi@cs.osakafu-u.ac.jp)

Abstract: This paper proposes a method to parallelize the process of generating fuzzy if-then rules for pattern classiﬁcation problems in order to reduce computational time. The proposed method makes use of GPGPUs’ parallel
implementation with CUDA, a development environment. CUDA contains a library to perform matrix operations inparallel. In the proposed method, published source codes of matrix multiplication are modiﬁed so that the membershipvalues of given training patterns with antecedent fuzzy sets are calculated. In a series of computational experiments, it
is shown that the computational time is reduced for those problems that require high computational efforts.
Keywords: fuzzy if-then rule, parallel computation, GPGPU, pattern classiﬁcation

I. INTRODUCTION

as binary ﬁles, some source codes of SGEMM (Single
precision General Matrix Multiply) algorithms have been
published by the developer.
In the proposed method, calculation of the membership values is parallelized by viewing them as matrix calculation, using two matrices which represent antecedent
fuzzy sets and training patterns. The published source
codes of the matrix multiplication in SGEMM are modiﬁed so that the membership values of given training patterns with antecedent fuzzy sets are calculated in parallel.
In a series of computational experiments, the computational time of the proposed method is compared with thatof the traditional method that only uses a CPU. It is shownthat the proposed method reduces the computational time
for pattern classiﬁcation problems that have high dimensionality and/or a large number of training patterns.

It is known that fuzzy systems based on fuzzy if-thenrules perform well for pattern classiﬁcation problems [1].
However, the computational cost of a fuzzy system is often huge when it is applied to high-dimensional problems
with a large amount of training patterns. This is mainly
due to explosive increase in the number of fuzzy if-thenrules that are generated to construct a classiﬁcation system. One solution to this problem is to reduce the numberof rules. For example, the number of rules to be generated can be restricted, or a small number of fuzzy if-thenrules can be selected by using genetic algorithms [2].
This paper proposes a method for speeding up the process of generating fuzzy if-then rules for pattern classiﬁcation problems without reducing the number of rules.
The proposed method is to implement the fuzzy-rule generation process on GPGPU (General Purpose computation on Graphics Processing Units) in order to reduce thecomputational time.
GPUs, which were originally developed for graphics
processing, have a lot of multiprocessors and have potential for high-speed parallel computation. Implementationof GPUs is usually done in C programming language using CUDA (Compute Uniﬁed Device Architecture) [3].
To implement with CUDA, it should be considered thata GPU has its own memories which are only accessible
from the GPU. GPU’s memories are composed of several types of devices with different access speeds and capacities. Thus the efﬁciency of parallel computing with
GPUs depends on the optimality of the memory access.
However, it is difﬁcult to design the memory access optimally without understanding the details of the hardware
architecture of GPUs. In this paper, we adapt existing
implementation to fuzzy classiﬁcation system.
A library, called CUBLAS, is included in the CUDA
package [4]. CUBLAS is an implementation of BLAS
(Basic Linear Algebra Subprograms) computation for
GPGPU. It allows us to develop parallel computing programs more easily without heavily modifying source
codes. While all algorithms in CUBLAS are published©ISAROB 2011

II. FUZZY RULE GENERATIONIn this paper, we propose a method to parallelize fuzzyrule generation that is formulated in the fuzzy system by
Ishibuchi et al [1]. It should be noted that the method canbe applied to any forms of fuzzy if-then rules because it
parallelizes only membership calculation. An overview
of the system in [1] is shown below.
In a pattern classiﬁcation problem with n dimensionality and M classes, we suppose that m training patterns,
xp = {xp1 , xp2 , ¢ ¢ ¢ , xpn }, p = 1, 2, ¢ ¢ ¢ , m, are given
and each attribute of xp is normalized to a unit interval[0, 1]. From training patterns we generate fuzzy if-thenrules of the following type:
Rq : If x1 is Fq1 and ¢ ¢ ¢ and xn is Fqn
then Class Cq with CFq ,q = 1, 2, ¢ ¢ ¢ , N,(1)where Rq is the label of the q-th rule, Fq = (Fq1 , ¢ ¢ ¢ ,Fqn ) represents a set of antecedent fuzzy sets, Cq a theconsequent class, CFq is the conﬁdence of the rule Rq ,and N is the number of rules. We use triangular membership functions as antecedent fuzzy sets. Figure 1

889

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Membership value ( µ k (x) )shows triangular membership functions which divide theattribute axis into ﬁve fuzzy sets. Suppose that an attribute axis is divided into L fuzzy sets. The membershipfunction of the k-th fuzzy set is deﬁned as follows:}{|x ¡ xk |, 0 , k = 1, ¢ ¢ ¢ , L, (2)µk (x) = max 1 ¡
v
xk =

1.0

0.0

k¡1
, k = 1, ¢ ¢ ¢ , L,
L¡1

(3)

Fig. 1. Triangular fuzzy sets

1
.
L¡1

(4)

III. GPGPU

v=

n
∏

µFqi (xpi ), q = 1, 2, ¢ ¢ ¢ , N,

(5)

i=1

where µFqi (xpi ) is the compatibility of xpi with the fuzzy
set Fqi and xpi is the the i-th attribute value of xp .
µFqi (xpi ) is calculated by equation (2).
Equation (5) implies that the same procedure is iterated
for calculating the compatibility of a training pattern witheach fuzzy if-then rule: First calculating the compatibility
for each attribute, and then multiplying them. Therefore,
we can view this process as a function of two matrices.
One matrix represents a set of fuzzy if-then rules. The
size of this matrix is N £ n and is composed of N row
vectors whose lengths are n and elements are antecedent
fuzzy sets Fq . The other matrix represents a set of training patterns. This matrix is n £ m and is composed of mcolumn vectors whose lengths are n and each column is a
training pattern xp . In the conventional matrix multiplication for two matrices, the (q, p) element of the product,
rqp , is represented as:
rqp =

n
∑

Fqi £ xpi .

(6)

i=1

We adapt the above calculation to the calculation of membership value µFqi (xpi ) with the same access order as
matrix multiplication. Thus the (q, p) element of the re0
sult, rqp
, is represented as:
0
rqp
=

n
∏

Fqi ¯ xpi ,

(7)

i=1

where ¯ denotes the membership calculation, i.e., equation (2). That is, the membership calculation (i.e., equation (7)) can be regarded as a matrix operation where
product operation is replaced with a membership function
and sum operation is replaced with a product operation.
The number of fuzzy rules to be generated is Ln . Thatis, the number of rules increases exponentially for the division number and the dimensionality.

©ISAROB 2011

1.0

GPUs have a lot of multiprocessors and thus have highpotential for parallel computation. The performance ofGPUs have been improved tremendously. NVIDIA havereleased CUDA, a development environment of GPGPU.
In CUDA, functions which are executed on a CPU arecompiled with a c-compiler (we used GCC in this paper). While functions which are executed on a GPU arecompiled with NVCC (NVIDIA CUDA Compiler). Dataand control structures available for GPUs are the same asthat of CPUs. However, a GPU has its own memories
which are only accessible from it. Thus data transportation between a CPU and a GPU is required before theactual computation. The basic procedure of GPU computing is composed of four steps: A CPU transports data
to a GPU, the CPU instructs the GPU to calculate, theGPU executes the calculation, and the GPU transports result to the CPU. Instructions for a GPU are composedof threads, blocks, and grids depending on the level ofparallelization. A thread is an atomic execution of the
instructions. A blocks is a set of threads, and a gridis a set of blocks. CPUs can only send instructions togrids. Upon receiving the instructions, threads in a block
execute the calculations parallely depending on instructions, and instructions of blocks in a grid are also executed parallely. During the calculations, threads which
belong to the same block can be synchronized and makeuse of shared memory whose access speed is higher than
the global memory. The number of thread per blockand the number of blocks per grid need to be determined. Thus the efﬁciency of parallel computing with
GPUs depends on the memory access and the composition of threads, blocks, and grids. However, it is difﬁcult to design the optimal memory access without understanding the details of the hardware architecture ofGPUs. As an optimized library of BLAS(Basic Linear
Algebra Subprograms) for CUDA, CUBLAS is publishedtogether with CUDA. In CUBLAS, SGEMM(Single precision General Matrix Multiply) and DGEMM(Double
precision GEMM) are implemented by Volkov et al [5].
Some source codes of SGEMM are published by them.
Since the memory access of CUBLAS is designed efﬁciently, user can implement their algorithms without any
concern about memory access diverting the memory access of it. In this paper, we modify the source code of
SGEMM so that the calculation of membership values

Compatibility of a training pattern xp with a fuzzy ifthen rule Rq is denoted by µFq (xp ) and is calculated asfollows:µFq (xp ) =Attribute value ( x )890

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011are parallelized in order to reduce the computational time.
Representation and processing of ﬂoating point on GPUs
follows IEEE754, and we suppose that real numbers on
CPUs and GPUs are both single precision in this paper.

And for a set of training patterns, we set a transposed
matrix B T as follows:
x11 . . . xm1
..  .
..
(10)
B T =  ...
.
. 

IV. IMPLEMENTATION

x1n

...

xmn

As mentioned in Section II, the formulation of membership values for fuzzy-rule generation is similar to thatof matrix multiplication. We modify the SGEMM algorithm introduced in section III, to be the algorithm to generate fuzzy if-then rules. Volkov et al [5] published the
SGEMM algorithm that calculates the following equation:
C new = α £ A £ B T + β £ C old ,(8)

By applying the calculation of compatibility modiﬁed
from matrix multiplication to the above two matrices, a
N £ m matrix C is computed as:µF1 (x1 ) . . . µF1 (xm )


..
..
..
C=
(11)
,
.
.
.

where A is a x £ y matrix, B T is a y £ z matrix, and C
is a x £ z matrix. α and β are scalar values. Equation (8)is calculated parallely by a GPU after initialization by aCPU. In this paper we specify that α = 1 and β = 0 to
consider only the matrix multiplication. By representing
elements of the matrices as A = (aij ), B T = (bij ), andC = (cij ) and a temporal variable as t, the procedure to
calculate an element of C new , cij , can be shown as thepseudocode in Fig. 2(a). The parallel procedure to calculate compatibility is shown in Fig. 2(b), where aik is
the label of the antecedent fuzzy set, and bkj is the input
value and µ(aik , bkj ) is the membership function of the
input value bkj for the fuzzy set aik . Thus the order toaccess the elements of each matrices is the same as thatof the original matrix multiplication. Therefore, the consistency of the parallel computation holds by replacing
addition and multiplication of the elements in matrix multiplication to multiplication and membership calculation
respectively. In addition, Volkov et al [5] employs 16 £ 4
threads per block and (x/64) £ (y/16) blocks per grid tomake the memory access efﬁcient. However, this limitation has no effect on the calculation of the equation. Now
we can parallelize the membership calculation on GPUs
by applying the above procedure to matrices which represent antecedent fuzzy sets and training pattern sets.
We suppose that xFqi is an element of A, where xFqi is
the mode of the fuzzy set Fqi computed by equation (3):
xF11 . . . xF1n..  .
..
A =  ...
(9)
.
. 

where a row vector of C corresponds to the compatibility of rules for each patterns, i.e., equation (5). The procedure to parallelize fuzzy-rule generation with a GPU
takes the following steps. First, by a CPU, A and B T are
made, and transported to a GPU. Second, the GPU calculates C using the matrix operation for A, B T . Finally,
C is transported to the CPU, and then it determines theconsequents.

xFN 1

...

µFN (x1 )

t=1
for(k = 1 to y){t £= µ(aik , bkj )}cij = t(a) Matrix multiplication(b) Proposed method

To verify the effect of parallelization with GPUs, thecomputational time to generate fuzzy rules with a GPU iscompared to that of a CPU. Table 1 shows the environment of the experiments. Although GeForce GTX 295
has a dual-chip structure, we use only one chip. In theexperiments, the computational time for solving a twoclass problem is compared. 100 classiﬁcation problems
with different number of training patterns and dimensionalities were used to evaluate the computational time. The
number of fuzzy sets for each axis is ﬁxed to two. The
results were averaged to compare the efﬁciency of theparallelization. The results of the experiments are shownin Figs. 3-6. Figure 3 shows how dimensionality of the
problem has an effect on the computational time when thenumber of training patterns is 64. Figure 4 shows how dimensionality of the problem has an effect on the computational time when the number of training patterns is 816.
In Fig. 3, the computational time with a CPU is shorter
than that with a GPU when the dimensionality is small.
As the dimensionality increased, the computational timewith a CPU increased drastically while that with a GPU
keeps short. In Fig. 4, the computational time with a GPU
is shorter than that with a CPU constantly. Figures 5 and6 show that the number of patterns has an effect on thecomputational time when the dimensionality of the problem is 12 or 18 respectively. In Fig. 5, the computationaltime with a CPU is shorter than that with a GPU when
the number of patterns is small. However, when the number of training patterns is large, the computational timewith a GPU is shorter than that with a CPU. In Fig. 6,
the computational time with a GPU is shorter than that

Fig. 2. Pseudocodes of the matrix multiplication and the
proposed method©ISAROB 2011

µFN (xm )V. COMPUTATIONAL EXPERIMENTS

xFN n

t=0
for(k = 1 to y){t += aik £ bkj}
cij = t...

891

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Computational time [s]with a CPU constantly. Thus the parallel computation forgenerating fuzzy rules with a GPU has the effect of reducing the computational time except in the case of lowdimensionality problems with small amount of training
patterns.
Table 1. Environment of the experiments
Intel Core i7 Extreme 945
3.20 GHz
5.8 GB667 MHzNVIDIA GeForce GTX 295
2401242 MHz896 MB
999 MHzLinux x86 64
CUDA(NVCC)2.2, GCC4.3

VI. CONCLUSIONSIn this paper, we proposed a method to parallelize
fuzzy-rule generation with a GPU using matrix multiplication that is optimized for CUDA. Computational experiments showed that the method reduced the computational time when the dimensionality of the problem
and/or the number of training patterns were large. For
future works, we will try to parallelize fuzzy inferencewith GPU, or resolve lack of memory on GPU when themethod is applied to problems with further dimensionality and/or the number of training patterns.

CPU
CPU + GPU

22
11
00

13
13

12
12

14
15
16
14
15
16
Dimensionality

17
17

18
18

60 60
50 50

CPU
CPU + GPU

40 40
30 30
20 20
10 10
0 0

12
12

13
13

14
15
16
14
15
16
Dimensionality

17
17

18
18

Fig. 4. Computational time for dimensions (816 training
patterns)

Computational time [s]REFERENCES[1] Ishibuchi H, Nakashima T and Nii M (2003), Classiﬁcation and Modeling with Linguistic Information
Granules. Springer[2] Ishibuchi H, Nozaki K, Yamamoto N, and Tanaka H(1993), Selection of Fuzzy If-Then Rules by a Genetic Method(in Japanese). The Transactions of theInstitute of Electronics, Information and Communication Engineers Vol.J76-A No.10:1465-1473

4 0.8
3 0.6

CPU
CPU + GPU

2 0.4
1 0.2
0 0

1
64

2
3
128
512
Number of patterns

4816

Fig. 5. Computational time for number of patterns (12dimensionality)

Computational time [s][3] NVIDIA CUDA, http://www.nvidia.com/
object/cuda home.html[4] NVIDIA
CUBLAS
Library,
http://developer.download.nvidia.com/compute/cuda/2 1/toolkit/docs/CUBLAS Library 2.1.pdf[5] Volkov V and Demmel JW (2008), LU, QR andCholesky factorizations using vector capabilities ofGPUs. Technical Report No.UCB/EECS-2008-49©ISAROB 201133

Fig. 3. Computational time for dimensions (64 training
patterns)

Computational time [s]CPU
Clock Frequency
Memory Size
Memory Clock
GPU
Processor Core
Processor Clock
Memory Size
Memory ClockOS
Development Environment

44

6060
5050

CPU
CPU + GPU

4040
3030
2020
1010
0 0

1
64

2
3
128
512
Number of patterns

4816

Fig. 6. Computational time for number of patterns (18dimensionality)892

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Interactive musical editing system to support human errors and offer
personal preferences for an automatic piano- A method for searching for similar phrases using DP matching –Kenichi Koga

and

Kentaro Minowa

andEiji Hayashi

Department of Mechanical Information Science and Technology
Faculty of Computer Science and Systems Engineering, Kyushu Institute of Technology
680-4, Kawazu, Iizuka-City, Fukuoka Prefecture, Japan
Abstract: We have developed a system that allows a piano to perform automatically. In order to play music in themanner of a live pianist, we must add expression to the piano’s performance. In the case of piano music, there are often
1000 or more notes in the score of even a short piece of music, requiring that an editor spend a huge amount of time toaccurately simulate the emotionally expressive performance of a highly skilled pianist. Therefore, we have developed
an interactive musical editing system that utilizes a database to edit music more efficiently. We have analyzed MIDI
data regarding the performances of highly skilled pianists in order to observe the stylistic tendencies of theirperformances. Our result showed that phrases having similar patterns in the same composition were performed insimilar styles. Therefore, we developed a system that searches for similar phrases throughout a musical score andevaluates the style of their performance. The method of searching for similar phrases uses DP (Dynamic Programming)
matching. Using the method of searching, we developed a phrase and music search engine. We thought that as long as it
is able to search for similar phrases, it would be able to search for a tune including the phrase from among various tunes.
Keywords: automatic piano, knowledge database, computer music, DP matchingevaluates the style of their performance. We propose amethod that uses DP matching as a way to search forsimilar phrases. This system converts notes into
character strings. In addition, the system runs DP
matching using character strings and calculates thedegree of disagreement between these strings. We use
these calculations as an index to determine whether thestrings resemble each other. At the end, we introduce aphrase and music search engine using the method ofsearching.
In this paper, we describe the results of searching forsimilar phrases using DP matching.

I. INTRODUCTION
We have developed a performance system for an
automatic piano. In this system, 90 actuators are
installed on the 88 keys and the 2 pedals of a grandpiano. These actuators operate key strokes and executepedaling on the piano. (See Figure 1.1)
Reproducing music with the piano is similar in someways to reproducing music on the computer. Essentially,variations in tempo, dynamics, and so on are needed toarrange the respective tones in the desired way.
However, in the case of piano music, there are 1000 or
more notes in a score of even a short piece of music,
and for this reason an editor must spend an enormous
amount of time working with an arrangement in order tosimulate the expressions of an actual performance.
Therefore, in this research, we have developed aninteractive musical editing system to edit music more
efficiently [1].
We have analyzed MIDI data from the performancesof highly skilled pianists in order to observe the stylistic
tendencies of their performances. Our results showedthat phrases having similar patterns in the same
composition were performed in similar styles. Moreover,
we found that the pattern of notes in a score sometimes
influences the expression of a piece of music.
In this research we developed a system that searches
for similar phrases throughout a musical score and©ISAROB 2011

Figure 1.1: View of the automatic piano

893

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

II. Musical Editing Support System

2.3.1 Musical Rules Database
This database contains the architecture of musicalgrammar necessary to interpret symbols in musicalnotation. It is composed of five tables containing“Dynamics marks”, “Articulation marks”, “Symbol of
Changing Dynamics or Changing Tempo” (symbol that
affects the speed of a note or the increase or decrease of
the volume), “Time signature”, and “Tempo marks”.
Analyzing a music symbol according to its usage
allows efficient information processing by the system.

2.1 System Architecture
The structure of the system is shown in Figure 2.1.
The user edits music via the user’s interface on acomputer display. The user can also access a databasethat has musical grammar, the user’s preferences, and soon. As a result, editorial work is reduced and efficient
editing becomes possible.

2.3.2 Score Database
This database has symbols including time signatures,notes, rests and so on in standard musical notation.
Symbols were pulled together in order of bars, and bar
symbols were arranged in a time series. Performance
expression in itself is only information such as pitch,strength, and length and concerns only the enumerationof a sound. Because the identification of each sound isdifficult, editing of the performance expression isdifficult. By adding the Score Database’s information to
performance expression, we can connect each note to its
enumeration. In doing so, it becomes easy to edit eachphrase.
This database consists of three tables, the “Element
table” (showing the position of the note and thecomposition of the chord), the “Symbol table” (showing
the position of the music symbol) and the “Same table”(showing the position of the repetition of the phrase).
The Element table contains the field “Note Value”.
Data in this field indicates the type of note, e.g., aquarter note, a triplet, and so on. “Note Value” isexpressed by three hexadecimal numbers, which are
shown in Figure 2.3.

Figure 2.1: Structure of the editing system
2.2 Format of Performance Information
The parameters of performance information are
shown in Tables 1 and 2. The automatic piano that wehave developed uses a music data structure that issimilar to MIDI. We defined performance information,
dividing it into two categories: the notes and the pedals.
The note information is comprised of the six parameters
involved in producing a tone: “Key” (note), “Velo”(velocity), “Gate”, “Step”, “Bar”, and “Time”. “Velo” isthe dynamics, given by the value of 1–127. “Gate” isthe duration of the note in milliseconds. “Step” is theinterval of time between notes, and it also exhibits
tempo. “Bar” is the vertical line placed on the staff todivide the music into measures.
The pedal information is comprised of fourparameters: “Key” (indicating the kind of pedal:“Damper” or “Shifting”), “Velo” (the pedaling quantity),“Time” (the duration for which the pedal is applied)”,and “Bar”.
2.3 Editing Support Process with Database
Our system can automatically apply a roughperformance expression using a Musical Rules Databaseand Score Database. (See Figure 2.2)
In addition, the system has Preference Database,which stores the editing characteristic of the user.

①
②
③
④
⑤
⑥

Figure 2.2: Automatic translation with database©ISAROB 2011

Note or Rest (0: Note, 1: Rest)
Note Value (000: A whole note, 001: A half note …etc)
Tie (If the note has a tie then this number is 1.)
Ornament(If the note has an ornament then this number is 1.)
The number of dots
(Exceptionally, if it is “11” then the note is tuplet.）
Additional Information
The number of tuplets. (Triplets: 0011)
The type of an ornament. (Trill:010）
etc…

Figure 2.3: Note Value

894

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

III. Searching for Similar Phrases

3.2.2
The Method of Searching
The flow of the similar phrase search is shown in
Figure 3.3.

As a result of the analysis, it was found that phrasesof the same pattern existing in the same tune are
performed in a similar expression. This time, we used
DP matching to search for similar phrases.
3.1 DP matching
DP matching is a technique used widely in the fieldof speech recognition, bioinformatics and so on. It has afeature that can calculate the similarity between two
words that are different in a number of characters fromeach other.
In Figure 3.1, the route of minimum cost in eachpoint is taken, and the route with the lowest cost is
assumed finally to be the optimal path. The cost at thattime is defined as the distance between patterns. In thissystem, this distance is handled as a threshold to judge
whether the phrases are similar to each other.
For example, if the cost moves up or to the right,
then it is increased by 1. If it moves to the upper right,
then it does not increase. Also, if the characters do notcorrespond in each point, then the cost is increased by 5.

Figure 3.3 The flow of the similar phrase search

Essentially, three rounds of search processing are
performed. The first processing round narrows down the
points to those having a resemblance in all searchranges. A pattern with the same number as the searchphrase is pulled out, and the distance between the twopatterns is calculated using DP matching. If they are incomplete accord (distance = 0), then the phrase is stored
in the Similar Table. If the distance is lower than thethreshold, then the phrase is stored in the Temporary
Table. (See Figure 3.4)

 g (i − 1, j ) g (i , j ) = min  g (i , j )  + d (i , j ) g (i , j − 1) Figure 3.1: DP matching
3.2 Searching with DP matching
In this passage, we describe a method of searchingwith DP matching. We had to convert a musical score
into character strings (a Note Pattern) before searchingfor similar phrases. This process is explained below.

Figure 3.4: The first process

3.2.1
Note Pattern
Our system converted a score into a Note Patternusing Note Values (See Passage 2.3) in order to perform
DP matching. Of the three columns of Note Values, weused the two columns on the left. The system replaces
numbers in the second column with letters of thealphabet (from G) because a letter, being a different
notation than that used in the first column, allows theexpression of one note with two columns. An example
of a Note Pattern conversion is shown in Figure 3.2.

In the second round of processing, DP matching is
performed again using the phrases in the Temporary
Table while increasing the number of characters. In
other words, the system looks for the most similarphrases in the surrounding phrases. The threshold of thissystem is decided by trial and error. (See Figure 3.5)

Figure 3.5: The second process
Figure 3.2: Example of Note Pattern©ISAROB 2011895

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

VI. Phrase and music search engine

Table 4.1: Search results

We can use this system to search for similar phrases
in a piece of music. Thereupon, we thought that as longas it is able to search for similar phrases, it would be
able to search for a tune including the phrase fromamong various tunes.
Therefore, we developed a phrase and music searchengine. It searches a tune from various tunes with one
phrase of voice data. The voice is first inputted; then theinterval between the sounds of the inputted voice data isconverted back to notes according to the tempo of music.
As a result, we thought would be able to search forsequence similarity to the phrase rhythm and notes. The
search strategy uses 3.2.2 The Method of Searching.

Name of sound4.1 Processing that inputs voice
It is shown that voice is input in the shape of waves
as shown in Fig. 1. The vertical axis of Fig. 4.1 shows
loudness (mdB), and the horizontal axis shows time. By
setting a threshold on the vertical axis, the presence ofvoice can be recognized. The interval of this sound ismeasured as the time. In other words, as shown in Fig. 1,
the interval of sound is the period between the initial
iterative peak and the next time the sound iterative peak.
The shapes of the waves tell us the note values (Seesection 2.3) of the phrase.

40
20
0
-20
-40
-60

1

2

3

4

5

17.20

Asita ga arusa
Agehatyou

128
9

16.98
18.33

Asuhe no tobira
Can you keep asecretHaruka

26

18.42

135

17.12

83

17.01

Pieces of a dreamTenntai kannsoku

47
13

17.91
19.62

sakura1

20.00

We designed methods of searching for similarphrases using DP matching and combined thesefunctions into a single system.
In a similar phrase search, the system was able tofind similar phrases using DP matching in a short time,and it was even possible to find phrases whose
resemblance might not be immediately apparent.
In the phrase and music search engine, we develop asystem to search for tune by inputting the data of avoice performing the tune. The interval between thesounds of the inputted voice data is converted it into anote according to the tempo of music. By this, we cansearch for sequence similarity to the phrase rhythm andnotes.
In this study, we were able to perform similar phrase
searches and searching for tune by voice. In our future
research we will perform evaluations with different
pieces of music and will evaluate the existing system. In
addition, we will develop the voice-input search system
to perform a search for changes in pitch.

60

0Sekai ni hitotu dake46
no hana

V. CONCLUSION

Interval of sound80

Number of searches The average distancefor phrases
between the phrase

Time[s]Figure 4.1: Result of sound input

REFERENCES[1] Hayashi, E. et al, “Behavior of piano-action in a grand piano.I”,
Journal of acoustical Society of America, Vol.105, pp.3534-3544,
1999.
[2] Hayashi, E et al, “Interactive musical editing system forsupporting human errors and offering personal preferences for an
automatic piano”, Proc. of the 7th International Symposium on
Artificial Life and Robotics, Vol.2, pp 513-516, 2002.
[3] Hikisaka, Y., Hayashi, E., et al, “Interactive musical editing system
for supporting human error and offering personal preferences foran automatic piano –Method of searching for similar phrases withDP matching and inferring performance expression-”, Proc. of the
12th International Symposium on Artificial Life and Robotics,
GS4-3, 2007.

4.2 Search Experiments and Discussion
We searched for music using the phrase and music
search engine. The search phrase was the phrase fromfourth sound of bar 24 to first sound of bar 27 of “asita
ga arusa” ULFULS. Results were set to output phrases
with a distance of 20 or less. Table 4.1 summarizes theresults. Looking at the results, “asita ga arusa” has thesmallest average distance. Thus, the system can find“asita ga arusa” to select smallest average distancesound.

©ISAROB 2011896

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Subsurface imaging for anti-personal mine detection by Bayesian super-resolution with
Smooth-gap prior
Satoshi Kozawa, Takashi Takenouchi and Kazushi Ikeda
Nara Institute of Science and Technology
8916-5 Takayama, Ikoma, Nara, Japan
Phone: +81-0743-72-5986, E-mail: {satoshi-ko,ttakashi,kazushi}@is.naist.jp

Abstract: Ground penetrating radars (GPRs) have been studied to reconstruct a subsurface image. Signal observedby the GPRs typically includes very strong noise and reconstruction of the image is a difﬁcult task. We propose a
new subsurface imaging method based on the framework of the Bayesian super-resolution. In the framework, we canincorporate additional information into the reconstructed image by considering a smooth-gap prior, which can representsmoothness of the subsurface image and gaps between materials, and improves the quality of the reconstructed image.
We investigated performance of the proposed method with a synthetic GPR dataset, and conﬁrmed the validity of the
proposed method.
Keywords: Subsurface Imaging, Inverse problems, Smooth-gap prior
1. Introduction
In the past, anti-personnel mines were mainly madefrom metals and hence metal detectors were typical tools
for detecting buried anti-personnel mines. However, recent
plastic mines are difﬁcult to detect because they include few
metal parts. Highly sensitive metal detectors may enable us
to ﬁnd plastic mines, but they also induce high false detection error caused by small metallic pieces such as nails orcans. To overcome this difﬁculty, ground penetrating radars(GPRs) have been studied. A GPR transmits electromagnetic waves into a ground surface, observes the reﬂected
waves, and reconstructs an image that represents the condition of the ground subsurface. An observed signal includes very strong noise that degrades the quality of thereconstructed image and the performance of detection. To
improve the image quality, some methods for subsurface
imaging with GPRs have been proposed.
Feng and Sato [1] tried to reconstruct the subsurface image by applying the pre-stack migration (which is a subsurface imaging method) for synthetic aperture radars (SARs).
A SAR has two or more antennas that transmit or receive
electromagnetic waves. The method improves the image
quality and shows clear shapes of objects, however, still has
noise in the reconstructed image, and the resultant image
causes high false positive error for mine detection. Gurbuz et al. [2] formulated the imaging problem as an inverse
problem based on the model used in [1], and applied theDantzig selector, which assumes sparseness with respect tothe existing probability of buried objects. The assumptionof the sparseness is a kind of regularization for the inverse©ISAROB 2011

problem, and the method for reconstructing images is more
informative for speciﬁcation of the location of subsurface
objects. However, the reconstructed images lose information on the shapes of objects, which makes it difﬁcult todistinguish mines from other objects.
We propose a new subsurface imaging method based on
the Bayesian inference. In the Bayesian framework, we canincorporate additional information into the reconstructed
image by considering a smooth-gap prior. The prior can
describe smoothness of the subsurface image as well as
gaps between materials, by introducing binary latent variables representing whether there is a gap or not between
two points. However calculation of the posterior distribution using the prior is not computationally feasible because
marginalization of latent variables for all pair of points requires exponential computation time. To overcome the difﬁculty, we employ the Variational-Bayes (VB) method, inwhich the posterior is assumed to be written in a factorized
form. We investigated performance of the proposed methodwith a synthetic GPRs datasets, and conﬁrmed the validityof the proposed method.
2. Bayesian Framework
The GPRs use electromagnetic waves to explore thesubsurface of the target. In this paper, we consider a bistatic
GPR, which has two antennas, one of which transmits
electromagnetic waves and the other receives the reﬂected
waves. For subsurface imaging, the GPRs observe the reﬂected signals yk ∈ RNt at k-th scan points and are moved
to the next scan point; and as a whole, a dataset {yk }K
k=1

897

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

is observed. We assume that the transmitted waves are reﬂected at only the boundary of two materials with different
dielectric constants and then the received signals include
information on the boundary in the subsurface. Additionally, we assume that the observed signals at the scan point
are represented as a linear superposition of reﬂected signals
from points over the boundary at a scan point. In a practical sense, there are interactions among the reﬂected signals
from points in the subsurface and then the assumption of
linear superposition cannot be appropriate; however, we use
the model for simplicity.
Let us consider a physical model that represents a relationship between the received signal reﬂected from a point
p and the transmitted signal s(t) as
ζk,p (t) =σp s(t − τk (p))
Ak,pwhere ² is measurement noise. Note that the n-th elementof x represents a reﬂectivity proﬁle corresponding to then-th point in the target subsurface space. In other words,
x corresponds to the subsurface image itself of the target
space and then our goal is to estimate x using the observed
dataset {yk }K
k=1 .
2.1. Probabilistic formulation
We assume that a probability distribution of the measurement noise ² is given by an isotropic Gaussian with amean vector 0. Then a conditional distribution of yk given
x is written asp(yk |x)(1)

where ζk,p (t) is the received signal reﬂected from the point
p in the subsurface at the k-th scan point, τk (p) is the totalround-trip delay between the antennas and the target point p
at the k-th scan point, σp is the reﬂection coefﬁcient of thetarget point and Ak,p is a scaling factor accounting for lossof the signal. Note that the reﬂection coefﬁcient is positive
when the target point p is on the boundary of two materials
with different dielectric constants, and otherwise is zero. A
whole model of the observed signal dk (t) at k-th scan pointis the superposition of reﬂected signals and then is writtenas∫∫∫dk (t) =ζk (x, y, z, t)dxdydz(2)= N (yk |Wk x, β −1 I),
)
(
β
1
2
exp − ||yk − Wk x||
=
Z
2

(7)
(8)

where β is a prediction parameter and Z is a normalization
constant. As a prior distribution of x, we employ the following smooth-gap prior distribution [3]p(x) =∑
ηp(x, η) =p(x, η),

(9)

( ρ
)
1
exp − E(x, η)
Z
2

(10)

where η is a vector of binary latent variables, each of which
represents the gap between two materials, ρ is a hyper paΩ
rameter that controls the strengh of the effect of the priorwhere Ω is a target region of interest. This is called the distribution, and E is an energy function deﬁned as follows:
point-target model [1][2].
∑()
To calculate the right hand side of (2), we discretize theE(x, η) =ηij (xi − xj )2 + (1 − ηij )λ .
(11)
integration in (2) as follows:i∼j∑
The summation i∼j is taken over all pairs of neighboring
pixels. The latent variable ηij represents the local characj=1
teristics of the prior and indicates whether a pair of pixels
where j is an index of a discretized point in the target re- take similar values or not. When η = 1, the pixels i andijgion Ω, ti = t0 + i/F s, t0 is an initial time of measurement j are smoothed due to the quadratic penalty, and there isand F s is a sampling frequency. We observe that a vector no effect for the smoothing when η = 0. Thus, this priorij
dk = (dk (t0 ), . . . , dk (tNt −1 ))t of dk (t) is written as
controls
dk = Wk x(4) whether there is a gap or not between two materials by thelatent variable η.
where x is an N -dimensional vector whose n-th element is
We can rewrite the joint distribution (10) as
σ(n)/A(k, n) and Wk is a matrix whose (i, j) componentisp(x, η) = p(x|η)p(η)(12)
dk (ti ) =N∑s(ti − τk (j))σ(j)
A(k, j)Wk (i, j) = s(ti − τk (j))

(3)

(5)

Here we assume that the GPRs measurement data at k-th
scan point is represented asyk = W k x + ²©ISAROB 2011

where,(6)898

p(η) = Ber(η|ν),p(x|η) = N (x|0, ρ−1 A−1
η ).

(13)
(14)

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Here, ν = 1/(1 + exp(−λρ/2))∏
is a parameter for the where
Bernoulli distribution Ber(η|ν) = i∼j ν ηij (1 − ν)1−ηij ,
(
)−1
K
∑
and (i, j) component of a matrix Aη is deﬁned byΣ =ρEη [Aη ] + β
Wkt Wk
,
 ∑
η
,
i
=
j,
k=1

k∈N (i) ik( K)[Aη ]ij =(15)−ηij ,i ∼ j,∑tµ = Σ βWk y k ,0,otherwise,

(23)

(24)

k=1

where N (i) is the set of neighboring pixels of the pixel i.
and,Then the posterior can be strictly calculated as follows:∏K
p(x) k=1 p(yk |x)(16)p(x|{yk }K)=
k=1
p({yk })∑
wherep(η|{yk })N (µη , Ση ), (17)
=
η

where

νij =

[
Ση

µη

=

(K
∑

ρAη + β

= βΣη

(K
∑

)]−1
Wkt Wkk=1,

1.
1 + exp(− ρ2 (λ − Ex [(xi − xj )2 ]))

(25)

(26)

We employ µ in (22) as the approximated variable of the(18) estimated subsurface image x.

)

Wkt yk

.

(19)

k=1

We employ the maximum a posteriori (MAP) estimate∑η p(η|{yk })µη of (16) for the estimated of subsurface
image x̂. However, since this estimate has the summation
over all pairs of neighboring pixels, it requires exponential
order of computational complexity and hence it is impossible to explicitely compute the estimate. To cope with thisproblem, we employ Variational-Bayes (VB) method to approximate the posterior in this study.
2.2. Variational-Bayes method
For the VB approximation, we introduce a trial distribution q(x, η) which maximizes the variationalenergy function∑∫
p(x, η, {yk })F (q) =q(x, η) lndx.
(20)
q(x, η)ηThe trial distribution can approximate p(x, η|{yk }) sincemaximization of the variational-energy function with respect to q is equivalent to minimization of Kullback-Leibler
divergence between p(x, η|{yk }) and q(x, η). Although
the trial distribution can be an arbitrary probability distribution for the unknown variables x and η in principle, for thesake of tractability, we assume that it can be factorized as:∏q(x, η) = qx (x)qηij (ηij ).
(21)
i∼j

Under the assumption, the optimal trial distribution
maximizing (20) is analytically given asqx∗ (x) = N (x|µ, Σ)©ISAROB 2011qη∗ij (ηij ) = Ber(ηij |νij )(22)

3. Experiments
In this section, we examined performance of the proposed methods by comparing with the existing methods(Feng and Sato [1] and Gurbuz [2] ) using a synthetic GPR
dataset. We created a target space as shown in Fig. 1(a):three objects (rectangle, diamond shape, “O”) are buried in
the target space. The GPR space-time observation {yk }K
k=1
is generated by (6), in which the SNR is 15 dB. In thisexperiment, the transmitter-receiver distance is 10 cm andboth antennae are a height of 10 cm. The number K of
GPR scan points is 400 and an interval of each scan point isuniform.
Results by the Feng and Sato method, and the Gurbuz method are shown in Fig. 1(b) and (c), respectively.
In our Bayesian approach, we applied three kinds of priordistributions, the non-informative prior, the smooth prior[4] and the smooth-gap prior deﬁned by (9). The noninformative prior is given by p(x) = N (x|0, Z −1 ) witha precision matrix ∑Z −1 = 0, whose∑posterior mean isKKwritten as x̂ = ( k=1 Wkt Wk )−1 k=1 Wkt yk . The
smooth prior is a special case of (9) and is given by setting ηij = 1(∀i, j), which results in x̂ = (ρAη |η=1 +∑K∑K
β k=1 Wkt Wk )−1 β k=1 Wkt yk . Figure 1(d), (e) and(f) respectively show the reconstructed subsurface images
with the non-informative prior, the smooth prior (β = 0.1,
ρ = 1.0) and the smooth-gap prior (λ = 0.04, β = 0.1, ρ =1). While the proposed method with the non-informative
prior (Fig. 1(d)) failed to reconstruct the subsurface image,the proposed method with the smooth prior (Fig. 1(e))
and the smooth-gap prior (Fig. 1(f)) could reconstruct thesubsurface image by which we can recognize shapes ofthree buried objects. Also, Fengs method (Fig. 1(b)) could
reconstruct the image; however, boundaries of target objects

899

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

(a)

(b)

(c)

(d)

(e)

(f)

Figure 1: (a) Target space in which three objects are buried: Horizontal axis represents a spatial position in a lateral
direction and vertical axis represents the depth. (b) Feng and Sato method, (c) Gurbuz method, (d) Proposed method with
the non-informative prior, (e) Proposed method with the smooth prior, (f) Proposed method with the smooth-gap priorwere obscure compared with Fig. 1(e), (f), and we cannot
ﬁnd the object shaped like a “O”. The resultant image by
the Gurbuz method in Fig. 1(c) distinctly indicated points ofexisting objects because of the assumption of the sparsenesswith respect to the existing probability of buried objects.
However, information on shapes of buried objects was lostand it is difﬁcult to determine whether the detected object
is a landmine or not.
To investigate the quality of the reconstructed subsurface image, we calculate a normalized SNR of the image reconstructed by the proposed method, with the smooth priorand the smooth-gap prior, and the method of Feng and Sato,respectively. We deﬁne a normalized SNR between the true
subsurface image and the reconstructed image as
10 log10||xtrue / max xtrue ||2[dB]||x̂/ max x̂ − xtrue / max xtrue ||2

In this paper, we proposed a new subsurface imaging
method based on Bayesian framework which enables us to
use prior knowledge such that represents the smoothnessof the subsurface and the gap between two materials. We
observed that the proposed method attained better performance compared with conventional methods for the synthetic dataset. Application of the proposed method for real
datasets will be a future work.
References[1] X. Feng and M. Sato: Pre-stack migration applied to
GPR for landmine detection, Inverse Problems, Vol. 20,
pp. 99-115, 2004.

(27) [2] A.C. Gurbuz, J.H. McClellan and W.R. Scott Jr: Compressive sensing for subsurface imaging using groundwhere xtrue is a vector associated with Fig. 1(a) and x̂ ispenetrating radar, Signal Processing, Vol. 89, pp. 1959an estimated vector. The result of Feng and Sato method is
1972, 2009.
-1.96 dB, Gurbuz method -0.119 dB, and the results of the
proposed method with the smooth prior and the smooth-gap [3] A. Kanemura, S. Maeda and S. Ishii: Super-resolution
prior are 5.96 dB and 6.176 dB, respectively. Then we obwith smooth-gap prior, IEICE Technical Report. (NC),serve that the proposed method signiﬁcantly outperformed
Vol. 105, No. 130, pp. 31-36, 2005.
the method of Feng and Sato, and also, the smooth-gap
prior which can represent gap information, improved per- [4] S. Kozawa, T. Takenouchi, I. Kazushi: Subsurface
formance compared to the smooth prior.
imaging by Bayesian super-resolution for anti-personalmine detection using ground penetrating radar, Journal
4. Conclusionof Signal Processing, Vol. 14, No. 4, 2010.

©ISAROB 2011

900

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Construction of a sense of force feedback and vision for micro-objects:
Development the Haptic Device's controlling

J. NodaE. Hayashi

Graduate School of Computer Scienceand System Engineering
Kyushu Institute of Technology
680-4, Iizuka City, Fukuoka, Japan, 820-8502
noda415@mmcs.mse.kyutech.ac.jp

Department of Mechanical Information
Science and Technology
Kyushu Institute of Technology
680-4, Iizuka City, Fukuoka, Japan, 820-8502
haya@mse.kyutech.ac.jp

Abstract. The purpose of this research was to develop a combined sense system that uses both force feedback andvisual feedback to determine the shape of the microscopic features of a microsample. Efficiency in performing
minute procedures should be improved if the operator can have a sense of force while using a manipulator. We used acantilever to touch a minute object and obtain a reaction force from the degree of bending. We constructed a haptic
device that gives a sense of that force to the operator when touching the sample with a cantilever. When the haptic
device was used in simulations, the user could feel a force as if he had touched the sample.
Key words: force feedback, haptic interface, simulation.

1. Introduction

via which the user can control and operate thesecomponents. The sample was fixed on the x-y stage by
an injector (Fig. 4) and a holding pipette (Fig. 5). Whenthe cantilever, which was fixed to the piezo stage,
touched the sample, the operator could maintain the
cantilever’s position by obtaining the value of thereaction force through the interface. The resolution of thepiezo stage is 1 nm. Table 1 gives the specifications of
the injector, and Table 2 gives the specifications of theholding pipette.

Technologies that can accurately perform minute work
are now being sought for both medical treatment and in
the field of manufacturing semiconductors. Such minutework is improved by using micromanipulators, but theiroperation is difficult because the operator has no sense offorce; he or she relies only on sight through a microscope.
As a result, a person skilled in the use of this technology
is needed for all minute work. The efficiency of minutework would be improved if the operator were able tohave a sense of force while using a manipulator.
Here we describe the development of a more efficientsystem for minute operations. Our aim was to develop asystem using not only the sense of sight through a
microscope but also a sense of force from the
manipulator. For this fundamental research, a system wascreated to assess the reaction force when a minutesample was touched. A cantilever was used to touch thesample, and the reaction force was obtained from thedegree to which the sample bent. In addition, we used ahaptic device and amplified the force feedback from aminute sample of a virtual object.

Fig. 1a. Photograph of the structure of the system.

2. System Structure
2-1. System summary
The structure of the system is shown in Fig. 1a, and aschematic view is shown in Fig. 1b. This system consistsof a microscope with an automatic x-y stage, a piezo
stage, a feedback stage controller to control the x-y stage,a piezo stage controller, a haptic device for transmitting
force feedback (Fig. 2), a cantilever (Fig. 3), and a PC

©ISAROB 2011

Fig. 1b. A schematic view of the system’s structure.

901

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

The actuator is controlled by a servomechanism on theactuator. Therefore, the system driving the actuator consistedof four actuators: a microcomputer, an inputting AD/DA port,
an outputting microcomputer, and a PC outputting ordervalue. The system controls the actuator during each part ofthe process. Figure 7 shows the structure of the haptic device.
Fig. 2. Haptic device.

Fig. 3. Cantilever.

Fig. 7. Structure of the haptic device.
Fig. 4. Injector.

Fig. 5. Tip of holding pipette.

The actuator, whose actions are governed by the PD
control, is operated through a digital differential calculus
device. A transfer function for the quadratic function system
shown in Fig. 8 is provided for the actuator servo system. The
role of each parameter of the control system is to adjust the
total offset to a master in Gi/Gif, to regulate the item
viscosity/resonance point in Gp/Gv, and to regulate the total
gain in Gm. Table 3 is a list of the control parameters of theservomechanism system.

Table 1. Specifics of the injector.
Drive distance

40 mmBraking distance

800 μ||

Size

200×80×30 mm

Weight2 kg

Table 2. Specifics of the holding pipette.
Length

60 mm

Inside diameter

13 μm
Fig. 8. Block diagram of the servomechanism system.

2-2. Haptic device

Table 3 Control parameters of the servomechanism system

Figure 6 is a diagram of the haptic device. It consists
primarily of a rotor, a laser, and a position-sensitive device(PSD). We installed a coil on the rotor with a polarity magnet,
which generated electromagnetic induction by an electric
current and a magnetic force. The angle of the rotor can bemeasured by the laser and the PSD. The rotor was able tofollow any input waveform.

3. Measuring the reaction force
The reaction force is used to calculate the force that is
applied by the minute object. In this experiment, we touched
the minute object with the cantilever shown in Fig. 3, and thereaction force was obtained from the degree of bend of thecantilever. The layout of the experiment is shown in Figs. 7
and 8, and the environment of the experiment is shown in

Fig. 6. Diagram of the haptic device.

©ISAROB 2011902

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

F = kxFigs. 9 and 10. As a result of this experiment, we obtained thereaction force applied by the minute object.

(1)

where x is the compression distance from the equilibrium
position, and k is the spring constant.

4. Deforming the sample in simulation
In this study, we aimed to build a working system using a
microscope, a haptic device, and a simulation. A fundamental
element was the simulation of the deformation of a minuteobject. Figure 13 shows the graphical user interface (GUI) of
the simulator. A graphic tool is created using OpenGL todraw the object and to choose the shape of the sample, forinstance, a cube or sphere. A dynamic model of the sample
consists of a spring-mass array of mass points in both thevertical and horizontal directions. An example of thearrangement of mass points is shown in Fig. 14. When aforce was applied at a mass point, the simulation calculated
the speed of all mass points that had been affected. The image
is renewed after every ten calculations.
We defined a spring as having a size but no weight, and amass point as having a size, a weight, and a rigid body. An
arbitrary mass object can be placed on a spring on a bitmap(Fig. 15). In addition, a sample can be seen from various
viewpoints, and the deformation of the sample, which is
impossible to observe by microscope, can be checked. The
shape of this object can be either a cube or a sphere, and any
point may be selected as a fixed point or an operating point.

Fig. 9. Environment of the experiment.

Fig. 10. Close-up of the environment of the experiment.

Fig. 11. Cantilever touching the tip of the holding pipette.

Fig. 13. Simulator.

Fig. 14. Arrangement ofmass points

Fig. 12. The cantilever detection program.
Figure 11 shows the cantilever touching the tip of theholding pipette. Figure 12 shows the experiment thatmeasures the reaction force of the downy hair. The image
processing speed of the cantilever was improved by the
tracking process. The bend of the cantilever is assumed to belinear-elastic so that Hooke’s law may be applied. The
restoring force, F, of the bend of the cantilever is given by©ISAROB 2011

Fig. 15. Placing an arbitrary object on a bitmap.

The calculation method for the displacement of each masspoint is based on Newton’s equation of motion (Eq. 1) using

903

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

6. Acknowledgement

the Euler method. A mass point is linked to an adjacent masspoint at both ends of a spring. When the spring between themass points is in the equilibrium position, the restoring force,
F, is the sum of the elastic force of the construction spring andthe shear spring, and the viscous force is given by

ma = ΣF

This research was partially supported by the Ministry of
Education, Science, Sports and Culture, Grant-in-Aid for
Scientific Research, 2010.

References(2)
1.

where m is mass and a is acceleration.
Pi-1j

2.

Pij-1
Fij-1
Pij

Fi-1j

Fij+1
Fi+1j

Pij+1

3.
Pi+1j

Fig. 16. Elasticity.
4.

ΣF is the sum of the elastic force of the construction
spring and shear spring, the viscous force, and the damping
force. To obtain F , we divided the restoring force into
F damping, F spring, and F viscous. Equation 2 gives thefollowing equations using the model shown in Fig. 16:

Fspring = ΣFij

(3)

Fdamping = −C d vi

(4)

Fviscous = −C v (vi − v j )(5)ΣF = Fspring + Fdamping + Fviscous

(6)

5.

6.

where C d is the damping factor, C v is viscosity, and v iand v j are the velocities.
7.

5. Conclusion
In the present study we evaluated whether the force
feedback could be amplified by touching a virtual object witha haptic device in a PC. We found that it is possible toamplify the reaction force, but we were unable to create alarge enough reaction force for a worker to feel its elasticity.
Future research should focus on building a system thatallows a reaction force to be detected and shown more
precisely. Such a system would make it possible to test
smaller samples.

©ISAROB 2011

8.

9.

904

Li X, Zhang Z (2008) Probe-type microforce sensor for
micro/nano experimental mechanics. Trans Tech
Publications, Stafa-Zurich, Switzerland
Zhang J, Xi N, Li G, et al. (2006) Atomic force
microscopy sensing using multiple modes. 2006
IEEE/RSJ International Conference on Intelligent
Robots and Systems, Beijing, China, 9-15 October.
IEEE, Piscataway, NJ
Hrouzek M (2005) Feedback control in an atomic force
microscope used as a nano-manipulator. Acta Polytech
Czech Tech Univ Prague 45(4):65-69
Ujihara R (2004) Developing the haptic device. Kyushu
Institute of Technology
Yaacoub F, Hamam Y, Abche A (2008) Computer-based
training system for simulating wrist arthroscopy.
Université Paris-Est, ESIEE, Laboratoire A²SI, Cité
Descartes, 93162 Noisy Le Grand, France. University ofBalamand, Department of Electrical Engineering,
P.O.Box 100, Tripoli Lebanon F’Satie – TUT, Private
Bag X680, Pretoria 0001, South Africa. Proceedings ofthe 21st IEEE International Symposium on
Computer-Based Medical Systems (CBMS 2008)
Tahmasebi AM, Taati B, Mobasser F, et al. (2005)
Dynamic parameter identification and analysis of a
PHANToM trade haptic device. School of Computing,
Queen’s University, Kingston, ON, Canada. Department
of Electrical and Computer Engineering, Queen’s
University, Kingston, ON, Canada. 2005 IEEE
International Conference on Control Applications(CCA) (IEEE Cat. No. 05CH37713)
Sonoda T (2006) Construction of super-micro sense offorce feedback and visual for an elastic body. Kyushu
Institute of Technology. Proceedings of the 12th
International Symposium on Artificial Life and Robotics,
Beppu, Oita, Japan, 2006
Miyamoto K (2007) Construction of super-micro senseof force feedback and visual for micro objects. Kyushu
Institute of Technology. Proceedings of the 12th
International Symposium on Artificial Life and Robotics,
Beppu, Oita, Japan, 2007
Uehara R (2010) Construction of super-micro sense offorce feedback and visual for micro objects. Kyushu
Institute of Technology. Proceedings of the 15th
International Symposium on Artificial Life and Robotics,
Beppu, Oita, Japan, 2010

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Trajectory Control of Biomimetic Robots for Demonstrating
Human Arm Movements
T. Kashima and M. Iwaseya
Tomakomai National College of Technology, Aza Nishikioka 443, Tomakomai, 059­1275, Japan(Tel/Fax : 81­144­67­8013)(kashima@me.tomakomai­ct.ac.jp)

Abstract: This study presents the trajectory control of biomimetic robots by developing human arm trajectory planning.
First, the minimum jerk trajectory of joint angles is analytically produced, and the trajectory of elbow joint angle ismodified by time adjustment of joint motion of the elbow relative to the shoulder. Next, experiments in which gyro
sensors are utilized have been conducted, and produced trajectories are compared with observed ones. According to theresults, the validity of this proposed trajectory control for demonstrating human arm movements has been evaluated.
Keywords: biomimetic robot, human arm movement, trajectory planning, gyro sensor

remains a crucial problem that a hand trajectory can not
be produced unless the human trajectory to be
duplicated is given.
This study presents the trajectory control ofbiomimetic robots. At the beginning, the minimum
jerk trajectory of joint angles is analytically produced,
and the trajectory of elbow joint angle is modified bytime adjustment of joint motion of the elbow relative tothe shoulder. As regards time adjustment, a newly
provided case in addition to a case reported in a past
literature[5] is taken into account. This time of elbow
joint motion to be adjusted is numerically determined so
as to reach the maximum hand velocity at the midpointof movement. Consequently, the hand trajectory derived
from the joint angle trajectories can be uniquely
produced once the initial and target positions andmovement duration are given. Subsequently,
experiments have been conducted in which gyro sensors
are utilized for measuring angular velocities with highaccuracy and high resolution. Then, trajectories are
numerically produced by use of the movement condition
observed by experiments, and they are compared withobserved ones in joint angle coordinates and Cartesian
coordinates. Finally, it is shown that the proposed
trajectory control is reasonable for biomimetic robots todemonstrating human arm trajectories.

I. INTRODUCTION
It is an important issue for biomimetic robots not
only to design appearance resembling a human arm but
also to move its arm along humanlike trajectories. An
effective method for controlling biomimetic robots
along such trajectories is to apply human arm trajectory
planning. Here, the criteria such as joint torquechange[1] and consumed energy[2] are proposed forcharacterizing human arm trajectory. The optimal
trajectories minimizing these criteria have good
agreements with human arm trajectories if movement
duration and arm parameters are properly set up. This
optimal trajectory is sensitively influenced by thechange of movement duration or external load. In
contrast, the hand path and velocity profile of humanarm trajectory is kept invariant when the movement
duration or the external load is changed[1]. In addition,
numerical calculations for producing optimal
trajectories become extremely difficult to converge
under specific movement conditions. Consequently,
such criteria have difficulty with these problems for the
trajectory planning of biomimetic robots.
Meanwhile, it is possible to formulate humanlike
trajectories which satisfy with the invariant property, if
we assume the trajectory planning geometricallydetermined. Here, the criteria defined by derivative of
the hand position including jerk in Cartesian coordinateshave been proposed[4]. The produced hand velocity
profile is always bell­shaped and shows property ofhuman arm trajectory. However, curved hand paths
occasionally observed in human arm movements can
not be demonstrated, since the produced path isconsistently straight. Such curved paths can be
represented by supposing a linear relationship between
the shoulder and the elbow angles in joint angle
coordinates. Furthermore, this idea leads to an important
fact that most of human hand paths can be duplicated byadequately setting a time delay of joint motion onset ofthe elbow relative to the shoulder[5]. However, there©ISAROB 2011

II. METHODS
A. Trajectory formation
Human trajectories sometimes show a linear
relationship between a shoulder and an elbow angles injoint angle coordinates[3],[5]. With respect to thisrelationship, the trajectory planning, so called“staggered interpolation” has been proposed. According
to this method, most of hand paths can be reproduced if
a linear relationship between these two angles is
initially supposed, and a time delay of joint motion
onset of the elbow relative to the shoulder isappropriately provided. However, this time delay can

905

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

joint motion onset of the elbow relative to the shoulder
is delayed, and only this case is considered in a past
study[5]. We newly set the second case that the elbow
joint motion stops before the shoulder joint motion does.
In the first case, we suppose the elbow joint motion
delayed time d with respect to normalized time s. Then,the elbow joint angle using Eq. (4) can be described by
q 2 ( s ) = q 2 s + (q 2 s - q 2 f )(15 p 4 - 6 p 5 - 10 p 3 ) (7)
where
0£s<d
ì0,
p=í
(8)
î( s - d ) /(1 - d ), d £ s £ 1
In the second case, when the elbow joint stops time dearly before the shoulder joint does, p in Eq. (7) isìs /(1 - d ), 0 £ s < 1 - d
P=í(9)
1- d £ s £1
î0,When the shoulder joint angle of Eq. (3) and the elbow
joint angle of Eq. (7) are substituted into Eq. (5), thehand velocity becomes function of s and d. Then, bydifferentiating it, we haved ( x& 2 + y& 2 )
dv
1
=
(10)
ds 2 x& 2 + y& 2
ds

not be determined unless the human trajectory to be
duplicated is given. Besides, the hand velocity profile,in some movement conditions, becomes distorted bell­
shape. In our study, these problems are overcome by
newly developed trajectory planning which reflects
properties of human arm trajectory in Cartesian
coordinates.
At the beginning, the performance criterion of aminimum jerk in a joint angle space is defines as
1 tf
J = ò (q&&&1 (t ) 2 + q&&&2 (t ) 2 ) dt ,(1)
2 0
where θ1(t) and θ2(t) are shoulder and elbow angles, andtf is a movement duration. When the initial condition of
movement is given byìïq i ( 0) = q is , q&i (0) = 0, q&&i (0) = 0
i = 1,2 ,(2)íïîq i (t f ) = q if , q&i (t f ) = 0, q&&i (t f ) = 0
the optimal trajectory becomesq i ( s ) = q is + (q is - q if )(15s 4 - 6 s 5 - 10 s 3 ), i = 1, 2 , (3)where s= t/ tf is a normalized time. The trajectory given
by Eq. (3) shows a perfect linear relationship between
two angles. However, the hand velocity profile in
Cartesian coordinates does not necessarily show the
maximum velocity at the midpoint of movement. In
other words, it does not, in a precise sense, implement
the property of human trajectory. Therefore, we need toprovide time adjustment which makes the hand velocity
profile reach the maximum at the halfway point.
A robot arm model is shown in Fig. 1. Here, thehand position is described byì x = l1 cos q 1 + l 2 cos(q 1 + q 2 )(4)íî y = l1 sin q 1 + l 2 sin (q 1 + q 2 )By differentiating Eq. (4), the relationship of velocity
between Cartesian and joint angle coordinates becomesìï x& = -l1q&1 sin q 1 - l 2 q&1 + q&2 sin (q 1 + q 2 )(5)íïî y& = l1q&1 cos q 1 + l 2 q&1 + q&2 cos (q 1 + q 2 )Accordingly, the tangential velocity of hand is((

Accordingly, a condition that the maximum velocity
exists at the midpoint of movement can be written as((+ 2l1l2 [(2q&1q&&1 + q&&1q&2 + q&1q&&2 ) cos q 2- q& q& q& + q& sin q }]]
=0
1 2

(

1

2

)

2

)(

)
(11)

s = 0 .5

Since the time difference is uniquely decided by Eq.
(11), the arm trajectory is perfectly determined once
initial and target positions are given. In this study, we
initially set as d=0, and time d satisfying with Eq. (11)has been searched by Newton­Raphson method.

))

B. Experiment
The important feature of human arm trajectory is the
dynamic behavior of joint angular velocity as well as
the hand trajectory in Cartesian coordinates. Therefore,
the joint angular velocity has to be measured with ahigh degree of accuracy since a time difference of joint
motion of the elbow relative to the shoulder is essential.
One of the most effective methods for measuring
angular velocity is to utilize gyro sensors[6]. In thisstudy, two gyro sensors are used on the upper arm andv = x& 2 + y& 2.
(6)In general, this velocity does not reach the maximum at
the halfway point. Hence, a time adjustment of elbow
has to be taken into account. The first is the case thaty)d x& 2 + y& 2
= [2l12q&1q&&1 + 2l22 q&1 + q&2 q&&1 + q&&2
ds
s = 0.5

Hand

l2
θ2

Gyro sensor

Elbow
Shoulderθ1xl1(a) Arrangement(b) Circuit board
Fig. 2 Gyro sensor

Fig. 1. Robot arm model©ISAROB 2011906

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

the forearm of a subject, and this gyro sensor ismounted in a circuit board as shown in Fig. 2. As
regards movement conditions, several sets of initial andtarget hand positions are provided for observing basic
properties of human arm trajectories. The trajectory ismeasured for movements in which a subject moves ahand from the initial to the target hand positions, andthis is named direct movement. Then, the initial andtarget hand positions are exchanged, and the trajectory
is also observed for movements which is called inverse
movement. Besides, the detail of experimental
procedure and data processing is described in a
literature[7]velocity in some movements and the phenomenon ofzero velocity crossing. However, these dissimilarities
are not serious drawbacks for representing human arm
trajectory because fluctuation within a small range ofangular velocity does not seriously influence thevelocity profile and the hand path.
With respect to an angular velocity, the elbow joint
motion in direct and inverse movements for a pair ofhand positions exhibits an interesting feature. Fig.3
indicates that the elbow joint motion starts after theshoulder joint motion does. Conversely, Fig. 5 for ininverse movement shows an opposite result that the
elbow joint motion stops before the shoulder joint
motion does. The same phenomenon can be seen in
direct and inverse movements of condition2. This timedifference between the elbow and the shoulder joint
motions takes an important role of human arm trajectory
planning.
The hand trajectory is one of the most important
issues for biomimetic robots demonstrating human arm
movements. Besides, it can be represented by the pathand the velocity profile of a hand. As can be seen from
Fig. 3 to Fig. 6, observed trajectories show such typical
properties that the hand path is either straight or simply
curved, and the hand velocity profile is bell­shaped withthe maximum value in the vicinity of the halfway point.
As for produced trajectories, the hand path of direct
movement of condition 1 is slightly different from thepath of observed trajectory. However, judging from theexperimental results conducted under plural subjects[9],a variation of this degree can be considered within arange of individual difference. In all movements otherthan that, the hand path nearly overlapped in theIII. RESULTS AND DISCUSSION
The hand path in human arm movements is either
almost straight or simply curved[1]­[3]. So, thesetypical trajectories of direct movement are shown in
Figs. 3 and 4. Results of inverse movement to eachdirect movement are also shown in Figs. 5 and 6. In theexperiment, a subject is instructed to move his hand
without concerning himself fine accuracy of the target
point. As a result, a pair of hand positions of direct
movement is not, in a precise sense, the same as a pairof hand positions of inverse movement.
As regards angular velocity, the observed trajectorybasically shows bell­shaped profile on motion in both
direct and inverse movements. Besides, most of theangular velocities overshoot near the termination time.
This phenomenon is called “zero velocity crossing” asintroduced in Ref. [8]. The angular velocity profile ofproduced trajectory well agrees with that of observed
one in all movements except oscillation of angular
Observed trajectory
Produced trajentory
4

Observed trajectory
Produced trajentory

Stop
Shoulder
4

rad/s

3

3

2

2

1
0
-1

0

0.5

1

Stop

0
-1 0
Start

(a) Shoulder angular velocity

Start

rad/s

1

sec
s
1.5

Shoulder

0.5

sec
s
1
1.5

(c) Hand path(a) Shoulder angular velocity(c) Hand path
4

3

rad/s

3

2

m/s

1

sec

0
-1 0

0.5

1

1.5

(b) Elbow angular velocity

-2

1
sec
0
0

0.5

1

1.5

-4

0.5

1

1.5

(d) hand velocity

1
0.5

s

-6

sec
0
0

(b) Elbow angular velocity

Fig. 3 Direct movement of condition 1©ISAROB 2011

0

m/s

sec

0

2

2

1.5

rad/s

0.5

1

(d) hand velocity

Fig. 4 Direct movement of condition 2

907

1.5

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Observed trajectory
Produced trajentory1

Observed trajectory
Produced trajentory

Start
Shoulder
1

rad/s

0
-1

Start

Shoulder Stop

rad/s

0
0

0.5

1

1.5
-1
sec
s

-2

0

0.5

1

sec
s

-2

-3

1.5

(c) Hand path

-3

Stop

(a) Shoulder angular velocity(a) Shoulder angular velocity(c) Hand path
1

3

rad/s

0
-1

0

0.5

1

1.5
sec

-2

6

m/s

4

2

sec

sec
0

(b) Elbow angular velocity

0.5

1

0

1.5

-2

(d) hand velocity

0

0.5

1

s
1.5

0.5
sec
0
0

(b) Elbow angular velocity

Fig. 5 Inverse movement of condition 1

0.5

1

1.5

(d) hand velocity

Fig. 6 Inverse movement of condition 2

produced and the observed trajectories. In addition tothis consistency of the hand path, the hand velocity
profile of produced trajectories is identical to that ofobserved trajectories in all movements. These results
lead to the conclusion that trajectories produced
according to the proposed trajectory planning welldemonstrate human arm property.

REFERENCES[1] Uno Y, Kawato M, Suzuki R (1989), Formation andControl of Optimal Trajectory in Human Multi­Joint
Arm Movement. Biol.Cybern.,61:89­101[2] Alexander R (1997), A minimum energy costhypothesis for human arm Trajectories. Biol. Cybern.
76:97­105[3] Atkeson C and Hollerbach J (1985), Kinematic
Features of Unrestrained Vertical Arm Movements. J. of
Neurosci., 5­9: 2318­2330[4] Richardson M and Flash T (2002), Comparing
Smooth Arm Movements with the Two­Thirds Power
Law and the Related Segmented­Control Hypothesis. J.
of Neurosci., 22­18: 8201­8211[5] Hollerbach J and Atkeson C (1987), Deducing
Planning
VariablesfromExperimental
Arm
Trajectories : Pitfalls and Possibilities. Biol. Cybern.,
56: 279­292[6] Zhou H and Hu H (2007), Internal sensors formotion detection of human upper limbs. Sensor review,27­2: 151­158[7] Sasada D, Kashima T, Iwaseya M (2008), Analysis
of Arm Trajectory Based on Measurements Utilizing
Inertial Sensors, Third Asia Inter. Sympo. on
Mechatronics, 399­402[8] Plamondon R, Feng C and Woch A(2003), A
kinematic theory of rapid human movement, Part Ⅳ: aformal mathematical proof and new insights. Biol.
Cybern., 89: 126­138[9] Kashima T, Isurugi Y (1998), Trajectory formation
based on physiological characteristics of skeletal muscle,
Biol Cybern., 78: 413­422

IV. CONCLUSION
In this study, the trajectory planning for biomimetic
robot has been conducted in kinematic coordinates, andproduced trajectories have been compared withexperimental results. Consequently, it is confirmed thatthe produced trajectories well demonstrate theproperties of human arm trajectories. According to thistrajectory planning, the trajectory can be uniquely
produced if initial and target positions and movement
duration are given. Furthermore, it always represents theproperties of human arm trajectory even though
movement duration or external load is changed. In
addition to these advantages, the numerical calculation
for producing trajectories is relatively simple. Thesefeatures show that the proposed trajectory planning iseffective for controlling biomimetic robots along
humanlike trajectories.
There are, however, some problems which remain tobe solved. The angular velocity of human arm trajectory
fluctuates near the end of movement, and zero velocity
crossing occurs especially when high angular velocity isrequired. This phenomenon can not be covered by our
trajectory planning. Further development of trajectory
planning reflecting such detail property of human
trajectories is the issue to be solved in the future.

©ISAROB 2011

m/s

1

2

1
0

-3

1.5

rad/s

908

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A Study on Reinforcement Learning Scheme in Cooperative Network-Systemsof Sensor Nodes and Mobile Robots
Koichi Hirayama†, Akihide Utani‡, and Hisao Yamamoto‡†Graduate School of Engineering, Tokyo City University‡Faculty of Knowledge Engineering, Tokyo City University
1-28-1, Tamazutsumi, Setagaya, Tokyo 158-8557, Japan{autani; yamahisa}@tcu.ac.jp
Abstract: Recently, there is growing expectation for a new network service by a wireless sensor network consisting of many sensor
nodes placed in an object area and not a few mobile robots as a result of the strong desire for the development of systems that function flexibly in dramatically changing environments. This study proposes a reinforcement learning scheme in a cooperative networksystem of many static sensor nodes and not a few mobile robots, named the Network-Robot System (N-R System) for adaptive cooperation of many static sensor nodes and not a few mobile robots. We evaluate the proposed reinforcement learning scheme by computer simulations on a benchmark problem. In the experiments performed, the performance of the proposed scheme is investigated to verify its effectiveness.
Keywords: Reinforcement Learning, Cooperative Systems, Static Sensor Nodes, Mobile Robots.
he proposed scheme is investigated to verify its effectiveness. This paper is organized as follows. In Section II,
the proposed scheme is outlined and the experimental results on a benchmark problem are reported. Finally, thepaper closes with conclusions and ideas for further study in Section III.

I. INTRODUCTION
Various network services have been provided. They
include inter-vehicle communication, which is a network service in intelligent transport systems, natural environmental monitoring, and emergent communication between mobile nodes in such the case of emergency as disaster. As a means of realizing the above network services, autonomous decentralized networks, such as a mobile ad-hoc network and a wireless sensor network, havebeen intensively studied with great interests. Especially,a wireless sensor network, which is a key network to facilitate ubiquitous information environments, has great
potential as a means of realizing a wide range of applications, such as natural environmental monitoring, environmental control in office buildings and factories, object tracking, and precision agriculture [1]. Recently, there is growing expectation for a new network service by
a wireless sensor network consisting of many sensor nodes placed in an object area and not a few mobile robots
as a result of the strong desire for the development of systems that function flexibly in dramatically changing
environments.
This study proposes a reinforcement learning scheme in a cooperative network-system of many static sensor nodes and not a few mobile robots, named the Network-Robot System (N-R System) for adaptive cooperation of many static sensor nodes and not a few mobile robots. We evaluate the proposed reinforcement learning
scheme by computer simulations on a benchmark problem. In the experiments performed, the performance of t-©ISAROB 2011

II. PROPOSAL AND RESULTS
In the existing studies, many reinforcement learningalgorithms have been proposed as agent-adaptive algorithms. In the existing reinforcement learning algorithms,
the agent is reinforced (trained) using only rewards from the environment. Reinforcement learning algorithms
can be classified into two approaches. One is “exploitation-oriented” approach, and the other is “exploration-oriented” approach. This study uses the “exploitation-oriented” approach. In the proposed N-R System, many static sensor nodes limited resource, which are compact and inexpensive, are placed in an object area. Each sensor node consists of a sensing function to measure the status of an object, a limited function on information processing, and a simplified wireless communication function, and operates on a resource of a limited power-supply capacity such as a battery. Not a few mobile robots have learning function and learn adaptive actions basedon information obtained from static sensor nodes placedin an object area.
The proposed N-R System, which is a cooperative network-system of many static sensor nodes and not a few mobile robots, consists of the following main modules: Perception Module, Learning Module.

909

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011a subgoal. In case that the warehouse functions as a subgoal, the profit is shared with each mobile robot by the
profit sharing plan [2,3] when it reaches the warehouse.
Fig.4 and Fig.5 show the experimental results on Pattern
4 and Pattern 5. From these results, it can be confirmedthat the subgoal effectively functions.

1. Perception Module
In the perception module, the sensor node perceives the task (request) and disseminates the task (request)
information and the route information to the task point to all nodes that include mobile robots.
2. Learning Module
In the learning module, each mobile robot learns adaptive actions based on information obtained from static sensor nodes placed in an object area. As the learning scheme, the profit sharing plan is used [2,3].
Through simulation experiments on the delivery problem by mobile robots in the plant shown in Fig.1, theperformance of the proposed N-R System is investigated to verify its effectiveness. The N-R System consisting of twenty-five static sensor nodes arranged in the object plant and two mobile robots is assumed. The conditions of simulation, which were used in the experiments
performed, are shown in Table1. In the initial stage, each mobile robot has fifty requirements.

Table2. Experimental patterns
The total number of requirements

DirectPattern1

0〜50

2

Pattern2

51〜1002

Pattern3

51〜100

1

Pattern4

101〜150

1

Pattern5

101〜1500Pattern6

151〜

0

250
Pattern1
The number of steps to Goal

Pattern2
R

R

R

Ware House
Sensor Node
Robot
Goal

200

Pattern3
Pattern4
Pattern5

150

100

50

0
1

100

200
300
The number of episodes

400

500

Fig.2. Learning curves

Fig.1. Simulation model
250

Pattern1

Table1. Conditions of simulation

Pattern2

50m × 50m

The number of sensor nodes

5× 5

The number of mobile robot

2

The number of warehouses

1

The number of requirement points

2

The number of episodes

500

Reward

6.0 × 105200
The number of steps to Goal

Simulation size

Pattern4
Pattern5

150

Pattern6
100

50

0

Experimental results on each pattern shown in Table2 are illustrated in Fig.2 and Fig.3. Fig.3 shows the learning curves of the case that the warehouse functions as©ISAROB 2011

Pattern3

1

100

200
300
The number of episodes

400

Fig.3. Learning curves (Subgoal is used)

910

500

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

III. CONCLUSIONSThe number of steps to Goal

160
subgoal is usedsubgoal is not used

In this paper, a reinforcement learning scheme in acooperative network-system of static sensor nodes andmobile robots, named the Network-Robot System for adaptive cooperation of static sensor nodes and mobile robots, has been proposed. In simulation experiments on
the delivery problem by mobile robots in the plant, theperformance of the proposed system was investigated toverify its effectiveness. Experimental results indicate that the proposed system has the development potential tofacilitate ubiquitous information environments. In futurestudies, we evaluate the performance of the proposed system in detail.

120

80

40

0
1

100

200
300
The number of episodes

400

500

Fig.4. Learning curves on Pattern 4

The number of steps to Goal

200

REFERENCES

subgoal is usedsubgoal is not used[1]I. Akyildiz, W. Su, Y. Sankarasubramaniam, and E.
Cayirci, “Wireless sensor networks: A survey,” Computer Networks Journal, vol.38, no.4, pp.393-422, 2002.
[2]K. Miyazaki, M. Yamanoto, and S. Kobayashi, “A theory of profit sharing in reinforcement learning,” Trans.
of the Japanese Society for Artificial Intelligence, vol.9,
no.1, pp.104-111, 1994.
[3]K. Miyazaki, H. Kimura, and S. Kobayashi, “Theory
and applications of reinforcement learning based on profit sharing,” Trans. of the Japanese Society for Artificial Intelligence, vol.14, no.5, pp.800-807, 1999.

150

100

50

0
1

100

200
300
The number of episodes

400

500

Fig.5. Learning curves on Pattern 5©ISAROB 2011911

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Towards the automation of cashew shelling operation
Son Doan Tran1, Naoki Uchiyama2, Suguru Kirita2, Norifumi Yamanaka2, Phat Minh Ho1
1

Ho Chi Minh University of Technology; 268 Ly Thuong Kiet Street, Dis. 10 HoChiMinhCity Vietnam(Tel : 84-90-382-5695; Fax : 84-83-866-0244 ;Email: Tdsonbk@Yahoo.com )2

Toyohashi University of Technology; 1-1 Hibarigaoka, Tempaku, Toyohashi, Aichi 441-8580, Japan

Abstract: Vietnam is one of the world's largest countries in terms of exporting cashew nuts in kernel form. Gross export
value in this field was $750 million in 2009. Domestic production of raw cashews is about 400,000 tons and they arealso imported from abroad about 100,000 tons. Hence, a large number of labors work for cashew processing industry.
Although most cashew processing operations have been automated, the shelling operation is still conducted manually.
Hence, not only shelling rate is very low compared to other operations, but also labor conditions are strenuous andunsafe. Currently, more than 200,000 labors engage in cashew shelling operations in Vietnam. This paper presents a
robotic system design for cashew shelling operation to improve working conditions and production efficiency. The
proposed design includes the milling operation of cashew shell and splitting process of the milled shell by wedge. In
addition, the conveyer system to transport cashews is also designed for increasing the production efficiency. For
designing the system, we consider the following three criteria to evaluate the efficiency [1]: (1) shelling rate (kg/h):weight of cashew kernels obtained per unit time, (2) shelling efficiency (%): ratio of shelled cashews over total cashews
handled, and (3) whole-kernel recovery (%): ratio of whole kernels obtained over whole and broken kernels. The
proposed system achieves 300kg/h, 95% and 80 %, respectively.
Keywords: cashew shelling; milling operation; splitting process; wedge; plate disk

I. INTRODUCTION
Vietnam is one of the world's largest countries interms of exporting cashew nuts in kernel form.
Although most cashew processing operations have been
automated, the shelling operation is still manuallyconducted. Hence, not only shelling rate is very low
compared to other operations, but also labor conditions
are strenuous and unsafe. Currently, more than 200,000
labors engage in cashew shelling operations in Vietnam.
There are mainly two methods for shelling cashews
as follows:• Cut the shell using form-milling cutter• Break the shell using hammer-like tool or split itby wedge-type tool
The first method is applied in countries such as
Vietnam, Indonesia, Thailand, India and some Africancountries. Cutters with two thin blades whose profile isbased on the cashew shape are used for shelling the
cashew (Fig. 1 and 2). Cashews have to be previouslyclassified by size in this approach. Cashew nuts aremanually placed between two blades. These blades areconnected with two foot pedals by links. Workers pedal
them to operate the blades. They use one pedal to cut
the cashew and the other pedal to rotate one blade by 90
degree to remove the shell from the cashew kernel.

©ISAROB 2011

Fig. 1 Cashew shell cutter

Fig.2 Manual cashew shelling

912

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

After this operation, workers also use a small knife totake the kernel out of the shell. This method has
advantages and disadvantages as follows:
Advantages:• Simple and inexpensive equipment• Easy operation
Disadvantages:• Low shelling rate (approximately 40-60 kg / shift)• Hard and unsafe working conditions• The whole-kernel recovery depends on worker’s
skill
Some countries employ hammer-like tools forbreaking the cashew shell. Workers break the cashew
shell and take the kernel out of the shell manually.
Although this method is simpler than the previous
method, the whole-kernel recovery is very low and it
largely depends on the worker’s ability.
Although wedge-like tools are also used forautomatic split of the cashew shell, shelling efficiency
and whole-kernel recovery are not sufficient.
This paper presents a robotic system design for
cashew shelling operation to improve working
conditions and production efficiency. The proposed
design includes the milling operation of cashew shelland splitting process of the milled shell by wedge. In
addition, the conveyer system to transport cashews is
also designed for increasing the production efficiency.

Fig.3 Cashew shelling machine structure
2. Shell cutting system
Figure 4 shows a schematic of the cashew shell
cutting system, which includes disc milling cuttersrotating with an appropriate speed.

II. SHELLING MACHINE DESIGN
1. Specifications and machine structure
The following specifications are required forautomatic shelling machine:• shelling rate (kg/h): 2000 kg / shift for each system.
• shelling efficiency (%): > 70%• whole-kernel recovery (%) > 90%
Figure 3 shows a schematic of the designed cashew
shelling machine. This machine consists of thefollowing five parts: (1) vibratory cashew feeder, (2)conveyer belt for transporting the cashew, (3) rollers toclamp the cashew tightly, (4) cashew milling cutter, and(5) cashew shell splitter.
The vibratory cashew feeder provides cashews
between the pair of conveyor belts continuously. The
conveyer belts transport cashews into a position
between a pair of disk cutters. After cutting, cashews are
put into a splitter that removes shells from kernels.

©ISAROB 2011

Fig.4 Cashew shell cutting system

913

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

larger groove for the separation of the shell from thekernel by wedges.) Because the shell has a wood-likematerial, we employ 9XC alloy steel for the cutter
whose hardness is 39-40 HRC, and the cutting speed is
v = 10.5 m / s.

The cutters are supported by springs. Cashews aretransferred into a position between two milling cutter
blades by conveyers.
3. Clamping system
Cashew nut is positioned between two V-shaped
conveyor belts (No. 2 in Fig. 5) and tightly clamped bysprings (No. 4) and pins. Rollers (No. 3) are employed
to reduce friction. Cashew nuts are transferred by theconveyor belt to the cutter (No. 1) position. Clamping
force of the spring must be large enough so that pins
penetrate the cashew shell and the belt settles the
cashew position. Cashew nut is kept fixed against the
cutting force.

C. Cutting conditions
Figure 6 shows forces in the cashew and cutters at
the beginning of cutting process. The following
notations are used.
R: thrust force
Fc: cutting force
Flx1, Flx2: spring force acting in the direction
perpendicular to lines AO1 and CO2, respectively.
N11, N21: forces acting from the cashew to plate
disks
N12, N22: forces acting from plate disks to the
cashew
Fms1, Fms2: frictions between plate disks and the
cashew (They have the same direction with FC).
α1 : Angle between N11 and the vertical line.
α 2 : Angle between N21 and the vertical line.
f’: Friction coefficient between the cashew andplate disks.

Fig. 5 Cashew clamping system
4. Design calculations
A. Force magnitudes
The cashew cutting force was measured byexperiment. The force is less than 5N for a 100 mm
diameter blade. The clamping force for cutting is
calculated theoretically based on the cutting force, andis estimated approximately as 60 N. The separating
force of the shell from the kernel is calculated
theoretically, and is estimated approximately as 9 N.
The clamping force for the separation is calculated
theoretically based on cutting force, and is estimatedapproximately as 18N for each side.
B. Cutting tool parameters
A cashew shell is a type of wood composed ofnatural organic compounds, plastic outer shell, spongy
middle layer, and hard and brittle inner layer. We havethe following parameter values: Number of teeth of thedisc cutter is z = 64, the cutter width is W = 2 mm (Thiscutter width has a larger value than that in [2] to create a©ISAROB 2011

Fig. 6 Forces in cashew and cutter

914

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Applying experimental parameters α1 = 21D , α 2 = 28D ,f ' = 0.25 and θ1 = θ 2 = 26 / 28D , we confirm thatconditions (7) and (8) are satisfied as follows:
For (7), we have
311 (N) > 23.7 (N) and 218.05 (N) >11.55 (N) (9)We have the following conditions for cutting the
cashew shell:⎧N11.cosα1 − f .N11.sinα1 ≥ Flx1.cosθ1⎧ N11.cos α1 − Fms1.sin α1 ≥ Flx1.cos θ1⎪⎪⇔⎨⎨⎪ N .sin α ≥ − F .cos α + F .sin θ⎪
N11.sinα1 ≥ − f / .N11.cosα1 + Flx1.sinθ1
1
ms1
1
lx1
1
⎩ 11
/

⎩

For (8), we have(1)

285.98 (N) > 23.7 (N) and 257.82 (N) > 11.55 (N)⎧N21.cosα2 − f / .N21.sin α2 ≥ Flx2 .cosθ2⎧ N 21.cos α 2 − Fms 2 .sin α 2 ≥ Flx 2 .cos θ 2⎪⎪⇔⎨⎨⎪ N .sin α ≥ − F .cos α + F .sin θ
⎪
/
2
ms 2
2
lx 2
2
⎩ 21
⎩N21.sin α2 ≥ − f .N21.cosα2 + Flx2.sinθ2(10)
5. Summary of experiment
The first prototype achieves the following
performance experimentally: (Test with 1 ton raw
cashew and 5 trials, 200Kg for each trial.)• Shelling rate (kg/h): 2200 kg / shift for eachsystem.
• Shelling efficiency (%): larger than 72%• Whole-kernel recovery (%): larger than 95%(2)
Consider the force balance in the horizontal direction,
we have
R + Fc .( cosα1 + cosα2 ) + Fms1. cos α1 + Fms1. cos α2 =N12.sinα2 + Ν22 sin α2⇔ R + Fc .( cosα1 + cosα2 ) = N12 (sinα1 − f ' . cos α1) + N22 (sinα2 − f ' . cos α2 )(3)and for vertical direction,

III. CONCLUSIONF.
c ( sinα1 − sinα2 ) + Fms1.sinα1 − Fms2.sinα2 = − N12. cosα1 + N22.cosα2⇔ Fc .( sinα1 − sinα2 ) = − N12.( cosα1 + f ' . sinα1) + N22 (cosα2 + f ' .sinα2 )(4)Lettingx = sinα1 − f ' .cos α1
y = sinα 2 − f ' .cos α 2
z = cosα1 + f ' .sinα1

and u = cosα 2 + f ' .sinα 2 ,and considering N11 = N12 and N 21 = N 22 , we obtain
N11 = [u. ⎡⎣ R + Fc . ( cosα1 + cosα 2 ) ⎤⎦ − y. Fc . ( sinα1 − sinα 2 )] / ( y.z + x.u )(5)

N21 = [ z. ⎡⎣R + Fc .( cosα1 + cosα2 ) ⎤⎦ + x. Fc .( sinα1 − sinα2 )]/ ( y.z + xu
. )

(6)

Substituting N11 and N21 in (5) and (6) into (1) and(2), we have⎧ ⎡⎣u.[ R + Fc (cos α1 + cos α2 )] − y.Fc .(sin α1 − sin α2 )⎤⎦(cos α1 − f / .sin α1 ) ≥ Flx1.cosθ1⎪( y.z + x.u)⎪⎪⎨⎪
⎪ ⎣⎡u.[ R + Fc (cos α1 + cos α2 )] − y.Fc .(sin α1 − sin α2 )⎦⎤ (sin α + f / .cos α ) ≥ F .sin θ
1
1
lx1
1
⎪⎩
( y.z + x.u)

ACNOWLEDGEMENTS
This work has been supported by the SUPREMHCMUT (Strengthening University Project for
Research-Based Education Model in Ho Chi Minh
University of Technology) with assistance provided by
the Japan International Cooperation Agency (JICA).
REFERENCES[1] T. Thivavarnvongs et al. (1995) Development ofcompact sized cashew nut shelling machinery (Part 1).
Synthesis of effective manual and semi-automatic
shelling methods. Journal of the Japanese Society ofAgricultural Machinery 57(2) 57-65.
[2] Đặng Đình Bôi (2005) Machine and equipment
processing wood, printed in Nong Lam University,
Vietnam.

(7)⎧ ⎡⎣ z.[ R + Fc (cos α1 + cos α 2 ) ] + x.Fc .(sin α1 − sin α 2 ) ⎤⎦(cos α 2 − f / .sin α 2 ) ≥ Flx 2 .cos θ 2⎪( y.z + x.u )⎪⎪⎨⎪⎪ ⎡⎣ z.[ R + Fc (cos α1 + cos α 2 ) ] + x.Fc .(sin α1 − sin α 2 ) ⎤⎦ (sin α + f / .cos α ) ≥ F .sin θ
2
2
lx 2
2
( y.z + x.u )⎩⎪(8)©ISAROB 2011

Automatic cashew shelling equipments are requiredin many countries that produce cashews in kernel form,
because shelling rate is not sufficient and labor
conditions are strenuous and unsafe in manual shelling
operations. This paper presents a shelling machine
design based on the milling of cashew shell and theconveyer system. A prototype system was developed toverify the performance of the proposed machine design.
This research has opened a new direction for the
automation of Vietnam's cashew industry.

915

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Research on the velocity variation method for precise motion transmission
Masaharu Komori, Jungchul Kang, Yukihiko Kimura and Fumi Takeoka
Department of Mechanical Engineering and Science, Kyoto University
Yoshidahonmachi Sakyo Kyoto 606-8501 Japan(Tel : 81-75-753-5858; Fax : 81-75-753-5858)(komorim@me.kyoto-u.ac.jp)
Abstract: A wide range of speed or torque is required for some types of robots. It is possible for the motors to realize
some range of speed or torque but the range is limited to some degree considering the size, weight and cost of the
motors. Therefore it is impossible to realize those requirements if a reduction device with a ratio is used. Velocity
variation devices using gears are widely used in the industrial fields. However, the motion transmission from the input
shaft to the output shaft is interrupted during the velocity ratio variation process. In order to solve this problem,
velocity ratio variation method that can transmit motion precisely in the changing process of velocity ratio is proposedin this report. The process of changing the velocity ratio in the proposed method is developed and the experimental
device that can change the velocity ratio is constructed. The experimental results show that the proposed device cantransmit motion precisely between the input shaft and the output shaft while changing velocity ratio and it is confirmedthat the proposed method is effective to realize the precise motion transmission.
Keywords: ratio variationtransmission from the input shaft to the output shaft isinterrupted during this process. Therefore it is
impossible to change the velocity ratio whiletransmitting rotation between the input shaft and theoutput shaft. In order to solve this problem, velocity
ratio variation method that can transmit motion
precisely in the changing process of velocity ratio is
proposed in this report. Design of the experimental
device of the proposed velocity variation method is
carried out. The process of changing the velocity ratio in
the proposed method is developed and the experimental
device that can change the velocity ratio is constructed.
The experimental results show that the proposed device
can transmit motion precisely between the input shaft
and the output shaft while changing velocity ratio and it
is confirmed that the proposed method is effective torealize the precise motion transmission.

I. INTRODUCTION
A variety of robots usually use some reduction
devices with high ratio because they must support thehigh load in their motion and they require high torque toaccomplish it. The reduction devices decrease thevelocity and therefore the velocities of the robots arelimited to some extent. However, a wide range of speed
or torque is required for some types of robots. Moving
robots used in the industrial fields are required to realizeboth outputting a large force when they carry loads andmoving with high velocity when they move to thedestination in order to shorten the moving period. It is
possible for the motors to realize some range of speed
or torque but the range is limited to some degree
considering the size, weight and cost of the motors.
Therefore it is impossible to realize those requirements
if the reduction device with high ratio is used. If thevelocity ratio between input and output shafts ischanged, a wide range of speed or torque can berealized even if the ability of the motor is limited in
speed or torque. Velocity variation devices using gears
are widely used in the industrial fields and there aremany research reports in terms of device control andmechanisms. Cylindrical gears such as spur gears haveadvantages in high torque capacity, precise rotationtransmission and high efficiency and that is the reason
why cylindrical gears such as spur gears are used in avariety of devices. In a geared transmission, it is needed
to change the working gear pairs to vary the velocity
ratio of the transmission. However, the motion©ISAROB 2011

II. VELOCITY VARIATION METHOD
Figure 1 shows the schematic model of the basic
structure of the velocity variation device. This device
accommodates two shafts, i.e. input shaft and output
shaft, and three gear pairs A, B and C, and threeclutches TA, TB and TC. The gear pairs A and B are
composed of typical circular gears, and gear pair C iscomposed of noncircular gears. Gears Ao, Bo and Co canbe connected to the output shaft by engaging theclutches TA, TB and TC, and then they rotate together
with the output shaft. In contrast, gears Ai, Bi and Ci are

916

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

III. EXPERIMENT

fixed to the input shaft, which rotate at the same speed
as the input shaft.
Suppose that the velocity ratio of the gear pair A isrA, and that of the gear pair B is rB. In the pitch curve ofnoncircular gear pair C, there are four sections as shownin Fig.2. The pitch curve of the noncircular gear pair C
is partly same as that of gear pair A, and partly same asthat of gear pair B. Those parts of pitch curve aresmoothly connected. In the four sections, the velocity
ratio is constant at rA, it changes to rB, it is constant at rB,
and it changes to rA respectively.
The proposed velocity ratio variation process fromrA to rB is explained. Under the condition that the clutch
TA is engaged, and the other clutches are disengaged,
the velocity ratio is rA. When the meshing of gear pair C
comes into the section corresponding to rA, the clutch TC
is engaged, and, after the engagement of clutch TC, theclutch TA is disengaged. Then the meshing of gear pair
C transits to the section corresponding to rB. In thesection corresponding to rB, the clutch TB is engaged,
and, after the engagement of clutch TB, the clutch TC isdisengaged. Through this process, the transition from rA
to rB is completed. At all steps in this process, at least
one of the clutches is engaged. Therefore, the rotationalmotion is precisely transmitted from the input shaft tothe output shaft.

The experimental device based on the proposed ratio
variation method is designed and constructed.
Experiment of the velocity ratio variation from rA to rB
is carried out. The experimental result is shown in Fig.3.
The velocity ratio is calculated from the rotary encoder
attached to the input and output shafts. The designed
value of rA is 0.8, and that of rB is 1.25. Figure 3 showsthat the velocity ratio is smoothly changed during thevarying process. In the conventional gearedtransmission, it is impossible to transmit the rotation
during the velocity ratio varying process. Thus, thevelocity ratio might become zero or negative in theconventional transmission. In contrast, the velocity ratioin Fig.3 dose not become zero or negative. This result
indicates that the proposed system solves this problem.

1.30
1.20
1.10
1.00
0.90
0.80
0.70

0.0

180.0
Rotation angle of input shaft [°]Fig.3. Experimental result when the ratio is changed

VI. CONCLUSION
In this research, the velocity variation method thatrealizes precise transmission of the rotation from the
input shaft to output shaft, is proposed. The
experimental device based on the proposed method isconstructed, and the fundamental experiment is carriedout. The result shows that the proposed device iscapable to transmit rotation during the velocity ratio
varying process.

Fig.1. Structure of the Velocity Ratio Variation Device

ACKNOWLEDGEMENT
This study was supported by Industrial Technology
Research Grant Program in 2009 from New Energy andIndustrial Technology Development Organization(NEDO) of Japan.

Fig.2. Pitch curve of gear pair C©ISAROB 2011917

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Towards the Enhancement of Biped Locomotion and Control Techniques Walking Pattern Classification
Basak Yuksel(1) (2) and Kemal Leblebicioğlu(2)(1)

Mitsubishi Electric Corporation, Advanced Technology R&D Center, Japan(Tel : 81-80-3595-0395)(basak@ieee.org)(2)

Department of Electrical & Electronics Engineering
Middle East Technical University, Ankara, Turkey(Tel : 90-312-210-2358)(kleb@metu.edu.tr)

Abstract: A new walking pattern classification method is proposed for uneven floor walking of 5-link 7 DOF bipedrobot. This method extracts the patterns as per the stance foot’s current floor position and swing foot’s transitioning
floor conditions during locomotion. When a global path composed of stairs, obstacles, etc. and certain walking
parameters, such as the speed of walking and total walking time, are given to the system, the guidance controller unit
determines footstep trajectory in terms of step patterns by using a genetic algorithm based optimization technique while
ensuring biped’s stability criterion. The demonstration of biped for different pattern classes is realized by a dynamic
simulator.
Keywords: Walking pattern classification, biped locomotion, uneven floor walking.

the stability of the system by using ZMP as a stability
measure. The generation of footstep trajectory is
realized by finding energy optimal foot step parametersof each walking step, such as step length, step speed, etc.
for a given desired global path as per the different
environmental conditions.
Implementation of optimal footstep planner isrealized by using the classified walking patterns
identified for the motion in uneven surface, as explainedin this paper in detail. Note that the biped motion in thiscontext is not necessarily periodic motion. The footstep
planning is realized to ensure that dynamic stability
measure (ZMP) is satisfied for non-periodic locomotion
on uneven surfaces, such as walking on an environment
with obstacles or stairs. After designing the optimized
footstep trajectory formed by the sequence of walking
patterns, the hip point and swing foot ankle pointreference trajectories in sagittal and frontal planes of
biped robot are obtained by using 3rd order (cubic)
spline interpolation and then, the reference joint angle
trajectory is calculated by inverse kinematics, for eachsequential step’s pattern class. In the supervisory
control unit level, PID and Computed Torque Control
methods are implemented successfully. The overallsystem, including the forward dynamics integrator, highlevel controller, trajectory planner, low level controller
and 3D biped dynamic animator is implemented in
MATLAB.

I. INTRODUCTION
Reference gait trajectory generation is one of thecritical issues and another challenging problem in thebiped locomotion and control study. The biped reference
trajectory synthesis is hard due to the fact that it
necessitates a complete comprehension and analysis of
the system characteristics. In the past 40 years,
considerable improvement has been realized in thebiped reference trajectory generation and in theliterature, many methods have been proposed. Most
common ones used are gait generation by analytical [1],[2], [3], center of gravity (COG) based [4], [5], [6], Zero
Moment Point (ZMP) based [7], [8], [9], computational
intelligence based [10], [11], [12], measured humanwalking data based [13], [14] and optimality based [15],[16] methods.
In this paper, full kinematical and dynamical modelfor 5-link 7 degrees of freedom (DOF) 3D biped robot,
derived by analytical calculations and Lagrangian
method, respectively, is used to implement and test the
proposed method. The overall system is composed of a
guidance unit (high level controller), trajectory plannerunit (reference trajectory generator), supervisory control
unit (low level controller) and biped model [17].
When a global path is given to the system, guidanceunit plans the footstep trajectory by using genetic
algorithm based optimization technique, while ensuring©ISAROB 2011918

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

II. WALKING PATTERN CLASSIFICATION
The aim of the reference trajectory generation in thispaper is to generate walking pattern classes so that thedesired gait trajectory of the biped system can begenerated similar to natural human locomotion. For thispurpose, a systematical method to generate reference
gait trajectory for a 5-link 7 DOF biped robot on
different floor conditions such as walking on evensurface, stepping over obstacles, stepping up andstepping down motions which results in a natural
human-like walking is generated. The proposed method
is a pattern classification method for the bipedlocomotion, whereas these patterns are chosen so thatthey cover stepping motions on different floor
conditions, such as even floor walking, stepping overobstacles, stepping up and stepping down motions
including staircase walking.
The pattern classes are extracted as per the biped’s
current and next steps’ floor conditions in sagittal plane.
In other words, the patterns are designed depending on
the stance foot’s current floor position and swing foot’s
transitioning floor conditions. The pattern classification
composes of 9 patterns as shown from Fig. 1 to Fig. 9 indetail. Note that stepping over an obstacle directly
corresponds to Pattern 1 with proper step heightadjustment.

Fig. 3. Pattern 3: Stair Up - Second Step

Fig. 4. Pattern 4: Stair Up - Second Periodic Step

Fig. 5. Pattern 5: Stair Down - First Step

Fig. 1. Pattern 1: Walking on Even Floor

Fig. 6. Pattern 6: Stair Down - Second Step

Fig. 2. Pattern 2: Stair Up - First Step

©ISAROB 2011919

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

IV. CONCLUSION
In this study, we developed a systematical methodthat classifies the biped walking patterns into 9 classes
for a 5-link 7 DOF 3D biped robot. It covers thewalking of biped on uneven surfaces composed ofrectangular shape obstacles and stairs. The ultimate
purpose is to generate enough walking patterns so thatthe walking steps of biped can be designed
systematically by only using the sequences of thesepatterns for any given environment to achieve a
successful biped walking. To achieve a complete patternclassification, this work will be extended to cover the
inclined surfaces so that design of biped reference
trajectories for any given environment will be achieved.

Fig. 7. Pattern 7: Stair Down - Second Periodic Step

VI. ACKNOWLEDGEMENT
This research work presented in this paper isconducted in Middle East Technical University with
Prof. Dr. Kemal Leblebicioglu.

Fig. 8. Pattern 8: Stepping on Up

Fig. 10. Simulated trajectory joint angle profile for Pattern 2

Fig. 9. Pattern 9: Stepping on Down

III. SIMULATION RESULTS
The walking is classified into 9 patterns, as per theenvironmental conditions that robot interacts taken into
account. The simulation results of the biped walking for
Pattern 2 are presented in Fig. 10 to Fig. 12. In Fig. 10,
the joint angle trajectories are given; in Fig. 11, the joint
angular velocity trajectories are depicted and in Fig. 12,
the simulation snapshots of the biped walking areshown for Pattern 2. To further elaborate the walking
patterns, the periodic staircase motion is also
investigated and the results for the periodic stair up caseare presented in Fig. 13 to Fig. 15 by means of joint
angle trajectory, joint angular velocity trajectory andsimulation snapshot of the biped walking.
©ISAROB 2011

Fig. 11. Simulated trajectory joint angular velocity profile for
Pattern 2

REFERENCES[1] Hurmuzlu Y (1988), Dynamics of bipedal gait part I:
objective functions and the contact event of a planar
five-link biped[2] Huang Q, Yokoi K, Kajita S, et. al. (2001), Planningwalking patterns for a biped robot, IEEE trans. on
robotics and automation 17(3):280-289

920

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig. 12. Simulated trajectory walking snapshot - sagittal (top) andfrontal (bottom) plane views - for Pattern 2

Fig. 15. Simulated trajectory walking snapshot - sagittal (top) andfrontal (bottom) plane views - for periodic stair up walking[6] Kajita S, Tani K (1991) Study of dynamic bipedlocomotion on rugged terrain-derivation and applicationof the linear inverted pendulum mode, Proceedings ofthe IEEE international conference on robotics andautomation, 2:1405-1411[7] Zheng YF, Shen J (1990), Gait synthesis for the SD2 biped robot to climb sloping surface, IEEE
transactions on robotics and automation, 6(1):86-96[8] Hirai K, Hirose M, Takenaka T (1998) The
development of Honda humanoid robot, Proc. of theIEEE int. conf. on robotics & automation[9] Kun AL, Miller WT(1999), Control of variablespeed gaits for a biped robot, IEEE robotics &automation magazine[10] Zheng YF (1990), A Neural gait synthesizer forautonomous biped robots, proceedings of the IEEE
international workshop on intelligent robots andsystems: towards a new frontier of applications, 2:601608[11] Arakawa T, Fukuda T (1996) Natural motion
trajectory generation of biped locomotion robot using
genetic algorithm through energy optimization, IEEE
international conference on systems, man, andcybernetics, 2:1495-1500[12] Cheng MY, Lin CS (1998) Genetic algorithm forcontrol design of biped locomotion, Journal of robotic
systems, 14(5):365-373[13] Hemami H, Farnsworth RL (1977) Postural andgait stability of a planar five link biped by simulation,
IEEE transactions on automatic control, 22(6):452-458[14] Dasgupta A, Nakamura Y (1999) Making feasible
walking motion of humanoid robots from human motion
capture data, Proceedings of the IEEE international
conference on robotics and automation, 2:1044 -1049[15] Chevallereau C, Aoustin Y (2001) Optimalreference trajectories for walking and running of a bipedrobot, Robotica, 19:557-569[16] Channon PH, Hopkins SH, Pham DT (1992)
Derivation of optimal walking motions for a bipedal
walking robot, Robotica, 10(2):165-172[17] Yuksel B (2008), Towards the enhancement of
biped locomotion and control, Dissertation, Middle East
Technical University

Fig. 13. Simulated trajectory joint angle profile for periodic stair
up walking

Fig. 14. Simulated trajectory joint angular velocity profile for
periodic stair up walking[3] Shih CL(1997), Gait Synthesis for a Biped Robot,
Robotica, 15:599-607[4] Kurematsu Y, Kitamura S, Kondo Y (1988),
Trajectory planning and control of a biped locomotive
robot - simulation and experiment, Robotics &manufacturing, recent trends in research, education &applications, 65-72[5] Zielinska T (1996) Coupled oscillators utilized as
gait rhythm generators of two legged walking machine,
Biological cybernetics, 4(3):263-273©ISAROB 2011921

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

DEA various method used Restricted Multiplier DEA
Shingo Aoki, Ryota Gejima, and Tomoharu Nakashima
Osaka Prefecture University, 1-1 Gakuencho Naka-ku, Sakai, Osaka 599-8531, Japan
Tel : 81-72-254-9353; Fax : 81-72-254-9915{aoki@, gejima@mis., nakashi@}cs.oskafu-u.ac.jp

Abstract: Data Enveloped Analysis (DEA) is used for evaluating management efficiency of Decision Making Units(DMUs). The traditional DEA has restrictions on weight for multiple objectives. This paper proposes a DEA
method that puts restrictions on not the weight but the multi-input and multi-output items in order to incorporate decis
ion makers’ subjective viewpoint. More DEA variant models can be derived from its basic model. Therefore, theproposed method allows to use evaluation criteria that cannot be used in the traditional method. Thus, the proposed
method is able to widely analyze for various efficiency of DMU. The numerical experiments show the performance ofthe proposed method where teams that participated in the RoboCup 2010 Soccer Simulation 2D are evaluated.
Keywords: Data Envelopment Analysis, Linear Programming, Decision Making Support, RoboCup

to incorporate a priori knowledge. The utility andeffectiveness of the proposed method are shown througha series of numerical experiments.

I. INTRODUCTION
Data Envelopment Analysis (DEA) is a method forevaluating management efficiency of entities. DEA
evaluates entities called DMUs (Decision Making
Units). DMUs evaluated by comparing with theircompetitors. The main characteristics of DEA are thefollowing:･DEA evaluates DMUs that are characterized bymulti-input and multi-output elements.
･The efficiency factors are calculated from multiinput and multi-output elements.
Therefore, DEA can perform evaluation from a lot of
aspects and also can be used in various problems suchas evaluation of universities, baseball players.
DEA calculates the weights for each of multi input
and output elements. Different DMUs have different
weights. The weights show the advantages of DMUs.
However, the more input and output elements DMU has,the more weights become zero. In this case, it is difficult
to clarify the difference between DMUs. Moreover,some elements that do not have a clear meaning forbusiness judgment might cause a significant difference
between DMUs.
Restricted Multiplier DEA (RM-DEA) was proposed
to solve the above problem by including a priori
knowledge in DEA. In this method, all weights are able
to have nonzero value. However, it is difficult to apply
DEA because RM-DEA is infeasible when the
restrictions of weights are too severe.
To remedy the disadvantage of RM-DEA, this paper
suggests a method that puts restrictions on not theweight but the multi input and output elements in order©ISAROB 2011

II. DATA ENVELOPMENT ANALYSIS
1. Outline of DEA
DEA was proposed by Charnes et al. in 1978 as amethod for management analysis [1]. The applicable
field of DEA is widely used in data mining such as theprediction of bankruptcy.
DEA regards each DMU as a production functionthat produces outputs from inputs. Then the efficiency
of a DMU is calculated by comparing with other DMUs.
There are two characteristics in DEA; (1) Weights are
assigned to each input and output data and virtual input
and output are generated. These weights are not fixed
but variable so that each DMU can employ suitableweights to be evaluated better. (2) Common index forevaluation is shown as efficiency value. The value ofthe most efficient DMU is one. On the other hand, theefficiency value is less than one if a DMU is notefficient compared with the others.

2. Formulation of DEA
While DEA has various models, this paper employs
efficiency model and inefficiency model [2]. The former
model evaluates the relative efficiency by the advantage
points. On the other hand, the latter model evaluates therelative inefficiency by the disadvantage points.
Let us assume that there are n DMUs and each
DMU is characterized by m input and s output. That is,
the input for DMUk has input expressed as x1k,…,xmk

922

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

and the output expressed as y1k,…, ysk . Here theefficiency value is calculated by solving the following
linear programming:or output elements that make DMU efficient are notconcerned in the standard business judgment. To deal
with these problems, RM-DEA was proposed. In the
RM-DEA, maximum and minimum of weights are
restricted using a priori knowledge. However, the RMDEA is infeasible when the restrictions of weights are
too severe. Due to the above problems, it is difficult for
decision makers to adjust the restriction using RM-DEA.

s

∑u y

Max

r rk

r =1
m

s.t.

(1)

∑v x = 1
i ik

i =1

r

−

s

∑

vi x j +i =1∑u y ≤ 0 ( j = 1,...,n)
r rj

r =1

ur ≥ 0 (r = 1,..., s), vi ≥ 0 (1,...,m)III. PROPOSED METHOD

The above equation signifies that weights are
assigned to the input elements of DMUk so that theweighted sum of the input elements equals to one. Thisguarantees that the efficiency value of other DMUs does
not exceed one. The objective function has the role for
maximizing the output of remarkable DMU. Moreover,
it is possible to analyze the advantage points of each
DMU by the assigned weights. This is because these
input and output elements that have nonzero weight areconsidered in evaluation. In other words, those elementsthat have nonzero weight are considered advantage
points.
The inefficiency value is calculated by solving thefollowing linear programming:

This section shows the proposed model that unifies
the efficiency model and the inefficiency model to solve
the problem that is described in the previous section. By
controlling the process of transformation from theefficiency model to the inefficiency model, the method
is considered so that a priori knowledge can beincorporated.

1. Reformulation of the efficiency model
The dual problem of Equation (1) is written as
follows:
Min θ kns.t. −

s

Min

∑u y

ij

k ik

∑y λ ≥ y
rj

(2)

i ik

r

∑
i =1

rk

λ j ≥ 0,θ k : freesvi x j +

j

j =1

i =1

∑

ur yrj ≥ 0 ( j = 1,...,n)Where, θ and λ are variables for the dual problem.
Equation (2) which ( d x+i , d x−i ), ( d y+i , d y−i ) are added as

r =1

ur ≥ 0 (r = 1,..., s),vi ≥ 0 (1,...,m)constraints for x and y respectively can be transformed
as follows:

The above equation allows to analyze thedisadvantage point of each DMU by weights. Therefore,
these input and output elements that have nonzero
weight are considered disadvantage points.

m

Min

∑

s

d x−i +

i =1

∑

m

d y−r + M (

r =1

s

∑

d x+i +

i =1

ns.t. −

3. Problem of DEA
In DEA, the number of evaluation criteria isincreased if the number of input and output elements isincreased. Therefore, even if the input or output
elements have only one advantage point, DMU isevaluated efficient. In this case, other elements becomes
weaker. Thus, the elements that are not advantageous
are not emphasized as more zero weights are assigned tomore input and output. Thus excessive number of input
and output lead the following two problems: (1) Many
DMUs are evaluated as efficient, which makes theevaluation meaningless. (2) Some advantages of input©ISAROB 2011(3)n∑v x = 1
−

j

j =1

r rk

r =1
m

s.t.

∑ x λ +θ x ≥ 0∑x λ + x +d −d = 0
ij

j

ik

+
xi

−
xi

∑d )
+
yr

r =1

(4)

j =1
n

∑ y λ +d −d = y
rj

j

+
yr

−
yr

rk

j =1

where M is a very big number, ( d x−i , d y−r ) are slack
vectors to hold the equality, and ( d x+i , d y+r ) are artificial
vectors to unify the model. This equation regards theefficiency value as the distance from efficiency frontier.
Thus, if a DMU is efficient, its efficiency value is zero.
If a DMU is not efficient, the efficiency value is greaterthan zero.

923

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

mixture of efficiency and inefficiency models when
given φxi are distinct primes for each input elements. In
addition, when φxi is less than π/2 and φxi is greater than0, the value of d x−i is not likely to increase comparedto the case when φxi is π/2. Then, input element of i-th
number is unlikely to search for an advantage ascompared with other input elements. Therefore, theproposed method is able to calculate difference of value
for each element.
From the above discussion, we can see that the
proposed method is able to show the various models
such as the mixture of the efficiency and theinefficiency models. Moreover, it is able to incorporatea priori knowledge by VDEC.

2. Reformulation of the inefficiency model
The inefficiency model is reformulated in a similar
way to the efficiency model. Reformulation of theinefficiency model is shown in the following:
m

s

m

s

∑ d + ∑ d − M (∑ d + ∑ d )
+
xi

Max

+
yr

i =1
n

s.t. −

−
xi

r =1

−
yr

i =1

r =1

(5)

∑x λ + x +d −d = 0
ij

j

+
xi

ik

−
xi

j =1
n

∑ y λ +d −d = y
rj

j

+
yr

−
yr

rk

j =1

where ( d x+i , d y+r ) are slack vectors to hold the equality,and ( d x−i , d y−r ) are artificial vectors to unify the model.

3. Unification of two models
Both the reformulated models have the same
constraint. Thus, it is possible to unify the efficiency
model and the inefficiency model by transforming theobjective function. In order to unify the models, wedefine the objective function as follows:

IV. NUMERICAL EXPERIMENTS
1. Experimental conditions and data
In order to show the effectiveness of proposed
method (Equation (7)) visually, we apply it to an
artificial data set. The data set consists of 16 inputoutput DMUs that are characterized by one input andtwo output elements (Table 1). We conducted four
experiments using VDEC as follows:(1)
φ are π/2 for all elements(2)
φ are -π/2 for all elements(3)
φx1 is -π/2 for input1, φx2 is π/2 for input2.
(4)
φx1 is π/2 for input1, φx2 is -π/2 for input2.
In the numerical experiments in this section,
characteristics of the method clarify to look at
differences of efficiency value. Experiments using dataof RoboCup will be shown in the presentation at the
symposium.

m∑ (d sin ϕ − d sin ϕ )
+
xi

Min

xi

−
xi

xi

(6)

i =1
s

+

∑

(d y+r sin ϕ y r − d y−r sin ϕ y r ),

−

r =1

π
2

≤ϕ ≤

π
2

where φ is the Value of Deciding Evaluation Criteria(VDEC). VDEC is used to control the degree of thetransformation between the efficiency and theinefficiency models. Thus, the proposed method isshown by calculating the following linear programming:mMin∑ (−d sin ϕ + d sin ϕ )
+
xi

xi

−
xi

xi

i =1

s

+

∑ (−d sin ϕ + d sin ϕ )
+
yr

−
yr

yr

yr

r =1

n

s.t. −∑

Table 1. Artificial data set(7)

xij λ j + xik + d x+i − d x−i = 0 (i = 1,2,..., m)

DMU input1 input2 output
1
1
1
1
2
1
2
1
3
1
3
1
4
1
4
1
5
2
1
1
6
2
2
1
7
2
3
1
8
2
4
1
9
3
1
1
10
3
2
1
11
3
3
1
12
3
4
1
13
4
1
1
14
4
2
1
15
4
3
1
16
4
4
1

j =1
n

∑y λ +d −d = y
rj

j

+
yr

−
yr

rk

(r = 1,2,..., s)j =1

λ ≥ 0, d x+i ≥ 0, d x−i ≥ 0, d y+r ≥ 0, d y−r ≥ 0,−

π
2

≤ϕ ≤

π
2

In this equation, optimal DMUs are evaluated with theefficient value of zero. On the other hand, the values for
non-optimal DMUs are greater than zero.
Equation (7) is the efficiency model when φxi and φyr
are equal to π/2 for all i and r. Equation (7) is theinefficiency model when φxi and φyr is equal to π/2 for
all i and r. Moreover, Equation (7) is regarded as the©ISAROB 2011924

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

DMU4 is inefficient for input2/output. Thus, theproposed method is able to evaluate DMUs by mixing
the of efficiency and inefficiency.

2. Experimental Results
The results are shown in Table 2. For instance, thevalue of the DMU2 is evaluated value as zero in the case(1), and as six in the case (2).

Input2/output

4

8

12

16

3

7

11

15

2

6

10

14

1

5

9

13

Table 2. Distance from optimal DMU
DMU
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

(1)
0
1
2
3
1
2
3
4
2
3
4
5
3
4
5
6

(2)
6
5
4
3
5
4
3
2
4
3
2
1
3
2
1
0

(3)
3
4
5
6
2
3
4
5
1
2
3
4
0
1
2
3

(4)
3
2
1
0
4
3
2
1
5
4
3
2
6
5
4
3

input1/output

Fig. 1. Visualization of case (1)Input2/output

3. Discussion
The numerical experiments of case (1), (2), and (3)are shown in the Fig, 1, Fig 2, and Fig. 3 in order toexamine the effectiveness of the proposed method
viscerally. All X-axes are input1 over output and all Yaxes are input2 over output. The grid points show
DMUs and the lines connecting them show DMUs
which have the same evaluated value.

8

12

16

3

7

11

15

2

6

10

14

1

5

9

13

input1/output

Fig. 2. Visualization of case (2)Input2/output

A. Case (1)DMU1 is the optimal DMU in this case, and its
evaluated value is “0”. The DMUs which areequidistance form DMU1 have the same evaluated value.
Moreover, as the distance from DMU1 becomes farther,
the evaluated value also become large. This shows thatDMU with smaller input and larger output is able tobecome optimal. Thus, given that φ is π/2 for all
elements, the proposed method is able to be regarded as
efficiency model.
B. Case (2)DMU16 is the optimal DMU in this case, and its
evaluated value is “0”. All evaluated values are inverted
in comparison with case (1). Thus, given that φ is -π/2
for all elements, the proposed method is able to be
regarded as inefficiency model.
C. Case (3)DMU4 is the optimal DMU in this case, and its
evaluated value is “0”. DMU4 is efficient when DEA
calculates it for input1 and output. On the other hand,©ISAROB 2011

4

4

8

12

16

3

7

11

15

2

6

10

14

1

5

9

13

input1/output

Fig. 3. Visualization of case (3)V. CONCLUSION
This study proposed a DEA model which unifies theefficiency model and the inefficiency model. The
proposed model is able to evaluate various criteria
which the traditional model can not evaluate. Moreover,
VDEC helps incorporating a priori knowledge. The
effectiveness of the proposed method was illustrated in
the numerical experiments.

REFERENCES[1] Charnes, A., Cooper, W. W. and Rhodes, E. L.
(1978), Measuring the Efficiency of Decision Making
Units. European Journal of Operational Research, Vol.
2, pp. 429-444.
[2] Yamada Y., Matsui T. and Sugiyama M. (1994), An
Inefficiency Measurement Method for Management
Systems (in Japanese). Journal of the Operations
Research Society of Japan, Vol. 37, No.2, pp.158-168.

925

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Efficient Distributed Ontology Management Scheme for Inferencein Surveillance Networks
S. Yang
Department of Information Engineering,
The University of Suwon, 445-743, KOREA(Tel : 82-31-229-8138; Fax : 82-31-229-8284)(Smyang@suwon.ac.kr)

Abstract: In this paper, we propose an efficient and scalable surveillance network providing better context inference
based on distributed ontology framework. Upon our distributed ontology management system, each agent can build andprocess ontology cooperatively. They share context ontology for cooperative combined inference. Data source servers
not only can get services from a region server, but also they can form and generate a P2P(peer-to-peer) network toprovide services to each other. For an efficient and scalable cooperation we adopt a P2P communication throughcommon API and adaptive caching.
Keywords: Surveillance network, Agent, Context ontology, Peer to peer network.

The rest of the paper is organized as follows, Section
2 we suggest a distributed inference framework forsurveillance networks. Experiments and implementation
is explained in Section 3. Section 4 reviews the related
work on distributed inference based on ontology,
particularly focusing on the user interface and theadaptive caching technique. Section 5 concludes thispaper with a short summary.

I. INTRODUCTION
In our network surveillance environment, eachsurveillance devices is equipped with a customized
agent. The distribution of demands for multimedia data
items is often skewed, and the surveillance devices havedifferent capabilities and data formats. These can lead topoor data communication and dropped messages
resulting in narrow reasoning and decision. Therefore
for more easy and efficient cooperation between theagents, we propose a more efficient and scalable
communication framework utilizing P2P computing andan adaptive cache scheme based on heterogeneous
devices and data network.
Context is any information that can be used tocharacterize the situation of environment entities. To
overcome the variety of data source servers, we build
agent for each and interface through common APIs. Our
network surveillance application can be thought ofsemantic web. Each agent tries to be context-aware
through integration, analysis and inference of data andinformation. For better inference, they need not only
data of their own but also data of other neighbor agents.
P2P computing utilizing common API provides flexible,
scalable and efficient ontology sharing method.
In this paper, we also propose a distributed adaptive
cache scheme based on heterogeneous device and data
network. Our scheme uses caching and conformity to
update and share data in a cooperative way.
Experiments and implementation are conducted toevaluate the effectiveness of our flexible cache scheme.

©ISAROB 2011

II. DISTRIBUTED INFERENCE
FRAMEWORK
Our surveillance network constructs a web ofinference systems. Each data source server is equippedwith an independent agent having local context
ontology and inference engine. Cooperative inferencesuch as object tracking requires the cooperation among
local data source servers. Data associations between
data sources are required to cover larger areas and solve
occlusion problems. Our distributed inference
framework manages ontology processing, ontology
location and ontology connection on behalf of eachagent.
Data source servers form a graph structure. For thehigh level inference, regional hierarchies are included init. Each data source server can communicate each otherfreely within access control permission forming P2P
computing networks. Equipped with agents, they
perform their own intelligent distributed inference basedon their own ontology knowledge base. In the case ofmulti-agent systems, association of data across the data

926

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

packet loss, the weight of data and the inverse of thenumber of hops.

source servers is indispensable for context awarenessand decision making for the integration of ontologies
and prevention of the narrow decision.
For each data request, data source server ni first tries
to find the required data item from its local cache. If it
encounters a local cache miss, it broadcasts a request
message to its neighbor peers via P2P broadcast
communication. Our ontology framework includes
common APIs for the operations. If a neighbor peer
does not have the needed data item, the agent will send
the request to the higher region server instead to obtainthe needed data. The representative common APIs are
getVideoSource(),getVideoSourceConfiguration(),getVideo(),setVideo(),getVideoStatistics(),
getVideoCaptureSize(),
getUsers(),
getVideoResolution(), requestSignal(), sendSignal(),getBitrateControlMode(), etc.
For the broadcasting, the latency is defined as thesum of the request and reply packets divided by network
bandwidth through the hops the data gets transmitted.
As peers can exchange such information as the cachesizes of their neighbors and the transmission ranges,
they can negotiate and come up with a cache strategy
based on the messages exchanged. As a result, betterusage of their limited cache space and higher system
performance can be obtained
The multimedia data and biometric feature
information caching will be distributed over the region
servers and data source servers. The data source servers
need to carry out the indexing and retrieval of theinformation distributed across the servers in an efficientmanner. To maintain the freshness and effectiveness ofthe data, we should cache data adaptively. We measure
the weight of cached data to describe its relative
importance as compared to the other data as proposed in[4]. The higher the weight, the lower is the probabilityof the data being replaced. We also use a policy basedon the size of the objects, in which the weight is
proportional to the size of the data. Therefore the weightis computed as the product of the access frequency, sizeof the data and the data recency:

III. EXPERIMENTS AND
IMPLEMENTATION
In a distributed surveillance environment,
multimedia data and biometric feature information are
produced continuously. Furthermore the multimedia
data transmission consumes large network bandwidth.
When the adaptive caching technique is applied,experiments show better performance as shown in
Figure 1. The data source servers might be grouped byregion. The caching skewed by region could bereflected through the variance in caching ratio.

cacheduncached

25

20

15
E
10

5

0
0.5
0.4
0.6
0.4

0.2
0.2
q

0.1

0

p

Fig.1. The expected packet transmission comparison
For the integration of ontologies between data
source servers, we developed common APIs according
to the standard [7]. The pseudo code for the
representative common APIs is as the following:
function grrc(key){
this.name = 'grrc';
this.key = key;
this.XmlHttpObject = function(){ //abridged}
this.validChk = function(){var add;
add = location.href;
add = add.split("//");
add = "http://" +add[1].substr(0,add[1].indexOf("/")) + "/";
host = 'http://localhost/grrc/';
URL = host + 'validChk.php?key=' + this.key+ '&add=' + add;
xmlhttpr = new XMLHttpRequest();xmlhttpr.open('GET', URL, true);
xmlhttpr.send(null);

Weight = Frequency f DataSize s Re cency r
The exponents are weighting factors. As a result, we
can expect the packet traffic would be proportional tothe probability of cache miss, latency, the probability of©ISAROB 2011

1
0.8

0.3

927

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

xmlhttpr.onreadystatechange ();//abridged}
this.GetUsers = function(){this.validChk();
host = 'http://localhost/grrc/';
URL = host + 'getUsers.php';
location.href = URL;}
this.GetVideoSource = function(){this.validChk();
host = 'http://localhost/grrc/';
URL = host + 'getVideoList.php';
location.href = URL;}
this.GetVideoSourceConfiguration =function(){this.validChk();
host = 'http://localhost/grrc/';
URL = host + 'getAllVideoList.php';
location.href = URL;}
this.GetVideoCaptureSize = function(){CaptureSize = new Array(1);CaptureSize[0] = new Array(2);this.validChk();
host = 'http://localhost/grrc/';
URL = host + 'VideoInfo.php';
xmlhttp = this.XmlHttpObject();
xmlhttp.open('GET', URL, true);
xmlhttp.send(null);
xmlhttp.onreadystatechange = function();//abridged
return CaptureSize[arrWidth][arrHeight];}
this.GetVideoResolution = function(){Resolution = new Array(1);Resolution[0] = new Array(2);this.validChk();
host = 'http://localhost/grrc/';
URL = host + 'VideoInfoResolution.php';

xmlhttp = this.XmlHttpObject();
xmlhttp.open('GET', URL, true);
xmlhttp.send(null);
xmlhttp.onreadystatechange =function();//abridged
return Resolution[arrResolutionWidth][arrResolutionHeight];}
this.getVideo = function(num){
this.validChk();
host = 'http://localhost/grrc/';
URL = host + 'getVideo.php';
xmlhttp = this.XmlHttpObject();
xmlhttp.open('GET', URL, true);
xmlhttp.send(null);
xmlhttp.onreadystatechange =function();//abridged}}
Our work is aimed at designing and deploying asystem for the surveillance and monitoring of provincearea containing about 15 cities. Final product is to be
included in the construction of the ubiquitous city. The
agent platform should be distributed across various dataacquisition systems such as CCTV, RFID sensors, soundmonitors, etc. We show the live video and the stored
video to monitoring clients and devices are controlled
via a remote GUI even from a smart phone. Figure 2
shows screenshot of map interface. It shows the stored
event data. On the left frame we can see the google map.
On the right frame, alarming events are listed first withfeatures. When we select the CCTV, we can see thespecific videos on the bottom side.

Fig.2. The sample user interface with google map©ISAROB 2011928

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011[2] J. Z. Pan, (2007), A Flexible Ontology Reasoning
Architecture for the Semantic Web, IEEE Transactions
on Knowledge and Data Engineering archive Volume
19 , Issue 2, 246-260[3] Anne Schlicht, Heiner Stuckenschmidt, (2008),
Towards Distributed Ontology Reasoning for the Web,
IEEE/WIC/ACM International Conference on WebIntelligence and Intelligent Agent Technology[4] L.Yin, G. Cao, (2006), Supporting Cooperative
Caching in Ad Hoc Networks. IEEE Tr. On mobile
computing, Vol. 5, No. 1.
[5]Zheng, J., Su, J., and Lu, X. (2005), A
Clustering based Data Replication Algorithm in Mobile
Ad Hoc Networks for Improving Data Availability,
Lecture Notes in Computer Science 3358
[6]
Chow, C., Leong, H., and Chan, A. (2005),
Distributed Group-based Cooperative Caching in a
Mobile Broadcast Environment, MDM 2005[7] ONVIF, 2009, Open Network Video Interface
Forum Core Specification version 1.01

IV. RELATED WORK
Several ontology reasoning systems have beendeveloped for reasoning and querying the semantic web
and they show good performance as seen in [1,2,3]. We
want to adopt, customize and improve them adequate
for our surveillance system. Especially all of them do
not use common API for communication. In [4], acooperative caching framework is introduced andclaimed to be effective to data availability. In [5], a
replica allocation method and clustering in distributed
networks are introduced to improve data accessibility ina mobile communication environment. Anothercooperative cache scheme for similar peers is describedin [6], which combines the P2P communication
technology with a conventional mobile system.
However they do not apply their technique to the
distributed surveillance system.

V. CONCLUSION
We describe an efficient ontology management
scheme for better context inference based on distributed
ontology framework. Data acquisition servers
communicate each other freely within access control
permission to perform their own intelligent distributed
inference based on integrated ontology knowledge baseutilizing common APIs. Our scheme uses P2P
computing using common APIs and data conformity to
update and share data in a cooperative way. For such a
P2P network, an effective cache framework that canhandle heterogeneous devices is required. We alsopresent a flexible cache scheme which is adaptive to theactual device demands and that of its neighbors. We
analyzed our distributed ontology management scheme
through experiments and implementation.

VI. ACKNOWLEDGMENTS
This work was supported by the GRRC program of
Gyeonggi province. [GGA0801-45700, Center for Ucity Security and Surveillance Technology]

REFERENCES[1] A. Flahive, W. Rahayu and D. Taniar, (2005), A
Distributed Ontology Framework in the Semantic Grid
Environment, 19th International Conference on
Advanced Information Networking and Applications(AINA’05)©ISAROB 2011929

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Recovery Technique from Classified Errors
in Adjustment Tasks of Domestic Appliances
Akira Nakamura and Yoshihiro Kawai
Intelligent Systems Research InstituteNational Institute of Advanced Industrial Science and Technology (AIST)
Central 2, 1-1-1 Umezono, Tsukuba, Ibaraki, 305-8568 Japana-nakamura@aist.go.jp, y.kawai@aist.go.jp
Abstract: Dexterous manipulation is an important function for working maintenance robots. Manipulator tasks such as
disassembly and reassembly can generally be divided into several motion primitives. The authors call such motion
primitives “skills” and explain how most manipulator tasks can be composed of sequences of these skills. In theirplanning to construct a maintenance robot for domestic appliances, the authors considered hierarchizing themanipulation tasks since the maintenance of such appliances has become increasingly complex. Additionally, they
considered grouping errors into several classes according to their possible causes. The reliability of the task
achievement was found to increase with the classification of errors. There are various kinds of tasks for maintenancerobots besides disassembly and reassembly, and adjustment tasks, which are often executed in the final stage of
maintenance, are also important. This paper presents a proposal for error recovery during adjustment tasks performed
by a robot in the maintenance of domestic appliances.
Keywords: manipulation skill, maintenance task, error recovery.

occurring. During actual
Hand-eye vision systemmanipulation,
however,
Manipulator
errors often occur forvarious reasons. Various
approaches
for
error
recoveryhave
beenreported [6]–[9]. However,
A-V System Manipulator
few methods for realistic
components
error recovery have been
proposed for the various
errors that could actually
Fig. 1 Maintenance robot
occur during maintenancefor audio-visualsystem components
tasks. We have previously
described our concept oferror classification and process flow using error recovery
in the task hierarchy [4], [5]. Errors are grouped into
several classes according to their possible causes. In thispresent paper, we describe a method of error recovery
during adjustment tasks such as rotating knobs when
maintaining domestic appliances. The reliability of thetask achievement increased with the stratification oftasks and classification of errors.
The next section explains manipulation skills andskills used in adjustment tasks. The stratification ofmanipulation tasks is shown in section 3. Ourclassification of errors and error recovery in the taskhierarchy intended for adjustment tasks are shown insection 4.

1. Introduction
Manipulation robots need to achieve various tasks
using special techniques to be useful in wide-ranging
fields. By analyzing the motions of human hands in tasks
such as disassembly and reassembly, we found that the
movements consisted of several significant motion
primitives. We call these motion primitives "skills" andhave shown that most of the tasks of a manipulator canbe composed of sequences of such skills [1]–[3].
We have already researched maintenance robots
working in various factories and power plants. In our
future research, we will consider manipulation robots
working closer to people and used for the maintenance ofhousehold appliances and consumer electronics. At
present, we are working toward producing a prototype ofa maintenance robot for home audio-visual systemcomponents and personal computers (Fig. 1). The
principle tasks of such maintenance robots will be not
only disassembly and reassembly as considered in [3]–[5] but also adjustment tasks. Such adjustment tasks
might involve rotating thumbscrews to tune various
settings such as frequency, amplitude and brightness.
Maintenance requires the use of many manipulation
skills, and the nature and range of the tasks may becomplex. However, stratification of the tasks makes
development more manageable.
Manipulation tasks using skills are performed intheory by sequences of visual sensing, geometricmodeling, planning and execution. In an ideal
environment, the tasks are achieved without any errors©ISAROB 2011930

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011(a) Thumbscrew(b) Thumbscrew(Type P)(c) Thumbscrew(Type S)(d) Wingbolt(e) Tuningpeg

Fig. 2 Thumbscrews and similar fasteners(a) Move-toinitialize
skill

Start

(c) Close-to-nip skill

Fig. 4 Task of fitting a gripper on a thumbscrew

Fitting task

θ > θrepeat(b) Move-toapproachskill- θrepeatNO

YES

θrepeat rotation task

θ rotation taskθ ← θ – θrepeatEnd

Re-fitting task(a) Open-todetachskill

Fig. 3 Process flow of adjustment taskusing a thumbscrew(d) Move-toapproachskill(e) Close-tonip skill

Fig. 5 Task of re-fitting a gripper on a thumbscrewθrepeat

2. Manipulation Skills
This section explains our concept of skills andexamples of the skills used in adjustment tasks. See
References [1], [2] for more details.

θ(a) Rotate-to-decrease-room skill

Fig. 6 Task of θrepeat rotation

2.1. Concept of Skills(a) Rotate-to-adjust skill

Fig. 7 Task of θ rotation

We analyzed human motions in such tasks as
disassembly and reassembly and found that the
movements consisted of several significant motion
primitives. We call such motion primitives “skills” [1],[2]. We considered three fundamental skills: move-totouch, rotate-to-level and rotate-to-insert, all of whichplay an important part in such tasks. A specific task iscomposed of sequences of skill primitives such as thesethree skills. Moreover, many related skills such as therotate-to-bite and rotate-to-loosen skills in tasks using
screwdrivers can be defined based on slightly modified
versions of these three fundamental skills. These skills
are frequently used for component replacement during
the repair of household appliances [3]-[5].

The knobs of adjustment screws used in maintenancetasks come in various head shapes. Figures 2(a), 2(b) and2(c) show various types of thumbscrews including
cylindrical type, “P” (plain) type, and “S” (shoulder) type.
There are also wing bolts (Fig. 2(d)) and tuning pegs on
string instruments (Fig. 2(e)). For simplicity, in thispaper we focus on screws with flat head projections (Fig.
2(b), 2(c), 2(d), 2(e)) that can be nipped by parallel
grippers.
Figure 3 shows the process flow for an adjustment
task using a thumbscrew. First, a fitting task using aparallel gripper on a plain knob is performed (Fig. 4).
Rotation tasks of the same angle θrepeat such as π, 2π(rad) are iterated (Fig. 6), while the re-fitting tasks of thegripper are performed between successive rotation tasks(Fig. 5). Finally an adjustment task comprising rotation
at a small angle θ ( ≤ θrepeat) is performed (Fig. 7).
However, the value of angle θ is hardly a suitablereference value. In most cases, the values of angles θ are
adjusted so that settings such as frequency, amplitude

2.2. Skills in Adjustment Tasks
In the maintenance of household electrical appliancesand home audio-visual equipment, a parts adjustment
step is often needed in addition to the main tasks of
disassembly and reassembly such as component
replacement [3]–[5].

©ISAROB 2011(b) Move-to- (c) Turn-backrelease
to-repeat
skill
skill

931

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011and brightness become the most suitable values.
Therefore, excess rotations may often occur, and will
require rotation in the opposite direction to be performed.
In this paper, this process will be treated as a patternusing error recovery as shown in section 4.

project (MAX)…

Project

project (2)project (1)task (max)…

Tasktask (2)task (1)task (0)3. Stratification of Tasks
Figure 8 shows a hierarchy of manipulation tasks
during maintenance [4]. If we ignore the servo layer, theskill layer, which consists of elements such as the moveto-approach and close-to-nip skills, is located in thelowest layer called the task(0) layer. Each skill is
performed by the processes of visual sensing, geometricmodeling, planning and execution. One tier above thetask(0) layer is the task(1) layer. Similarly, task(i+1) iscomposed of sequences of task(i) elements. The top layer,
where the error recovery loop is closed, is called task(max)and one tier above task(max) is called the project layer. The
project layer might also be hierarchized, but we will not
discuss it in this paper.

SkillprimitiveFig. 8 Manipulation Hierarchy

Start
Sensing(All-over task (max) )Modeling(All-over task (max) )Planning(All-over task (max) )

(0)

※1

task (1)※1(0)

task (2)Classification of error

task (0)( i0 )

※1

(0)

※1

task ( i0 )Class T (max+1)
Stop the process and executeall necessary changes.

※3
Re-planning(task( (1)i1 ) )Class 1(max)

task ( imax )In an ideal environment, tasks are achieved without
any errors occurring. In actual manipulation, however,errors often do occur from various causes. Our conceptof error classification and process flow with error
recovery in the task hierarchy are described in thissection. See References [4] for more details with respect
to component replacement tasks. In this present paper,
errors occurring in adjustment tasks during maintenanceof domestic appliances and electronic equipment are
explained and error recovery techniques in theadjustment tasks are shown.

(1)

4. Error Recovery in Stratified Tasks

task ( imax , …………… , i1 )※3Modeling(task( (1)i1 ) )Class T (2) Execution ofnecessary changes※3

Class 2 Change ofplanning parameters
Class 3 Change ofmodeling parameters
Class T (1) Change ofsensing parameters

Sensing(task( (1)i1 ) )

Execution※2

task (0)( i0 )

Sensing
Modeling
Confirmation

error

correct

End

4.1. Classification of Errors

Accurate expression:※2

The causes of failures can be attributable to several
kinds of errors. We group error states into several classes
according to their possible causes as follows. The classesof errors are described in detail in Reference [4].

(0)task (0)( i0 ) ＝ task ( imax , ……… , i0 ) ,(1)task (1)( i1 ) ＝ task ( imax , …… , i1 )Fig. 9 Process flow with error recovery

Merely remedying the causes of these errors does notalways solve the problem. It may be necessary to return
to a previous step when the working environment isgreatly changed by the error.

･Execution error: a mechanical error caused in the
manipulator mechanism such as a gear backlash.
･ Planning error: an error caused by inaccurate
parameter values in planning. For example, a rotation
task fails because of an incorrect torque setting.

4.2. Error Recovery based on Classificationin Adjustment Tasks･Modeling error: an error caused by differences in
the real object and the geometric model in the software.
For example, failure occurs because of the incorrecthead shape of the thumbscrew.

A generalized process flow of stratified tasks thattakes error recovery into account has been shown in [4].
Figure 9 is an illustration of the central portion of Fig. 10
in [4]. At the confirmation step in each skill primitivetask(0)(i0), an automatic process or a human operator
judges whether the result is correct or a failure. Error
recovery is performed using the following error
classification.

･ Sensing error: an error occurring during visual
sensing. For example, the environment model aroundthe thumbscrew cannot be derived correctly due to asensing error in calibration and luminous intensity.

©ISAROB 2011

※3

932

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Class 1: When the error is judged to be an execution
error, task(1)(i1) is executed again without a correctionin the parameters.

In the future, we will further research the optimum
adjustment methods for the various parameters used inerror recovery and a fully automatic method to confirm
task achievement composed of skills. We will attempt toapply our technique to actual maintenance robots.

Class 2: When the error is judged to be a planningerror, task(1)(i1) is executed again with a change in theplanning parameters.

References

Class 3: When the error is judged to be a modelingerror, task(1)(i1) is executed again with a change in themodeling parameters.

[1] Hasegawa T, Suehiro T, Takase K (1992), A modelbased manipulation system with skill-based execution.
IEEE Trans Robotics Autom 8(5):535-544[2] Nakamura A, Ogasawara T, Suehiro T, Tsukune H(1996), Skill-based backprojection for fine motion
planning, Proceedings of the IEEE/RSJ International
Conference on Intelligent Robots and Systems(IROS ’96), Osaka, Japan, Nov 4-8, 1996, pp.526533[3] Nakamura A, Ogasawara T, Kitagaki K, Suehiro T(2001), Using robust and simplified geometricmodels in skill-based manipulation, Proceedings ofthe IEEE/RSJ International Conference on Intelligent
Robots and Systems (IROS 2001), Hawaii, USA, Oct
29-Nov 3, 2001, pp.138-145[4] Nakamura A, Kotoku T (2009), Systematization ofError Recovery in Skill-Based Manipulation,
Proceedings of the 14th International Symposium on
Artificial Life and Robotics (AROB 14th ’09), Oita,
Japan, Feb 5-7, 2009, pp.610-613[5] Nakamura A, Kawai Y (2010), Recovery Techniquefrom Classified Errors in Skill-Based Manipulation,
Proceedings of the 15th International Symposium on
Artificial Life and Robotics (AROB 15th ’10), Oita,
Japan, Feb 4-6, 2010, pp.1010-1013[6] Donald BR (1989), Error detection and recovery inrobotics. Springer-Verlag, Berlin Heidelberg New
York, pp.1-256[7] Seabra Lopes L, Camarinha-Matos LM (1995), A
machine learning approach to error detection andrecovery in assembly, Proceedings of the IEEE/RSJ
International Conference on Intelligent Robots andSystems (IROS ‘95), Pennsylvania, USA, Aug 5-9,
1995, vol.3, pp.197-203[8] Yamazaki K, Tomono M, Tsubouchi T, Yuta S (2006),
Motion planning for a mobile manipulator based on
joint motions for error recovery, Proceedings of theIEEE/RSJ International Conference on Intelligent
Robots and Systems (IROS 2006), Beijing, China,
Oct 9-15, 2006, pp.7-12[9] Scheutz M, Kramer J (2007), Reflection andReasoning Mechanisms for Failure Detection andRecovery in a Distributed Robotic Architecture forComplex Robots, Proceedings of the IEEE
International Conference on Robotics andAutomation (ICRA 2007), Roma, Italy, Apr 10-14,
2007, pp.3699-3704

Class 4 ( = Class T(1) ) : When the error is judged to
be a sensing error, task(1)(i1) is executed again with achange in the sensing parameters.
Class T(2): task(2)(i2) is executed again after the
execution of the necessary changes and returns to thestart of one tier above the layer task(1)(i1).
::Class T(max): task(max)(imax) is executed again after the
execution of the necessary changes and returns to thestart of (max - 1) tier above the layer task(1)(i1).
Class T(max+1): When it is judged that too manychanges will be required, the process being executed isinterrupted and the process returns to the start of theoverall task.
Parameters judged in any given Class are revised anderror recovery is performed according to a flowchart. In
most adjustment tasks, the target angle θ is decided sothat another setting value such as frequency or brightness
becomes the most suitable value as shown in section 2.2.
Therefore, it is often necessary to rotate the thumbscrewin the opposite direction because of excess rotation. In
such cases, the required change in skill sequence isperformed through Class T(2). This means re-planning in
one tier above the lowest task layer. In general, theadjustment task converges while the required rotationand the opposite rotation are being repeated. On the other
hand, it may be necessary to return to a smaller angle due
to excess rotation, for instance when adjustment ispossible only in processes in which the frequency varies
from low to high as in the adjustment of a musicalinstrument.

5. Conclusions
In recent years, dependability has been a frequent
topic in robotics research. It is necessary to increase thereliability of the maintenance tasks performed by robots
working on household appliances. Since error recovery isimportant, we considered a method that uses the conceptsof both task stratification and error classification. In thispaper, we have described our error recovery techniquefor adjustment tasks performed by maintenance robots.
The capability to recover from errors during such tasks is
improved by stratification of tasks and classification oferrors.

©ISAROB 2011933

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

A Visual Debugger for Developing RoboCup Soccer 3D Agents
Yosuke Nakamura and Tomoharu Nakashima
Osaka Prefecture University, 1-1 Gakuen-cho,Nakaku,Sakai,Osaka(Tel : 81-72-254-9351; Fax : 81-72-254-9351)(nakashi@cs.osakafu-u.ac.jp)

Abstract: In this paper, we introduce a visual debugger that helps us develop soccer agents for RoboCup Soccer 3D
Simulation. The visual debugger enables us to graphically monitor the internal state of a soccer agent and the soccer
field such as joint angles, the position of objects, and text messages. We employ a server/client framework where thedebugger acts as a server while the agent acts as a client. A soccer agent connects to the debugger using TCP/IP and sends
the information about the field and the internal status. The information that is sent from the soccer agent to the visualdebugger consists of three parts: visible objects of the soccer field, the joint angles of the soccer agents, and text messages
from the agents. These are shown in separate components on the screen of the debugger. The debugger draws the current
pose of the soccer agent from the information on the joint angles that is sent from the soccer server. Text messages areused as a debugging message. The developer of soccer agents are allowed to check if the developed agent works properly
through the screen of the visual debugger. A soccer agent that is manually controlled using a game-pad is also included
as a part of the debugger. Each of the above features is explained in detail.
Keywords: RoboCup, soccer robot, multi-agent system

I. INTRODUCTION

soccer monitor, which is included in the package of thesoccer server[7]. Although the soccer monitor is useful to check the movement of joints of soccer agents, it
is not possible to check the internal status of the soccer
agents such as vision, ground force, body rotation, andtheir decision making process. Furthermore the soccer
monitor does not record the movement of soccer agentsduring the course of the game. Thus it is not possible toplay back the match even if we want to retrospectively
watch the behavior of the soccer agents. These things
make the development of 3D soccer agent very difficult.
Even if we use files to replay a match, the internal statusof agents such as sensory information and the intentionof agents cannot be shown and we only have to guessthem from the behavior of the agents.

Soccer simulation league is one of the oldest leagues
in the RoboCup competitions. The main aim of the soccer simulation league is to develop decision making systems that achieve intelligent behavior in both high andlow levels. It is expected that successful decision making systems are translated to that of real robots. The developers of soccer agents in the simulation league haveproposed various techniques from both top-down andbottom-up manners. For example, Stone[1] proposeda layered approach to achieve a complex decision system. FC Portugal proposed a flexible formation modelthat is based on the ball position[2]. These are categorized as top-down approaches. On the other hand, Stoneet al.[3] proposed a neural network-based approach forlearning low-level skills. Sean et al.[4] also showed thatthe team formation is successfully evolved by a genetic
programming. A fuzzy reinforcement learning method
is proposed for a ball intercept task by Nakashima etal.[5].
In the early years of the RoboCup soccer simulation
league, soccer matches were played on a virtual twodimensional field. The first prototype of 3D simulation
was proposed by Oliver[6] in 2003. The community of
the 3D simulation has rapidly grown up since then. In
2005 the first competition of 3D simulation was held
in Osaka, Japan. The soccer agents were modeled asa sphere object with a kick device. In 2007 bipedal humanoid robot model was employed for soccer agents.
This made the development of soccer agents very challenging because not only intelligent decision making but
also the movement of joints have to be considered when
implementing even lower level skills. The standard way
to check the behavior of developed agents is to use the©ISAROB 2011

In the 2D simulation league, several high functional debuggers have been proposed. For example,
soccerwindow2[8] has useful features such as monitoring internal status, modeling team formation based on
the ball position. The developer of soccer agents canalso check the decision making process inside the soccer agents through the messages that are sent from thesoccer agents to soccerwindow2. Furthermore, soccerwindow2 has a play-back function where the developer
can check the behavior of the soccer agents at the pasttime steps. SoccerScope[9] has similar useful features
as soccerwindow2, though SoccerScope can analyze thegame logs and give statistics of the game such as thenumber of successful pass, ball possession rate, and thetotal number of pass, dribble or shoot. In contrast, in
3D simulation league,there are no such tools available inpublic. Therefore, we have developed a visual debugger
as a development support tool that connects to an agent,
displays sensor information and checks the agent’s behavior.

934

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

II. ROBOCUP SOCCER 3D SIMULATIONLEAGUEthe position of objects from as broader area as possible.
An aural sensor gets one message from both teams every 3 time steps. Agents make a decision based on theabove information. Furthermore, the decision made byan agent must be converted to the necessary changes tothe joint angles by the agents themselves. Agents send
joint change rates to the server. There are 22 joints ina Nao robot. To control this robot perfectly, an agent
program is required to control 22 joint angles every time
step.
The RoboCup Soccer 3D Simulation Server is available in a Sourceforge project[7]. The package includes
the soccer server, the sample agent, and the soccer monitor. The soccer monitor is the standard way to checka behavior of agents. But the monitor shows the soccer
field in the third person viewpoint and we can only checkthe state of joints from their pose. Therefore, a softwaretool that shows the internal state of agent is required. In
this paper, we develop a visual debugger as a development support tool that connects with an agent, displays
the value of sensors and checks the agent’s behavior. We
also develop a soccer agent that can be controlled by agame-pad as a part of the debugger. In the next section,
we introduce the visual debugger in detail.

RoboCup Soccer 3D simulation league is the project
where autonomous agents play soccer in a virtual 3D
soccer field. The first prototype of 3D simulation was
proposed in 2003. In 2005 the first competition of 3D
simulation was held in Osaka, Japan. In this competition, the soccer agents were modeled as a sphere object
with a kick device. In 2007, a bipedal humanoid robot
model was employed for soccer agents. This made thedevelopment of soccer agents very challenging because
not only intelligent decision making but also the movement of joints has to be considered when implementing
even lower-level skills. The humanoid robot which wasused in the RoboCup world competition 2008 is given in
Fig. 1. This robot model is based on the Nao[10] which
is used in the standard platform league. In the RoboCup
world competition 2009, one team consisted of threerobots. Six-vs-six games were played in RoboCup 2010.

III. VISUAL DEBUGGER
1. Overview
The visual debugger is connected to a soccer agent
via TCP/IP and receives information on the field and theagent’s internal status every cycle. The functions that thevisual debugger provides make it easier for us to develop
agents. A screen shot of the visual debugger is shown in
Fig. 2. The soccer agent can send a one-line message tothe debugger at each time step. The message contains
the information on the state of the field and the agent
graphically. The visual debugger graphically shows theinformation on the display.

Fig. 1. Humanoid robot
Each soccer agent is autonomously controlled in the3D simulation league. A game is divided into two halvesand each half has 300 seconds according to the official
rule. Since one time step is 20 ms, a half consists of
15000 steps. During the simulation, the soccer server
sends a message to agents every time step. The message
includes joint angles, the value of gyro sensor, the valueof foot force resistance sensor, the position of visible
objects, and say-messages from other agents. The gyro
sensor is equipped in torso. It senses the orientation of
the torso in the global coordinates. The foot force resistance sensor informs about the force that acts on a foot.
This sensor is equipped in the bottom of feet. A soccer agent can also receive the position of visible objects(the ball, other agents, goal posts, and landmark flags)
from the soccer server. An agent is equipped with thecamera on its head and the server sends the position of
objects which are in the sight of the camera. The angleof sight had been 360 degrees until the world competition 2008 and agents could see all objects from everywhere. However, in the 2009 competition, the angle ofthe sight was changed to limit to 120 degrees and thus
agents have to turn neck and search for objects to receive©ISAROB 2011

Fig. 2. The visual debugger

935

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

2. Debug Message
The one-line message from the soccer agent includes
four pieces of information, that is, game state, visual information, joint angles, and gyro information. We explain each of the above information in detail.
• Game StateThe game state information consists of the game
time, the playmode, each team’s name, and eachteam’s score.
• Visual Information
The agent receives the visual information from thesoccer server in three dimensional polar coordinates. The visual information is then sent to thevisual debugger as a three dimensional vector in
Cartesian coordinates. This information provides
the position of visible objects.

Fig. 3. Visible object in the soccer field

4. Pose of the Agent
The debugger can also draw the pose of the agent andthe positional relation between agent’s feet and the ballas shown in Figs. 4 and 5. In Fig. 4, the left image
shows each joint position from the frontal view and theright sagittal view. In both images the pose of the agent
is shown by nodes and edges. The top circle indicatesthe head, the under circles indicate neck joint and torso.
Square nodes indicate leg, heel, and toe joints. In theleft images, the right and left circles means arm joints.
These figures are drawn based on the message from theagent that contains the joint angles of each time step.
Shaded area is the eyesight of the agent. In the left image of Fig. 4, the horizontal axis and the soccer field areassumed parallel, and in the right image of Fig. 4, thevertical axis and the soccer field are orthogonal. The developer of the agent checks the pose of the agent fromthe two images in Fig. 4.
The positional relation between agent’s feet and theball is shown in Fig. 5. Figure 5 is the mapping of eachjoint position to the horizontal plane. The white circle
indicates the ball, the black circles indicate the nodes of
the agent; in this figure they represent the upper bodyof the agent, and the black squares indicate the lower
body of the agent. When the ball is in the eyesight of
the agent but is too far to draw in the display, a white
triangle locator appears to indicate the direction toward
the ball.

• Joint Angles
Joint angle includes the joint angles of the agent at
the current time step. One message format corresponds to one joint angle.
• Gyro Information
Gyro information includes the value of the gyro
sensor. The gyro sensor shows the angular velocity
of agent’s torso.
3. Visible Objects in the Soccer Field
The debugger shows visible objects in the soccer fieldwithin the agent’s view cone as explained in subsection
3.2. The agent sends the debugger a message that contains field information about whether each object is in
the eyesight of the agent or not and if it is visible, its
position is also included in the message. The positionof visible objects is given in the local coordinates of theagent. The debugger converts it into the absolute coordinates (i.e., the origin is set to the center of the field)
and draws it in the overhead view of the soccer field.
Figure 3 shows an example of the overhead view of thesoccer field. In Fig. 3, the arrow in the rightside of the
field indicates the position and direction of the agent’s
torso, two lines from an arrow to rightside and to topof the figure shows the eyesight of the agent, the whitecircle indicates the ball, the light gray square indicates a
teammate player, and the dark gray square indicates anopponent player. The debugger draws each of the above
objects when it is in the agent’s eyesight. The positionof landmarks such as flags and goal posts are describedby gray circles or black circles. The positional relation
among landmarks are pre-specified by the soccer server
according to the official rule of the league. Therefore, if
at least two landmarks are visible to the agent, the position of landmarks which are out of sight can be identified. Lighter circles in Fig. 3 are visible, while black
circles are out of sight.

©ISAROB 2011

Fig. 4. Pose of the agent

936

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

IV. CONCLUSION
In this paper, we introduced a visual debugger thathelps develop a soccer agent of the RoboCup soccer 3D
simulation league. We also showed a manually-operable
agent by game-pad for debugging. Using the debugger,
the visible object on the soccer field and agent’s internal state are graphically shown. The debugger can also
replay the game on itself by loading a debug message
file so that we can check agent’s behavior in detail. The
source code of the visual debugger will be available in
the near future. We hope the debugger makes the agent
development more efficient for all teams of soccer simulation 3D league.

Fig. 5. Position of feet and ball

5. Replay Function
The debugger can replay the game without running
the soccer server by loading a text file which includes
all debug messages sent by the agent during a game.
We call this text file “debug message file” in this paper.
The main difference between a debug message file anda log file is that the debug message file contains the internal information from the viewpoint of an agent whilea log file just records the joint angles each time step.
While replaying a game from a debug message file, extra operations such as pause, go-to-next-step, or go-toprevious-step are available. By using these functions,
we can examine an agent’s behavior in detail during agame. These functions are also available while a soccer
match is played.

REFERENCES[1] Stone P (2000), Layered learning in Multiagent
Systems: A Winning Approach to Robotic Soccer.
MIT Press[2] Reis L P, Lau N, Oliveira E C (2001), Situation Based Strategic Positioning for Coordinatinga Team of Homogeneous Agents. Balancing Reactivity and Social Deliberation in Multi-Agent Systems, LNCS 2103:175–197[3] Grasemann U, Stronger D, Stone P (2007), A
Neural Network-Based Approach to Robot Motion
Control. RoboCup 2007: Robot Soccer World Cup
XI:480–487

6. Manually-Operable Agent[4] Luke S, Hohn C, Farris J, et al. (1997), CoEvolving Soccer Softbot Team Coordination with
Genetic Programming. RoboCup 1997: Robot
Soccer World Cup I:398–411

The behavior of an agent sometimes requires the interaction with other agents. Passing the ball to a teammate and avoiding opponent agents that are approaching
the agent are examples of such behaviors. Even though
there are released binaries of those teams who participated in previous competitions, it is hard to examine
those behavior using the released binaries because soccer agents are autonomously moving and never manually controlled. To overcome this problem, it is helpful
to prepare an agent that can be manually controlled. For
this purpose, we have developed a manually-operable
soccer agent. To control the agent, we use a USB Gamepad with 12 buttons and 2 sticks. Figure 6 shows thegame-pad for manually operating the agent.

[5] Nakashima T, Udo M, Ishibuchi H (2003), A
Fuzzy Reinforcement Learning for a Ball Interception Problem. RoboCup 2003: Robot Soccer
World Cup VII:559–567[6] Kogler M, Obst O (2003), Simulation League: The
Next Generation. RoboCup 2003: Robot Soccer
World Cup VII: 458–469[7] Spark - A generic physical simulator, Sourceforge
project,
http://sourceforge.net/projects/simspark/files[8] rc-tools, released web page,
http://rctools.sourceforge.jp/pukiwiki/[9] SoccerScope2003, released web page,
http://ne.cs.uec.ac.jp/˜newone/SoccerScope2003[10] Nao, Aldebaran web page,
http://www.aldebaran-robotics.com

Fig. 6. Game-pad©ISAROB 2011937

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Development of Pulse Control Type MEMS Micro Robot with Hardware
Neural Network
Kazuto Okazaki†, Tatsuya Ogiwara†, Dongshin Yang†, Kentaro Sakata†, Ken Saito†1, Yoshifumi Sekine††2and Fumio Uchikoba†3†

Department of Precision Machinery Engineering, College of Science & Technology, Nihon University††
Department of Electronics & Science, College of Science & Technology, Nihon University
7-24-1, Narashinodai, Funabashi-shi, Chiba, Japan(Tel. +81-47-469-5343)
1
Email:kensaito@eme.cst.nihon-u.ac.jp
2
Email:ysekine@ecs.cst.nihon-u.ac.jp
3
Email:uchikoba@eme.cst.nihon-u.ac.jp

Abstract: This paper presents the MEMS (Micro Electro Mechanical Systems) micro robot which demonstrates thelocomotion, controlled by the HNN (Hardware Neural Network). The size of the micro robot fabricated by the MEMS
technology was 4×4×3.5 mm. The frame of the robot was made of silicon wafer, equipped with the rotary type actuator,
the link mechanism and 6 legs. The rotary type actuator generated the rotational movement by applying the electricalcurrent to artificial muscle wires. The locomotion of the micro robot was obtained by the rotation of the rotary type
actuator. Same as the living organisms HNN realized the robot control without using any software programs, A/D
converters, nor additional driving circuits. CPG (Central Pattern Generator) model was implemented as a HNN systemin this micro robot to emulate the locomotion pattern. The MEMS micro robot emulated the locomotion method andthe neural network of the insect by the rotary type actuator, link mechanism and HNN. The micro robot performed
forward and backward locomotion, and also switched the direction by inputting the external trigger pulse. The
locomotion speed was 19.5 mm/min and the step width was 1.3 mm.
Keywords: Micro Robot, MEMS, Hardware Neural Network, Pulse-Type Hardware Neuron Pair Model, CPG
model

about implement the neural networks of lower
organisms to HNN for the purpose to construct theflexible micro robot system.
In this paper, the millimeter size micro robot is
proposed. The fabrication process is based on MEMS
technology. Also, the locomotion system is controlled
by HNN which is composed of CPG model. We will
explain about the micro robot system from the view
point of the mechanical system, the fabrication process
by MEMS technology, the HNN system, and theevaluation of the action of the robot.

I. INTRODUCTION
Many studies have intensively been done on micro
robots for several applications such as medical field,
precise manipulations, and so on. Although theminiaturization of the robot has conventionally been
progressed by mechanical machining and assembles,some difficulty has appeared in order to achieve further
miniaturizations. Instead of the conventional
mechanical machining, MEMS technology based on the
IC production lines has been studied for making thecomponents of the micro robot [1],[2].
Programmed control by a microprocessor has been
the dominant system among the robot control. However,some advanced studies of artificial neural network havebeen paid attention for applying to robots. A lot of
studies have reported both on software models andhardware models [3]-[6]. It is difficult to implement thecomplicate neural network such as higher animal to
artificial neural network systems. In contrast, the neuralnetworks of lower organisms realize the excellent
sensory information processing and locomotion control
by simple neural networks. Therefore, we are studying©ISAROB 2011

II. MECHANISM OF MEMS MICRO ROBOTThe design of the fabricated MEMS micro robot isshown in Fig.1. The number of the legs of the micro
robot is 6. The structure and the step pattern of the robot
is emulated those of the insect. The micro robot consistsof frame parts, rotary type actuators and linkmechanisms. The rotary type actuator generates thelocomotion of the robot by applying the electricalcurrent to the artificial muscle wires [4]. The size of the938

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

0.225mm

Artificial muscle wires

0.46mm
Frame

2.0mm

0.21mm0.45mm

3.5mm

2.84mm0.23mmt=0.2mm
0.6mm(a) Size of link
4mm

4mm

0.5mm
(b) Size of leg

Links

Fig.1 Design of the fabricated MEMS micro robot.
robot fabricated by the MEMS technology is designed
as 4×4×3.5 mm.
The rotary type actuator is shown in Fig.2. The
rotary type actuator is composed of the rotor and 4 pieceof artificial muscle wire. The frame is assembled fromthe front frame, rear frame, center frame, and top frame.
The both ends of artificial muscle wires are fixed to theframe. The wire shrinks at high temperature and extends
at low temperature. In this study, the wire is heated byelectrical current flowing, and cooled by stopping theflowing. The rotational movement of the each actuator
is obtained by changing the flowing sequence.
The link mechanism is shown in Fig.3. The front leg
and the rear leg are connected to the center leg by link
bars, respectively. The center leg is connected to therotor by the shaft. Therefore, the rotational phase issame as the rotor. On the contrary, the other two legs areconnected by the link bar that generates 180 degree
phase sift. Also, backward step is obtained by thecounter rotation of the actuator.
Figure 4 shows the schematic diagram oflocomotion method. In the case of heating the artificial
Top frame
2.2mm

Front leg

Rear legCenter leg(c) Fabricated link mechanism

Fig.3 Design of the link mechanism.
muscle wire from EI1 to EI4, the micro robot moves
forward. In contrast, heating the artificial muscle wirefrom EI4 to EI1, the robot moves backward. The
locomotion pattern is 180 degree phase sift on each side
to represent the locomotion of insect.

III. COMPONENT FABRICATION BY
MEMS TECHNOLOGY
The fabrication process of the component of micro
robot was based on MEMS technology. The designed
shape was formed by the photolithographic process. ICP
dry etching process realized high aspect ratio machining.
The components of the MEMS micro robot are
shown in Fig.5. The starting material was silicon wafer
with various thickness (100, 200, 385, 500 µm) which
was used depending on the shape.
The size of the fabricated component is shown in
Fig.6. The designed diameter of the joint was 450 µm.
The designed diameter of the receptacle was 460 µm.
The error of the joint was 0.23 %, and the error ofreceptacle was 1.38 %. Therefore, the link mechanism
was made within the error of ± 2%. The link bars andArtificial muscle wires

3.8mm

Rear frame

3.2mm

Direction of locomotion
3.2mm
EI4
3.8mm
Center frame

Shaft Rotor

EI3
EI2
Power-off

2.2mm

1

Front frame

4

2
3
4
Shrunk artificial muscle wires
Extended artificial muscle wires

Fig.4 Schematic diagram of locomotion method.

Fig.2 Design of the rotary type actuator.

©ISAROB 2011

EI1

939

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Excitatory neuron model

Inhibitory neuron model

Excitatory synaptic model

Cell body model

RSE2(a)Frame(b)Rotor (c)Legs(d)Links (e)Spacers

Vout(t)
RSE4

CSE
RSE1

Fig.5 Components of the MEMS micro robot.

RSE3

CG
RCin

RG
VA

CG
M1
RM

CM
b

RG

M2

RSI6

R a CM
VA

RSI4

Rb

RC

RSI5

CSI
RSI3
RSI1

cia

(b) Receptacle of the link
Cell body model

Fig.6 Size of the fabricated component.

Inhibitory synaptic model

Fig.8 Circuit diagram of pulse-type hardware
neuron pair model.

the legs were assembled in the gap between 15.6 µm.
Figure 7 shows the fabricated MEMS micro robot.
The size of our MEMS micro robot was 4×4×3.5 mm,same as designed size as shown in Fig.1. The wire fromthe micro robot was connected to the HNN.

adder includes an inverting amplifier using an
operational amplifier (Op-amp), the amplification factorof the inverting amplifier varies according to synaptic
weight. The temporal summation characteristics are
realized by the Op-amp RC integrator. The circuit
parameters of the excitatory synaptic circuit were as
follows: RSE1=RSE2=RSE3=RSE4=1 MΩ, CSE=1 pF. The
inhibitory synaptic circuit is obtained by reversing theoutput of the excitatory synaptic circuit. The excitatory
synaptic circuit and the inhibitory synaptic circuit are
single input but in the CPG model we use multi-inputs
including excitatory inputs and the inhibitory inputs.
The circuit parameters of the inhibitory synaptic circuit
were as follows: RSI1=RSI2=RSI3=RSI4= RSI5=RSI6=1 MΩ,
CSI=1pF. The cell body circuit consists of a voltage
control type negative resistance and an equivalent
inductance, a membrane capacitor CM. The voltage
control type negative resistance circuit with theequivalent inductance consists of a n-channel
enchantment-mode
MOSFET,ap-channel
enchantment-mode MOSFET, a voltage source VA,
resistors RM, RG, and a capacitor CG. The cell body
circuit has the negative resistance property whichchanges with time like a biological neuron, and enables
the generation of a continuous action potential by a selfexcited oscillation and a separately-excited oscillation.
Moreover, the cell body circuit can switch between bothoscillations by changing VA. The separately-excitedoscillation occurs by the direct-current voltage stimulusor the pulse train stimulus. The circuit parameters of thecell body circuit were as follows: CM=470 nF, CG=4.7

IV. HARDWARE NEURAL NETWORK FOR
MOTION CONTROLThe HNN generates driving pulse of the rotary type
actuator without using software programs, A/D
converters, nor additional driving circuits. The pulsetype hardware neuron pair model compose of theneuron pair of excitatory and inhibitory and each neuron
compose of the synaptic circuit and the cell body circuit.
Figure 8 shows the circuit diagram of the pulse-type
hardware neuron pair model. The synaptic circuit has
the spatio-temporal summation characteristics similar to
those of living organisms. The spatial summation
characteristics are realized by the adder. Moreover, the

3.5mm

4mm
4mm
Fig.7 Fabricated MEMS micro robot.

©ISAROB 2011

Ra R

RSI2

451.03µm(a) Joint of the leg

M1

RM

466.63µm

M2

940

EI1
E1

EI2

I1

I2

I4

I3

E2

DrivercircuitTime 1s/div(a) Forward locomotion
Drivercircuit

E4

E3

EI4

EI3

DrivercircuitTime 1s/div(b) Backward locomotion

Fig.10 Output waveform of the generated locomotion
pattern
waveform of the locomotion pattern was generated by
the CPG model without using any software programs.
The fabricated micro robot was performed forward andbackward locomotion, and also the locomotion switchedby inputting the external trigger pulse. The locomotion
speed was 19.5 mm/min and the step width was 1.3 mm.

Fig.9 Schematic diagram of the CPG model.
µF, RM=10 kΩ, RG=680 kΩ, Ra=20 kΩ, Rb=15 kΩ. The
voltage source VA=3.5 V.
Figure 9 shows the schematic diagram of CPG
model. In this study, four sets of the neuron pair wereconnected inhibitory then the four output ports wereextracted. The CPG model consists of four pulse-type
hardware neuron pair models (EI1, EI2, EI3, and EI4). E
and I represent an excitatory neuron circuit and an
inhibitory neuron circuit, respectively. The solid circle
indicates an inhibitory connection. We denote the pulsetype hardware neuron pair model by EI. The hardware
neuron pair model was mounted on a print circuit board
and source power was supplied externally.
Figure 10 shows the output waveform of thegenerated locomotion pattern. It is shown that ourconstructed CPG model can output the waveform of theforward locomotion and the backward locomotion
corresponding to Fig.4. Thus, the constructed CPG
model is effective to generate the locomotion of the
MEMS micro robot. As a result, our fabricated robot
performed forward and backward locomotion, and also
the locomotion switched by inputting the external
trigger pulse. The locomotion speed was 19.5 mm/min
and the step width was 1.3 mm.

ACKNOWLEDGMENTS
The fabrication of the MEMS micro robot was
supported by Research Center for Micro Functional
Devices, Nihon University. We appreciate the support.

REFERENCES[1] E. Edqvist, N Snis, R C. Mohr et al(2009),
Evaluation of building technology for mass producible
millimeter-sized robots using flexible rinted circuitboards. J. Micromech. Microeng 19:11[2] H. Suematsu, K. Kobayashi, R. Ishii et al (2009),
MEMS Type Micro Robot with Artificial Inteligence
System. Proceedings of International Conference on
Electronics Packaging:975-978[3] Ken Saito, Kazuto Okazaki, Tomonari Kawakami etal (2010), Pulse-Type Hardware CPG Model for MEMS
Type Micro Robot. AVLSIWS 2010:201[4] K. Matsuoka(1987), Mechanism of Frequency andPattern Control in the Neural Rhythm Generators.
Biological Cybernetics 56:345-353[5] T. Ikemoto, H. Nagashino, Y. kinouchi, T.
Yoshinaga(1997), Oscillatory Mode Transitions in a
Four Coupled Neural Oscillator Model(in Japanese).
International Symposium on Nonlinear Theory and its
Applications:561-564[6] K. Nakada, T. Asai, Y. Amemiya(2003), An Analog
CMOS Central Pattern Generator for Interlimb
Coordination in Quadruped Locomotion. IEEE
Transaction on Neural Networks 14:1356-1365[7] Dai Homma (2003), “Metal Artificial Muscle'BioMetal Fiber' (in Japanese). RSJ Vol. 21:22-24

V. CONCLUSIONSIn this paper, we fabricated the 4×4×3.5 mm size
micro robot by MEMS technology. The locomotion of
micro robot was controlled by the HNN. The CPG
model was implemented as a HNN system in this micro
robot to emulate the locomotion pattern. The MEMS
micro robot emulated the locomotion method and theneural network of the insect by the rotary type actuator,
link mechanism and CPG model. As a result, the output©ISAROB 2011

Output voltage 5V/div

DrivercircuitOutput voltage 5V/div

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011941

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Design of the transmitting device for motion control of robots
Masaharu Komori, Jungchul Kang, Fumi Takeoka and Yukihiko Kimura
Department of Mechanical Engineering and Science, Kyoto University
Yoshidahonmachi Sakyo Kyoto 606-8501 Japan(Tel : 81-75-753-5858; Fax : 81-75-753-5858)(komorim@me.kyoto-u.ac.jp)
Abstract: It is possible for the motors to realize some range of speed or torque but the range is limited to some degree
considering the size, weight and cost of the motors. Moving robots used in the industrial fields are required to realize
outputting a large force in some cases, and moving with high velocity in other cases. Therefore a velocity variation
system is an ideal method in which the velocity ratio between input and output shafts is changed. However, the motion
transmission from the input shaft to the output shaft is interrupted during the velocity ratio variation process in theconventional velocity variation system. In order to solve this problem, velocity variation method that can transmit
motion precisely is proposed. The principle of this velocity ratio variation method is explained and its components are
considered. The required conditions to the design of the transmitting element of this variation device are clarified inorder to realize transmitting motion precisely. By this analysis, it is confirmed that the proposed ratio variation method
can change the velocity ratio while transmitting the motion precisely between the input shaft and the output shaft.
Keywords: ratio variation

I. INTRODUCTION
II. VELOCITY VARIATION METHOD

It is possible for the motors to realize some range of
speed or torque but the range is limited to some degree
considering the size, weight and cost of the motors.
Moving robots used in the industrial fields are required
to realize both outputting a large force when they carryloads and moving with high velocity when they move tothe destination in order to shorten the moving period.
Therefore a velocity variation system is an ideal methodin this point. Velocity variation devises using gears arewidely used in the industrial fields. Cylindrical gears
such as spur gear have advantages in high torquecapacity, precise rotation transmission and highefficiency and that is the reason why cylindrical gears
such as spur gears are used in a variety of devices. In ageared transmission, it is needed to change the working
gear pairs to vary the velocity of the transmission.
However, the motion transmission from the input shaft
to the output shaft is interrupted during this process.
This leads to an important problem in the field of robots.
In order to solve this problem, velocity variation methodthat can transmit motion precisely is proposed in our
research. In this method, each transmitting element mustsatisfy each condition, which is different among eachelement. The required conditions to the design of thetransmitting element are clarified in order to realizetransmitting motion precisely. By this analysis, it isconfirmed that the proposed ratio variation method canchange the velocity ratio while transmitting the motion
precisely between the input shaft and the output shaft.

©ISAROB 2011

In the proposed velocity ratio variation method,
there are two shafts, i.e. input shaft and output shaft,three gear pairs A, B and C, and three tooth clutches TA,
TB and TC as shown in Fig.1. The gear pairs A and B are
composed of typical circular gears, and gear pair C iscomposed of noncircular gears. Gears Ao, Bo and Co canbe connected to the output shaft by engaging theclutches TA, TB and TC. In contrast, gears Ai, Bi and Ci
are fixed to the input shaft. Suppose that the velocity
ratio of the gear pair A is rA, and that of the gear pair B
is rB. In the pitch curve of noncircular gear pair C, there
are four sections. The pitch curve of the noncircular
gear pair C is partly same as that of gear pair A, andpartly same as that of gear pair B. Those parts of pitch
curve are smoothly connected. In the four sections, thevelocity ratio is constant at rA, it changes to rB, it isconstant at rB, and it changes to rA respectively.
The proposed velocity ratio variation process fromrA to rB is explained. Under the condition that the clutch
TA is engaged, and the other clutches are disengaged,
the velocity ratio is rA. When the meshing of gear pair C
comes into the section corresponding to rA, the clutch TC
is engaged, and, after the engagement of clutch TC, theclutch TA is disengaged. Then the meshing of gear pair
C transits to the section corresponding to rB. In thesection corresponding to rB, the clutch TB is engaged,
and, after the engagement of clutch TB, the clutch TC isdisengaged. Through this process, the transition from rA

942

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Thirdly, the output shaft is rotated NB times, and thenclutch TC is engaged. To engage clutch TC, considering
the difference between the rotational angle of the output
shaft and that of gear Co, following condition must be
satisfied, where J3 is arbitrary integer.
2πJ 3
(3)
2πN B ( rB − 1) =to rB is completed. At all steps in this process, at least
one of the clutches is engaged. Therefore, the rotationalmotion is precisely transmitted from the input shaft tothe output shaft.

ndFinally, the output shaft is rotated by π, and then clutch
TA is engaged. To engage clutch TA, considering thedifference between the rotational angle of the output
shaft and that of gear Ao, following condition must be
satisfied, where J4 is arbitrary integer.
2πN B (rB − rA ) + π (1 − rA ) =III. CONDITIONS OF TRANSMITTINGELEMENTSnd (1 − rA )∈Z
2
nd (1 − rB )∈Z
2In the proposed velocity ratio variation method,
tooth clutch is used to transmit the motion precisely.
However, to use the tooth clutch, it is required that thedog teeth on the output shaft side and that on the gearside are in appropriate condition so that they can bemeshed. Suppose the numbers of the dog teeth of tooth
clutches TA, TB and TC are nd. The case that the velocity
ratio is changed from rA to rB, and returned form rB to rA
is considered. At the initial state, all of tooth clutches TA,
TB and TC can be engaged. In addition, gear pair C
meshes in the middle of the section corresponding tovelocity ratio of gear pair A.
First, suppose that the input shaft is rotated NA times,
and then the velocity ratio variation process starts, inother words, clutch TC is engaged, where NA is natural
number. After the rotation of NA, the difference between
the rotational angle of the output shaft and that of gear
Co is 2πN A ( rA − 1) . To engage clutch TC, this value must
be integral multiple of pitch between the teeth of the
clutch. This condition is expressed as follows, where J1
is arbitrary integer.
2πJ 1
(1)
2πN A ( rA − 1) =In the system proposed in our research, precisetransmission of the rotation from the input shaft to
output shaft is realized. The required conditions to thedesign of the transmitting element are clarified. It is
confirmed that the proposed ratio variation method canchange the velocity ratio while transmitting the motion
precisely between the input shaft and the output shaft.

ACKNOWLEDGEMENT
This study was supported by Industrial Technology
Research Grant Program in 2009 from New Energy andIndustrial Technology Development Organization(NEDO) of Japan.

Secondary, the output shaft is rotated by π, and thenclutch TB is engaged. To engage clutch TB, considering
the difference between the rotational angle of the output
shaft and that of gear Bo, following condition must be
satisfied, where J2 is arbitrary integer.

©ISAROB 2011

(5)

VI. CONCLUSION

nd

2πJ 2
nd

(4)

These equations are organized, and the following
conditions to the design of the transmitting element areclarified, where Z is set of integer.
nd (rA − rB ) ∈ Z

Fig.1. Structure of the Velocity Ratio Variation Device

2πN A (rA − rB ) + π (1 − rB ) =

2πJ 4
nd

(2)

943

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Construction and basic performance tests of underwater monitoring network
Chunhu Liu, *Bin Fu, Han Zhang, Lian Lian
Shanghai Jiao Tong University* b_fu@sjtu.edu.cn，+(86)-21-34207293
Abstract:

With the rapid development of economy, environmental pollution has become one of the major problems incoastal areas and cities along the river. Real-time observation of the water quality along the river has been
considered as an efficient way to control wastewater emission and manage environment of water quality.
Traditional ways to observe ocean environment, including satellite telemetry, radar, investigation ship, ocean
observation station and etc, are not applicable for water quality observation along river because of their high cost,
Poor real-time, low accuracy and so on. Based on Wireless Sensor Networks, the study discussed in this paper
proposes a new observation system using under-water multisensory information. After processing multisensory
data of each sensor the system transmits it to hub node through wireless sensor networks, and then transmits it
to land data center through GPRS wireless network. In order to check the basic performance of this system, theauthors have completed the node positioning experiment based on GPS module and the communication
experiment based on ZigBee. This paper reports the hardware design and the experimental results.
Keywords: Wireless Sensor Networks(WSN), Underwater, ZigBee, Communication
the influence of antenna swaying on communication
performance and organize a network in ahundred-meter range area between several hundred
meters. In addition, the properties of networkorganization
mode,
communication
protocol,optimum topology structure and etc, are not clear yet
when the height between antenna and water surface
is low. Therefore, based on theoretical calculation andanalysis on extreme conditions mentioned above, thepaper proposes a design scheme of system networkorganization, and reports the test of basic
performances of the system by installing it in thewater area of Shanghai, China.

1. Introduction
With the rapid development of economy in theseyears, environmental pollution has become a major
fact of invading people’s life and influencing global
climate change. Especially, wastewater emission andleakage from the cities along the river and ocean leadto pollution of water quality, and it could bring global
environmental deterioration impact. Nowadays, themajor ways to observe ocean environment, mainly
depends on satellite telemetry, radar, investigation
ship, stationery ocean observation station and etc,
which are not applicable for water quality observation
along river because of their high cost, poor realtimeness, low accuracy and so on. The study
introduces WSN technology of land field toobservation system of water quality. By setting
continuous underwater sensor nodes, the water
quality environment in a large area can be real-timeobserved. By setting up wireless communication
network among multinodes networks, physicalinformation, including water temperature, PH value,
oil leakage, noise and etc, can be converted toelectrical signal, and then transmitted to surface unit.
In this system, each unit is not only a node, but also adata transmission relay of other nodes. Node’s
information can be exchanged and transmitted to landdata management center through wirelesscommunication in order to assure that observed data
is real-time, actual, universal and continuous.
However, the biggest technology difficulty of gatheringand sending underwater information is to overcome©ISAROB 2011

2. wireless sensor network design
2.1 system summary
As shown in figure 1, the system is composed of somesurface nodes, hub nodes, and land data terminal. The
carrier of each sensor node is buoy. The buoy carries
surface wireless communication unit, data processing
unit and many underwater sensors. The nodes
communicate with each other by wirelesscommunication, and they are composed of wirelesssensor network with node topology structure. The
network converges information at hub node. The hub
node is located along river shore. First, it transmits
information by GPRS network of cell phone， and thentransmits the information to land data terminal
through remote wireless communication.

944

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Buoy
Sea surface

Hub node

Sensor 1
Sensor 2
Sensor n

Sea bedWater area

AB
-12
3

land

000000 0 00 00000 0000
000000 0 00 00000 0000
000000 0 00 00000 0000
000000 0 00 00000 0000
000000 0 00 00000 0000

000 00000 0000 0000 00 0 00000 0000 00000 0
000 00000 0000 0000 00 0 00000 0000 00000 0
000 00000 0000 0000 00 0 00000 0000 00000 0
000 00000 0000 0000 00 0 00000 0000 00000 0
000 00000 0000 0000 00 0 00000 0000 00000 0

0 00
0 00
0 00
0 00
0 00

000000 0 00 00000 0000
000000 0 00 00000 0000
000000 0 00 00000 0000
000000 0 00 00000 0000

000 00000 0000 0000 00 0 00000 0000 00000 0
000 00000 0000 0000 00 0 00000 0000 00000 0
000 00000 0000 0000 00 0 00000 0000 00000 0
000 00000 0000 0000 00 0 00000 0000 00000 0

0 00
0 00
0 00
0 00

000000 0 00 00000 0000
000000 0 00 00000 0000
000000 0 00 00000 0000
000000 0 00 00000 0000
000000 0 00 00000 0000

000 00000 0000 0000 00 0 00000 0000 00000 0
000 00000 0000 0000 00 0 00000 0000 00000 0
000 00000 0000 0000 00 0 00000 0000 00000 0
000 00000 0000 0000 00 0 00000 0000 00000 0
000 00000 0000 0000 00 0 00000 0000 00000 0

0 00
0 00
0 00
0 00
0 00

Data terminal

Fig. 1 System Composition(this module is Only installed

GPS Module&Antenna

in hub node）
Wireless communication

GPRS communication

modulemoduleObserving and controlling

Solar paddle

Sensor 1

unit

Sensor 2

Sensor N

Processing Circuit

Memory module

Memory unit

Storage battery
Fig. 2 Schematic Diagram of Surface node
2.2 sensor node design
As shown in figure 2, in our design, each node in thesensor network has an observing and controlling unit,
memory unit, battery unit cell of solar energy, GPS©ISAROB 2011

positioning unit, wireless communication unit andsome sensor interfaces. Observing and controlling unit
is capable of data collecting, digital control, datatransmission and so on. Hub node includes remote

945

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

wireless communication unit which communicateswith land data terminal. Micro embedded GPS modulewith Patch Antenna is used in Node positioning in the
study; Zigbee is used in wireless communication
among nodes because it is highly reliable, free andnetwork organizable; Because GPRS mode is low costand has a wide coverage of network, it is used in along-distance wireless communication between nodes
and land data terminal. Power supply of nodes takes
form of combination of solar energy and storage
battery; in order to assure the integrity of collected
data, each node is self containing. Which in otherwords, the collected data is saved before it isretransmitted. In order to collect data from different
underwater sensors, observing and controlling unit
contains modulation function of analog signals andserial port communication function; the collected dataneed to be preliminarily processed to reduce
redundancy node.
2.3 Deign of data transmission rule
ZigBee module itself doesn’t support error correction
and automatic repetition mechanism, so in order toincrease success rate of data transmission, the systemneed the following functions:a) short frame transmission ruleb) automatic error correction and repetition function
For example, transmitting end transmits data todesignated node, and then receiving end responds
after receiving data; if latency is overtime,transmitting end will retransmit the data. If it doesn’t
receive any response from receiving end after 3-time
retransmitting, the system will consider it a data-link
failure between the transmitting end and receiving
end and abort current data transmission; at the sametime, the system saves all data for the subsequent
data analysis and processing.
Data of each node are composed of node position
information and sensor information. Each node isequipped with GPS positioning module, and canposition each node itself. Positioning information istransmitted in a format as shown in figure 3(a), andthe transmission time interval of this format is longer
than the format shown in figure 3(b). If position of thenode changes a lot, the position information will betransmitted to control center where staff can timely
process it. Comparatively, data information of eachsensor is transmitted in a data format as shown informat 3(b.)⑥E: East Longitude, W: West Longitude;⑦CRC16 Verification; ⑧End of Frame.
(a) Format of Position Information①Address; ②Sensor Data Flag; ③Frame Info.；④ Sensor data; ⑤ CRC16 Verification; ⑥ End of
Frame.
(b) Data Information of Sensor Information
Fig.3 Data Format
3. Performance Experiment
3.1 Objective
Node positioning, efficient working distance andcommunication between nodes are key technologies
in this system. The following questions remain to bevalidated in this experiment:a) Communication performance of wireless sensor net
work in a open water area.
b) Positioning accuracy of the node drifting within asmall area
3.2 Contentsa) The neighboring water area in shanghai is taken to
be the subject in this experiment. An experiment pointis respectively chosen from Huangpu River, East China
Sea and Qiantang River to test positioning accuracy of
sensor node.
b) Take offshore area of East China Sea for an example,
actual efficient communication distance of ZigBee ismeasured and compared with theoretical distancevalue. Also, the effect of antenna’s oscillation angle
change on communication distance is tested.
c) The relation between transmission bit error rate
and transmission distance of ZigBee mode is tested.
Short-frame and long-frame transmission error rate
are also reported.
3.3 Resultsa) Positioning Accuracy of Node
The theoretical positioning accuracy of GPS module
used in this experiment is 3[m], and actual positioning
accuracy of the 3 tested point are respectively 9.47[m],4.77[m], and 3.86[m]. The positioning accuracy ofhorizontal direction is shown as figure 4. The node
positioning error is within 10[m]，which is goodenough to meet the need of the system.
b) Error Rate
Table 1 shows comparison of error rate between
long-frame and short-frame communication when 10K-byte date is transmitted. The data length of long
frame is more than 30 bytes, while the data length of①Address; ②Flag of GPS data; ③Latitude;④N:North latitude, S:South Latitude; ⑤Longitude;©ISAROB 2011946

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011short frame is no more than 10 bytes. Apparently,
error rate of short frame communication is much
lower than that of long frame’s communication.
Efficient communication distance is about 600[m]when error rate is less than 4‰.

c) Efficient Communication Distance
Theoretically,, the free-space power received byreceiving unit is determined by equation(1)[1,2]Where，

[dB] is transmitting power,

receiving power,[dB] and….(1)[dB] is[dB]are respectively

antenna gain of transmitting and receiving,[m] isthe transmission distance,  [m] is wavelength.
Then transmission loss[dB] can be obtained by[2]
the following equation .
 ….(2)In this experiment,(a) Qiantang River (2DRM=9.47[m])receiver is -94[dBm],=104[dB]. Then theoreticalvalue of transmission distance is 1658[m]. Not only ancertain margin is allowed, coefficient of transmission
attenuation is bigger due to big air humidity aroundsea. Besides, energy loss is caused by imperfecttransmitting circuit and receiving circuit. As revealedby actual measurement, when bit error rate is lessthan 4‰, efficient communication distance is about
600[m] which is only 36% of theoretical transmission
distance. However, we can increase efficient
communication distance by boosting antenna gain ortransmitting power.
In additional, because of the antenna directivity of
ZigBee module, antenna mounting height have to be
increased to raise communication efficiency in a roughwave environment(b) Huangpu River (2DRM=4.77[m])4. Summary
The key technologies, which are used for datacollecting, data processing, node positioning,communication among nodes and etc, are proposed inthis study and have been validated throughexperiments. And we are trying to go on with along-term observation test about water quality. In
additional, the technologies mentioned are expected
to be used in intelligent observation with large
information transmission such as underwater
acoustics, video and etc.
ACKNOWLEDGMENTS: The study is supported by
Public Welfare Project of Science and Technological
Department, Zhejiang Province, China.

(c) East China SEA (2DRM=3.86[m])Fig.4 Positioning accuracy of sensor node
Table 1 Comparison on bit error rate between long
frame and short frame communication
Distance

Long frame

Short frame

300

0

0

400

2.4e-41.8e-4500

1.3e-38.4e-4600

2.4e-2

3.6e-3700

6.3e-24.3e-3800

2.4e-16.2e-3©ISAROB 2011=10[dBm] ， sensitivity ofReferences
1. Rappaport, Theodore S, Wireless Communications
Principles and Practice, Prentice Hall, 2001, P70-74.
2. Shreharsha Rao, Estimating the ZigBeetransmission-range ISM band, EDN, May 2007, P67-72

947

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

On the Use of Human Instruction for Improving the Behavior of RoboCup
Soccer Agents
Yosuke Nakamura and Tomoharu Nakashima
Osaka Prefecture University, 1-1 Gakuen-cho,Nakaku,Sakai,Osaka(Tel : 81-72-254-9351; Fax : 81-72-254-9915)(nakashi@cs.osakafu-u.ac.jp)

Abstract: In this paper, we propose a behavior generation approach from human instruction to improve the strategy
of RoboCup soccer 3D simulation team. Many teams implement their strategies based on the programmers’ ownknowledge about soccer. That is, the programmers have to write action rules that cover any situations of the soccer
field. Although it is clear that this is not the best approach, there are only a few research works that tackle this problem.
In this paper, we solve this problem using human instruction to improve the manually implemented behavior of soccer
robots. It is shown that the team performance is improved by the generated rules by this approach.
Keywords: RoboCup, soccer robot, learning, multi-agent system

I. INTRODUCTION

optimal defensive positioning as a multi-criteria assignment problem and demonstrated that pareto-optimal collaborative positioning yields good results. Kalyanakrishnan and Stone [3] introduced a policy search method
for a keepaway task, which is a popular benchmark for
multiagent reinforcement learning from the simulationsoccer domain.

RoboCup is an international project which aims atbuilding autonomous soccer robots. RoboCup has somemain leagues such as Soccer, Rescue, @Home and Junior. We focus on the soccer simulation league, which isa subleagues of the RoboCup Soccer League. The soccer simulation league is one of the oldest leagues in the
RoboCup competitions.

Fig. 2. 3D Simulation
Fig. 1. 2D Simulation
On the other hand, the 3D simulation league includes
the concept of height and can simulate the real world
better than the 2D league. Figure 2 shows a game of the3D simulation league. We can watch the game of the
3D soccer simulation league through the soccer monitor, which is included in the package of the soccer server[4]. The first prototype of the 3D soccer agent was proposed in 2003 [5]. In the early stage of the 3D simulation league, the soccer agents were modeled as a sphereobject with a kick device. In 2007, a bipedal humanoid
robot model was employed for soccer agents for the firsttime in the league. This made the development of soccer agents quite challenging because not only intelligent
decision making but also low level skills such as the
movement of joints have to be considered when devel-

There are two categories in the soccer simulation
league. One is 2D league where all objects such as theball, players, flags, and goal posts are modeled as a circle. The other is 3D league where humanoid robots with
22 degrees of freedom are autonomously controlled in athree-dimensional field. Figure 1 shows the snapshot of
the 2D simulation game. In the 2D simulation league,
all objects are realized in a two-dimensional space. This
league is valuable as a test bed for high level decision
making systems. There are many famous papers about
2D league. Gabel et al.[1] considered a defense scenario of crucial importance and employed a reinforcement learning methodology to autonomously acquire an
aggressive duel behavior. Kyrylov and Hou [2] treated©ISAROB 2011948

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

oping the controller of the robot. Shafii et al.[6] employed a truncated fourier series approach for a stablebiped walking of a humanoid robot and optimized it byusing particle swarm optimization. Warden et al.[7] proposed a framework for spatio-temporal real-time analysis of dynamic scenes to improve the grounding situation of autonomous agents in physical domains. Recently low level skills have been significantly improved
by top teams in the world. In addition, the number of
agents in one team is increasing: one team had three
agents in RoboCup 2009, and it was increased to six in
RoboCup 2010. It will finally become 11 in the near
future. Therefore, it is getting more important to implement team strategy to win a game. In this paper, we propose a method that generates action rules automaticallyfrom human instruction. A human instructor is expected
to give more appropriate actions to the soccer agents. In
the proposed method, the instructions are recorded andconverted to action rules after selecting any useful instructions.

Fig. 3. Human instruction

II. BEHAVIOR GENERATION USINGHUMAN INSTRUCTION
1. Overview
Many teams implement their strategies based on the
programmers’ own knowledge about soccer. That is, theprogrammers have to write action rules that cover any
situations of the soccer field. Although it is clear that
this is not the best approach, there are only a few research works that tackle this problem. In this paper, we
solve this problem using human instruction to improve
the already implemented behavior of soccer robots. For
this purpose, we developed a human-agent interface. In
this system, a gamepad is used to send human instructions to agents. The human instructions are then converted to a set of action rules that are used to modify
the behavior of the soccer robots. The process of ourapproach is the following:

Fig. 4. 3D Monitor for instruction

with sensory information that the soccer agent receivesat that time. A user interface is used to monitor thesensory information that the soccer agent is currently
receiving. The above process is graphically shown in
Fig. 3. The 3D monitor in Fig. 3 is developed for the
purpose of this paper. The snapshot of the 3D monitor
is shown in Fig. 4. The 3D monitor allows human instructors to check the internal status of the soccer agents
since the sensory information sent to the soccer agent islimited to the front area of its head.
When the human instructor sends an instruction to
the soccer agent, the 3D monitor records it along with
the internal status of the soccer agent. The internal status recorded with the action instruction consists of thepositions of the ball and five soccer agents (three opponents, the other mate attacker, and itself). There arethree actions available for the human instructors: kick,dribble, and wait. For the kick and the dribble actions,the human instructors also have to send the action direction. Although the human instructor can specify any
direction for the two actions, the 3D monitor quantizes
it into one of the eight directions such as up, down,right, left, up-right, up-left, down-right, down-left, andtoward-opponent-goal. The nearest direction out of thenine to the specified one is selected and recorded in the
3D monitor.

i. Recording instructions.
ii. Reducing and clustering instructions.
iii. Generating action rules.
In the following subsection, we explain our approach indetail.
2. Recording instructions
The action of the soccer agent is semi-automaticallydetermined. That is, the soccer agent has its own decision based on the sensory information. However, if thehuman instructor thinks that the action currently takenby the soccer agent is not appropriate, the action of thesoccer agent is overruled by the instruction from the human instructor. Each time a human sends an instruction to the soccer agent, the instruction is recorded along©ISAROB 2011949

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

3. Reducing and clustering instructions
Since a huge number of instructions are sent from humans during a match, it is not practical to use all the
instructions. Also, some instructions are useful whileothers are not helpful for better strategies (e.g., scoringa goal). In this paper, we only use the recorded instructions that led to a goal while discarding the other instructions. Thus only helpful action rules are generated
to improve the behavior of the soccer agent.
Since there are still a large number of instructions
after removing not successful instructions, we apply
a clustering method to compress the information contained in the instructions. We apply an incremental clustering method to the field status for each action. In theincremental clustering method, a pair of two instructions
with the minimum distance in the field status space is
combined and the average is used as the representativeof the pair. This process is iterated until the number ofclusters becomes a pre-specified number. In the computational experiments of this paper, we applied the clustering method to obtain 100 clusters (i.e., 100 representatives of the cluster) for each action. During the paring
process, the distance between two clusters is measured
as the minimum distance among all possible combinations of the elements within the clusters.

Fig. 5. Instructing scene

Calculate current field state Pcur .

Find a rule R , where the distance D ! P
is minimum.

If D !

Pcur .

then perform action A,

else perform the intelligent decision making.

Fig. 6. Decision making process

4. Generating action rules
agent’s action if the action is thought to be not appropriate for the instructor. In this paper, the action taken by
the main attacker (i.e., the player nearest to the ball) is
the focus of the overrule by the human instructor.
Figure 5 shows a snapshot of the interface for the human instructor. This interface is used to send instructions that overrule the currently executed actions of themain attacker. The instructions are then converted to action rules as described Section II. The generated action
rules are added to our team, opuCI 3D 2010, which participated in RoboCup 2010 Singapore. The agent firstlooks at the rules that are manually written. Then thefittest rule with the current field status is chosen to select an action. The rule R which has the nearest P tothe current field status is selected and the distance D
between P and the current field status is calculated asfollows:

The representatives of each cluster obtained in theprevious subsection are converted into a set of action
rules. As described in the previous subsection, only successful instructions leading to a score are converted toaction rules after clustering. For each cluster center, an
action rule of the following form is generated:
R: If the current status is P then the action is A,
P : (xself , yself , xball , yball , xopp1 , yopp1 ,(1)
xopp2 , yopp2 , xopp3 , yopp3 , xmate , ymate )A: Instructed action,
where P is the status of the field. The status of the fieldcontains the positions of the opponent agents, the othermate agent, and the ball. The action in the consequent
part of the action rule is the instruction that was specified
by the human instructor.

D

III. PERFORMANCE EVALUATION
1. Experimental settingsi∈Objwhere Pcur is the current state vector and Obj includes
self, ball, opp1, opp2, opp3, and mate. The distance D
is calculated in Euclidean distance. If D is smaller thana certain threshold value θ, the behavior A is executed.
Otherwise, an agent makes a decision according to manually written action rules. Figure 6 shows the decision
making process of an agent using action rules.

In the computational experiments in this paper, a setof action rules are generated from human instructionsin 3-on-3 soccer matches. That is, a team consists ofthree soccer agents (two attackers and a goal keeper).
The strategy of the team is manually written beforehand.
While the soccer agents autonomously play according to
the written strategy, a human instructor can overrule the©ISAROB 2011

= ||P − Pcur ||
(2)
X
cur 2
cur 2 12= [{(xi − xi ) + (yi − yi ) }] (3)950The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Original team

Table 2. The result of t-test

With rulesαt-value
P (T ≤ t)

16
14
Frequency

12

0.05
1.674
0.049410
8

say that the proposed method effectively decreased thetime necessary to score a goal.

6
4
2

IV. CONCLUSIONS

0
0䌾30

30䌾60

60䌾90 90䌾120 120䌾150 150䌾180(sec.)In this paper, we introduced the behavior generation
approach from human instruction. The proposed method
enables us to improve the behavior of autonomous soccer agents through human instructions. The results ofcomputational experiments showed that the agents withhuman instructions are superior to the original ones interms of time from kickoff to the first score. Now top
level teams which participate in the world competition
have developed highly sophisticated skills, and it becomes more important to improve the team strategy. Our
method can be expanded to apply to defender’s behavior
to improve the defense ability.

The time from kickoff to the first score

Fig. 7. Performance comparison between the originalteam and the experimental team
Table 1. Mean and variance of the time necessary forscoring
PP
PP
Original team
PP
P
P
Mean
90.20
Variance
2333.1

With rules
75.04
946.92

REFERENCES

In the performance evaluation, the team with the action rules generated by the proposed method played
against team opuCI 3D 2010. A game starts by the experimental team’s kickoff and the time from kickoff tothe first score by the experimental team is measured. 40
games are played for the performance evaluation.

[1] Gabel T, Riedmiller M, Trost F (2008), A Case
Study on Improving Defense Behavior in Soccer Simulation 2D: The NeuroHassle Approach.
RoboCup 2008: Robot Soccer World Cup XII:61–
72

2. Results[2] Kyrylov V, Hou E (2009), Pareto-Optimal Collaborative Defensive Player Positioning in Simulated
Soccer. RoboCup 2009: Robt Soccer World Cup
XIII:179–191

The results of the performance evaluation are shownin Fig. 7. In Fig. 7, the vertical axis means the frequency
and the horizontal axis means the time necessary to geta score. Table 1 shows the mean and the variance ofthe time necessary for scoring a goal. From Fig. 7 andTable 1, we can find that the generated rules, that is generated behaviors, lead to the decrease in the time fromkickoff to a goal. In order to show the statistical significance of our method, we performed a one sided t-testin order to show that there is a significant difference between these two means. The null hypothesis Ho and thealternative hypothesis Ha are the following:
Ho : u t = u r(4)

H a : u t > ur

(5)[3] Kalyanakrishnan S, Stone P (2009), Learning
Complementary Multiagent Behaviors: A Case
Study. RoboCup 2009: Robt Soccer World Cup
XIII:153–165[4] Spark - A generic physical simulator, Sourceforge
project,
http://sourceforge.net/projects/simspark/files[5] Kogler M and Obst O (2003), Simulation League:
The Next Generation, RoboCup 2003: Robot Soccer World Cup VII:458–469[6] Shafii N, Nezami O M, Aslani S et al. (2009), Evolution of Biped Walking using Truncated Fourier
Series and Particle Swarm Optimization. RoboCup
2009: Robot Soccer World Cup XIII:344–354

where ut is the mean of the original team’s first scoretime and ur is the mean of the rule experimental team’s
first score time. The result of the t-test is shown in Table 2. We can find the fact that the p-value P (T ≤ t)
associated with the t-test is smaller than α from Table 2
and there is evidence to reject the null hypothesis Ho in
favor of the alternative hypothesis Ha . Therefore we can©ISAROB 2011[7] Warden T, Lattner A, Visser U (2008), Real-Time
Spatio-Temporal Analysis of Dynamic Scenes in
3D Soccer Simulation. RoboCup 2008: Robot
Soccer World Cup XII:366–378

951

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Development of an autonomous-drive personal robot“An environment recognition system using image processing and an LRS”
Yasushi Kibe and Hideki Ishimaru and Eiji Hayashi
Department of Mechanical Information Science and Technology
Faculty of Computer Science and Systems Engineering, Kyushu Institute of Technology
680-4, Kawazu, Iizuka-City, Fukuoka Prefecture, Japan
Abstract: We are developing an autonomous personal robot able to perform practical tasks in a human environment
based on information derived from camera images and an LRS (a laser range sensor). It is very important that the robot
be able to move autonomously in a human environment, and to select a specific target object from among many objects.
For this reason, we developed a system by which these functions would become possible. This environmental
recognition system is composed of an autonomous driving system and an object recognition system. First, theautonomous driving system calculates the driving route from the visual information provided by the CCD camera. The
robot is driven by this system. The object recognition system proceeds by identifying the specified object using image
processing and an LRS. The robot can grasp the object using this system. An environment recognition system isessential to both of these functions. Here we explain the algorithm by which the robot recognizes the surrounding
environment. In addition, we apply this system to the robot, evaluate its performance and discuss our experimental
results.
Keywords: Personal robot, monocular camera, Image processing, LRS, Autonomous driving, Object recognition
camera. The robot performs a grasping operation for theobject according to this system.

1. Introduction
In the near future, autonomous self-driving robots
are expected to provide various services in human living
environments. For this to occur, the robots will need togain a grasp of the human environment. Therefore,systems to provide environmental recognition based onimage information are being widely studied. However, it
is very difficult to recognize all driving environmentsfrom image information only; so far, no prospects forsuch a system have emerged. Here, we report on thedevelopment of an autonomous personal robot able toperform practical tasks in a human environment basedon information derived from camera images and an LRS(a laser range sensor), which is used to acquire twodimensional distance information.
The system for this robot is composed of anautonomous run system for movement and an object
recognition system for the recognition and grasping ofan object. First, the autonomous run system decides
upon a robot driving command based on information in
the limited space map. Information such as walls andbarricades are set to the map, and the data obtained fromthe CCD camera are compared against the map data.
The route is decided, and the robot drives. The object
recognition system is composed an object-recognition
processing part and a location-information acquisition
processing part, both of which use the monocularcamera and the LRS. An object is recognized andidentified using range information obtained from LRS
in addition to the processed image data provided by the©ISAROB 2011

2. System for robot
Our robot has a drive mechanism consisting of two
front and two back wheels. The front wheels areattached to a motor that operates the wheels on eitherside independently, while the back wheels function as
castor wheels. This method has the advantage ofallowing a small turning radius. In addition, to acquire
image information, both a single CCD camera withapproximately 2,000,000 pixels and an LRS are
installed on the head of the robot and can be rotated toall sides by two motors. DC servo motors are used forthe robot’s drive mechanism, and position and speed
control are achieved by the control system of the drive
mechanism. The robot also has two arms and hands
equipped with sensors, which enable it to respond to thevarious demands of humans. Finally, an installedwireless LAN can provide remote control for humans.
All devices are controlled by a PC, and lead batteries
supply the robot’s electric power.

Fig. 1952

Our developed robot

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

and danger zones. When there is a wall and an obstacleon the course to a goal, the robot travels along the
middle point between them. The robot always takes theshortest route and removes other routes. The system isshown in Figs. 4 and 5.

3. Specification of the LRS
LRS is a noncontact laser measurement system; ourLRS is made by HOKUYO AUTOMATIC CO., Ltd.,.
The maximum detection distance of this LRS is 4m.
Moreover, the horizontal plane space is scanned by 270°
at intervals of about 0.36° (360°/1024) to detect both the
distance and the direction of the target body. This LRS
requires a time of only 100 msec for a single scan.
Therefore, a reduction in the distance acquisition time
was enabled by using LRS to gain details about thetarget object recognized with the monocular camera. Fig.
2 shows the externals of the LRS.

Fig. 2

Fig. 4

Removed routeFig. 5 Correct route

II. Course correction system
Using this system the robot corrects its path bymeasuring and equalizing the distance on the right andleft to prevent it from crashing into a wall.
(i) The data are stored in a database.
The data are the pattern of the slope of the line on
the image, and they are made and stored in database.
The data are calculated from the width of the course andthe CCD camera angle. The width of the course,
direction of the robot, slope of the line and the distancefrom the center of the course are stored in the database.
(ii) Image processing
The robot acquires the image, and it is processed byedge-based binarization and noise removal. After that,straight lines are extracted, and the image is processedby the Hough transform into straight lines. This process
is shown in Fig. 6.
(iii) Correcting self-position
The robot estimates its position and direction andcorrects the latter by a straight line matched with thedata from the database.

Scanning laser range sensor

4. Autonomous driving system
4.1 Outline of the system
We developed an autonomous driving system for
robots that can move with image information captured
by monocular CCD camera. It has two subsystems: aroute searching system, which decides the course of therobot, and a course correction system, which traces asafe course during the actual run.
4.2 Method for autonomous driving
In this section, we explain the method forautonomous driving. The flow for the autonomous
driving is shown in Fig. 3.
START
Searching route

Original image

Correcting straight a course

Fig. 6
No
Arrived?

Follow route

4.3 Experiment of system evaluation
I. Method of experiment
In order to evaluate this system, we experimented at
Research building 3F of Kyushu Institute of Technology.
The total distance that the robot ran was about 35m, andwe verified that its systems worked successfully.
The images used in the experiment were acquired
from a camera mounted on the robot. This camera wasat a height of 875mm from the floor and its depression
angle was 20 degrees.

Yes
Arrived
END

Fig. 3 Autonomous driving system flow
I. Route searching system
In this system, the robot searches for routes based ona limited space map. This map includes informationsuch as the start position of the robot, the goal, walls,©ISAROB 2011

Noise remove
Hough transform
Image processing

953

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

“sees”. Fig. 9 shows an example of the results of theobject recognition processing.

II. Results of the experiment
The robot could reach the goal thanks to thesuccessful operation of the systems. The course
correction system was at work during the run. A pictureof the course that the robot followed is shown in Fig. 7.
This shown that robot ran in a crooked diagonal line,
but the course correction system worked, so that the
robot corrected the gap.

START
Object recognition processing
Measurement of distance using LRSThree-dimensional coordinate transformation
Matching the camera image
Angle of depression change of LRS
Position detectionof recognition object

No

Yes

End condition(Detection of two places)

No
Angle of depression calculation

Yes
Display of location information
Holding operation of object
END

Fig. 7

Results of experimentFig. 8

Object recognition system flow

Fig. 9

Object recognition processing

5. Object recognition system
5.1 Outline of the system
We developed an object recognition system for
robots that can acquire the target object position withimage information captured by monocular CCD camera
and range information obtained by LRS. This system
can acquire the object position on the assumption, forexample, that the object is placed on a desk. The systemthen acquires the location information of the object byusing LRS with the recognized object. Afterwards, thearm is driven based on the location information, and theobject can be grasped and held.

II. Measurement of distance using an LRSThe LRS then acquires range information on theobject recognized in the object recognition processing.
The LRS is fixed to the robot head, and moves with thecamera. Fig. 10 shows the situation in which the range
information is acquired.
LRS

Recognized object
Robot

5.2 Method for object recognition
In this section, we explain the method for object
recognition. The flow for the object recognition isshown in Fig. 8.

Desk

Fig. 10

III. Three-dimensional coordinate transformation
The range data of LRS is range information from theirradiation point to the measurement point of the LRS.
Therefore, this range data is converted into threedimensional coordinates. This conversion is derived in
the provided data based on the distance data and thehorizontal and perpendicular angles.

I. Object recognition processing part
This processing part can search for objects withimage information captured by a monocular CCD
camera. The system then searches for the object with theshape and color of the object registered in the database.
This processing part notes the shape and color of theobject and then step by step narrows down the objects it©ISAROB 2011

Position information acquisition situation

954

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

IV. Matching the camera image
The system successfully matched the LRS data to
the camera image by integrating the camera image with
the LRS data.

object and the distance of the hand to the object. The
arm is driven again based on the calculated driving
amount, and the grasping operation of the object isperformed.

V. Position detection of recognized object
Whether the LRS data accurately describes thelocation information of the object was judged. This
involved determining which position of the camera
image provides the best measurement point for LRS todetect the object. If the object area recognized in theobject recognition processing corresponds to the image
coordinates of the LRS measurement point, themeasurement can be considered successful. This system
acquires three-dimensional coordinates in the vicinity of
the center of the object as the location information,
provided that the LRS measurement point is at the 60%
height level, from the bottom of the object. In addition,
to prevent false detection and to provide good, accurate
detection, data acquisition is done in a second place inthe area (below the 60% mark).

5.3 Experiment of system evaluation
I. Method of experiment
We performed the following experiment to evaluate
the performance of this system in the grasping operationof an object, in which it searches by object recognition
processing. The angle of depression of the camera wasset at 10°. Fig. 11 shows the experimental environment
when the object is a PET bottle. Its location was on asingle-color desk in the laboratory.

Fig. 11 Experiment environment of the target PET bottle
II. Results of experimentFig. 12 shows the result of the robot’s actually
performing the grasping operation of the target object.
The target object was a PET bottle.

VI. Angle of depression calculation method
The system acquires location information on theobject at two places (an upper part and a lower part).
After the first location information is acquired in theupper part, the angle of depression of LRS is calculated
according to this information. If the distance of theobject and the robot can be found, the angle ofdepression can be calculated from the image
information by a geometrical calculation.

Fig. 12 Result of arm drive experiment
The arm was able to be driven to the location
coordinates of the object, and the grasping operation by
the hand was able to be performed. However, thelocation of the hand varied a little in relation to thelocation of the object. Therefore, it is necessary to
improve the accuracy by which the arm is driven.

VII. Display of location information
When the location information of the recognition
object in two places can be acquired, final location
information on the object is displayed in two places.
Because it is preferable that the acquired positional data
reflect the exact position of the object, a part in thecenter of the two examined places is acquired as theobject location information.

6. Conclusions
We propose a system that recognizes the driving
environment of a robot using image processing and an
LRS. This environmental recognition system iscomposed of an autonomous driving system and an
object recognition system. The driving environment of
the robot can be processed by these systems, and the
behavior pattern of the robot can expand. At present, it
is possible to move in a preselected area, and to locate
and grasp a target object. Expansion of the action area
and the transportation operation of the robot with theobject are in our sights and will be developed in the
future.

VIII. Grasping operation of object
The target object is located within a certain distance(2 feet) of the robot. Target coordinates are first set infront of the object, based on the acquired object position,and the arm is driven to the coordinates. An image isacquired after the arm arrives, and the remaining drive
distance of the arm is calculated from the position of the©ISAROB 2011955

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Design of robotic behavior that imitates animal consciousness－Emotion expression of robotic arm based on eyeball movement－－
Kouichirou Kurogi

Kei Ueyama

Eiji Hayashi

Department of Mechanical Information Science and Technology
Faculty of Computer Science and Systems Engineering, Kyushu Institute of Technology
680-4, Kawazu, Iizuka-City, Fukuoka Prefecture, Japan

Abstract: In this research, with the aim of “user affinity”, a trial which gives a robot “consciousness” like people

or an animal is performed. Here, the goal of user affinity suggests the capability for a robot to inspire a sense of
closeness in the user, such that the user is not bored with its use. Our laboratory previously conceived a model ofthe mechanism of consciousness and action and a software architecture by which this model can be used tocontrol the action of an artificial animal based, called the Consciousness-based Architecture (CBA). Here, we
newly built a “motivation model” which assumes that certain motives inhere in the actions of an animal, andtherefore ascribes to the robot the motive of action. In this motivation model, the dopamine-generating
mechanism of an animal is modeled. Moreover, an eye display for the robot to simulate the expression of feelings
was developed. As a first step to coherent, “conscious” expression, the speed of the blink was changed according
to the motivation model.
Keywords: CBA, consciousness of the robot, Motivation of the robot

I. INTRODUCTION

function for behavior selection, and controls the robot’s
behavior.
In the present study, we developed a robotic arm thathas six degrees of freedom, with the aim of providing
the robot with the ability to autonomously adjust to atarget position. Fig. 1 shows an overview of the roboticarm. The robotic arm that we used has a hand consistingof three fingers in which a small monocular WEB
camera is installed. The landmark object is detected in
the image acquired by the WEB camera, enabling it toperform grasping and carrying tasks. As an autonomy
action experiment, CBA was applied to the robot arm
and the behavior then inspected.
This research was intended to develop the“emotional” display of a robot. Thus, it paid attention toeyes, where feelings appear most easily in theexpressions.

In recent years, the development of non-industrial
robots in such fields as medical care and welfare and forlife in general has flourished. The operation of theserobots requires not only the basic functions of robots,
such as a high level of intellectual activity, but also thefunction of user compatibility or affinity so that a user
can feel close to the robot as a result of its appearance
and behavior. User compatibility implies that the user iseasily able to operate the given robot, without gettingbored with its use, and can easily develop a sense of
closeness with it; ultimately, the user can receive thehealing benefits experienced in relationships with otherconscious beings.
Although a robot may gain in user compatibility by
being genuinely modeled after a face, it is far moredifficult to achieve user compatibility through its
behavior and actions, including human-like “capricious
behavior”. The attempt to give robots “consciousness”
such as that identified in humans and animals is a partof these requirements.
Our laboratory has studied animals’ adjustments totheir environments in an attempt to emulate animal
behavior. We constructed a hierarchic structural modelin which consciousness and behavior were
hierarchically related. In regard to this, we developed asoftware architecture we call Consciousness-based
Architecture (CBA). CBA introduces an evaluation©ISAROB 2011

Fig.1. Overview of Robotics arm

956

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

2. Situation recognition with a Web camera

II. SYSTEM STRUCTURE

The first step of the “humanized” robot’s system is torecognize a situation. For this purpose we devised alabeling image which we divided into green, blue, andflesh color parts from an image of web camera installedon the robot hand. Then we divided the green, blue, andflesh color into separate blobs and extracted the shape,size, and center of gravity position. From thisinformation and the posture of the robot arm, the robot
could recognize the position and the distance of thetarget colored object. Furthermore, the systemmemorized the central point for three frames.

Fig. 2 shows the appearance of the robot arm, and Fig.
3 shows its degrees of freedom. The robot arm has 7
levels of flexibility {shoulder (Joint1, Joint2), elbow(Joint3, Joint4), wrist (Joint5, Joint6), and finger(Joint7)} at its full length of 450 [mm]. The hand parthas 3 fingers with one flexibility; the weight of the mainpart is about 0.8 kg. A small Web camera, equipped at
the tip of a robot arm, can recognize the externalsituation. The web camera and the robot arm’s actuator
are controlled by USB communication.

Fig.5 A Web camera image and a labeling Central
image point

Fig. 2 robot arm

Fig. 3 degree of robot arm

III. AUTONOMOUS BEHAVIOR

Fig.6

1. The motivation of the robot
Most robots are pleasing to people because of theirunique movements. However, the action choices ofrobots are too mechanical. Action choices that resemble
those of human beings and animals are needed toactualize user compatibility. Therefore, we at firstthought about a human action.
When an animal, including a human being, takes some action, it can be represented by a flow chart such as "Recognition → Comprehension → Motivation → Action".
On the other hand, the action of the robot repeats a sim
ple flow such as "Recognition (Comprehension) → Action”.
The flowchart of this system is shown in Fig. 4, and the details of each item are given below.

Central point

2-1. The objective move direction prediction
A picture is divided into eight domains in order topredict the movement direction. The movement
direction is predicted only when an object is missed at
the time of situation recognition. The prediction
direction corresponding to the eight domains and thearea as a whole is shown in Fig. 9.

Situation recognition with a Web camera
Calculate the motivation
Fig.7. A division domain and the prediction direction
Moved eyeball as the situation demands

2-2. Virtual object creation
The virtual object corresponding to the predictedmovement direction of the object is created in thevisual angle. Since it is possible to pursue the obje
ct within the visual angle, a virtual object is created, and the robot runs after a real object by running
after the virtual object.

Choose the action motivation
Control the motor
Fig. 4 Flowchart of this system©ISAROB 2011957

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

3. Calculation of dopamine and motivation based on
the situation
When a man and an animal interact, changes occur inthe dopamine level in the brain. The dopaminegenerating locus is regarded as the robot's motivation
model, and the generating locus was copied using thecontrol model. A control model is shown below. In theωn in ωn ,ζ and T are shown.
graph, sample changes

4. Choice of the action that accepted motivation
The action level was set up by dividing a robot's
motivation by a fixed value. In this way, the action
which can be chosen according to an action level was
limited.

2
2
Rising y′′ + 2ωnζ y′ + ωn y − ωn u( t ) = 0･･･(1)Decaying y = e−t･･･(2)

Tωn：natural angular frequency：earliness of a risingζ ：braking rate：height of the peak of a rising
T：time constant：attenuation performance
1 .6

2
1 .8
1 .6
1 .4
1 .2
1
0 .8
0 .6
0 .4
0 .2
0

ω n = 1 .0
ω n = 3 .0

1 .4

1 .2

1

0 .8

0 .6

0 .4

0 .2

0

5

10

15

20

25

30

5. Consciousness architecture (CBA)
Fig. 11 shows a diagram of a hierarchical structure
model called CBA (Consciousness-based Architecture)
which relates consciousness to behavior hierarchically.
The characteristic of this model is that theconsciousness field and behavior field are built
separately. In a dynamic environment, this model
determines the consciousness level to the environmentthat a robot most strongly consider, and the robot thenselects the behavior corresponding to that consciousness
level and performs the behavior. This model ischaracterized in that the consciousness level approaches
an upper level so robot can select advanced behavior
when certain behavior corresponding to theconsciousness level is discouraged by some external
environmental factor.
Additionally, an upper-level consciousness can make achoice of low-level behavior. The mechanism of thismodel is that it selects the optimum behavior within the
low-level behaviors , so the robot aims for goals.

ζ =0．1
ζ = 1 .0
ζ = 1 0 .0

00

Fig. 10 Choice of the action based on motivation

1(a) change ωn

2

3

4

5

(b) change ζ

1 .2
T=10
T=50

1
0 .8
0 .6
0 .4
0 .2
0
0

2

4

6

8

10

(c) change T
Fig.8 Motivation model

In this study, we captured the trace of this dopamine as
the motivation of the robots and termed this function the“Motivation model”. Fig.9 shows this Motivation model.

time: Dopamine of plus
: Dopamine of minusp: Motivation

Fig.9 Motivation modelFig.11 Consciousness-based Architecture (CBA)©ISAROB 2011958

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

IV. The eyeball system which operate in

V. INSPECTION OF AUTONOMOUS

response to external conditions

BEHAVIOR

Facial expression is an important factor in the role ofconveying feelings. This study focuses on the eyes,
which greatly influence emotional expression. A pair ofeyes was made in 3-D graphics using the Open GL(Open Graphics Library). Fig. 12 shows the eye display
we created.

1. Validation experiment
We conducted a validation experiment to synchronize
the robot arms with the eye system. The experimentconfirmed whether the eye would follow a green ballwhen someone moved the green ball before the cameraon the robot arm. Another experiment confirmed
whether the number of blinks changed in response to therobot arm’s motivation.
Fig. 14 shows validation results. It demonstrates thatthe eye operates in response to external conditions.

Fig.12 The eyeball created
1. The eyewink system
The study implemented the eye-blinking system foundin the human eye. Blinking is among the most important
expressive human eye movements. In order to imitate
the actual state of mind, we measured the timing ofblinks of humans in different psychological states. Fig.
13 shows the measurement result of blinking in a human.
Fig.14 Validation results of the eye display system

VI. CONCLUSION
In this paper, we created the eye diaplay of a robot arm
using Open GL. It became possible to synchronize theexpression depending on the situation outside.
In the future, we will try to improve the eye display
system because the eyeball’s movement was delayed.
Although we focused on only one color of ball at present,
we will expand the robot’s repertoire to respond toanother color in a different way; i.e., a system which
performs eye movement that refuses to follow a blue
ball’s motion.

VIII. Acknowledgement
Fig.13 Measured result of the wink of humanThis research was partially supported by the Ministry of
Education and Science,Sports and Culture, Grant-in-Aid for Scientific
Research, 2010.

In this figure, the horizontal axis represents one minuteof time and the three timelines show the timing of blinks
in different psychological states. The upper graph of
Fig.13 shows the usual state. The second graph shows astate of concentration and the third graph shows a stateof agitation. This result can be confirmed that eyeblinking rate is suppressed with concentration and isincreased with excitation. This provides clues to aperson’s psychological state. Based on this result, aneye-blinking system was developed to synchronize withthe robot’s assigned “motivation”..

©ISAROB 2011

IX. References[1] Motoki Shimono, Eiji Hayashi, Design of robotic
behavior that imitates animal consciousness (AROB
13th ’2008), Bepu, Oita, Japan, 2008[2]Takahiro Yamasaki, Eiji Hayashi, Design of robotic
arm’s action to imitate -the mechanism of an animal’s
consciousness-, (AROB 15th ’2010), Beppu, Oita,
Japan, 2010

959

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Development and Experimental Study of a ovel Pruning Robot
S. Ueki1), H. Kawasaki2), Y. Ishigure3), K. Koganemaru2), and Y. Mori4)
1)Toyota ational Colleges of Technology, 2-1 Eiseicho, Toyota, Aichi, 471-8525
2)Gifu University, 1-1 Yanagito, Gifu, Gifu, 501-1193
3)Marutomi Seikou Co., Ltd., 3147-7 aza ikuda, kurachi, Seki, 501-3936
4)Hashima karyuu kougyou Ltd., Hashima, Gifu
Abstract: This paper presents the development progress of a timberjack-like pruning robot. The climbing principal is an
imitation of the climbing approach of timberjacks in Japan. The robot’s main features include having the center of its
mass outside of the tree and an innovative climbing strategy fusing straight and spiral climbs. This novel design brings
both lightweight and high climbing speed features to the pruning robot. We report our progress in developing the robot,
focusing on straight climbing, the behavior for uneven surface, and pruning.
Keywords: Pruning Robot, Climbing Robot,

I. ITRODUCTIO

II. DEVELOPED PRUIG ROBOTThe timber industry in Japan has gone into a decline
because the price of timber is falling and forestry
workers are rapidly aging. This has caused the
dilapidation of forests, resulting in landslides following
heavy rainfall and the dissolution of mountain village
society. However, a pruned tree in a suitably trimmed
state is worthwhile because its lumber has a beautiful
surface with well-formed annual growth rings. The
development of a pruning robot is important for thecreation of sustainable forest management. The researchand development of a pruning robot [1,2,3,4,5] has been
scarce and only one commercial product was available
in Japan [6]. The machine climbs a tree spirally and cuts
brunches using a chainsaw. This machine’s weight (25kg) and slow speed hinder it from being an optimal
solution to resolve the forest crisis. A lightweightplatform is required, because most of the mountains inJapan have high slopes, and transportation of a pruning
robot is a demanding task. To advance the state of theart of pruning robots, we present an innovative pruning
robot that locates the center of its mass outside of a tree.
The wheel mechanism is designed for a hybrid climbing
method, i.e., the robot is able to switch between straightand spiral climbs. This method ensures both lightweightand high climbing speed features of the robot.
In earlier paper [7], we introduced the basic designconcept and described experiments with the prototype
robots in detail. Moreover, hybrid climbing method
have proven that the proposed pruning robot can climb
up and down at high speed [8]. Here, we report ourprogress in developing the robot, focusing on straightclimbing, the behavior for uneven surface, and pruning.

With the ultimate goal of building a lightweightpruning robot, we have developed a novel climbing
method that uses no pressing or grasping mechanism
but relies on the weight of the robot itself, like aJapanese traditional timberjack does when climbing atree (Fig. 1). The timberjack uses a set of rods and ropes,
which is called "Burinawa," and does not hold or grasp
the tree strongly while his center of mass is locatedoutside of the tree. That is, the timberjack can stay on
the tree using own weight.

©ISAROB 2011

Fig. 1 Tree climbing method using “BURINAWA”
Based on this new design concept and therequirements from the forestry industry, the pruning
robot has been developed. As shown in Fig. 2, the robot
is equipped with four active wheels. Wheel 1 and wheel
2 are located on the upper side, and wheel 3 and wheel 4
are located on the lower side. Each wheel is driven by a
DC servomotor and a warm wheel reduction mechanism
which has non-back-drivability. The steering angle of960

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

The controller is constructed using CPU board,which is equipped with wireless LAN. The controller isable to communicate data/command with personal
computer via wireless LAN. Each wheel is controlled
by velocity PI control. Feedback input, velocity throughhigh-pass filter, is appended.
By comparison with 2nd prototype[8], 3rd prototype
is lightweight expect controller and batteries. Also,
controller and electrical source were located outsidewhen the 2nd prototype. Also, the 3rd prototype isequipped with wireless LAN and a chainsaw. Althoughdetail of the chainsaw is omitted in this paper, an
experiment will be performed to show an ability of
cutting branch using the 3rd prototype.

each wheel is also driven by the DC servomotor andwarm wheel reduction mechanism. Based on theanalysis[7,8,9], the center of mass was located outsideof the tree with the help of weight of controller andbatteries. The center of mass was located with margin,
because the friction coefficient is unclear and theposition of center of mass may be moved by disturbance.
For example, the robot will be tilted when the robot
climb up on uneven surface. In the Fig. 2 (a), the centerof mass was located with parameter as follows: H is
0.3[m], W is 0.22[m]. As shown in Fig. 3, where, H isdistance between upper side wheel and lower side wheel,
W is distance between trunk surface and center of mass.
The robot is robustness from the analysis when D is
0.25[m], even if the robot is tilted about 0.1[rad].

III. EXPERIMETS
Three experiment were performed to evaluate the
3rd prototype. The 1st experiment was performed toevaluate basic performance of the 3rd prototype. The
2nd experiment was performed to evaluate therobustness for uneven surface. The 3rd experiment was
performed to show whether the robot can prune abranch. All experiment were performed using substitutetree in doors. The diameter of substitute tree was
0.25[m]. The frictional coefficient of the substitute tree
was about 0.4, which was smaller than natural tree. To
collect the experimental data, motor current, position of
the robot, and orientation of the robot were measured.
The motor current was measured using shunt resistance.
The position were measured by a 3-D position
measurement device (OPTOTRAK, Northern DigitalCo.). The orientation were measured by a 3-Dorientation sensor (InertiaCube2, InterSense Inc.).

(a) Photo image
Wheel 1

Chainsaw

Wheel 2angle 2angle 1

Wheel 4

Wheel 3

3.1 Basic performance
Straight climbing experiment was performed toevaluate basic performance. The desired speed of four
wheels was given by trapezoidal profile. A acceleration
was 0.2[m/s2], a speed was 0.2[m/s] as 0.075[m] of
wheel radius.
The experiment result is shown in Figs. 4, 5, and 6.
Fig. 4 shows the speed of the robot. The speed of eachwheels was calculated from values of rotary encoder.
The robot was able to be climbed up at 0.2[m/s].
Although the delay of start was about 0.5[sec] owing tocontrol law, it was not a problem. Fig. 5 shows the
movement distance. ”3D” is measured value by a 3D
position measurement device, and movement distanceController and batteries(b) CAD image
Fig. 2 3rd prototype of pruning robot(a) side view(b) top viewFig. 3 3D figure of pruning robot on a tree

©ISAROB 2011961

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

experiment were performed to evaluate the robustnessof the developed pruning robot for bump on trunk. This
experiment was performed substitute bump. The bumpwas made of ABS plastics, which was bigger size thannatural bump. Also, the desired speed of four wheels
was given by trapezoidal profile. A acceleration was
0.2[m/s2], a speed was 0.2[m/s] as 0.075[m] of wheel
radius.

of each wheel was calculated from value of rotary
encoder. In the Fig. 5, we found three kind of error as
follows: (E1) errors of movement distance between eachwheel and 3D position measurement device, (E2) error
between wheel 1 ( or 3) and wheel 2 (or 4), (E3) error
between wheel 1 and wheel 3 (and error between wheel
2 and wheel 4). We considered two reasons. The 1st
reason is difference of deformation of each wheel. The
movement distance of each wheel was calculated as
0.075[m] of wheel radius. The wheel was composed ofurethane and inner tube, which was deformed by acting
force. The deformation volume is depend on the force
magnitude. From the theoretical analysis[7,8,9], theforce magnitude of 3rd prototype tends toward asfollows: the normal force at nearer the center of mass
becomes bigger than it of opposite side. Hence, Fn4 =
Fn2 > Fn3 = Fn1 was considered, where Fni ismagnitude of normal force of wheel i. (E1) and (E2) canbe explained by 1st reason. Also, we considered thatreason of (E3) was slippage of wheel on the trunk. Fig.
6 shows the electric current of wheel motors that were
measured by shunt resistance. The theoreticalanalysis[7,8,9] also has shown that the tangental force atlower side become bigger than it of upper side. Fig. 6
tends toward the theoretical analysis.

wheel 1wheel 3current [A]8wheel 2wheel 4

6
4
2
0
0

2

4

6
time [sec]

8

10

12

Fig. 6 Electric current of each wheel

angle [rad]0.2

angle 1angle 2

4

6

0.1
0

-0.1
-0.2
0

2

time [sec]810(a) Wheel 1 goes over the bump
3.2 Behavior for uneven surface
To use the robot safely, the robot must be robustness
for uneven trunk. The bump is caused by the growth ofremnant of pruned branch. Therefore, straight climbing

angle [rad]0.2

angle 1angle 2

4

6

0.1
0

-0.1
-0.2

desiredwheel 3wheel 1wheel 40wheel 2

2

0.2

0.1
0
-0.1

8

10

angle 1angle 2

0.1
0
-0.1
-0.2

0

2

4

6
time [sec]

8

10

12

0

Fig. 4 Climbing speed
3D

wheel 1wheel 2wheel 32

4time [sec]

6

8

10

(c) Wheel 3 goes over the bump
0.2

wheel 4

distance [m]angle [rad]

1.4
1.2
1
0.8
0.6
0.4
0.2
0

angle 1angle 2

4

6

0.1
0

-0.1
-0.2
0

0

2

4

6
time [sec]

8

10

2

time [sec]8(d) Wheel 4 goes over the bump
Fig. 7 Roll angle and pitch angle each case

12

Fig. 5 Climbing distance©ISAROB 2011

time [sec](b) Wheel 2 goes over the bump

0.2

angle [rad]velocity [m/s]

0.3

962

10

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

the tree of afforested area is smaller than it in general.
Moreover, pruning experiment has proven that the 3rd
prototype can prune a branch.
In the future work, we would like to test in the realenvironment, and improve the robot more.

The experimental results are shown in Figs. 7. Figs. 7
shows the trajectories of angle 1 and angle 2 (Aboutangle 1 and 2, see in Fig. 2(b)). The angle 2 rotatedtoward the plus direction in all cases, that is thecontroller box was uplifted. This means that the centerof mass moved toward the tree. Equally, the center ofmass moved toward the tree when angle 1 rotates
toward the plus direction. It means decreasing of thefriction force to stay the robot on the tree. However, theelectric currents of wheel 2 and 4 were bigger thancontinuous current in the experiment. Therefore, there
were no danger of falling down. On the other hand,those angles were returned back to former angles,
although both angle 1 and angle 2 were changed whenthe each wheel get over the bump . This results shows
the good robustness.

V. ACKOWLEDGMETS
The authors’ thanks are addressed to the members of
the pruning robot research project of Gifu University.
This research is supported by NEDO (The New Energy
and Industrial Technology Development Organization)
in Japan.

REFERECES[1]M. Takeuchi et al., Development of Street Tree
Climbing Robot WOODY-2, Proc. of Robomec 200
9, 1A2-D07, 2009 (In Japanese)[2]Y. Kushihashi et al., ”Development of Structureof Measuring Grasping Power to Control Simplificat
ion of Tree, Climbing and Pruning Robot, Woody1”, Proc. of the 2006 JSME Conference on Robotics and Mechatronics, 2006 (in Japanese).
[3] Y. Suga et al., ”Development of Tree-climbing
and Pruning Robot .WOODY- Actuator Arrangementon the End of Arms for Revolving Motion”, Proc.
of SI2006, p. 1267-1268, 2006 (in Japanese).
[4] T. Yokoyama, K. Kumagai, Y. Arai, and M. So
ga, ”Performance Evaluation of Branches Map Building System for Pruning Robot”, Proc. of the 2006
JSME Conference on Robotics and Mechatronics,
2006 (in Japanese).
[5] T. Yamada, K. Maeda, Y. Sakaida, and T. Tamari, ”Study on a Pruning System Using Robots - Development of Prototype Units for Robots - ”, Proc.
of the 2005 JSME Conference on Robotics and M
echatronics, 2005 (in Japanese).
[6]S. I. Co., http://www.seirei.com/products/fore/b232
r/ab232r.html.
[7]
H. Kawasaki, S. Murakami, H. Kachi, and S.
Ueki, ”Analysis and Experiment of Novel Climbing
Method”, Proc. of SICE Annual Conference 2008, pp.
160-163, 2008.
[8] H. Kawasaki, S. Murakami, K. Koganemaru, W.
Chonnaparamutt, Y. Ishigure, and S. Ueki, "Develo
pment of A Pruning Robot with the Use of Its Ow
n Weight", Proc. of Clawar 2010, pp.455-463, 2010.
[9] T. Kato, K. Koganemaru, A. Tanaka, S. Ueki,
Y. Ishigure, Y. Mori, H. Kawasaki, Development of a Pruning Robot with the Use of Its Own Weight,
Proc. of RSJ2010, Nagoya, 2010(in Japanese).

3.3 Pruning Experiment
An experiment was performed to show whether the
3rd prototype can prune a branch. An attached chainsaw
was driven by DC motor with 24[V] battery. The robot
climbed up spirally at 0.03[m/s] speed. The diameter oftarget branch was 0.01[m].
The experimental scene is shown in Fig. 8. As theresult, the branch was cut with short remnant, which
was less than 0.005[m]. Also, the trunk was not injured.

Fig. 8 Pruning experiment with pruning robot

VI. COCLUSIOThe development progress of a timberjack-likepruning robot is presented, focusing on straightclimbing, behavior for uneven surface, and pruning. The
straight climbing experiment has proven enough basic
performance of 3rd prototype. The experimental resultof climbing up on the uneven surface has proven thegood robustness for the bump, because most bumps on©ISAROB 2011963

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Design of the android robot head for stage performances
Dongwoon Choi, Dong-Wook Lee, Duk Yeon Lee, Jun Young Jung and Hogil Lee
Department of Applied Robot Technology, Korea Institute of Industrial Technology(Tel: 81-031-8040-6317; Fax: 81-031-8040-6370)(cdw｜dwlee｜proldy｜paran1｜leehg@kitech.re.kr)

Abstract: In this paper, we propose an android robot head for stage performances. As you know, an android robot is oneof humanoid robots but more like human. It has human like joint structures and artificial skin, so the android robot is
the nearest creature to human appearance. To date, there are several android robots are developed but most of them are
made for research purpose or exhibitions. We had our attention to commercial values of android robot, especially in theacting field. EveR-3, our android robot, already had commercial plays in the theater and through these; we could learn
what requisite points for robot as an actor are. The new 9 D.O.F head is developed for stage performances. The D.O.F
is reduced for using larger motors can make exaggerated expressions, because exaggerated expressions are more
important than detail, complex expression on the stages. L.E.D lights are installed in the both cheeks to emphasize
emotion expression by color exchanging like make-up. From these trials, new head is more suitable for stage
performances.
Keywords: Android, robot, head, emotion, stage, performanceI.

INTRODUCTION

were. The first point is beauty, the second is durability
and the third is exaggerated expressions. Finally, we
make new 9 D.O.F head which has reduced but large
motors, L.E.D lights beneath the skin. This head hasless expression but it can express more clearly and with
L.E.D lights, it can emphasize expressions.

For a long time, the main issues of robotic researches
have been applications in the industrial fields, but now,
the advancement of robot technology can expand
applying of robots to variable fields. One of new field
what we focus is the arts, in detail, area of the stage
performances. Though the applications of robots to arts
have been nonmainstream, some attempts have been
trying continuously. TAREK M. SOBH made robotmusicians to play real instruments [1]. It is very
realizable application of robot on the stage, but it wasmore likely to automation. Some researches tried toinvestigate relationship between autonomous robots andintelligent environment [2] [3]. The main concerns ofthese researches are about interaction with human,
environment and robots. Cynthia Breazeal made cyberflower and intelligent theater to study interactive effects
between robot performer and audiences [4]. These
works are very meaningful to know interaction ofhuman with robots in the arts, but our interest is more
likely to the commercial performances. We madeAndroid robot EveR-3 for stage performances which
has silicon skin covered face, humanoid body andmovable lower body. Our first android robot EveR-1
was made for exhibitions [5] as a secretary but we findother possibility as an actor, so our recent hardware
EveR-3 was made for stage performances. We alreadyhad several commercial performances and through these,
we could find what requisite points for robot as an actor©ISAROB 2011

II.
1.

Hardware design

Design of the android faceIn the part of making face, our goal is to makebeautiful face but “beautiful” is very subjective. To
solve this problem, we survey many comments ofaudiences who enjoyed our performances and find some
important factors of beautiful face. One is the size of
face. Most people consider small face as beautiful facethan large one, so we should reduce the size of head.
The other factor is age. The young face seems to be
more beautiful than old face. From these considerations,
we try to make smaller, younger face than exist one.
There are no models to our face. We designed ourmodel by 3-D program (Cinema 4D) (Fig.1), because itis easier to make beautiful character by imagination
than using a real model and our goal is commercial
performance, so portrait right is very important problem.
With 3-D design, we can have beautiful face freely andno problem of portrait right even if it is unreal character.
After design, we made mold for real skin. The mold ismade by RP (RAPID PROTOTYPING). The silicon

964

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

complex is used as skin because it is the nearest
material to human skin. With 3D design and mold, we
can duplicate and modification our face easily.

Fig. 2 Inner frame and motor frame

Fig.1 Face and 3D image
2.

3.

Mechanical part

The L.E.D lights are used to emphasize emotion
expression (fig.3). People often use color of skin toemphasize emotions unconsciously. When people feelshame their face become red, for example, and fear
make the color of skin white. To change all colors ofskin is not easy, so we choose both cheeks as the L.E.D
point, because cheeks are highlight points of the face.
The L.E.D lights are designed which can express allcolors by combing three basic colors (red, blue, green).
These L.E.D lights are made as panels and they areattached under the silicon skin. These are connected tomain controller to synchronize with emotions. Though
we made our efforts to use effectively, we faced one big
problem. The L.E.D modules works well without skin,
but when it is attached beneath the skin, it can makeonly red color. We can find the color of skin andpenetration ratio disturbs the color of L.E.Ds. As aresult, we can express only red color and our next goal
is to solve this problem.

The mechanical parts are divided an inner frame and a
motor frame. The inner frame is designed by 3D CAD
program (3D MAX) and its shape is based on face
image (Fig.2). We use strings to pull and push the skinto make emotions. The routes of strings guided by
Kevlar tubes and they are located at inner frame and theinner frame roles as connector between skin and motor
frame. The inner frame is made of urethane to havestrength of structure. The motor frame has 9 RC servo
motors (Hs85mg, HiTec). It is bigger size than our
existing hardware EveR-1’s, so it can make bigger
movement of skin to express exaggerated expressions.
The locations of motors are decided by FACS [6] [7],but we should reduce the number of motors, because
small size head, large motors and simplifying structure
for durability. The D.O.F is one of important factors fordurability. If there are many motors in the head, it’s hard
to maintain and the possibility of disorder is becoming
highly. From these reasons, we choose 9 essential points
and they are shown in table 1. These purposes of design
brought less number of expressions, so we used L.E.D
lights to emphasize the expressions to solve thisproblem.

ActionEyelid open-close
Eye raise-down
Eye rotation
Chin open-close
Lip corner stretch-press
Lip stretch-press

D.O.F
2
1
2
1
2
1

Fig. 3 L.E.D light module

Table.1 D.O.F for actions©ISAROB 2011

L.E.D lights

965

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

III.

IV.

Experiment and result

The experiment took to check 3 aims. The first iswhether it can make 7 expressions (pleasure, smile,
surprise, fear, pain, wink, and sorrow) or not, the second
is the comparison with exaggerated expression andexisting expression and the third is how L.E.D effects
when it is used with expression. We attached new headto our hardware, android robot EveR-3, and makementioned expressions. Suggested seven expressions
are working well. We can also the exaggerated
expressions are more clear and easy to recognize thanexisting ones. Of course, it is very subjective to judge
the mount how much exaggerated, so we need to makekinds of standards of judgment. The comparison of twomodels is shown fig. 4. We also compared same
expression with L.E.D and without L.E.D. All seven
expressions are tested and we can verify that L.E.D
lights can emphasize the emotion expression (fig. 5).

CONCLUSION

The 9 D.O.F android head which is made for stage
performances is presented. In the stage, importantfactors are beauty, durability and exaggerated
expressions. To satisfy these factors, we reduce D.O.F
of head to make small head, use large motors and raise
durability by simplifying. The problem of lessexpression is caused, by reducing D.O.F, so we use
bigger motors to make exaggerated expression and
L.E.D lights to emphasize expressions. After several
experiments, we can confirm our goals and find our
future works which are L.E.D problems with skin andsome kinds of expression standard to judge.

Acknowledgement
This research is supported by Ministry of Culture,
Sports and Tourism(MCST) and Korea Creative Content
Agency(KOCCA) in the Culture Technology(CT)Research & Development Program 2010

References[1] TAREX M. SOBH and BEI WANG (2003),
Experimental Robot Musicians. Journal of Intelligent
and Robotic Systems 38: 197-212[2] Mari Velonaki and David Rye (2010), HumanRobot Interaction in a Media Art Environment. HRI
Workshop 16-20[3] Kwang-Hyun Park, Kyoung Jin Kim, Sang-Rok Oh
and Il Hong Suh A Miniature Robot Musical Using
Roboid Studio. HRI Workshop 47-48[4] Cynthea Breazeal, Andrew Brooks, Jesse Gray, Matt
Hancher, Cory Kidd, John McBean, Dan Stiehl, Joshua
Strickon (2003), Interactive Robot Theater. IROS2003.
[5] Taegeun Lee, Moosung Choi, Taeju Kim,
Kwangung Yang (2006), Development of an Android
Robot: K-1004, CASS2006.
[6] P. Ekman and W. Friesen (1978), “Facial Action
Coding System (FACS). A technique for themeasurement of facial action,” Palo Alto, CA:
Consulting.
[7] Tingfan Wu, Nicholas J. Butko, Paul Ruvulo,
Marian S. Bartlett, Javier R. Movellan (2009), Learning
to Make Facial Expressions. IEEE 8TH international
conference on development and learning.

Fig. 4 The effect of exaggerated expression (surprise)Fig.5 The L.E.D effect in same expression (smile)©ISAROB 2011966

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Implementation of IMU using wavelet transform and variable IIR filter
Woojin Seo1 , Bongsu Cho1 , Jaehong Lee2 , and Kwang Ryul Baek1
1

Department of Electrical Engineering, Pusan National University, Busan, Korea
2
Hyosung Industrial Machinery, Anyang, KyeongGiDo, Korea(Tel : +82-51-510-1697)(E-mail: seowoojin@gmail.com)

Abstract: Preprocessing to original signal of IMU is used for getting more reliable information. Generally,
LPF(Low-Pass Filter) which has low cut-off frequency is designed to reduce noise of inertial sensors. However, in thecase that an object travels with acceleration, IMU cannot determine motion of the object due to this LPF. Therefore,
LPF with low phase delay has to be designed.
In this paper, suggest following algorithm to solve this problem. Design two LPF with 1Hz and 4Hz cut-off frequency.
Then apply 1Hz cut-off frequency LPF at a constant velocity or stationary and 4Hz cut-off frequency LPF atacceleration. It is possible to implement IMU which has suitable response by selecting suitable filters to the conditions.
Wavelet transform to angular velocities from gyroscopes can be applied to measure the characteristic of the motion.
Wavelet transform suggests the way to observe the area of interest at frequency domain through multi-level
decompositions. The changes of motion characteristics are measured by the detail coefficients. Detail coefficients ofgyroscopes can be used to find out start and finish points of motions. Threshold value is needed to determine whichcharacteristic the system has. This threshold value can be used to divide condition of acceleration and a constantvelocity or stationary condition. The aim of this algorithm is to get reliable control information and quick response.
In this paper, used Stewart platform in order to simulate movement of an object. We compared traditional IMU and
IMU applied suggested algorithm for an acceleration condition and a constant velocity or stationary condition.
Suggested algorithm shows better result in acceleration. Because LPF with 4Hz cut-off frequency shows faster response
due to lower phase delay at acceleration.
Keywords: IMU, robotics, artificial life, wavelet, variable filter.
resistance response but high phase delay. Another has
quick response but sensitive response about noise. Eachone is applied at stationary or constant velocity
condition and acceleration condition. Wavelet transform
is used for dividing motion of an object into stationary
or constant velocity condition and acceleration
condition.

I. INTRODUCTION
Today, as the advancement of MEMS
(MicroElectroMechanical
Systems)
technology,
research into systems using SDINS(Strap Down Inertial
Navigation Systems) is being developed. An
IMU(Inertial Measurement Units) consists of tri-axial
accelerometers and tri-axial gyroscopes. The IMU isused for determination of velocity, position and attitude
by integration of acceleration and angular velocity of an
object. Since it relies on Newton’s law of motion, sensor
biases and noise are unbounded due to the integral
windup. Noise from IMU affects velocity, position andattitude information. Therefore, the object has
unbounded velocity, position and attitude errors.
Preprocessing to original signal is needed for
reducing noise and bias to estimate position and attitude.
This is for providing more reliable information.
LPF(Low-Pass Filter) is generally used forpreprocessing. Low cut-off frequency or high order LPF
causes high phase delay. And high cut-off frequency or
low order LPF causes bad frequency response.
Therefore, suitable order and cut-off frequency have to
be chosen.
In this paper, suggest an algorithm switching two
LPF according to condition of an object. One has noise©ISAROB 2011

II. ALGORITHM
It is applied to switch two LPF which have different
cut-off frequency by using wavelet transform. Then,
LPF which has noise resistance at stationary or constantvelocity and quick response at acceleration can be
designed.
1. LPF(Low Pass Filter)
IIR LPF which has 6th order and 1Hz cut-off
frequency is designed to apply at stationary or constantvelocity. This filter has noise resistance. IIR LPF whichhas 6th order and 4Hz cut-off frequency is designed toapply at acceleration. This filter has quick response.
2. Wavelet transform
Wavelet transform represents original signal byusing basis function which has finite length. Basis

967

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

functions of wavelet transform are scale function called
φ and detail function called ψ.
 j ,k ( x)  2 j / 2  (2 j x  k )(1)
 j ,k ( x)  2 j / 2 (2 j x  k )  ( x)dx  1
 ( x)dx  0

4. Recognition of acceleration condition
Sampled data with 50 Hz frequency are applied
wavelet transform. Wavelet transform of 6th level gives
information that shows changes of low frequency datafrom gyroscope. Detail coefficient of 6th level is used
for information that shows which condition the system
is in. Two LPF are changed according to the conditionof a system from detail coefficient of 6th level.
Threshold value of detail coefficient is determined
experimentally.

(2)

Scaling parameter is j and translation index is k. If j
increases then width and height of basis function isgoing to be narrow and high. If k increases then basis
function is going to be translated to right side.
Original function is represented by linear
combination of basis function.
(3)f ( x)   c j ,k j ,k ( x)   d j ,k j , k ( x)kjk

Scale coefficients are c j , k and detail coefficientsare. They are calculated by following pyramidalgorithm.
c j ,k  f ( x),  j ,k ( x)   hn  2 k c j 1,nn
(4)
d j ,k  f ( x), j ,k ( x)   ( 1) n h n  2 k 1c j 1,nn‘h’ is a transform coefficient of basis function and‘n’ is a time index of a coefficient. Original signal isscale coefficient of the highest level. And calculate scaleand detail coefficient of lower level by using equation(4). Decomposition of wavelet transform is same atevery level.

Fig.2. Decomposition of multi-level 6.

III. EXPERIMENT
A system has gyroscope, accelerometer and compass
sensor. Stewart platform is used for measuringperformance of suggested algorithm. In this paper use
IMU made by Crista to compare with suggested system.
The IMU has at most 200 Hz data output rate and 1 kHz
internal A/D rate. It consists of 300 degree/s gyroscopes
and 10g accelerometers at 3-axis. Stewart platform for
HILS(Hardware In the Loop Simulation) is used formodeling movement of the system. Stewart platform has
resolution of 0.1 degree and was proved its accuracy.

Fig.1. Decomposition of wavelet
3. Haar wavelet transformIn this paper use basis functions of Haar wavelet
transform. Haar wavelet transform is easy to implement
and fast to calculate. Following functions are basis
functions of Haar wavelet transform.
1, (0  x  1)
 ( x)   0, ( o therw ise11, (0  x  2 )1 ( x )    1, (  x  1)2

 0, ( o therw ise©ISAROB 2011

1. Component of system
A. Gyroscope
ADIS16100 gyroscope is used for measuringcondition of an object. The gyroscope can measure
angular velocity at most ±300 degree/s. SPI interface
is used for accessing gyroscope. Each 3 gyroscopeconsist of 3-axis.
B. Accelerometer
SCA3000-D01 accelerometer is used for measuring
acceleration. An accelerometer can measure acceleration

(5)

968

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

coefficient of raw data is -0.026 at a normal situationand -0.068 at a special situation. Therefore, -0.025 isapplied as suitable threshold value. If IMU switches
their LPF, hold for 1500 cycle of their control period.
(a) and (b) in Fig.4. show raw data from gyroscopeat each situation. (c) and (d) are detail coefficient fromgyroscope. Flat section appears because detailcoefficient is not necessary when IMU holds theiracceleration condition.

of 3-axis. Maximum measurable acceleration is ±2g.
SPI interface is used for accessing accelerometer.
C. Compass sensor
CMPS03 compass sensor module is used for
implementation of IMU. The compass sensor is notimportant in this paper. This is a component of IMU.
D. Controller
TMS320F2812 32bit DPS made by Texas
Instrument is used for IMU. It operates at 150 MHz
frequency.
2. Result of experiment
A. External noise
In this paper supposed two situations. One is normal
situation with no vibration. Another is special situation
with vibration when Stewart platform moves as fast as
possible. Vibration occurs at start and stop point ofmoving.
Table 1. Setting of situation
Velocity Acceleration
Movement[mm/s][mm/s2](angle of roll)
Normal48.6
50
0→30→0→30
DegreeSpecial
48.6
1504

Fig.4. Transient section of the two conditions
C. Result of experimentFig.5. shows difference between Traditional IMU
and IMU with suggested algorithm. Two IMU arealmost same at normal situation. However, IMU withsuggested algorithm is better at special situation. (b)shows a lot of noise at transient section. (d) shows
vibration with reduced noise.

Fig.3. Comparison of 1Hz and 4Hz LPF
B. Transient section of the two conditions
Define transient section of the two conditions byapplying suggested algorithm. The lowest detail©ISAROB 2011

Fig.5. Comparison of the two IMU

969

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

V. CONCLUSIONREFERENCES

In this paper, suggested IMU switching two LPF.
We used data from gyroscope to improve performanceof accelerometers. Detail coefficient is used for
estimating condition of a system from angular velocity.
The detail coefficient was calculated by using wavelet
transform. Suitable LPF was chosen according tocondition of a system. Then, noise of accelerometer was
reduced and vibration was remained. Since inertial
sensor is very sensitive to noise, reducing noise is very
important at preprocessing. Furthermore, vibration has
to be remained because it is very important to estimate
attitude.
Stewart platform is used to estimate performance of
IMU with suggested algorithm. We made static
condition like stationary or constant velocity anddynamic condition like acceleration. In addition to this,vibration was added at each condition. Then, we
compared traditional IMU and IMU with suggested
algorithm. The result shows two IMU are almost sameat stationary or constant velocity. However, IMU withsuggested algorithm shows better performance at
acceleration. Noise has to be removed. However,vibration doesn’t have to be removed. Actually, it isvery difficult to distinguish noise and vibration. Noiseand vibration are high frequency area at frequency
domain. Therefore, noise can be reduced by low cut-off
frequency LPF at stationary or constant velocity
condition because there is only noise at high frequency
area. However, in acceleration condition high frequency
data must be remained because vibration is important
information to estimate attitude of a system.
The result of experiment shows IMU with suggested
algorithm has low noise at stationary or constantvelocity. It is better result than IMU with 4Hz LPF. In
addition to this, IMU with suggested algorithm has
quick response in acceleration. Therefore, attitude canbe estimated from vibration. It is better result that IMUwith 1Hz LPF. As a result IMU with suggested
algorithm has low noise like 1Hz LPF and quick
response like 4Hz LPF. This satisfies the aim of thispaper.
This work was supported by the Grant of the Korean
Ministry of Education, Science and Technology" (TheRegional Core Research Program/Institute of Logistics
Information Technology)[1] Jeong Hwan Lee(2005), Chungnam National
University, Design and Implementation of GPS/INS
Navigation System of UAVs”[2] Lee Ki Seong, Chungnam National University,“The Operation System Design for Unmanned Aerial
Vehicle”[3] D.H Titterton, J.L. Weston, American Institute of
Aeronautics and Astronautics. "Strapdown Inertial
Navigation Technology - Second Edition"[4] http://www.koreauav.com, "Korea Robot Aircraft
Competition"[5] http://www.uavcenter.com, “UAV Center”[6] http://www.cloudcaptech.com, "Variety informationof IMU"[7] JaeHyun Lee, Jijyun Kim, JiBin Hwang, Sung-Shick
Kim(2007), Korea University, “ A Study on Fault
Detection of Cycle-based Signals using Wavelet
Transform”[8] Jun-Seok Yoon(2010), Pusan National University,“Development of 6 Degree-of-Freedom Motion Table
for Performance Verification of Unmanned Helicopter
Flight Control Computer”[9] Seongmin Baek(2001), Postech, “ Posture-based
Motion Conversion, Evaluation and Advice System”[10] N.El Sheimy and S.Nassar and Aboelmagd
Noureldin(2004 October) "Wavelet De-Noising for IMU
Alignment" IEEE A&E SYSTEMS MAGAZINE,[11] Seong-Baek Kim, Seung-Yong Lee, Ji-Hun Choi,
Kyong-Ho Choi, Byung-Tae Jang(2003), Tangible GIS
Research Team, Spatial Information Technology center,
ETRI, "Low cost IMU/DGPS Integration using
Wavelet"©ISAROB 2011970

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Development of Crawler type Rescue Robot with Slide Mechanism
Go Hirano

Seiji Furuno

Dep. of Electrical and Communication Eng.
Kinki University
11-6 Kayanomori,Iizuka,Fukuoka 820-8555
Japan(Tel:81-948-22-5655; Fax:81-948-29-8936)(hira@fuk.kindai.ac.jp)

Dep. of Control and Information Systems Eng.
Kitakyushu National College of Technology
5-20-1 Shii,Kokuraminami-ku,Kitakyusyu,Fukuoka,802-0985
Japan(Tel:81-93-964-7261; Fax:81-93-964-7236)(furuno@kct.ac.jp)

Abstract: In this paper, we develop a rescue robot that searches for the victim in the disaster scene. To go in
to a narrow space in such a scene, size of the robot should be small. But the ability to pass over the trench
and the rubble is important. We propose the crawler type robot which has the expandable side crawler. Thismechanism can enlarge mobility of the robot by expanding the side crawler when it is necessary. And this r
escue robot pass the rubble by using the side crawler as the arm. So that the swing mechanism for the sidecrawler is also installed. Details of the developed robot and several experiments are denoted in this paper.
Keywords: Rescue Robot, Expandable Side Crawler, Narrow Environment

and transformed corresponding to the environment in
the rubble. We propose a novel slide mechanism. Thismechanism can enlarge mobility of the robot by
expanding some crawlers when it is necessary. And theswing mechanism is also installed. The robot can pass
the rubble by using expandable crawlers as the arm.
This paper organized into four sections. In section 2, we
describe concept of the rescue robot and details of
proposed mechanisms for the side crawler. The
experimental results are shown in section 3. Finally, theconclusion is given in section 4.

I. INTRODUCTION
When the disaster such as earthquakes occurs, there
might be the survivors left in collapsed buildings. The
life rescue in the disaster scene has hope for 72 hours,
then the survival rate after that decreases remarkably.
Therefore, quick and safe rescue operation for thesurvivors are important. But the rescue team can not go
into the rubble immediately, because there is danger ofthe second disaster. Moreover, it takes much time for the
rescue in a narrow space where is difficult for human toenter. In such a situation, it is important to knowwhether the survivor is trapped in the rubble.
To search for the survivors in a narrow space in the
rubble, snake-like or multi-connected robots aredeveloped by many researchers. Mori et al developed
the snake-like robot "ACM-R3 [1]. They proposed
three-dimensional serpentine motion to realize omnidirectional locomotion. The multi-vehicle connectedcrawler type rescue robot are developed [2],[3],[4].
These robots connect several units in serial so that total
length of the robot is long. A turning motion in the
narrow environment might become difficult for the
robot with long length. The size of the robot should besmall to go into a narrow space in such a scene. But theability to pass over the trench and the rubble isimportant.
In this paper, we develop the rescue robot withcompact body to search for the survivors trapped in anarrow space. This robot is composing four crawlers©ISAROB 2011

Fig.1. Rescue robot with Expandable Side Crawler

II. CONCEPT OF PROPOSED ROBOT
We consider the searching task in the narrow spaceof the rubble. We propose the crawler type rescue robot
which has the expandable side crawlers shown in Fig.1.

971

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Four crawlers are arranged in parallel, two crawlers
fixed to the main body are called the center crawler.
Other two crawlers installed in both sides are named theside crawler. Each crawler can be controlled
independently. The robot moves by these driving forcesin searching filed. Position of the side crawler is Fig.2(a) usually. Under such a condition, the robot goes into
the inside of the rubble and can change the direction ofmove on. When the robot can not move because of stackor the existence of high obstacle, the side crawler is
expanded as shown in Fig.2 (b). Then the side crawler
can rotates as shown in Fig.3. As a result, the sidecrawler can be used like the arm to improve theperformance of the robot temporarily. The side crawlerof the robots developed by other researchers doesn't
have the slide mechanism and only have rotate one.
Therefore, the space in the direction of height to rotate
is necessary. Our robot expands the side crawler inparallel to the robot body direction even in a low ceiling.
The total length can be freely adjusted without changing
the total height of the robot.
It is possible to transform the configurations of therobot even in the narrow and tight space. The degree of
freedom of the robot is eight, four degree of freedom foreach crawler drive, two degree of freedom for slide
mechanism and two degree of freedom for swing
mechanism. The DC motors that installed in the robot
drive these mechanisms. However, it is necessary tothink about the center of gravity of the robot when
expanding the side crawler. So as not to influence even
if the position of the crawlers are change, the positionsof the DC motors are arranged in symmetry as shown in
Fig.4. As a result, it doesn't depend on the position of
the side crawler, and the center of gravity of the robot
can be located at the center of the total length at any
time. We explain the mechanism, the design, and the
hardware organization of the proposed rescue robot as
follows.

1. Swing mechanism
The side crawler and the main body are united onlywith the rotation shaft. This shaft is also united with theslide mechanism. Then, the side crawler will rotate
around this shaft. In this robot, two DC motors are used
for crawler drive and for swing motion. And each driven
mechanism is built into the inside of the side crawler. As
a result, the side crawler becomes heavy. Because therobot passes over the obstacles and the rubbles, the sidecrawler must be maintained in arbitrary posture.
Sometimes a big impact and the reaction force act on
the side crawler while moving. It is necessary toconsider the breakage prevention of the motor and thegear. Then, a warm gear is adopted in the swing
mechanism so as not to rotate reversibly mechanicallyby a high reduction ratio.

Fig.3. Swing motion of side crawler

Fig.4. Installed location of DC motors

2. Slide mechanism
The side crawler moves in parallel along the centercrawler. The range of movement is from the position in
Fig.2 (a) to the position in Fig.2 (b). In a word, it ispossible to slide only in the wheelbase length of thisrobot. The slide mechanism and its actuators are
installed in the main body of the robot. The rotation
shaft of the swing mechanism is connected with theslide mechanism. Therefore, the impact forces areapplied to the slide mechanism when the robot ismoving on the rubble. A linear guide is used as a slide

Fig.2. Slide motion of side crawler©ISAROB 2011

972

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

mechanism in consideration of these forces andmoments. Weight of the side crawler, resultant force andmoment that act on the rotation shaft are estimated. The
trapezoid screw is used for the drive of the slide
mechanism, and installed it inside of the center crawler
as shown in Fig.5.

Fig.6. Overview of rescue robot in initial position

Fig.5. Arrangement of slide mechanism
3. Crawler mechanism
The crawler driven mechanism is composed of the
sprocket with the chain. Two sprockets are used for eachcrawler, and the DC motor is installed and driven it. The
aluminum boards are installed in the chain with theattachment that transmits the driven power to theground. In addition, rubber is disregarded on thealuminum board to improve the frictional force with thefield.

Fig.7. Overview of expanding side crawler
Table 2. Specifications of rescue robot
Length[mm]444

4. Developed Rescue Robot
The rescue robot that we developed here is shown in
Fig.8 and Fig.9. Figure 8 shows the normal
configuration of the robot, and Fig.9 denotes theconfiguration of expanded in the maximum position.
This robot has eight degree of freedom and use eight
DC motors. Details of the DC motor and the reduction
ratio, etc. used are shown in Table 1. All the same
motors are used for four crawlers. The rotary encoder isequipped in all DC motors, so that rotational speed canbe measured. The turning angle speed of each crawler,
the amount of the slide length and the swing angle ofthe side crawler can be calculated. The space is open
between center crawlers of the main body, the battery,the power supply circuit, sensors and CCD camera, etc.
can be built in.

Amax26,12V

GP32A,66:1

MR-500slide
crawler

RE13,12V
Amax26,12V

GP13A,17:1
GP32A,86:1

MR-256
MR-500©ISAROB 2011

763

Width[mm]Height[mm]Weight[kg]

408
125
16

III. EXPERIMENTS
We show some experiments to confirm the mobilityof the proposed rescue robot. The specifications of thedeveloped robot are shown in Table 2.
1. Basic motions
At first, we demonstrate straight and turning motionof the robot. The straightforward motion is able to drivewith stability regardless of the expansion of the sidecrawler. The maximum speed of the robot is
581[mm/sec], the time that finishes sliding is 24[sec]and the time to swing the side crawler up to 90[degree]is 24[sec]. However, the turning motion is unstable
operation according to the condition. It is thought thatfriction to the road of four crawlers is large. But theTable 1. Specifications of selected DC motors
DC motor
Gear headencoderswing

Max. length[mm]973

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

robot turns as the crawler slipping. This friction forceschange depending on the contact state of the crawler
and ground. The center of rotation is depending on theamount of the slide of the side crawler and the speed ofeach crawler. Therefore it is necessary to derive thoserelational expressions of the motion to do the turnoperation with stability.
(a)

(b)

(c)

(d)

(e)

(f)

2. Step climbing
Next, we try to confirm the ability of the step
climbing of the robot. The procedure is shown below.
Here, the step height is 170[mm], and the side crawlers
are shrinks at initial position.
Step1. Move just before the step (Fig.8 (a)).
Step2. Expand the side crawlers to the maximum
position (Fig.8 (b)).
Step3. Rotate the side crawlers until it exceeds the step,then move forward the robot and contact the sidecrawlers with the edge of the step (Fig.8(c)).
Step4. Rotate the side crawlers in the opposite direction
to Step2 to lifts the main body and centercrawlers (Fig.8 (d)).
Step5. Move ahead until the whole of the side crawlers
get on the top of the step (Fig.8 (e)).
Step6. Rotate the center crawler body by swing
mechanism until the main body and the sidecrawlers become straight. Then shrink the centercrawlers by expand mechanism (Fig.8 (f)).

Fig.8. Experiment of step climbing motion

REFERENCES[1] M. Mori and S. Hirose (2002), Three-dimensional
serpentine motion and lateral rolling by Active Cord
Mechanism ACM-R3. Proc. of IEEE/RSJ International
Conference on Intelligent Robots and Systems: 829-834[2] A. Masayuki, T. Takayama and S. Hirose (2004),
Development of "Souryu-III":Connected Crawler
Vehicle for Inspection inside Narrow and Winding
Spaces. Proc. of IEEE/RSJ International Conference onIntelligent Robots and Systems: 52-57[3] H. Miyanaka, N. Wada, T. Kamegawa et al (2007),Development of an unit type robot "KOHGA2" withstuck avoidance ability. Proc. of IEEE International
Conference on Robotics and Automation: 3877-3882[4] K. Osuka and H. Kitajima (2003), Development ofMobile Inspection Robot for Rescue Activities: MOIRA.
Proc. of IEEE/RSJ International Conference onIntelligent Robots and Systems: 3373-3377

This robot can also climb the continuous stairs as sameprocedure.

IV. CONCLUSION
In this paper, we develop crawler type rescue robotthat has the expandable side crawlers. The main aim of
this robot is to search survivors trapped in the rubble
made by an earthquake or a disaster. Proposed rescue
robot is designed to go into a narrow space in such ascene. Then, the side crawlers are expanded when it isnecessary to enhance the mobility of the robot. Someexperiments are performed, and effectiveness and the
problem of this mechanism are shown. As future works,
we will improve the stability at the turn running, derive
the control method based on kinematics, and construct
the remote control systems.

©ISAROB 2011974

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Proposal of Intelligence Module Type Robot that can Exchange It in
Seamless
Shunta Takahashi, Kohei Miyata, Yuhki Kitazono, Lifeng Zhang, Seiichi Serikawa
Kyushu Institute of Technology, 1-1 Sensui-cho Tobata-ku Kitakyushu-shi Fukuoka 804-8550, Japan(Tel : 090-6278-9441)(Email takahashi@boss.ecs.kyutech.ac.jp)

Abstract: Although industrial robot has been developed well from the end of the last century, and has been well used invarious fields, robotics is still being actively researched in Japan now. Chasing after national research institute, theeducational institution and robot manufacturer also cost a lot of resource to develop a new generation robot. Moreover,the society gives a high expectation to robot. To provide a low price robot that can be widely used in home, in medical
treatment, in welfare, and in disaster relief, etc., a high function, low cost module that can be easy exchanged formaintenance is very necessary. To achieve this aim, it is undesirable to develop the robot based on an original
standard in the research institute, in the educational institution, and in the robot manufacturer, etc. separately. Therefore,
a common standard in the robotic development needs to be established, and develop the robot based on the common
standard. In this research, a new concept for next generation robot development is proposed. This concept contributes ahigher intelligence to module, a lower manufacture cost and an easy exchange operation.
Keywords: Robot, Module, Concept, Command, Communication, Intelligence.
standard in the robotic development in 2005. The
specification of the project is shown as follows.
•
Robot is equipped high functional Central
Processing Unit (CPU) in it and works withdepending on Operating System (OS) such as
Windows and Linux, etc.
•
The command of the communication between
robot body and module is made at the level ofsignal.
•Module need to be installed the middleware forthe module when module is exchanged.
•
Module need to be wire physically between robot
body and module when module is connected.
•
Module works with depending on the commandof robot body.
Here, a signal-level command indicates the physicalvalues of the distance, the speed, and weight, etc.
In this research, a new concept for next generation
robot development is proposed. This concept contributes
a higher intelligence to module, a lower manufacture
cost and an easy exchange operation. The specificationof the concept is shown as follows.
1. The robot works without depending on Operating
System (OS).
2. The command of the communication between
robot body and module is made at the level thathuman perceives and works.
3. Module has its own intelligence by equipping the
microcomputer in it.

I. INTRODUCTION
Japan is leading the world as a country where robot
is actively researched now. Robot is researched anddeveloped in various fields such as national research
institute, the educational institution and robot
manufacturer now. Many of robots that has been
researched and developed in Japan are industrial robots.
These are still active around the manufacturing industry
now[1]. Moreover, the society gives a high expectation to
robot day by day, that the robot should active in thesefields which are close to our daily life such as homehelp, medical treatment, welfare, and disaster relief, etc,
in the future. It is required that the robot composition
module should be a high function, low cost, and be
easily exchanged for a good maintenance ability so thatit can expand the application field with a low restriction.
And, to realize this aim, it is undesirable to develop therobot based on an original standard in the research
institute, in the educational institution, and in the robot
manufacturer, etc. separately. Therefore, a common
standard in the robotic development needs to beestablished, and develop the robot based on the common
standard[2].
Then, as a typical research example concerning
sharing the standard in the robotic development, New
Energy and Industrial Technology Development
Organization (NEDO)[3] executed the project of next
generation's robotic development aiming at sharing the©ISAROB 2011975

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

and the module is at human perceive/act-level command,the program of robot body is simple. Therefore, even if
the robot body break down, it is possible to take a
backup operation by using the module.

4.

The communication between the robot body andthe module is made wireless.
5. The module works by the unit even if it separates
from the robot body.
6. The brain of the robot can be dispersed from therobot body to each module[4] ~[6].
Moreover we produced a prototype robot that works
simply based on the concept of 2, 3, 4, 5. As averification experiment result, the robot worked
correctly. Therefore, validity and the effectiveness of
this concept were confirmed, then we described thedetail following.

III. INTELLIGENCE MODULE TYPEROBOT THAT CAN EXCHANGE IT IN
SEAMLESS
1. Outline of intelligence module type robot that can
exchange it in seamless
To be realized the concept of 2, 3, 4, 5, we produced
a prototype robot that move on the line, and recognizing
the object that is on the line experimentally. This robot
can work correctly with depending on the samecommand even when we put and change the different
module that has the same function on the robot body.

II. OUTLINE OF CONCEPTThis section explains the specification of the
proposal concept.
A. Proposal concept 1
Because of robot body is not depending on
Operating System (OS), it is possible to make the robot
work even if a high-end Central Processing Unit (CPU)is not installed in the robot body. According tocircumstances, it is possible to make the robot workwith the one-chip microcomputer.
B. Proposal concept 2
The human perceive/act-level command is acommand that uses the language, which just shows
recognition and action by the robot and the module. In
the communication between the robot body and themodule, it is possible to correspond to a different
module that has the same function by sending andreceiving the human perceive-level command.
Moreover, because the command is easy, the driver
software need not be installed in the module.
C. Proposal concept 3
It is possible to convert the human perceive/act-levelcommand into the signal level command by installingthe microcomputer in the module.
D. Proposal concept 4
By using the wireless communication between robot
body and module, it is possible to connect easily themodule to the robot body by putting and changing it
only.
E. Proposal concept 5
It is possible to use the module individually byseparating the module from the robot body. Moreover, if
necessarily, it is possible to use the module that isconnected to the robot body as robot.
F. Proposal concept 6
Because the communication between the robot body

©ISAROB 2011

2. Composition of intelligence module type robot thatcan exchange it in seamless
The prototype robot that we produced
experimentally this time is composed of the robot body,
distance measurement module and movement module.
As the robot body, we used small note computer. As
distance measurement module, we used PSD modulethat is composed of infrared LED and Position Sensitive
Detector (PSD) and ultrasonic module that is used
ultrasonic sensor. As movement module, we used tire
type line tracer and caterpillar type line tracer. The
composition of the robot is shown as Fig.1. The robot
body (Refer to Fig.2 (a)), distance measurement module(Refer to Fig.2 (b), (c)) and movement module (Refer toFig.2 (d), (e)) is shown as Fig.2.
3. Communication between robot body and module
The state of the communication between robot bodyand module is shown as Fig.3. The communication isserial communication. The command that is the levelthat human perceives and works is send and received
periodically. About the communication between the
robot body and the distance measurement module, whenrobot body send the command “Measure the distance
between the robot and the object” to the module, themodule understand the command and measure thedistance with the microcomputer that is installed in themodule. After the module measure the distance, it
recognizes the distance as “Near” or “Far”. The result of
the recognition is send to the robot body as character
string. About the communication between the robot
body and movement module, when the robot body send

976

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

“Measure the distance between
the robot body and the object”

Robot
Robot Body
Small notebook computer“Near” or “Far”+

Distance measurement module
PSD module
Ultrasonic module

Distancemeasurement
module

Robot
body

+

Movement module
Tire type line tracer
Caterpillar type line tracer“Moving” or “Stopping”“Move” or “Stop”

Fig.1. Composition of robot

Movement
moduleFig.3. Communication between robot body and module

IV. EXPERIMENTS
For verification of working for the robot, a small
notebook computer was equipped on the robot body tocontrol the robot and to send the human perceive/actlevel command to each module. Moreover, distancemeasurement module and movement module wasdocked to the robot body separately for the robot
moving on the line.
First, the line was drawn like circle on the floor andthe object was put on the line. The robot move on theline while measuring the distance with the object. Afterit approaches the object, it recognized the distance as“Near” and stopped on the line.
Second, the object was removed on the line. The
robot recognized the distance as “Far” and moved on
the line again.
Four kinds of module groups that change
combination of modules (Refer to Fig.4 (A), (B), (C),(D)) are shown as Fig.4. About validation methodologyof robot, the robot was worked by putting and changing
the robot body into four kinds of module groups that ischange combination of the distance measurement
module and the movement module. Putting andchanging four kinds of module groups on the robot body
(Refer to Fig.4 (A), (B), (C), (D)), using the exactly
same human perceive/act-level command, the robot
worked correctly.

(a)Robot body(b)PSD module(c)Ultrasonic module(d)Tire type line tracer (e)Caterpillar type line tracer
Fig.2. Picture of Robot body and module
the command “Move” to the module, the module
understand the command as well as distancemeasurement module and move on the line by the
microcomputer that is installed in the module. Moreover,the module sends the state of working of the module to
the robot body at the same time.

©ISAROB 2011

977

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

robot development is proposed. Moreover, we could
experimentally produce robot that works easily based on
the concept of 2, 3, 4, 5. And we could confirm that theconcept of 2, 3, 4, 5 was effective.
In the future, we are going to produce robot thatworks easily based on the concept of 1, 2, 3, 4, 5, 6
experimentally and confirm that the concept of 1, 2, 3, 4,
5, 6 was effective.
(a)PSD module andTire type line tracer (A)REFERENCES[1] H. Asoh, et al: Jijo-2:”An office robot that
communicates and learns”, IEEE Intelligent System,
Vol.16, No.5, pp.46-55, SEPTEMBER/OCTOBER(2001)[2] Fumio Kanehiro, Kiyoshi Fujiwara, Shuuji Kajita,
Kazuhito Yokoi, Kenji Kaneko, Hirohisa Hirukawa,
Yoshihiko Nakamura and Katsu Yamane: ”Open
Architecture Humanoid Robotics Platform: OpenHRP”
Journal of the Robotics Society of Japan, vol.21, No.7,
pp.785-793, (2003)[3] NEDO, http://www.nedo.go.jp/english/index.html[4] Kenji Kaneko, Fumio Kanehiro, Shuuji Kajita,
Kazuhiko Yokoyama, Kazuhiko Akachi, Toshikazu
Kawasaki, Shigehiko Ota, and Takakatsu Isozumi:“Design of Prototype Humanoid Robotics Platform for
HRP”, In IEEE/RSJ International Conference onIntelligent Robots and Systems, pp. 2431-2436, (2002)[5] R. Bischoff and V. Graefe: “Integrating, Vision,
Touch and Natural Language in the Control of a
Situation-Oriented Behavior-Based Humanoid Robot”,
In IEEE International Conference on Systems, Man, andCybernetics, pp. 999- 1004, (1999)[6] Fumio Kanehiro, Ikuo Mizuuchi, Kotaro Koyasako,
Youhei Kakiuchi, Masayuki Inaba, and Hirochika
Inoue: “Development of a Remote-Brained Humanoid
for Research on Whole Body Action” In IEEE
International Conference on Robotics and Automation,
pp. 1302-1307, (1998)(b)Ultrasonic module andTire type line tracer (B)(c)PSD module andCaterpillar type line tracer (C)(d)Ultrasonic module andCaterpillar type line tracer (D)Fig.4. Four kinds of module groups
As a result, the robot worked correctly according to
the same human perceive/act-level command even use
the different module group.

V. CONCLUSIONSIn this research, a new concept for next generation©ISAROB 2011

978The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Designing Practical Omni-directional Mobile Module in the Robot
Hardware Platform for Common Use
Moosung Choi, Sulhee Lee, Byungkyu Ahn, Euncheul Shin, Kwangyoong Yang, Hongseuk Kim
Korea Institute of Industrial Technology, Ansan Korea(Tel : 82-31-8040-6332; Fax : 82-31-8040-6399)({moosung, leesury, bk.ahn, unchol, page365, hskim}@kitech.re.kr)

Abstract: As a holonomic omni-directional mobile robot is useful with its high mobility in a narrow or crowded area, it
has been studied for a long time. But until now most of the omni-directional mobile robot developed in the laboratorylevel, universities and research institutes for study. Commercially available omni-directional mobile robots are mostly
educational robots, toy robots, and omni-directional forklift. This study looks forward to presents commercialization ofomni-directional mobile robot with many advantages and developed prototype. In this paper, we introduce the details of
this progression and use it throughout the experiment and practical application proved.
Keywords: Omni-directional, Mobile robot, Practical design, Modularization

advantages, but it is a matter of practicality and there
are not commercially available. In this paper, thestructural simplicity, usability, scalability and modular
design to move forward by suggesting considerationcommonization robot platform can be used as thepossibility of getting the look. In Chapter 2, design
direction, simulation processes are described to explain
the overall concept. The component selection process,mechanical designing are depicted in Chapter 3. In
Chapter 4, the performance and specifications areverified through the experiment with a prototype.

I. INTRODUCTION
Differential drive, most mobile robots are used in,have benefits such as rotation in place and simple
structure, but cannot move laterally and haverestrictions on the movement. Omni-directional steering
mechanism can be applied to perform tasks in a small,complex environment due to the ability of 3DoF
movement(front and back, left and right, rotated), whichmakes possible to travel in any direction from theposition, in two-dimensional planar.
Omni-directional mechanism is largely divided into
using a structure without omni-directional wheels andusing omni-directional wheels. There are several typesof omni-directional wheels like universal wheel,
mecanum wheel, double wheel, alternate wheel, halfwheel, orthogonal wheel, ball wheel, etc[1][2][3].
In Synchro-drive mechanism, the representatives ofthe structural omni-directional implementation, eachwheel drives and steers at the same time. In other words,
a driving motor drives all wheels and a steering motor
steers all wheel-assemblies. The advantage of thisstructure, all wheel driving force is equal which allows
excellent friction and odometry performance. Little
couple relation between the driving and steering canminimize the heading error and make easier to control.
And because there is no turning radius, learn tooperability. Less destructive force against the floor lets
high-efficientandalso
reducespower
consumption[4][5].
The concept of the omni-directional movement,
which has already been established long ago, has many

©ISAROB 2011

II. CONCEPTUAL DESIGN
1. Point of Design
Products to be used can be applied to a variety offorms, and compact size, reliability, scalability anddirection were the main design.
First, moving from general indoor environment,
there is no problem, because you should not badefficient to forward the implementation of structural andslip ring in order to simplify the structure was used
actively. Most existing methods of synchro-drive mobile
robot with a driving axis to synchronize steer axis andthe axis between the two couple to resolve the problem
requires a complex mechanical mechanism[4]. Due toits complexity, this approach reduces efficiency andreliability were also problems. Low contact resistancewas used as a communication line, even when there are
no problems using the slip ring to allow rotation of each979

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

2. Specifications of Joints
Specifications for a driving axis are obtained asfollowing.

15
45

t = t a +t m

45(cm)

J
f
´p ´
g
t
3000kg × cm p ´ 1.59rev / sec=
×980cm / sec 2
0.5sec= 30.5kg × cm
t m = mWD / 8

Omni-direction

Actuators

30

ta =

상위제어기

상위제어기

통신 device센서(LRF등)
Battery interface& power device이동모듈 interface

Driving axis
Steering axis
Turret axis

= ( 0.1´ 60kg ´ 20cm ) / 8 = 15kg × cm

t = 45.5kg × cm = 4.46 N × m
Fig.1. Conceptual Design of Omni-directional
Mobile Module,where t a is acceleration torque, t m is friction
torque,momentofinertia22
2
J = WD / 8 = 60kg ´ (20cm) / 8 = 3000kg × cm , desired
angular velocity f = 1.59rev / sec (linear velocity
1m / sec ), desired acceleration time t = 0.5sec , W isweight of load and D is diameter of a wheel.(Fig.2)
Joint specifications are calculated from the upper
formulas like 4.46 N × m,128rpm for a driving axis,
forasteering
axis,
and
3.5 N × m,120rpm
7.96 N × m, 60rpm for a turret axis.

axis, the infinite couple minor fixes and simplificationof power delivery for each axis are available.
In addition to the scalability of the main drive submodule a variety of sizes to accommodate the demand
was for. Payload, even if a similar application area ofrobots to the base part may differ. In such a case the sizeof the base plate needed to produce the driving axis submodule by attaching the desired shape of the omnidirectional mobile base can be built.
Payload of a human scale dual arm, head, battery,
2~3kg lightweight enough to mount around the body is
65cm wide and narrow areas of the body moves withoutchange of direction is possible, the size of a person's
normal walking speed (3km/h) was faster than the speed
targets. Wheel configuration hill driving or high speed
and cause to stability in the face of slowing excellent 4wheel synchronous structure was adopted as shown in
Fig.1.
Homing direction for reducing time-axis turret axis
absolute encoder is used. If you use the proximity
sensor and because there is no absolute position in yourrobot, the robot may move up to 180 degrees andhoming time can take a very long time.

3. Simulation
As shown in Fig.3, a commercial simulator(RecurDyn V7R1) was used to validate the joint
specification calculated from the formula in Chapter II2. The results derive the required joint torque and thecurrent is consumed.
Fig.4 shows the result of simulation. Besides theresult of steering axis, which might be caused by thefriction condition, we got the similar values to one of
the formula.

W

f

D

Fig.2. Basic Structure of Omni-directional Mobile
Module©ISAROB 2011

Fig.3. Simulation with RecurDyn

980

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 1. Specifications of Parts

Front-Left Driving

Rear-Left Driving
Rear-Right Driving

Time(sec)

Actuation

1.00E+02
Front-Right Steering

0.00E+00-5.00E+01

0
0.751
1.502
2.253
3.004
3.755
4.506
5.257
6.008
6.759
7.51
8.261
9.012
9.763

)m·Nm(euqroT

Front-Left Steering

5.00E+01-1.00E+02

Time(sec)

Rear-Left Steering
Rear-Right Steering
Sum of Steering

Slip ring-1.50E+02-2.00E+02
1.50E+04

Wheel

1.00E+04-1.00E+04

Turret
0
0.526
1.052
1.578
2.104
2.63
3.156
3.682
4.208
4.734
5.26
5.786
6.312
6.838
7.364
7.89
8.416
8.942
9.468
9.994

)m·Nm(euqroTAbs.
encoder

5.00E+03
0.00E+00

Model

EC-powermax 30(200W) +GP32C(66:1)
EC-powermax 30
Directing
Maxon(200W) + SHD
axis
motor,
14(50:1) +HDSpulley(1.5:1)
EC-powermax 30(200W) + SHD
Turret axis
17(100:1) +pulley(1.2:1)
Directing
SRS1802-12
Rotac
axis
(12 conductors)
630K
Turret axis Mercotac(6 conductors)
TPA 150/12K
Blickle(Dia. 150mm)
MAE3
For origin US Digital(12bit/turn)
BLDC 10A 1-axis
Solubot
controller
Co. Ltd.
(CAN interface)

Driving axis

Front-Right Driving

1.50E+02-5.00E+03

Manufacturer

Items

0
0.751
1.502
2.253
3.004
3.755
4.506
5.257
6.008
6.759
7.51
8.261
9.012
9.763

)m·Nm(euqroT

5.00E+03
4.00E+03
3.00E+03
2.00E+03
1.00E+03
0.00E+00-1.00E+03-2.00E+03-3.00E+03-4.00E+03

Motor
controller

Time(sec)-1.50E+04

Fig.4. Results of Simulation
Sub-module: driving axis

Steering axis
Slipring
Driving axis
Motor controller
Driving axis
motor+reducer

III. DETAILED DESIGN
1. Mechanical Design
The selection of each component was carried out on
the basis of joint specification obtained from thesimulation and the formula. Motors of maxon motor Co.,
Ltd. are used for the purpose of the small volume overpower. For the driving axis, EC-powermax, which has
small diameter and is BLDC-typed, is used to be
mounted inside the wheel and for compact design. For
the reduction gear of steering axis, a SHD model ofHarmonic Drive Systems Inc. is selected, because it has
lower overall height, but a little defect on the precision.
For the slip ring, one of the important elements for thesimplification of the structure, 630K model of Mercotac
Inc. is used due to relatively small contact resistance.
However, for the steering axis, SRS model of Rotac Inc.
is selected, because there is not enough space. The
wheel made from the material of thermoplastic rubber
elastomer tread is used, because there should be small
friction with ground in steering, and small reaction force.
The details are shown in Table 1.
CAD image of three kinds of sub-modules areshown in Fig.5. The used tool is Solid Edge ST2.

©ISAROB 2011

Wheel

Sub-module: steering axis

Steering axis motor
Steering axis
Motor controller
Steering axis
Harmonic Drive

Sub-module: turret axis

Turret axis motor
Turret axis
Harmonic Drive
Turret axis
Motor controller
Turret axis slipringFig.5. Sub-modules and Detailed Structure

981

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

2. Electrical Design
Components of this mobile module are divided bytheir functions like motion controllers, absolute
encoders, IMU sensors, and so on. CAN interface and
CANopen communication protocol are used forconnecting with one another as shown in Fig. 6.

Fig.8. Payload Test of the Mobile Module

V. CONCLUSION
In this paper, a practical form of omni-directional
mobile module is described in the design and prototype.
It is true that the omni-directional mobile module isuseful, but complex and expensive relatively to thedifferential drive mobile robot. However, as proposed inthis paper, if the structure is simplified by using theparts helpful for simplifying aggressively and themodularization is considered for the scalability, it ispossible to produce on a large scale by being able torespond to the various demands, to decrease the cost of
manufacturing, and to be used widely in the end.
In the near future, the mobile module designed inthis study will be used for research of robot hardware
platform in common use, and should be improved forthe reliability by the more experiment and complements.

Fig.6. Electrical structure

IV. EXPERIMENTThe experiment is carried out to verify theachievement of the desired specifications in payload andvelocity, which are main indicator. For the experimentof payload, the frame is mounted on the mobile moduleand the load, including the battery, of 50kg is added asshown in Fig.8. In this state, round trip moving test is
progressed in the velocity of 1m/sec in three metersintervals like Fig.8.

REFERENCES[1] Kenjiro Tadakuma, Riichiro Tadakuma, Jose
Berengeres (2007), Development of Holonomic
Omnidirectional Vehicle with “Omni-Ball”: Spherical
Wheels. Proceedings of the 2007 IEEE/RSJ
International Conference on Intelligent Robots andSystems:33-39[2] I.E. Paromtchik, H. Asama, T. Fujii, L. Endo (1999),
A control system for an omnidirectional mobile robot.
Proceedings of the 1999 IEEE International Conference
on Control Applications:1123-1128[3] Giovanni Indiveri (2009), Swedish Wheeled
Omnidirectional Mobile Robots: Kinematics Analysis
and Control. IEEE Transactions on Robotics 25(1):164171[4] Suk Hyung Jung, O Sang Kwon, Byung Soo Kim,
Seung Hong Hong (1999), Local obstacle avoidance forsynchro-drive mobile robot. Proceedings of the IEEE
Region 10 Conference TENCON 99:1524-1527[5] M. Wada (2000), A synchro-caster drive system forholonomic and omnidirectional mobile robots.
Industrial Electronics Society 2000:1937-1942

Fig.7. Prototype of Omni-directional Mobile Module
Table 2. Specifications of Module
wheel config.
Max.
velocity(m/sec)
Payload(kg)Weight(kg)Dimension(cm)Communication©ISAROB 2011

Specifications
omni-directional
1
50
16.2
45×45×23.5
CAN

982

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

New Acoustic Positioning System for Under Water Robot Using Multiple Frequencies
XiuJing Gao , Feifei Zhang , Masanori Ito(Tokyo University of Marine Science and Technology, Japan)
Abstract: Recently, positioning system for under water robot becomes a very important subject. The conventional
positioning system is based on the time difference or phase lag. However, the systems are expensive and complex, andwe proposed a new system using sound propagation loss. In this system, we separated the direct wave from received
wave, and measure the amplitude or power of it. Using this data we calculated the distance. However, the signal
accepted the influence of reflected waves, and it couldn’t obtain correct distance for long distance where the direct
wave can’t be separated from reflected waves. In this study, we propose a new analysis method based on multiple
frequencies. This method uses several different frequency sounds, and sends them sequentially. The distance can becalculated from taking the average power of received signals. The advantage of this method is not necessary ofseparating process, and can measure longer distance.
Keywords: Underwater Positioning，Propagation Loss，Multiple Frequencies，Direct Waves，Mixture waves
distance, the reflected wave appears at the first stage ofreceived wave.

I. INTRODUCTION
In recent years, the research for the oceanic survey ordevelopment of oceanic natural resource is actively
advanced. The needs of the underwater robots used for
those purpose have been becoming higher and higher.
For the autonomous underwater robot, positioning
system is one of the important subjects to exercise theirfull performance. The conventional positioning systemin the water is based on the time difference or phase lagof transferred sound from source to receiver. It measuresthe distance between moving object and station where
the position has already fixed. When 3 stations areprepared, we can fix 3 dimensional coordinate of object.
However, the systems are expensive and complex.
Then it is difficult to measure the position in real timeand expand work area of the robot. From 2006, we areproposing a new system using sound propagation loss.
In this system, the distance can be calculated using therelation between the propagation distance and soundpropagation loss. In the former study, we separated the
direct wave from received wave, and measured theamplitude or power of it. Using this data we calculated
the distance. However, we cannot separate direct wave
and reflected waves from the received signal in usualcase and it couldn’t obtain correct distance. Figure 1 isthe simulation result with this method for 5m and 50m
distance of 1m depth. Top graph shows the received
waves without reflected wave. Second graph shows thereceived waves of 5m distance and third graph is theresult of 50m. It shows that the part of direct wave isclear in case of short distance, but in case of long

©ISAROB 2011

Fig.1Simulation Results for Propagation of Sound Wave
In that case, the distinction and the separation of thedirect wave from the received wave are impossible. So,
in this study, we paid attention to the reflected waves,and used them actively in the data analysis. In thissystem, instead of the single-frequency sound, we use
several different frequency sounds, and send themsequentially. In the step of data-analysis, we use mixed
wave which contains direct wave and reflected wave,
and take the average power of received wave. Thissignal is used to calculating the distance. The
performance of this method was proved with simulationand experiment. The advantage of this method is that it
can measure longer distance.

II. Experimental system
It is called propagation loss that the sound pressurelevel damps depending on a distance when a sound983

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Ⅲ. Simulation and Experimentpropagates in the water/air. The relations of the
propagation loss PL[dB] and propagation distance R[m]can be presented as follows.

PL=20×log10(R)+α×Ｒ

1. Simulation(1)

Before experiments, we made simulation study forthe decision of sound frequency.
At the condition of constant depth and propagation
distance, we suppose the amplitude of each transmitting
wave is 3[v], the amplitude of the direct waves at thereceiver is 2[v] and the amplitude of the delayed andreflected waves is setting by random, and we examined
about each received signal.

Here, loss coefficient α [dB/m] is calculated with SOAP
equation as follows

α=0.11 × f 2
44 × f 2++ 3 × 10 4 × f 2
1+ f 24100 + f 2(2)

On the other hand, in this system, PL can becalculated with

PL = 20log10(Tv/Rv) + Tg + Ts + Rs + Rg (3)The parameters are defined as follows:
Tv[V] : Transmitted signal voltage
Rv[V] : Received signal voltage
Tg[dB]: Transmission gain
Rg[dB]: Reception gain
Ts[dB]: Transmission sensitivity
Rs[dB]: Reception sensitivity
It is clear that the propagation distance R can becalculated from the transmitted and received signal (Tvand Rv).
Fig.3 Received Signal (40 kHz)

Fig.2 Experimental System
Figure 2 shows the experimental system. The
transmitting unit consists of a function generator to
generate the multiple frequencies sounds with same
voltage of Tv, a power amp to amplify the level of thesignal with gain Tg and transducer to transmit the signal
into the water with the sensitivity of Ts. The receiving
unit consists of a transducer to catch the signal withreceiving sensitivity of Rs, a pre-amplifier to amplify
the received signal to a necessary level, a band-passfilter to clearing the noises and a computer to record thedata.

©ISAROB 2011

Fig.4 Received Signal (38 kHz)
Figure 3 and figure 4 show the received signals
when the depth is 1[m] and the propagation distance is
3[m].

984

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Ⅳ. Experiment and Results
2.1

ExperimentFig.5 Comparison of Powers (1)Figure 5 is the results of simulation with changing
the propagation distance at same depth. It shows thatthe power of direct waves and average power of themixed waves are nearly equal on every propagation
distance.

Fig.7 Water Tank in TUMSAT
We made the experiments in the water tank in
TUMSAT which has the size of 10m(W)×50m(L)×2m(D) as showed in Fig. 7.
All instruments are put on the moving train. We put
down the transducer for transmitting sound into thewater at depth of 1m and fixed it at center axis of thewater tank (Equidistance from both side wall: 5m) witha crane. The transducer for reception is put into thewater at depth of 1m and fixed on moving train. We
moved the train with every 1m to change the distance
between transmitter and receiver.

Fig.6 Comparison of Powers (2)2.2
Figure 6 is the results of simulation with changing
the depth of transmitter and receiver at same depth
under same propagation distance. It shows that, at thesame propagation distance, the power of the direct
waves and average power of the mixed waves are alsonearly equal with changing the depth of the transmitterand receiver who are at the same depth.
The results say that the level of received wave
changes because of the change of phase lag dependingon the frequency of transmitted wave. However, when
we used multiple frequencies of 40kHz, 39kHz, 38kHz,
37kHz, 36kHz, 35kHz, 34kHz, 33kHz and 32kHz astransmission signals, the average level of their received
signal is corresponding to the level of received signal ofthe direct waves. And there exist many groups of
frequency that shows such kind of characteristics. So,
we can decide the useful combination of frequency withthe simulation.

©ISAROB 2011

Results

Fig.8 Example of Received Wave
Figure 8 shows a example of received wave. The firstpart of this wave is direct wave which is arrived firstly.
The following part is mixed wave that contains direct
wave and reflected wave. In this study, the received

985

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

signal is measured with taking the average of power
gained through FFT to mixed signal.

Figure 10 and 11 show the distance calculated by theconventional method that uses the power of the direct
wave from received signal. In these results, theinfluence of reflected waves appears clearly and theaccuracy becomes not stable.

Fig.9 Comparison of Power for Received Waves
Figure 9 shows the results measured the power ofreceived wave from direct wave of single frequency (32kHz and 40 kHz) and average of the mixed wave ofmultiple frequencies for 1 to 5m distances. There exists
no clear difference.

Fig.12 Calculated Distance Based On Mixed Waves ofMultiple Frequency
On the other hand, we show the distance calculated
from average power of the mixed wave that contains the
direct wave and the reflected wave in Fig. 12.
Comparing those results, we can say that new method
has higher accuracy than conventional method.

Ⅴ. Conclusion(1) In this study, we propose a new method to get thecollect distance that is transmitting several frequency
wave and taking average of power including direct andreflected wave from received signal. We confirmed theavailability and performance of this method withexperiment and simulation.
(2) As the next tasks, we will make a experiment in thereal sea area for long distance and prove the practicalityof this method.
(3) On the basis of this method, we will construct a
sensor network system and establish real time
positioning system for underwater robot.

Fig.10 Calculated Distance Based on Direct Waves ofEach Frequency (32kHz～36kHz)

References[1] The Marine Acoustics Society of Japan, Fundam
entals and Applications of Marine Acoustics, Seizan
do 2004[2] Yosikawa,Fluid Acoustics of Sound Source, Tokyo:
korona, 2007
[3] FuBin, A New Positioning System for Under Water
Based on Sound Propagation Loss and Sensor Network,2008

Fig.11 Calculated Distance Based on Direct Waves ofEach Frequency (37kHz～40kHz)©ISAROB 2011986

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Development of under water use humanoid robot
Yunyi Li, Eturo Shimizu, Masanori Ito

Tokyo University of Marine Science and Technology
2-1-6,Ecchujima,Koto-ku,Tokyo,Japan(Tele: 03-5245-7300-8756Fax: 03-5245-7300-8756)(iamliyunyi2000@yahoo.co.jp, shimizu@e.kaiyodai.ac.jp, itom@kaiyodai.ac.jp,)

Abstract: In this research, we have developed a swimming robot with flutter kick of two legs, which can swim freely
both on the surface of water and in the water, and established the control method for all kinds of motion of this robot, w
e considered a dynamic model of undulatory fins and it has been used to construct the dynamic model of pr
opulsion and control system and analyze its motion velocity, and propulsion efficiency. We proposed and desi
gned the corresponding control algorithms to generate appropriate thrust force and to decrease disadvantatageo
us influences of the interference of flutter kick. We can realize free posture control and get various motion f
orms.
Keywords: Humanoid, Underwater Robot
oscillating fin. The robot is wearing a waterproof suit on
the body. And it is designed to get neutral buoyancy also.
The neutral buoyancy is the condition that the gravity
equal to buoyancy. And the center of buoyancy andgravity are arranged to be collinear along the body z-ax.

I. INTRODUCTION
With the great development of science, the robot
technology
hasbeen
experiencing
rapid
advancement .As the intercrossed subject of under water
engineering and robot technology, under water robot s
are among current interest all over the world.
However the research on underwater humanoid robot
has not been particularly investigated. The authors thinkthat the humanoid type underwater robot is convenient
for underwater works as on the ground. The underwater
environment is so dangerous for human, that manykinds of robot have been extensively used forunderwater work, such as underwater resources
exploration,
oceanographic
mapping,undersea
wreckage salvage, ocean engineering survey, dam
security inspection, and so on. However, there has
been no underwater robot, which can take the place ofthe diver by now. Considering those situations, theauthors are putting the focus on developing Underwater
Humanoid Robot.

Fig.1 Image of Underwater Humanoid Robot

II.DESIGN OF ROBOTIn this section, we briefly present a design ofunderwater humanoid robot prototype, describing its
propulsive mechanism and mechatronics design.
As illustrated in Fig.1, the propulsive structure
treated here is a free-swimming humanoid robot. It is
composed of three parts: a body with two arms and twolegs. Each of the legs is composed of two links and one©ISAROB 2011

Fig.2 Underwater Humanoid Robot
Fig. 2 shows prototype of radio-controlled
underwater humanoid robot. The body of the robot is
wrapped with gum. In the body, following devices are
installed in a water shielded package. 1)Microcontroller
board: Motion Greator for TTL and PS/2 Bluetooth
controller, 2)Communication devices: Parani ESD-200
Bluetooth Serial Adapter,3)Arm servo motors,

987

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

the body is taken to be x axis, the lateral axis is to bey axis, and z axis to be positive upward. The bodyfixed translational and angular velocities are denoted by
the vectors V and  . The velocity of the body isgiven by4)Batteries
and 5)Underwater camera. The total
weight is approximately 1.5 kg. And the length is
400mm. We can control the swim of robot with wirelesscommunication. Its speed is adjusted with frequency
and amplitude of oscillating signal, and its turning
motion is controlled with arm motion.

ˆR  Rx  RV

III.MOTION ANALYSIS 0ˆ  
 3
  2In this section, the authors built the dynamic modelon the basis of undulated fins and the drag model influid mechanics. The dynamic model of undulated fins
can make clear the relation between the forces/moments
and propulsive wave parameters, geometric parameters
as well as swimming velocity. We can study about the
motion of the robot, control method and efficiency of
propulsion. This dynamic model for undulated fin has
been validated with experimental tests in thrust, andpropulsive velocity of the underwater robot.

 3
0

1

 2 
 1 
0 (2)where  i is the joint angular rotation rate.
In order to calculate the moment of inertia and added
mass simplicity, we approximating the body as anellipsoid, the arms and legs as columns.
3.

Potential forces
The robot is assumed to be neutral buoyancy withcenters of buoyancy and gravity that are noncoincident
but are collocates along an axis parallel to the z axis. By
the effect of gravity and buoyancy we can get a torque.

1.

Forces on the system
Given the composition of this robot, the set of
external forces on this robot is lift and drag from thelegs, lift and drag from the body, lift and drag from thearms, the force of buoyancy, the force of gravity, andthe moments resulting from these forces. These forces
are shown in Fig.3.

1
T   mR T k  rb
2Tk  0 0  1(3)

rb  0 0 h There  is the magnitude of the gravitational
force, h is length from the center of mass to the centerof buoyancy.
4. Forces on legs
To simulate the dynamics of the swimming, we
simply considered an equation of force with flutter kick.
It includes the lift and drag on the legs.

1
cl A(v12  v 22 ) cos(   )221
D  c d A(v12  v 22 ) sin(   )2
2
LFig.3 modeled forces
2. Unforced equations of motion
The position and orientation of the body segment of
the robot are denote by g which can be written inhomogeneous matrix notation as R xg
 0 1

where L is Lift force, D is Drag force,  isdensity of fluid, cl and c d are the lift and dragcoefficients, A is the planform area of the leg in the xy plane, v1 is tangential velocity, v 2 is normal velocity, is the angle between leg and body. As the influencesof the interference of flutter kick is very complex, we
use the approximation and calculate
under ideal
condition in this paper.

(1)

Where R is the orientation of the body, and x is the
position of center of mass of the body, both relative to a
fixed inertial reference frame .the longitudinal axis of©ISAROB 2011

(4)

988

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 20115.

ine the control method that will generate desired motion primitives.
1. Forward locomotion
Since the robot moving in straight course is described
as a posteriorly propagating wave, the desired motion
for the robot links is expected to approximate the wave
to generate forward thrust, the desired motion can be
expressed asForces on the body and arms

We assume the shape of body is not plate but an ellipsoid, and then the resistance force on the bo
dy is assumed to be generated in the usual steadyflow. Under this assumption, the drag and lift force
becomes as follows
1
Db  c db Sv 2
2
1
Da  c da (   )  Sv 2
(5)
2
1
Lb  clb Sv 2
2
1
La  cla (   )  Sv 2
2
where v is velocity of the body, c D b and c D a are2
 x x    xf x，t   c1  c 2   sin k  t l l l    lwhere f is transverse displacement of the body, x is
displacement along the main axis. t is time, k is bodywave number,  is body wave frequency, l is lengthof the robot. In order to generate the body wave, we
set the joint angles as fallows

the drag coefficients of the body and arms, cl b a
nd cl a are the lift coefficients of the body arms,

 is attack angle, s is projected area of body in the y--z plane,  is the angle between body and arms.
6.

 1  0,  2   0 sin(t ), 3   0 sin(t   )Force Equations of motion

1
r
2

 r 
g M (r , g )    U ( r , g ) g 1 
r   2  3 2.

Turning
When the robot is required to turn uniformly withgiven angular velocity w and turning diameter D, acentripetal force F should be offered asF1m 2 D2(9)

If we set the robot arms making angle between bodyas  , we can get the centripetal force from thrust force
and resistance force. Using the analysis results we canget the relationship between  and w.

(6)

Where U ( r , g ) is potential energy, and V and M
correspond to the body-fixed velocity and mass matrix
in the robot system. r is the matrix of the joint angles
of arms and legs.

V. SIMULATION
In order to evaluate the applicability and capabilitiesof humanoid robot, we made simulations study withusing developed model.
The results of simulation forforward swimming and turning are shown in Fig.4 andFig.5. The condition for simulation is shown in Table 1.

IV. CONTROLIn order to evaluate the validity of the model in t
he previous section to actual robot, we must determ©ISAROB 2011(8)

We can achieved the forward locomotion, here  0is amplitude.  is the phase difference. Using theanalysis results, we can get the relationship between
velocity and frequency of flutter kick. Then withchanging flattering frequency, we can get a desired
swimming speed.

For systems such as this one where forces on thesystem are independent of the body position andorientation, Lagrange equation can be used to describe
the system dynamics. A mechanical system of the typeconsidered here can be described by the states of the
body g , termed group states, in combination with thevalues of the states that describe the relative placementof moving components to the body, denoted R andtermed the shape states. The shape state states R are
the joint angles of legs and arms.
The Lagrangian for this robot with statesr , g , r, g can be written as

L (r , g , r, g ) 

(7)

989

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Table 1 Condition of Simulation
Dimension of Robot:
Body Width: 23cm, Body Length:40cm
Body Depth; 11cm, Leg Length;20cm
Fin Size; 10cm*6cm
Weight;1.5kg
Frequency of Leg Motion;
2Hz

Base robot

Controller

Waterproof suit

Fig. 6 Robot System for Experiment
45
40
35

2. Experiment Results

30

y(cm)

25

We would like to report those results on the symp
osium as much as possible.

20
15
10
5
0

0

2

4

6

8

10

12

VI. CONCLUSION

14

x(cm)

In this paper, we considered a dynamics model andcontrol method. Concretely, we introduced mainly the
propulsion system, such as principal, the structuraldesign or control algorithms to generate appropriate
thrust force. This mechanism has a disadvantage that the
flutter kick of both leg interfere each other This
propulsion system can also generate turning moment,
upward going moment or downward going moment
adding thrust force with selecting angular velocity offluttering and center angle of fluttering. And withintegrating basic control to cooperated control, we canrealize free posture control and get various kind of
motion. Of course, more accurate parameters remains tobe seen, and we would like to develop the system tohigher level.

Fig.4 simulation result for forward swimming
0

-0.2

-0.4

y(m)

-0.6

-0.8

-1

-1.2

-1.4

0

0.1

0.2

0.3

0.4

0.5
x(m)

0.6

0.7

0.8

0.9

1

Fig.5 simulation result for turning

VI. EXPERIMENTFor the confirmation of the simulation results andfurther study, we made experiments with actual robot.

1.

REFERENCES[1] YAMAMOTO I, TERADA Y (1995) Propulsion
system with flexible / rigid oscillating fin [ J ]. IEEE
Journal of Ocean Engineering 20 (1): 23-30.

Experiment system

In this experiment, we used a high performance
humanoid robot as a base machine and covered it withwaterproof suit. It can be controlled with gamecontroller remotely or with program autonomously. The
photograph of is shown in Fig.6. This robot was tested
in the water tank in our laboratory.

©ISAROB 2011[2] Colgate J E, Lynch KM(2004), Mechanics andControl of Swimming .A Review [J ] . IEEE J . Oceanic
Eng,29 (3) :660 - 673.
[3] J.Yu, Wang, andM. Tan, (2007), Geometric
optimization of relative link lengths for biomimetic
robotic fish, IEEE Trans.Robot 23(2): pp.382-386.

990

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Linear Estimation Method for Position and Heading with RDOA
Measurements
G. H. Choi*, H. S. Cheon**, J. B. Park*, and T. S. Yoon***Department of Electrical and Electronic Engineering, Yonsei Unversity, Seoul, Korea(Tel: 82-2-2123-2773)(E-mail: {choigh99 & jbpark)@yonsei.ac.kr)**Department of Electrical Engineering, Changwon National University, Changwon, Korea(Tel: +82-55-213-3633)(E-mail: {starchon & tsyoon}@changwon.ac.kr)

Abstract: Passive localization in the sensor network has been studied in many areas. Especially, the estimation of the position andheading for the target is an important subject in the navigation problems. We estimate the position and the heading information onlywith range difference of arrival measurements. The proposed algorithms are based on the pseudo linear measurement equationtransformed from the nonlinear one and uses the instrumental variable method to remove estimation errors. It does not needadditional computational burden so that it will be advantageous to real time applications. To show the usefulness of the proposition,
we simulate it in the various positions and headings comparing with a nominal least squares method and the robust least squares
method.

Keywords: RDOA, Localization, Heading estimation, Least squares, Instrumental Variable.

measurement matrix which causes an estimation error
by correlation with itself or with measurement noise [57]. The other problem is a bias of the measurement
noise. These problems are concerned with the RDOA
measurement noise and they may be neglected under theassumption that the variance of the noise is small [2-5].
However, if the condition of SNR is not good, theestimation error can be increased rapidly.
The IV (instrument variable) method can be a proper
solution, because the method uses an instrumental
variable in the measurement matrix for fleeing from thecorrelation [8]. It does not require additional
computational burden so that it will be advantageous toreal time applications.
Therefore, we propose the position and heading
estimators based on the IV method and assume thestochastic information is unknown. To treat the bias ofmeasurement noise of the pseudo linear equation, we
estimate the bias by augmenting to the state variables
based on the bias common model [9]. The useful aspectsof the proposition are shown by simulation in thevarious positions and headings and by comparison witha nominal least squares method and the robust leastsquares method in [6] which can be adapted to thislinear uncertain problem.

I. INTRODUCTION
Passive localization in sensor network has beenstudied in many areas such as control, communication,
signal processing and etc. For the localization, theutilized measurements of sensor network are such as
TOA (time of arrival), TDOA (time difference ofarrival), RDOA (range difference of arrival), or RSS(received signal strength). TDOA or RDOA-based
localization methodologies have been applied forfinding target’s position because it does not needsynchronization between the target and the network.
Additionally, if one can get two different positions
concerned on-board of the target, heading estimation ispossible. This method has been used in a navigation
problem [1]. In this paper, we are focusing on theheading and position estimation with RDOA
measurements.
In a plane, the localization problem with RDOA
measurements is regarded as an MLE (maximum
likelihood estimation) problem which decides a crossing
point between two parabolic functions [2]. However,there is always a possibility that the MLE method is notconverged to the global minimum but converged to a
local minimum depending on the initial point.
Additionally, its solution is often derived withnumerical analysis methods which can be a burden forcomputation.
To convert nonlinear estimation problem to linear one,an intermediate variable method has been proposed withadding a new state variable [3]. This transformed linear
equation is called as a pseudo linear equation. With thisequation, it can be relaxed for the initial guess problem.
However, there are still two problems for the linear
equation. The one is that there is an uncertainty in the©ISAROB 2011

II. POSITION AND HEADING ESTIMATIONMODEL WITH RDOA MEASUREMENTS
1. Position Estimation Model
We assume the target has two transmitters. The
transmitters generate some signals which the sensor
nodes of a network can realize the target. The network
does not know the burst time of the signal but it can

991

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

measure the arrival time. In this case, one can use a timedifference to localize the target’s position and the timedifference can be transformed to the RDOA by
multiplying a propagation velocity.
(1)
r1,2  v p (t1  t2 )linear estimator can be applied to find the transmitters’
positions. If one can obtain two positions for the target,
its center position and heading can be derived by using
the positions. The center position isx  xBy  yB, yc  A
xc  A(5)
22, and the heading is
x x  y  yA 
(6)
 c  cos 1  B A   sin 1  B

l
 l



By using this idea, we can build a linear model for the
position and heading. Using (5) and (6), the linear
equation is given by

where v p is the propagation velocity, t1 and t 2 arearrival time at each sensor node, and r1,2 is the RDOA.
To make this problem simple, we derive it in the 2-D
case as Fig. 1.
y  axissensor4

sensor5( x4 , y4 )( x5 , y5 )

 2  x2  x1 
  xc 
0

 

0
 2  y2  y1   yc  y1B  y1A l  x2  x1    cos( c )   v1B  v1A 0 2 
 


 B
A
0
l  y2  y1    sin( c )   v1B  v1A 
1  y1 
y





 r A   r A r A   r A   d

z1
n1
1,2
1,2
1,2
 1,2B  1, A 
BBB r1,2   r1,2 r1,2   r1,2   d1, B  
T xc y  c
 c sensor3( x3 , y3 )sensor2( x2 , y2 )sensor6( x6 , y6 )

c

d2 A

d1B

l

sensor1

d1A( x1 , y1 )x  axis( xA , y A )(7), and its generalized form when there are n+1 sensors is

Fig. 1. Positions of the sensor nodes and the target
in the 2-D plane   1   z1    
 n1 
1
 z         n 2 2     2   
X  2           zn     n    n   nn 



 

z
n

 


By using the intermediate variable method [4], themeasurement model is given by(r1,2A ) 2  d 022  d 012 
y1A

 2 x j  x1

 2 y j  y10
  2 j
0  r1,Aj
B  r1, j

X

where
d1 A  ( x A  x1 ) 2  ( y A  y1 ) 2 ,(3)

d 01  ( x1  0) 2  ( y1  0) 2

(8)

where

T

 ( x2  x1 )   x A (2)2  ( y2  y1 )   y A   2r1,2A  r1,2A  ( r1,2A ) 2



  r1,2A   r1,2A   d1 A 
v1A
 
 


A
H1A



1

( xB , y B )




T


 0

 0

0


 0
l x j  x1  ,  j  2  0
l y j  y1   r A 1,Bjr1,Aj  r1, jr1,Bj

0







0 
0 
0 

0 
A 
 r1, j
 r1,Bj (9), and  r1,2A is the RDOA measurement noise and its
stochastic property is a white and zero mean. The
measurement noise of (2), v1A can be assumed to be awhite noise but it cannot be assumed zero mean.
(4)
E  v1A   ( 2 )1A
where( 2 )1A

isthevarianceoftheIII. LINEAR ESTIMATION METHODS FORPOSITION AND HEADING
1. Instrumental Variable Algorithm
The stochastic compensation solution, the RoLS
(robust least squares) in [6] is an useful method because
its formulation resembles with the general least squares
method and computational burden is low. However, if
the sensor network has a characteristic of stochastic
information being varying, the estimation results may
be incorrect. To overcome this problem when thestochastic properties are unknown, the IV method canbe an alternative solution. The IV method uses
instrument variable for avoiding the correlations which
are between the measurement matrix uncertainties,

RDOA

measurement noise,  r1,2A . In a low SNR condition, thisfactor can cause an estimation error, so that it needs tobe removed from the measurements y1A or to be
estimated by setting it as a new state variable.
For the other transmitter, B, the measurement equation
can be derived likewise.
2. Position and Heading Estimation Model
The intermediate variable method makes the nonlinear
localization problem to the linear problem so that a©ISAROB 2011

T

992

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

cor (  ,  ) or between the measurement matrix
uncertainties
andthemeasurement
noise,cor (  , n ) [8]. This method is unlike the RoLS
algorithm which removes the correlations by using thescale-factor error compensation term, W and the biased
error compensation term, V but flees from thecorrelations by replacing the measurement matrix with
an IV matrix. The IV matrix can be built simply with
one-step delayed measurement matrix because we
assume the measurement noise is white.
cor   (k  1),  (k )   02 n 2 nXˆ ( k )  () k 1 Tkk 1 T1  k 1 T  k( Aug ) ( Aug )   TAug  Augk
1  k 1 T k
Tplim (Aug ) ( z )   Aug  Aug X Aug
k  k k 1 )T k
 plim  (
Aug
Aug

k  

1 0  1 0

T

 

IV
 k 1 )T  k 1 ( k 1 )T z k
Xˆ Aug( k )  (
Aug
Aug
Aug





 ( Aug )T  Aug©ISAROB 2011(k )   TAug  Aug) zk 
k 1 T
Aug

(20)

 (
1

Aug



)T X Aug  X Aug

IV. SIMULATION RESULTS
To show the performances of the position and heading
estimators, we compare them with the nominal leastsquares method through simulations. The sensor
network contains 6 fixed sensor nodes at (0, 0), (-1.015,0), (-1.015, 1.015), (-1.015, 2.03), (0, 2.03), and (0,1.015) [ m ] . The variances of the RDOA measurements(12)

are all set to 0.02 [ m 2 ] . For batch type comparison, we
accumulate 5000 measurements and calculate the error
performance by using 100 iterations. In this simulation,
we fix the transmitter A and rotate the transmitter B
every 10 degree to check the estimation performances atdifferent target’s center position and heading.
The mean error (ME) for the position is shown in Fig.
2 and its RMSE is shown in Fig. 3. As referred in [6],the NLS (nominal least squares) method has a large
estimation error by the measurement matrix uncertainty
and the bias of the measurement noise. On the contrary,
the RoLS, IV and augmented IV methods have low
estimation error within 0.02 [ m ] . The bias of the IV
method caused by non-zero mean of the measurements
noise is not relatively serious. However, the augmented
IV method compensates properly the error by estimatingit.
The estimation results for the heading are shown in
Fig. 4 and Fig. 5. The trend of heading results of the
NLS resembles with the position results. The heading
error from the cosine is bigger than the one from thesine like the position error. The reason for this is that theheading from the cosine depends on the x-position of
the target and the other is vice versa as shown in (6).
The heading estimation by the augmented IV method
also shows good performance as the RoLS estimator.
Therefore, the augmented IV method can be a practicalmethod because it does not need any stochastic

(13)

(14)

□
Proof:
Since the measurement noises are i.i.d., the autocorrelation of the measurement matrix uncertainties andthe cross-correlation between the measurement matrix
uncertainties and the measurement are derived asfollows:
Aug

1

■

k 

T

(19)

 1  k 1 T  k 
 1  k 1 T k  plim  (( Aug ) z Aug )  Aug   plim k   kk   k

Theorem 1. (Convergence to true position and headingof the proposed algorithm) If there are enough sensors
to observe the position, heading and the biases, and thenoises of sensor network are i.i.d. (independent andidentically distributed), then the estimation results of the
proposed method converge to true position and headingin probability.
IV(15)
plim Xˆ Aug(k )  X Aug  (
1and n ( k ) is the zero mean and white noise which
eliminates the bias from n in (8). With (12), theaugmented position and heading estimator is



(18)

IV
plim Xˆ Aug(k )where I is a 2(n-1) dimensional vector which is (k  1)
E 
Aug

(17)

With these convergences, (18) and (19), the estimation
result, (14) is also able to converge to true position andheading by Slutsky’s theorem [8].

(11)

k



 
  X (k )  n (k )z ( k )     ( k ) I     ( k ) 0  b(k ) 


 






  Aug ( k ) X Aug ( k )



X

k 

However, an error still exists in the estimation result
by the measurement noise bias in (4). Because we
assumed the stochastic properties of the RDOA
measurement are not available, the bias which is thevariance of the RDOA noise cannot be removed fromthe measurements. If the sensor network is able toassume that the RDOA noise variances are identical for
all measurements as the bias common model in [9], it ispossible to estimate the common bias. To estimate thebias, we set the bias as a new state variable and derive
the measurement equation as follows:

I 2( n 1)1  1 0

Aug

k 

 ( ) z 
1

T
Aug

plim

Therefore, the position and heading estimator with the
IV algorithm isIV

TTherefore, each correlation can be rewritten as
convergence in probability.

(10)

cor   (k  1), n(k )   02 n1  z (k )   

 (k  1)
E 
Aug

(16)

993

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

information and additional computational burden.
Estimation error [deg]ME for target heading from cos

V. CONCLUSION
The target’s position and heading can be estimated in
the RDOA measured sensor network with linear
estimator. However, since a nominal least squares
estimator causes estimation error by the measurement
matrix uncertainty and the bias of the measurement
noise, there needs a proper compensation algorithm. We
propose the position and heading estimator and it does
not need an additional compensation procedure and thestochastic information of the RDOA measurement noise.
The proposition is based on the instrumental variable
method and estimates the bias of the measurement noise
under the bias common model assumption. The
estimator works well in noisy environment like therobust least squares method and shows better
performance than the nominal least squares estimator.
The useful aspects that it has low computational burden
and does not need any stochastic information make itutilized for the practical applications.

300

Estimation error [deg]Estimation error [deg]Estimation error [deg]350

Estimation error [cm]Estimation error [cm]

4
NLS
RoLS
IV
IV Aug

3
2
1
0

0

50

100

150
200
Heading [deg]

250

300

350

Fig. 2. Mean error for the target’s center position
Estimation error [cm]RMSE for target x-position
20
NLS
RoLS
IV
IV Aug

15
10
5
0
0

50

100

150
200
250
Heading [deg]RMSE for target y-position

300

350

Estimation error [cm]

4
NLS
RoLS
IV
IV Aug

3
2
1
0

0

50

100

150
200
Heading [deg]

250

300

350

Fig. 3. RMSE for the target’s center position©ISAROB 2011

300

350

NLS
RoLS
IV
IV Aug

10
5
0

0

50

100

150
200
Heading [deg]

250

300

350

30
NLS
RoLS
IV
IV Aug

20
10
0
50

100

150
200
250
Heading [deg]RMSE for target heading from sin

300

350

15
NLS
RoLS
IV
IV Aug

10
5
0

0

50

100

150
200
Heading [deg]

250

300

350

[1] Agihili, F., and Salerno, A. (2009), Attitude determination andlocalization of mobile robots using two RTK GPSs and IMU,
IEEE/RSJ Int. Conf. Intelligent Robots and Systems, 1:2045-2052.
[2] Abel, J. S., and Smith, J. O. (1989), Source range and depth
estimation from multipath range difference measurements, IEEE
Trans. Acoustics, Speech, and Processing, 37(8):1157-1165.
[3] Chan, Y. T., and Ho, K. C. (1994), A simple and efficientestimator for hyperbolic location, IEEE Trans. Signal Processing,42( 8): 1905-1915.
[4] Cheung, K. W. et al. (2006), A constrained squares approach tomobile positioning: algorithms and optimality, EURASIP J. Appl.
Signal Process., 2006: 1-23.
[5] Huang, Y. et al. (2001), Real-time passive source localization: apractical linear-correction least squares approach, IEEE Trans.
Speech and Audio Processing, 9(8):943-956.
[6] Ra, W. S., et al. (2007), Recursive robust least squares estimator
for time-varying linear systems with a noise corrupted
measurement matrix, IET. Control Theory & Applications,
1(1):104-112.
[7] Ra, W. S., Whang, I. H., and Yoon, T. S. (2009), Constrainedrobust Kalman filtering for passive localization, ICROS-SICE
Int. Joint Conf., 1: 4516-4521.
[8] Dogancay, K. (2006), Bias compensation for the bearing-only
pseudo-linear target track estimator, IEEE Trans. Signal
Processing, 54(1): 59-68.
[9] Picard, J. S., and Weiss, A. J. (2008), Localization of networks
using various ranging bias models, Wirel. Commun. Mob.
Comput., 9(1): 553-562.

0
250

150
200
250
Heading [deg]ME for target heading from sin

REFERENCES

5

150
200
Heading [deg]ME for target y-position

100

Fig. 5. RMSE for the target’s heading

NLS
RoLS
IV
IV Aug

100

50

15

0

ME for target x-position

50

0

RMSE for target heading from cos

20

0

10

Fig. 4. Mean error for the target’s headingThis research was supported by Basic Science
Research Program through the National Research
Foundation of Korea (NRF) funded by the Ministry of
Education, Science and Technology (2010-0005318).

10

NLS
RoLS
IV
IV Aug200ACKNOWLEDGMENT

15

30

994

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Omnidirectional State-Changing Gripper Mechanism for Various Objects
Kenjiro Tadakuma1, Riichiro Tadakuma2, Hiroki Tanaka1, Takuto Fukuda1, Mitsuru
Higashimori1, Makoto Kaneko1 , 1.(Osaka University, Japan), 2(Yamagata University, Japan)
This paper describes the morphing omnidirectional gripper which is able to grasp various objects with low
melting point alloy, functional fluid, dilatancy fluid, and so on. The deformable part of the gripper changes its shapeby covering all direction of objects and makes the contacting area higher. This time, we especially focus on the "HotIce" phenomena to realize higher grasping motion. The basic performance of fluid of CH3COONa has been observed.
Key Words: Mechanism, Deformable, Hot-Ice Phenomenon, Omni-Gripper, Conformable

1. Introduction

from the gripper. Our proposed configuration (in 2008) can grasp
soft objects effectively without any large damages, andconfiguration of the gripper itself is totally different from theprevious any researches. And we already proposed the mixtures ofthe plural numbers of the particles and air or liquid as the contentsof the grippers sack in that paper.

1.1: Mechanism of the former morphing omnidirectional gripper
The former morphing omnidirectional gripper had a mechanicalproblem of poor response when the low melting point alloy,
enclosed substance, changed its phase from liquid to solid. In thispaper, we propose a method to improve gripping response making
use of the phenomenon of supercooling.
As the enclosed substance, functional fluids like magneto
rheological and electro rheological fluids, dilatant fluid, mixture ofparticulate and its solvent, and mixture/composite of all thesesubstances, we speculate, may be possible candidates.
In our former prototype, we used a low melting point alloy which
solidifies at room temperatures. Though that exhibits relatively highrigidity when it grasps objects compared to the candidates listed
above, it has a problem of poor response caused by rather long timerequired to the phase change from liquid to solid which needs tooccur after conforming to objects. Thus, in this report, we propose
two alternatives for applications which impose higher importanceon quick response than on high rigidity when gripping objects.

2. Morphing omnidirectional gripper
Quick response does not seem, in our experience, to becompatible with high rigidity at solid- or almost solid state.
Here, we examine the adaptability of the morphing gripper withbag-shaped structure continuously extending from inner- to external
parts to longitudinal and angular offsets. High degree of tolerance tooffsets may be favorable when objects to be grasped are vibrating,objects set on moving bed are vibrating and gripper itself isvibrating. In addition to the present gripper with bag-shaped
structure continuously extending from inner- to external parts, weare developing a double-layered structure that may be folded.

1.2: A proposal to improve response
One is a mixture of gas and particulate substance (glass beads) andanother saturated water solution of sodium acetate CH3COONA
which exhibits supercooling known as “Hot Ice” phenomenon.
When trihydrate of sodium acetate is cooled below 58 degree
Celsius, the melting point, it solidifies suddenly in response toadded nucleus or to stimulation done by metal fragment, forexample. This is the Hot Ice phenomenon we refer to. The
triggering for solidification may be replaced by electric stimulation,
like application of voltage and we are examining this method. In
addition to that, mixture of water solution of sodium acetate andother enclosed substances such as functional fluids described in
Chap. 2 are being studied as well.
In this report, the two items regarding to the basic characteristics
of our morphing omnidirectional gripper will be examined. One isthe basic gripping performance when angle offsets are given togripped objects and another how to use the supercooling of water
solution of sodium acetate which is aimed at improvement ofgripping response.
Just only the jamming phenomena have already proposed in the
1982[12]. In recently(in 2010), Cornell University, University of
Chicago, and iRobot reported the jamming gripper, but theconfiguration is almost the same of that work by G. Bancon et al. It
can grasp various kinds of object, but it needs the pushing force todeform. It means, the grasped object should take the same force©ISAROB 2011

Fig. 1: Example of Previous Mechanical Gripper

Inner-Outer Flexible Sack
Linear Guide

Outer Shell(with Heater and Cooler)
Inner Rod(with Heater and Cooler)
ObjectLow Melting Point Alloy

Fig. 2: Basic Concept of the Morphing Omni-Gripper

995

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Angle

Fig. 5 Gripping Experiments with Offest
Results obtained by analyzing the amount of paint are shown in
Frs. from 4 through 8. These figures demonstrate high conformingof the gripper we proposed.

Fig. 3: Grasping Motion of the Morphing Omni-Gripper
Figure 2 depicts the basic sequence of conforming gripping. By
direct displacement of only outer shell (an element of gripper)
toward the grasped object, gripper may catch it eliminating thedisplacement of the object. It may be possible, conversely, to grasp
object drawing it toward gripper.

3. Gripping characteristics when offset exists
We have measured the gripping characteristic for tilted objects.
Kamakura et al. [2] coated paint on objects and observed thelocation of paint attached in the hand of person who grasped it.
While in our experiment, we coated paint on gripper taking
consideration on large contact area of grasped object.

Triangle Square Cylinder Trigonal Square Circular
Pole

Pole

Solid

Pyramid PyramidCone

SphereGripping Rate (No offset)
Fig. 6 Experimental Result without offset(1) Ready(2) Grasping (3) Finish (4) Release

Fig. 4 Gripping Experiments
A

B

Triangle PoleA

B

Square PoleA
CylinderSolid

B

Trigonal PyramidA

Gripping Rate (5mm)©ISAROB 2011

996

B

Square Pyramid

Circular
Cone

Sphere

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig. 7 Experimental Result with offset

Inner Shaft
Outer Shell

Supporting Module
Linear Guide

Inner-Outer
Continuous Sack

Table 1: Specification of the Morphing Omni-Gripper

Linear Spiral Copper Pipe
Cartledge Heaterfor Function of CoolingFig. 8 Prototype Mechanical Design of the Omni-Gripper

1

2

3

Fig. 10 Phenomenon of “Hot-Ice”

1

Fig. 9 Overview of the Omni-Gripper

2

3

4. Taking advantage of Hot-Ice phenomenon
Here, we discuss how to improve the response performance ofsolution (exhibits supercooling described above) as the enclosed
substance in gripper with bag-shaped structure continuously
extending from inner- to external parts. Triggering to solidification
may be done by:1) mechanical stimulation
2) introduction of nucleus
3) electric stimulation

4

In our experiment this time, we took the third of electric
stimulation in view of simple implementation. The result of ourpreliminary experiment is shown in Fig. 9. Solution of 200ml
volume solidified in some four seconds. This speed depends on themethod of introducing electric stimulation. We are examining
response performance and rigidity of solidified substance varying
concentration of sodium acetate. Thus, we are working to improve
the performance of enclosed substance in gripper which is
conforming to grasped object and grips it firmly at the same time.

©ISAROB 2011

5

Fig. 11 Grasping Motion of the Omni-Gripper

997

6

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

1

2

4

[7] N. Yamano, S. Takamuku, and K. Hosoda, Development ofunderactuated humanoid robot hand with conforming grip,
1A1-A11, Robotics-Mechatronics Conference, 2008[8] H. Iwata, T. Hayashi, H. Shiozawa, K. Kimura, S. Ishii, K.
Sakagami, T. Ohta, K. Iwamoto, and S. Kanno, Structuraldesign of high skill and multi-fingers equipped with passive
flexibility on finger and palm, 1A1-A01, RoboticsMechatronics Conference, 2008
[9] K. Takagi and T. Omata, Electric artificial hand withaugmentation mechanism of gripping force, 12th Robotics
Symposium, 2B4[10] K. Tadakuma, R. Tadakuma, S. Teshigawara, Y. Mizoguchi, H.
Hasegawa, K. Terada, T. Takayama, T. Omata, A. Ming, and
M. Shimojo, Morphing omnidirectional gripper-Proposal ofbasic concept and first prototype of mechanical structure,
1E1-01，26th Conference of Japan Robotics Society, 2008[11] M. Wakabayashi, T. Mitsuda, and S. Kawamura, Developmentof a force display for leg motions using adhesive particles withvariable constraints, 1P1-F06, Robotics- Mechatronics
Conference, 2002.
[12] G. Bancon and B. Huber: Depresion and Grippers with Their
Possible Applications, 12th ISIR, Paris (1982) pp. 321-329.

3

5

6

Fig. 12 Grasping Power of the Gripping in Solid State

5. Summary
We have carried out an experiment on angular offset of grasped
objects to examine the basic performance of our morphingomnidirectional gripper. It proved the good conforming
performance of the gripper. And some description was made on
application of supercooled sodium acetate to improve the grasping
performance of the gripper.

References[1] Name of invention: Device to grasp objects, Number ofapplication: Applied patent 1995-306353, Date of application:
November 1, 1995, Publication number: Publicized patent
1997-123082, Date of publication: May 13, 1997, Applicant:
Kao Corporation, Inventor: K. Uehara and another[2] N. Kamakura, M. Ohmura, H. Ishii, F. Mitsuboshi, and Y.
Miura，Gripping pattern of healthy hands - An approach to theircategorization-, Japanese Journal of Rehabilitation Medicine,vol.15, no.2, June, 1978
[3] N. Kamakura, F. Mitsuboshi, N. Asami, and M. Nakata,Moving pattern of healthy hands when they manipulate object,
Japanese Journal of Rehabilitation Medicine, vol. 23, no.2,
March, 1986
[4] K. Harada and M. Kaneko, Conforming gripping of objects
more than one – Kinetics and conforming grasp，J. Japan
Robotics Society, Vol.16, No.6, pp.860-867, 1998
[5] T. Yoshitomi, Story of fish squama, editorial supervisor,
Japanese Society of Fisheries Science, ISBN978-4-42585261-1[6] S. Hirose, Biomechanical Engineering, Kogyo Chosakai
Publishing, 1987

©ISAROB 2011

998

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Object Co-occurrence Graph for Object Search in 3D Environment
Puwanan Chumtong, Yasushi Mae, Kenichi Ohara, Tomohito Takubo, Tatsuo Arai
Graduate School of Engineering Science, Osaka University
1-3, Machikaneyama-cho, Toyonaka, Osaka, Japan 560-8531(Tel: 81-66-850-6365){c_puwanan, k-oohara, takubo}@arai-lab.sys.es.osaka-u.ac.jp{mae, aria}@sys.es.osaka-u.ac.jp

Abstract: We propose a method of using object co-occurrence graph for mobile service robots to search for small-scale
objects in 3D environments without pre-defined map. Object co-occurrence graph describes co-occurrence relations
between objects in scenes. The presence of other large objects in the environment, which are easier to be found,provides cues to search for a small object related to the large object. If the target object is often near to a large object,robot firstly finds the large object, and then searches around the large object for the target object. Thus, using object cooccurrence graph makes object searching task easier and more efficient. The object co-occurrence graph is
automatically constructed by many tagged images from the WWW. Our experiment shows a robot searching for a target
object using object co-occurrence graph.
Keywords: Object Co-occurrence Graph, Object searching, Mobile robot.

I. INTRODUCTION
The object search is an essential task for mobile
robot which helps human in everyday environment.
Many methods for object search in 3D environment
have been proposed. For example, David et al. [1],Masuzawa et al. [2] and Jeremy et al. [3] use color
information to verify the candidate of target object inthe scene. However robot may face some difficulties tofind the small target object in the case of occlusion
problem and insufficient current image’s resolution.
On the contrary, some methods focus on existingcues in the observing environment for object search.
Dominik et al. [4] proposed the usage of object’s
relationship to search for target objects in unknown
environment. Cipriano et al. [5] use the position
probability of the observing map and found objects intheir searching task. Ksenia et al. [6] utilize position
probabilistic with the given hint in motion planning forsearching task. Importantly, the utilizing of cues candecrease the difficulty of searching task when target
object, especially small object, cannot be observed in
the current robot’s view.
In our approach, we focus on employing object’s cooccurring relation as our cues because this relation isrepresented for the frequency for how often object A
and object B co-occurr together. Thus we propose the
Object Co-occurrence Graph (OCG) describing the cooccurring property between objects. That is, we utilize
the existing of cue objects to search for target object inunknown environment. This idea can be illustrated inFig. 1. The benefit of using object co-occurring cues
among previous mentioned researches is that ourapproach can be easily constructed and applied as
robot’s knowledge.

©ISAROB 2011

Fig. 1: The utilizing of cues in observing environment
for robot’s searching task

II. OBJECT CO-OCCURRENCE GRAPH
In our approach, the concept of Object Cooccurrence Graph for robot’s searching task can beillustrated in Fig. 2. One arbitrary object oi (e.g., desk,
chair, etc) may have many related objects according to
its reference database. Moreover, in term of cooccurring relation, each related objects of oi may havesome relationship with any objects in OCG database.
For each couple of objects oi and ok , we will obtain
2 values; poi | ok  and pok | oi  . At here, poi | ok 
is the probability to find target object oi when cue
object ok is found and vice versa.
In order to utilize these probability values, we selectthe next visiting object candidate based on the priority
ranking among others in database. That is, if our target
object, ot , is not observed while cue objects are found,robot then can determine next visiting object o j byusing the following concept:oˆ j  argmax p ot | o j(1)
o j Obj

999





The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig. 2: The main concept of OCG structure
We know that Obj  o1 ,...,or  where r is number ofobjects. It is assumed that the higher possibility value,
the higher chance to find target object nearby that cue.

Fig. 3: Proposed method for OCG construction
computation of these 2 construction methods are basedon different searching concept as follow.

III. INTERNET-BASED OCG
CONSTRUCTION

1. Dependent Co-occurring based Construction
This method constructs database based on therelationship of all objects. By using name of oi assearching keyword, we will obtain wki ,i  W oi  . The
standalone occurrence noi  and co-occurrencen oi  o j
arecountedfromallimages,wkt ,t  W ot  : t  1,...,r . In order to gain the cooccurrence possibility value, we employ the definitionof conditional probability:p(oi  o j )(2)p(oi | o j ) p (o j )The OCG can be constructed by manually learningfrom observing actual situation. However this learning
method is not appropriate since it is time consuming. Asa learning method, one interesting research [7] uses theobject co-occurrence gathering from many internet
images as prior knowledge for robot. As a result of this,the expected length of the path to the object isminimized. Due to benefits of internet data, the similar
concept was also applied in our work. The different
point of our approach from others is that our approach
does not require pre-defined map for robot’s searching.
To construct OCG, we employ image hosting
website that contains a form of metadata as ourreference database. Instead of employing well-known
web search engine such as Google, the related work [8]describes the difficulty of using images from Google
search as training data because most of returned images
may be visually unrelated to the intended keyword.
Moreover Google’s images do not contain tagged
information. Thus we use website such as Flickr thatgives us word-tagging images provided by millions ofinternet users. Most of those provided images are
related to searching keyword since its image searchingprocess is based on word-tagging of each image.
The concept of OCG construction is shown in Fig. 3.
The name of oi is used as keyword to search forrelevant images within main page, W oi  . Based ontags of these images in sub-webpage, wk i ,i W oi  , thestandalone object occurrence and object co-occurrence
will be obtained. In this work, we have proposed 2
methods; dependent and independent co-occurring
method, for OCG construction. For dependent cooccurring method, possibility values depend on tagged
information in all searching images but the adding of
new item or the deleting of existing item can affect thechanging of possibility values of all object couple indatabase. Thus the recalculation of all values is needed.
On the other hand, for independent co-occurring method,
it is assumed that probability values between couples
are independent. Thus probability values of new adding
items with others can be separately calculated. The©ISAROB 2011



We employ the following definition:
 n
n Ck   k  (3)Thus, we will have r C 2 pairs of object cooccurring couple. We will finally have 2   r C2  possib
ility values for OCG database.
2. Independent Co-occurring based Construction
In the case of considering relationship of   th pairbetween o i and o j , the standalone occurrencen (oi ) and n (o j ) are calculated from images
wki ,i W oi  and wk j , j  W o j . At here, we define that  1,...,r C2  because we will obtain r C 2 pairs ofobjectco-occurring
couple.
We
also
canfrom those images. When theobtain n oi  o j
relationship between o i and o c is needed, we just focus
on the information in wki ,i and wk c,c . Unlike previous
mentioned method, recalculation of entire values is notnecessary.

 





IV. TRAINING RESULTS AND DISCUSSION
We design the training set based on images in Flickr
website. The word “period” defines the image-searchingof oi  Obj : i  1,...,r within M images. We
define that:
r

M

m

i

(4)

i 1

At here, m i can be varied according to the number ofreturned images in 1 webpage when name of o i isused as searching keyword. Thus the searching of M

1000

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig. 4: Training results for OCG construction. Training results of dependent co-occurring method are shown in (a), (b),and (c) while training results of independent co-occurring method are shown in (d), (e), and (f).
40 searching periods returns us 9,571 related images. It
can also be noticed that data-lines provide lessfluctuation when number of searching images is morethan 4560 images (after 19th period).

images is so called one period. In our experiment, we
take a consideration to 4 sample objects which are desk,
chair, book, and beverage; that is, we will have r  4 .
For our OCG construction, we train data up to 40
periods and examine how possibility values change as
the number of searching images increases. As can beseen in Fig. 4, we also provide the following definition:- 1st phase: searching keyword of desk ( o1 )- 2nd phase: searching keyword of chair ( o 2 )- 3rd phase: searching keyword of book ( o3 )- 4th phase: searching keyword of beverage ( o 4 )2. Experimental Results of Independent Cooccurring-based Construction
According to the method in section III-2, we obtain
6 pairs of co-occurring relationship as can be seen in Fig.
4(d). Their change is based on the appearance of oi inrelated images. For the 1st pair (   1 ) whose members
are desk and chair, n1 o1  changes during the 1st and2nd phase where we use searching keywords of desk andchair, respectively. From Fig. 4(e), n1 o1  o2 
increases during 1st and 2nd phase and remains constantin other phases because this method concerns only
images of wki ,i W o1  and wk j , j W o2  .
According to Fig. 4(f), the training process of 40
periods returns us 9,578 searching images. It can benoticed that data-lines in Fig. 4(f) have similar
characteristic as data-lines in Fig. 4(c).

1. Experimental Results of Dependent Cooccurring-based Construction
According to the method in section III-1, During 1stphase of Fig. 4(a), it can be noticed that no1  rapidly
increases until the number of searching images reaches
m1 images because the name of o1 is used. During
2nd phase, no1  slowly increases although name ofo 2 is used. This shows that there are some cooccurrences of o1 and o 2 during 2nd phase. The cooccurrence of desk ( o1 ) & chair ( o 2 ) can be easily
observed in Fig. 4(b). On the contrary, no2  does notchange during the 4th phase of Fig. 4(a). This shows that
there is no co-occurrence of o 2 and o 4 during thisphase. We can also see the co-occurrence of chair ( o 2 )& beverage ( o 4 ) in Fig. 4(b) whose data-line remains
constant during 4th phase.
For 40 period-results in Fig. 4(c), data-lines ofprobability values, p oi | o j where oi , o j  Obj : i  j ,provide less fluctuation as the number of searching
period increases. In our experiment, the data training of©ISAROB 2011

3. Comparison Between 2 Construction Methods
The probability results of each couple of objects
after 40-period training are shown in Table 1. Although
the numerical results of probability values between 2
proposed methods are different, their ranking order ofhypothesis cue objects convey the same ranking orderand they indeed accord to the fact of object’s
relationship in realistic environment. From Table 1, it
can be noticed that this knowledge lets robot know
visiting location for object search. That is, if robot is
assigned to search for book but it cannot be found in the



1001

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

TargetObject
desk

chair

book

beverage

Cue
ObjectDependent
Co-occurring

Independent
Co-occurring

chair
book
beveragedesk
bookbeveragedesk
chair
beverage
desk
book
chair

0.08900.06430.03250.08150.03640.0176
0.05750.03550.02500.02960.02550.01750.09100.06490.02980.08460.03200.01490.05980.03090.02080.02790.02190.0151

Fig. 5: Simulation of object search using OCG. (a)shows initial robot’s position. (c) shows robot’s
positions after approaching to desk. (b) and (d) show
robot’s views with recognition results.

Table 1: The comparison of experimental results
between 2 proposed methods

environment. The OCG is automatically constructedbased on searching images within Flickr website. Two
proposed methods for OCG construction; dependent andindependent co-occurring methods are also presented. In
this paper, we also show the successive result of robot’s
simulation of object search using OCG without predefined map.

1st scene, robot then has to determine where it has to go
next to search for it. In case that desk and chair are
observed in the 1st scene, robot will know that desk is 1stplace to be visited while chair is the 2nd place to bevisited. This robot’s knowledge is obtained from OCG.
Although both construction methods provide us thesame meaningful results, the usage of independent will
be more appropriate for real experiment because it canbe easily adjusted when new object is added or existing
object is removed from current database.

REFERENCES[1] David Meger, Per-Erik Forss´en, Kevin Lai, Scott Helmer,
Sancho McCann, Tristram Southey, Matthew Baumann, James J.
Little, David G. Lowe, and Bruce Dow, “Curious George: An
Attentive Semantic Robot”, in Robotics and Autonomous
Systems, 2008[2] Hiroaki Masuzawa amd Jun Miura, “Observation Planning for
Efficient Environment Information Summarization”, inInternational Conference on Intelligent Robots and Systems(IROS), Louis, USA, 2009[3] Jeremy Ma and Joel W. Burdick, “A Probabilistic Framework for
Stereo-Vision Based 3D Object Search with 6D Pose Estimation”,
in IEEE Internal Conference on Robotics and Automation(ICRA), Alaska, USA, 2010[4] Dominik Joho and Wolfram Burgard, “Searching for Objects:
Combining Multiple Cues to Object Locations Using a
Maximum Entropy Model”, in IEEE International Conference on
Robotics and Automation (ICRA), Alaska, USA, 2010
[5] Cipriano Galindo, Juan-Antonio Fernández-Madrigal, Javier
González, and Alessandro Saffiotti, “Robot task planning using
semantic maps”, in Robotics and Autonomous Systems, Volume56, 2008, pp.955-966[6] Ksenia Shubina and John K.Tsotsos, “Visual search for an object
in a 3D environment using a mobile robot”, in Computer Vision
and Image Understanding, Volume 114, 2010, pp.535-547[7] T. Kollar and N. Roy, “Utilize object-object and object-scene
context when planning to find things”, in IEEE Internal
Conference on Robotics and Automation (ICRA), Kobe, Japan,
2009[8] R. Fergus, L.Fei-Fei, P. Perona, A.Zisserman, “Learning Object
Categories from Google’s Image Search”, in International
Conference on Computer Vision (ICCV), Beijing, Chaina, 2005
[9] David G.Lowe, “Local Feature View Clustering for 3D Object
Recognition”, in Computer Vision and Pattern Recognition(CVPR), Hawaii, USA, 20012-15[10] Pedro F. Felzenszwalb and Daniel P. Huttenlocher, “Efficient
Graph-Based Image Segmentation”, in International Journal ofComputer Vision, Volume 59, 2004, pp.167-181

V. OBJECT SEARCH USING OCG
We show a simulation of object search using OCG
and example of object recognition. In our object
recognition method, range data and color image
gathering from single stereo camera are employed as
our input data for robot’s exploration. The experimental
set up can be viewed in Fig. 5(a). In this test, book is
assigned as target object for robot’s searching task. The
robot employs well-known SIFT [9] for target object’s
detector. As can be seen in Fig. 5(b), robot cannot
observe book in the 1st scene due to low image’s
resolution and occlusion from other objects. However
robot found some cues; desk and chair, labeling with
red ellipse and green ellipse, respectively. Cue objects,
considered as large objects, are detected based on range
data and color segmentation result [10]. Consequently
robot obtains cue objects in the observing environment.
Thus, robot can determine the sub-goal’s location basedon OCG knowledge. As a result of this, robot chooses tovisit the desk’s location and continue searching for bookas can be seen in Fig. 5(c). From Fig. 5(d), it can benoticed that robot can finally verify the existing of bookwhich is placed on desk (cue object).

VI. CONCLUSION
This paper has described the usage of OCG as
robot’s knowledge for searching task in unknown©ISAROB 2011

1002

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Study on the wall climbing robot driven by the caterpillar tracks
Jiwu Wang1

Yisong Wang1 Weijie Gao 1

Department of Mechanical and Electrical Engineering, Beijing Jiaotong University, Beijing, China, 100044 1
Sugisaka Masanori2
Nippon Bunri University, Oita, Japan2(jwwang@bjtu.edu.cn)
Abstract: With the development of economy and various techniques, the development of the wall climbing robot isgetting more and more attention. This is because various jobs are not appropriate for human beings, such as some tasks
in some special or dangerous environments, wall cleaning of high building, etc. In view of such problems, it isnecessary to develop a wall climbing robot moving in various working conditions. In this paper, we developed a wall
climbing robot driven by the caterpillar tracks. It can pass some simple obstacles. At the same time it can capture theenvironment pictures with a CCD camera and send the image data directly to the server. Moreover, we analysis its
structures and some functions.
Keywords: Wall climbing robot, caterpillar track.
work on different planes.
Compared with these three types, we find they use
the same way to against the gravity—some mechanicaldevices. Consequently, a big part of energy was used bythese devices.
Because of the technology development, people
have higher requirements of climbing robots. Small size,
mobility, environmental adaptability, low noise, lowenergy consumption and long cruising time become thefurther goals.
Because of the traditional climbing robot has many
shortcomings, therefore the future climbing robot
structure should towards practical direction.
Xuming Cui, Yingfei Sun, Fujun He [2]. shows thatin this few years, scientists discovered the secret of how
does gecko walk on the wall. The secret is the Van der
Waals force. It is weak electromagnetic attraction
produced by neutral molecules, when they are
extremely close. This means can be used intodeveloping a new kind of adsorption device. And theenergy problem has been considered. We need a battery
with high-capacity and light, or by remote control
approach for robots to provide energy and signal control.
To solve the problems above, we discovered a easymeans of using helium balloon. We can use thebuoyancy of the helium balloon to against the gravity.
Then we can save a lot of energy to lengthen the robots
working time. And it is safe for robot, because we don’t
need worry about it drop from the wall or the bottom ofbridges when it power off suddenly.
Helium is one of the six inert gasses. When somehelium leak from balloon and mixed with air, it resultsin blast or fire. So helium is very safe for practical using.

I. INTRODUCTION
Climbing robots can attach to the wall and movewith carrying capacity of the working tool. And work insome special or dangerous circumstances to substitutehumans to complete the relevant tasks. Xiaodong Chenet al [1]. shows that they are used primarily for wallcleaning, maintenance of nuclear power stations, hull
painting, sand blasting, anti-terrorism and testing of
petrochemical oil tank. In recent years, with thedeveloping of the railway, bridge inspection becomes a
new direction of climbing robots.
They can be classified by the type of moving. There
are wheel-type, crawler-type and sucker-type.
Robots with sucker-type can go across small
obstacles, but slowly. And they have lest two suckers
and legs for movement. Then not only the weight will
increase by setting more legs and suckers, but thesystem of control maybe more complex. It is bad forstability.
Wheel-type provides a faster, flexibility-control
means. But it is difficult to holding a continued
adsorptive power. And robots need a big sucker toprovide absorptive power. When robots are working onsome bumpy plane, it is a little trouble of control. We
can’t ensure the wheels have rolled the same distant. If
we add some system to correct routes, the energy
consumption will be raised.
Caterpillar haves strong adaptability to wall and its
landing area is large, but not easily turn. To work on thewall or under bridges, sucker devices are necessary. The
caterpillar is heavier than wheel-type structure. It ismean that caterpillar expends more energy. But it can©ISAROB 2011

1003

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

II. ELEMENTS AND FUNCTIONS
Most of climbing robots must be working with somemechanical parts to provide the power to against thegravity. But when we are designing a climbing robot,
decreasing the weight is an important qualification. As
we know that more weight consumes more energy. So
we have to balance the two aspects----using moremechanical parts to provide more power anddiminishing more structure to lengthen the working time.
Now we present a new means to against the gravity.
It is the helium.
Helium is a very light colorless gas that is one of thesix inert gasses; the most difficult gas to liquefy; occurs
in economically extractable amounts in certain naturalgases. And the helium balloon was used for aviation in
the primary period. Now it can play the same role inclimbing robot field to provide buoyancy.
We use the buoyancy in two ways, some part ofbuoyancy to counteract the weight of the robot. And theother part is used to create enough pressure for friction.
Then the friction can transform into the impetus ofplane motion under the bridge.
Here are some basic equations.
(1)
Fb  M  g  Fp

Fig.1.The illustration of the structure

Fb ——the buoyancy;
M ——the weight of whole robot;

Fp ——the pressure;g —— the gravity acceleration;
Fm  Ff    Fp
Fm —— the impetus of plane motion;

Fig.2.The detail of the structure(2)

 —— the coefficient of friction;

Ff —— the friction between the travelling mechanism
and the bottom of bridge;
Here are some figures to show the structure.

Fig.3.The walking structure©ISAROB 20111004

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Fig.4.Inset a part of walk structure in balloon

III. ANALYSIS AND EXPERIENCES
For proving the means can be used into reality, we
made some analysis about it.
According to the equations below, we drew the
curves to describe the relation between volume ofhelium and buoyancy.
The density of helium is 0.1786g/L (0°C, 1atm).

Fig.3.The relation between volume and buoyancy

IV. CONCLUSION
After analysis, we can see that this means can beused in some robots which have a big body type. And
when the length of a side is above 3m, the helium
balloon can provide appreciable buoyancy. And thepower is clear, without any energy consumption. What’smore, this means can avoid some accident. Such as,
when the robot shut down suddenly or power off inworking time, the helium balloon can protect the robot
from crashed. Consequently, this means is safety andenergy-saving. But we find that wind bring a bad effecton the big balloon. Now we are working on finding a
way to decrease the bad effect.

1. Equations
We assumed the shape of balloon is cube. Then we
set three thickness of balloon. They were 0.0005m,
0.0007m and 0.001m.

Fb   a  Va   h  Vh   b  Va  Vh  (3)Fb ——the buoyancy (kg); a —— the density of air ( kg / m3 ); h —— the density of helium ( kg / m3 );Va —— the volume of helium ( m 3 );

 b —— the density of balloon ( kg / m3 );Vb —— the volume of balloon ( m 3 );Va  Vh  ra  rh
33

REFERENCES(4)[1]Xiaodong Chen et al.(2007) , Police robot. Science
press, Beijing.134-135

ra —— the radius of the volume of balloon replaced air[2]Xuming Cui, Yingfei Sun, Fujun He.(2010),
Research and Development of Wall-climbing Robot ,Science Technology and Engineering. China academic
journal electronic publishing house, P.R.China.26722677( m );
rh —— the radius of the volume of helium ( m );ra  rh  hb

(5)

hb —— the thickness of balloon ( m );2. Figures
After we did some calculation, we draw the curves
between volume and the buoyancy.

©ISAROB 20111005

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Authentication of the reconstructed image from Computer Generated Hologram:
To use the digital watermark.
Norihiro Fujii, Ken-ichi Tanaka
School of Science and Technology, Meiji University
1-1-1 Higasi-Mita, Tama, Kawasaki-City, Kanagawa 214-8571, Japan
E-mail: ce91055@isc.meiji.ac.jp, tanaken@isc.meiji.ac.jp
Abstract: Computer Generated Hologram has been targeting the application areas such as a reconstruction three
dimensional image, project the image of virtual object, optical information processing and optical memory. Laser beam
is not required when synthesis and reconstruct CGH. In addition, it is expected to function as a digital watermarking.
Digital watermarking is a technology to embed data into digital content. Digital content needs information security.
Digital watermarking is used for information security. In this paper, the correlation between original image andwatermark image are taken. Watermark image is made from CGH. Phase Only Correlation is adopted as a correlation
method. As a result, have expanded the size of the image, little negative impact did not appear on the correlation value.
It is possible to identify self and others using the image correlation value a state of the image of expanded size. These
results suggest that the using of this proposal can be considered easier to identify the state between the auto correlationand cross correlation. In addition, we found that even large images are identified. Therefore, this proposal will be able
to increase the possibility of adaptation to the authentication of the reconstructed image calculated by CGH.
Keywords: Computer Generated Hologram (CGH), Complex Hadamard Transform (CHT), authentication

I. INTRODUCTION
Inverse Hadamard Transform is Eq.(5).
Computer Generated Hologram has been targeting the(5)f  nHFH T
application areas such as a reconstruction threeComplex Hadamard transform is used to complex
dimensional image, project the image of virtual object, Hadamard matrix. Complex Hadamard matrix is using theoptical information processing and optical memory.[1] real and imaginary parts. Complex Hadamard matrix is
Laser beam is not required when synthesis and reconstruct described as follows.
CGH. In addition, it is expected to function as a digital(6)
H 0  1
watermarking. Digital watermarking is a technology to H n  1 iH n  1
embed data into digital content. Digital content needs(7)
H n   information security. Digital watermarking is used foriH n  1 H n  1 
information security.[2] CGH is considered to be theComplex Hadamard Transform is as similar processing
practical application of information security as a firstas Eq.(3), and use complex conjugate of transposed matrix.
purpose. Therefore, authentication of the reconstructed
The character similar to Eq.(4) is approved to Complex
image from CGH is necessary for digital watermarking. In
Hadamard Transform because complex conjugate matrix isthis paper, the correlation between original image andsimilar to inverse matrix.
watermark image are taken. Watermark image is made(8)
F  1 n HfH from CGH. Phase Only Correlation is adopted as a(9)f  nHFH
correlation method.
H*: complex conjugate Hadamard matrix

II. EXPERIMENT
1.

Complex Hadamard Transform
Hadamard transform is the orthogonal transformation. 2. Sample images
512 × 512 pixels on the image of the black, starting at
Processing can be performed by the sum of the product ofHadamard matrix and the input data.[3] The basis of the top left of the image of the alphabet A ~ Z of the one
Hadamard transformation is a Hadamard matrix. which created the characters are written in white letters.
Image files are created with 256 colors bitmap format. This
Hadamard matrix is described as follows.
image as original image, used to convert those who(1)
H 0  1
applied.
H n  1 
 H n  1
(2)
Under the same conditions, 256 × 256 pixels images are
H n    H n  1  H n  1
made.
Each images shown in Fig.1, Fig.2 and Fig.3.
At this time, the size of the input image should be thesame as that of Hadamard matrix. Two dimensional
3. Experiment
Hadamard Transform is described as Eq.(3) of follows.
Correlation values of images are measured in theT(3)F  1 n HfH
following combination of image groups. Correlation values
HH T  I(4)
in type 1 and type 2 are measured with a combination ofbrute force. FT is Fourier Transformf: input
H: Hadamard matrix
F: output©ISAROB 2011

HT: transposed Hadamard matrix

1006

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

In the experiment No.2, the width between maximum
value and the second highest correlation value is aboutType 1
Type 20.08941236. Set of threshold is difficult.
Meanwhile, result of experiment No.3 is the best value.
Reconstructed image from1
Original image
And, result of experiment No.4 is the second best value.
CGH made by FT
In the experiment No.4, the width has been reduced toReconstructed image from2
Original image
86% of the width of experiment No.3. This result is better
CGH made by CHT
Reconstructed image from Reconstructed image from than the result of experiment No.5. Threshold setting is3easily if the width of this level can be secured.
CGH made by FT
CGH made by FT
Reconstructed image from Reconstructed image from4
CGH made by CHT
CGH made by CHT
5Original image
Original image
Table 1. Combination of image groups

Fig.1. Original image (A)Fig.2. Reconstructed image
from CGH made by FT (A)Fig.3. Reconstructed image from CGH made by CHT (A)III. RESULTS AND CONSIDERATION

Fig.4. Combination of image groups and correlation value

The threshold is set to identify “self” and “other”.
VI. CONCLUSION
Threshold is set as the value shall be uniform in each of theReconstructed
image from CGH made by CHT iscorrelation. Threshold is set between “autocorrelation”
comparedwithreconstructed
image from CGH made by(maximum) value and the second highest correlation value.
FT
at
the
pointsofthedetectionaccuracy with correlation
The width between maximum value and the second highest
value.
And
thisstudy,
toexpandthe size of the originalcorrelation value should be broad as possible. The second
image
and
the
watermarkimage
aretried.
highest correlation values were as follows.
Usingacombinationofautocorrelation
can be obtained.
In the experiment No.1 and No.2, maximum valuesItispossibletoidentify
selfandothersusing
the imagewhich a state of “autocorrelation” are not in 1.0. Therefore,
correlation
value.
These
resultssuggestthat
theusing ofin the process of dividing the value of each element in that
this
proposalcanbeconsideredeasiertoidentify
the stateseries the maximum value of the series (normalization
betweenthe
auto
correlation
and
cross
correlation.
process) are done.
Therefore, there is a good possibility that the adaptation
Table 2. Combination of image groups and correlation to authentication, in the case of this proposal using for
calculation method.
value: The second highest correlation valuea. 512 × 512

b. 256 × 256

a-b

REFERENCES

1

0.89104747

0.80709695

0.08395052

20.91058764

0.83366690

0.07692074

30.03422546

0.03426302-0.00003756

40.16488647

0.14813232

0.01675415

50.69196076

0.56340830

0.12855246[1]T.Yatagai, (2004), Optical Computing, Kyoritu Shuppan
Ltd (in Japanese).
[2]N.Komatsu, K.Tanaka (2004), Digital-watermarking
technology, The Institute of Image Electronics Engineersof JAPAN (in Japanese).
[3]Z.Kiyasu (1983), The Hadamard matrix and its
application, IEICE (in Japanese).

©ISAROB 2011

1007

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

An aerial handwritten character recognition based on motion direction
and ratio of stroke length
Y. Nishida1, K. Ogura2, H. Miura2, N. Matsuda2, H. Taki2 and N. Abe3
1

Fukui University of Technology, Japan (nishida@fukui-ut.ac.jp)
2
Wakayama University, Japan
3
Kyushu Institute of Technology, Japan

Abstract: This paper describes a method to recognize a character handwritten in the air. This recognition method
evaluates the motion direction instead of positions of the device. It also doesn't evaluate the motion direction of pen's
up-down, so it is one-stroke cursive character recognition. It selects character candidates with dynamic programming(DP) matching algorithm. We made one-stroke cursive character recognition software prototype based on mousepointer trajectory. We adopted Genetic Algorithm in order to define matching parameters. We improved recognition
rates by comparing “motion direction and ratio of stroke length” of detected data with dictionary. As a result, weachieved correct recognition rate about 90% for categories of Hiragana characters and numeral characters.
Keywords: on-line character recognition, one-stroke cursive character

I. INTRODUCTION

II. RELATED WORK

Recently, we have a lot of opportunity to use Cloud
services. However, it is necessary to input text data inorder to use many Cloud services. It is difficult for us to
input text or character without the desk. If we can use a
new device which provides the input function by
handwriting in the air at anytime and anywhere, it ismore convenient for us to use Cloud services. So, we
study on-line aerial handwritten character recognition
system. There are many problems to realize such
recognition system. Our issues are "detection of pen's
up-down in the air" and "limitation of handwriting area,
character size and writing speed".
We proposed an on-line one-stroke cursive character
recognition method based on motion direction. This
recognition method regards a character as one-strokeand evaluates the motion direction instead of position
information. The recognition system obtains the
direction data (called "direction code") when a user
moves an input device. And it selects charactercandidates with DP matching algorithm. One of theadvantages of using DP matching algorithm is that therecognition system obtains the pattern of the inputtedcharacter and avoids individual differences.
We made one-stroke cursive character recognition
software prototype based on mouse pointer trajectory. In
this paper, we present an improvement of an aerial
handwritten character recognition based on both motion
direction and ratio of stroke length.

In the preceding researches, these character
recognition systems use the pen lifting information(pen's up-down information) [1] [2]. The pen's up-down
is natural operation to write the character on the paper.
But when we write the character in the mobile
environment in which there is neither tablet nor paper,
the system should be designed so that the pen's up-down
operation is omitted from the user job. Therefore, we
adopted one-stroke cursive character recognition.

©ISAROB 2011

III. CODING METHOD
In our previous work [3], we adopted the series ofthe stroke direction code "a"-"h" and “S” code whichmeans the start point, the end point and slightmovements, to find the characteristic of the Hiragana
pattern (as shown in Fig. 1). In this case, the sample
code of Hiragana character "し" shown in Fig.2. was"SdcbaS".
And we applied Genetic Algorithm in order todefine penalty codes and dictionary parameters for DP
matching algorithm. As a result, we got general penalty
codes which don't depend on users' habit of writing andthe dictionary. The system's correct recognition rate hascome to 85% for categories of Hiragana characters. But
there are some recognition errors in similar shapecharacters, for example "う","ら" and "ろ". This isbecause our previous system is not based on theinformation of stroke length.

1008

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Our new method accumulates same direction code
based on proportion of stroke length.
For example, if Hiragana character "し" consists of
50% code "d", 10% code "c", 10% code "b" and 30%
code "a" and if total number of coding data size is 30,
the sample code of Hiragana character is"dddddddddddddddcccbbbaaaaaaaaa".
Table1 shows dictionary examples of new method(total number of coding data size is 50).

Fig.3. A screen-print of the prototype system’s GUI
Table2. Improvement of ratio of stroke length
Previous method
New method
Rec. rate
Error
Rec. rate
Error
い
8/10
り,し
10/10
う
8/10
ら
9/10
ら
こ
9/10
て
10/10
ら
6/10
う
9/10
す
ろ
6/10
ら,う
10/10
Total
400/470 (85.1%)
436/470(92.8%)

Fig.1. Direction code data

Code : dRatio of stroke
length : 50%

V. CONCLUSION

Code : cRatio of stroke
length : 10%

Code : b
Ratio of stroke
length : 10%

This paper has shown the on-line one-stroke cursive
character recognition and its improvement method of
the aerial handwritten character recognition based on
both motion direction and ratio of stroke length. Not
only characters " う ", " ら ", and " ろ ", the correct
recognition rate grow up as a whole. The system's
correct recognition rate has come to about 90% forcategories of Hiragana characters and numeralcharacters. The new method can ignore the detection
noise caused by the slight movement code “S”. Thismethod is more robust and effective for addition of new
characters than the previous system.

Code : aRatio of stroke
length : 30%

Fig.2. The shape of Hiragana character "し"

Char.
あ
い
う

Table1. A part of dictionary codeCodebbbbbbbbbggggeedddddccaaeeefhaaabbbbccddee
dddddddddcccccaaaaaaaaaaaaaaaaaaacccddddddddd
cccccccffffeeeeeaaabbbbbbbbbcccddddddeeeeeeeee

REFERENCES
IV. EVALUATION[1] Sudou T, Nakai M, Shimodaira H et al (2002), Online Overlapped Handwriting Recognition Based on
Substroke HMM (in Japanese). IEICE PRMU2001-65,
pp.163–170[2] Oki T, Nakagawa M (2003), Implementation and
Evaluation of a Combined Recognition System using
Normalization of Likelihood Values (in Japanese).
IEICE PRMU2002-196, pp.43–48[3] Nishida Y, Ogura K, Miura H et al (2010), AnAerial
Handwritten Character Recognition Using Moving
Direction Only (in Japanese). HIS Vol.12 No.3:91-98

We made one-stroke cursive character recognition
prototype system based on mouse pointer trajectory andevaluated correct recognition rate by comparing our
new method to our previous method.
Fig.3 shows a dialog of the prototype system's GUI
view and Table2 shows comparison between theprevious method and the new method with correct
recognition rates and recognition failure characters.

©ISAROB 2011

1009

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Gamer robot for the rock-paper-scissors game by hand motion recognition
Ho Seok Ahn, Dong-wook Lee, and Hogil Lee
Korea Institute of Industrial Technology (KITECH), Republic of Korea(Tel : 82-31-8040-6318; Fax : 82-31-8040-6330)(hoseoka@gmail.com, dwlee@kitech.re.kr, hglee@kitech.re.kr)

Abstract: This paper presents a gamer robot system for the rock-paper-scissors game by hand motion recognition
without any additional units. The proposed system consists of three parts: game management part, hand motion
recognition part, and robot hand control part. 1) The game management part decides a motion of robot hand amongrock, paper, and scissors. After gaming, it makes a reaction about the result of the game using speech or facial
expression. 2) The hand motion recognition part realizes the hand motion of the opponent. It does not use any
additional unit on body, and only uses a camera on a robot. 3) The robot hand control part shows the decided motion ofrobot hand. A robot hand has four fingers which are controlled independently.
Keywords: Hand motion recognition, gamer robot, rock-paper-scissors game, finger design & control.

I. INTRODUCTION

II. Hand Motion Recognition System

Humans usually communicate with computer or
robot using additional input devices, such as keyboard,
touch screen. To make simple communication
environment, many researchers have studied about
Human Computer Interaction (HCI) and Human Robot
Interaction (HRI) [1-3]. Especially, communication
using hand motion, which is the most definite
expression in human body, is studied for interaction
with robot systems. Moreover, the speech-impaired andthe hearing-impaired are able to communicate withrobots using sign language [4, 5].
There are various researches about hand motion
recognition, for example, motion detecting by wearing
glove with sensors, vision processing of hand motion,etc. In the motion detecting by wearing glove withsensors, although it is easy to detect and recognize handmotion in real time, additional input device is needed[6]. On the other hand, in the vision processing of handmotion, although additional input device is not needed,
recognition rate is dependent on environment, such as
shadow, illumination.
In this paper, we present a gamer robot system forthe rock-paper-scissors game by hand motion
recognition using vision processing. The gamer robot
system consists of three parts; game management part,
hand motion recognition part, and robot hand control
part. In Section 2, we introduce a hand motion
recognition system. In Section 3, we explain a fourfingered robot hand system. In Section 4, we show theexperimental results. Finally, we conclude this paper in
Section 5.

Fig. 1. The diagram of the proposed hand motion
recognition system©ISAROB 20111010

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

As humans make various hand motions, it is difficult
to understand all hand motions. The hand motion
recognition system by vision processing, we introduce
in this paper, understands only three hand motions; rock,
paper, and scissors. Fig. 1 shows the diagram of the
proposed hand motion recognition system. There are
two steps; the first step is hand detection and the second
step is motion recognition. In the hand detection, we get
the biggest hand image from the camera of robot system.
In the motion recognition, we classify hand motion
among rock, paper, and scissors. We use HSV color
space for the hand motion recognition system, because
it is conceptualized in human vision in terms of othercolor-making attributes, such as hue, lightness, andchroma [7].
2.1. Hand Detection
We extract candidate pixels of hand using skin color

detection. From Fig. 2 to Fig. 5 shows the results of skincolor detection process. 1) We get the input image fromthe camera of robot as shown in Fig. 2. 2) We set thethreshold range for skin color. 3) We get the image
shown in Fig. 3 from the hue image of Fig. 2 using
threshold range. The white colored pixels are detected
pixels as skin color. 4) We get the image shown in Fig. 4
from the saturation image of Fig. 2 using thresholdrange. The black colored pixels are detected pixels asskin color. 5) We get the skin colored image shown in
Fig. 5 by logical operator conjunction between Fig. 3
and Fig. 4. The white colored pixels are detected pixels
as skin color. As skin color detection extracts similarcolored pixels with skin color, noise is mixed in Fig. 5.
Therefore, we reduce noise of Fig. 5 by Morphology
algorithm [8]. Then, we classify the candidate pixels ofhand by well-known Labeling algorithm. We assume
that the biggest lump of pixels is the hand.

Fig. 2. The input image from camera

Fig. 5. The candidate pixels of hand (white)

Fig. 3. The hue image of skin color (white)

Fig. 6. The edge image of hand (white)

Fig. 4. The saturation image of skin color (black)Fig. 7. The significant features of hand

©ISAROB 2011

1011

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

VI. EXPERIMENTS

2.2. Motion Recognition
We detect the edge of the hand to recognize handmotion as shown in Fig. 6. Then, we extract significant
features such as convex points for the end points offingers and concave points for the interdigital points offingers as shown in Fig. 7. We calculate the length offingers using convex points and concave points offingers. We classify the hand motion using the
calculated result among three candidate hand motions;rock, paper, and scissors.

We tested the explained hand motion recognition
system. Fig. 11 shows the recognition result about rock.
Fig. 12 shows the recognition result about scissors. Fig.
13 shows the recognition result about paper. Althoughbackground is disordered, the explained hand motion
recognition system recognized correctly.

III. Four-Fingered Robot Hand System
We design the four-fingered robot hand system forplaying the rock-paper-scissors game. The robot fingers
are designed using small gears instead of wires toaccurate control without compensation. Each finger isable to control independently to express scissors. Fig. 8
shows the designed four-fingered robot hand system.
Fig. 9 shows the 3 DOF (Degree Of Freedom) designed
robot finger with three knuckles and three robot fingers
are designed identically. Fig. 10 shows the 4 DOF
designed thumb with three knuckles. Thumb has one
more DOF due to the saddle joint to lift some objects.

Fig. 11. The recognition result about rockFig. 12. The recognition result about scissors
Fig. 8. The four-fingered robot hand system

Fig. 9. The designed robot finger
Fig. 13. The recognition result about paper
We implemented the designed four-fingered robot
hand system based on the heterogeneous reconfigurable
modular architecture [9]. Fig. 14 shows the expressionof rock. Fig. 15 shows the expression of scissors. Fig.
16 shows the expression of paper.

Fig. 10. The designed thumb©ISAROB 2011

1012

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

In the four-fingered robot hand system, we designed
palm, thumb, and finger using small gears instead ofwires to accurate control without compensation. Then,we implemented the designed robot hand system of
their fingers are controlled independently. Two systems
are integrated based on the heterogeneousreconfigurable modular architecture for playing therock-paper-scissors game. In the future, we will
research about recognition of sign language.
Fig. 14. The expression of rock

REFERENCES[1] Bronstein, A.M., Bronstein, M.M., and Kimmel, R.,“Rock, Paper, and Scissors: extrinsic vs. intrinsic
similarity of non-rigid shapes,” In Proceedings of the
11th IEEE International Conference on Computer
Vision, pp. 1-6, 2007.
[2] Hasuda, Y., Tshibashi, S., Kozuka, H., Okano, H.,
and Ishikawa, J., “A robot designed to play the game"Rock, Paper, Scissors"”, In Proceedings of the IEEE
International Symposium on Industrial Electronics, pp.
2065-2070, 2007.
[3] Lae Kyoung Lee, Sungshin Kim, Young-Kiu Choi,and Man Hyung Lee, “Recognition of hand gesture tohuman-computer interaction,” In Proceedings of the
26th Annual Conference of the IEEE Industrial
Electronics Society, vol. 3, pp. 2117-2122, 2000.
[4] Maebatake, M., Suzuki, I., Nishida, M., Horiuchi, Y.,
and Kuroiwa, S., “Sign Language Recognition Based on
Position and Movement Using Multi-Stream HMM,” In
Proceedings of the Second International Symposium on
Universal Communication, pp. 478-481, 2008.
[5] Bauer, B., and Hienz, H., “Relevant features forvideo-based continuous sign language recognition,” In
Proceedings of the IEEE International Conference on
Automatic Face and Gesture Recognition, pp. 440-445,
2000.
[6] Heumer, G., Amor, H.B., Weber, M., and Jung, B.,“Grasp Recognition with Uncalibrated Data Gloves - A
Comparison of Classification Methods,” In Proceedingsof the IEEE Virtual Reality Conference 2007, pp. 19-26,
2007.
[7] Michael W. Schwarz, William B. Cowan, and John
C. Beatty, “An experimental comparison of RGB, YIQ,
LAB, HSV, and opponent color models,” ACM
Transactions on Graphics, vol. 6, no. 2, pp. 123–158,
1987.
[8] Peters, R.A., “A new algorithm for image noise
reduction using mathematical morphology,” IEEE
Transactions on Image Processing, vol. 4, no. 5, pp.
554–568, 1995.
[9] Ho Seok Ahn, Young Min Beak, In-Kyu Sa, Woo
Sung Kang, Jin Hee Na, and Jin Young Choi, “Designof Reconfigurable Heterogeneous Modular Architecture
for Service Robots,” In Proceedings of the 2008
IEEE/RSJ International Conference on Intelligent
Robots and Systems (IROS 2008), pp.1313-1318, 2008.

Fig. 15. The expression of scissors

Fig. 16. The expression of paper

V. CONCLUSION
As hand of human is the most definite expression
part in human body, it can be useful way for theinteraction between human and robot. In this paper, we
proposed the hand motion recognition system andintroduced the four-fingered robot hand system. In thehand motion recognition system, we found hand by skincolor detection and then, recognized the hand motion byedge detection and analysis. As the proposed handmotion recognition system classified hand motion into
three categories, it had high recognition rates in our
experiments.

©ISAROB 2011

1013

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Analysis method of tooth meshing condition and motion of gears
Edzrol Niza Mohamad*1, Masaharu Komori*1, Hiroaki Murakami*1 and Aizoh Kubo*1*1

Department of Mechanical Engineering and Science, Kyoto University
Yoshidahonmachi, Sakyo-ku, Kyoto-shi, Kyoto, 606-8501 JAPANkomorim@me.kyoto-u.ac.jp

1. Introduction

 ycly ( y ) =  l  cy 

The fluctuation in rotational motion of gears is a
critical issue for gear designers. Effective methods ofanalyzing motion error are needed. Based on the author’s
developed method [1], this study proposes a new
function of stiffness considering actual contact ratio (c.f.
Fig. 1). By comparing the obtained results of motion
error from the new method and the previous method, theeffectiveness is validated.

v(4)

2.2 Tooth meshing stiffness
The non-linear change of stiffness against the variationsin the actual contact ratio is represented as follows:

K ( x, h1 , ε ) = K 0 stx ( x) stc 2 ( x, h1 , ε ) x − xk 0 stx ( x ) = 1 − (1 − k x )
 lkx 

(5)

i

(6)
ii

Fig. 12h − ε  x− 12 
stc 2 ( x, h1 , ε ) = 1 − (1 − k 2 )
(7)
ε




2


where K0 controls the magnitude of the unit stiffness andε is actual contact ratio. Figure 3 shows the definitions of
the proposed function of unit stiffness.

Definition of actual contact ratio

2. Novel general model of tooth meshingcondition
2.1 Tooth flank form deviation

Fig. 3 Functions to define stiffness
2.3 Formulation of motion error
From the expressions mentioned, the absolute amount of
motion error Te is represented as:
Fig. 2v
1  v +1r(8)Te = R0 rid (x ) + ζclx ( x ) v +1  s (x )s (x, h , ε )t ( x) 2 


 tx
1

tc 2
r



Definition of ridge curve of tooth flank form
and its curve on contact line

Figure 2(a) shows a schematic diagram of the infinitely
expanded ridge curve on tooth flank [1]. The ridge curveR(x) is represented as: x − xr 0 R ( x) = R0  lrx s

(1)

The normalized motion error Te0 is defined:

Figure 2(b) shows a schematic diagram of the deviation
form on the contact line at position x. The form of thecurve on the contact line is expressed:

C ( x, y ) = C0clx ( x)cly ( y )u x − xc 0 clx ( x) = 1 + (c x − 1) lcx ©ISAROB 2011v

v

C  l   c ( x) stx ( x) stc 2 ( x, h1 , ε )t r ( x) 2  v +1 (9)ζ = 0  cl   lxR0  2lcy  r

Te = R0Te 0(10)

3. Effect of changes of parameters in
distribution function of unit stiffness(2)

Figure 4 shows the distribution function of unit
stiffness on x-direction under each value of parameter kx.
The conditions that, compared with the stiffest contact

(3)

1014

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011line (at the center of tooth flank), the stiffness of othercontact lines are lower can be expressed by parameter kx.
The changes of the actual contact ratio at peak andvalley and also the amplitude of motion error at peakagainst parameter kx is shown in Fig. 5. It can beconcluded that parameter kx has an influence on theactual contact ratio at peak and valley and also theamplitude of motion error at peak position. Figure 6
shows the distribution function of unit stiffness on
x-direction under each value of parameter i.

Fig. 4

Relationship between parameter
position of contact line, x

kx

Figure 7 shows the changes of against theamplitude of motion error at peak position against
parameter i. Overall, the effect of parameter i is verysmall. Based on these results, previous method and the
newly proposed method show the identical trend.

Fig. 6

Relationship between parameter
position of contact line, x

Fig. 7

Relationship between parameter i and motion
error at peak and valley positions

i

and

and

4. Conclusions
In this report, by using a formulation for gear
motion error based on a general model for meshingcondition, the effectiveness of the newly proposed
method to represent the stiffness was analyzed. The
results obtained by both methods show that the analyzed
characteristic of the motion error were in a goodcorrespondence between the proposed method and theprevious one.

Acknowledgment
This study was supported by Industrial Technology
Research Grant Program in 2009 from New Energy andIndustrial Technology Development Organization(NEDO) of Japan.

References[1] E. Niza Mohamad, M. Komori, Hiroaki Murakami,
A. Kubo, and Suping Fang (2009), Analysis ofGeneral Characteristics of Transmission Error ofGears With Convex Modification of Tooth Flank
Form Considering Elastic Deformation Under Load,
Journal of Mechanical Design, Vol. 131, No. 6
Fig. 5©ISAROB 2011

Relationship between parameter kx and motion
error at peak and valley positions

1015

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Research for an Adaptive User’s Intent-Detection Method
for the Use in Rehabilitation Robots
J. Y. Jung, D. Y. Lee, D. W. Choi, I. H. Jang, H. G. Lee, H. S. Park and D. W. Lee
Department of Intelligent Robotics, University of Science and Technology, Daejeon, Korea(Tel : 82-31-8040-6314; Fax : 82-31-8040-6370)(paran1@kitech.re.kr)
Division of Applied Robot Technology, Korea Institute of Industrial Technology, Ansan, Korea(Tel : 82-31-8040-6292; Fax : 82-31-8040-6370)(hsubpark @kitech.re.kr)
Abstract: In this paper, research on how to make an adaptive algorithm for user’s walking intent detection is presented.
User’s intent detection is the most critical problem in rehabilitation robots designed to help paraplegic patients. It isvery difficult to overcome this problem in the dynamic environments. The idea of adaptive method is the combinationof machine learning technics. Grammatical evolution is used to evolve C++ codes related user’s intent detection.
Dynamic time wrapping is used in appraiser which is in charge of a fitness function for the evolutionary process.
Keywords: Rehabilitation Robot, User’s Intent Detection, Adaptation, Genetic Algorithm, Symbolic Regression,
Dynamic time wrapping‘the decision at what time which action should be
performed is dependent on the users and the actions
selected by the user are simply conducted by robot’. In
other words, the robot’s role can be limited to detecting
signals WHAT and WHEN from users and thenexecuting the according behavior. The problem ofdetecting behavior intention could thus be regarded as auser’s intent detection problem. It can be definedmathematically as follows:

I. INTRODUCTION
Recently, many researches related to helping thedisabled and elderly are conducted. Most popular is thearea of wearable robots. A wearable or exoskeleton
robot is a robot which one can wear like a second
skeleton. Representative researches are HAL[1], andReWalk[2]. HAL was initially designed to augment
muscle power of the elderlies but its abilities wereextended to walking assistance for paraplegia patients.
Another well-known robot is ReWalk, developed by
Argomedtec, Israel. ReWalk uses tilt sensors, attachedto the torso, and force sensors, located in the robot’s
soles, to detect the user’s intent. The robots mentioned
above succeeded in detecting the user’s intent, but they
have their limitations. The reason for this is that the
robots have only been applied to experimental scenarios.
Daily life or dynamic environments are much moredifficult since several unexpected situations can arise.
In this paper, we propose an adaptive method todetect user’s intent in dynamic and daily environments.
In order to acquire a proper method, we combined two
methods, symbolic regression and dynamic time
wrapping.

f(𝑥𝑡 , s𝑡 ) → 𝑥𝑥+1
𝑥𝑡 ∈ *𝑠𝑡𝑎𝑡𝑒1 , 𝑠𝑡𝑎𝑡𝑒2 , 𝑠𝑡𝑎𝑡𝑒3 , … , 𝑠𝑡𝑎𝑡𝑒𝑛 +𝑠𝑡 = ,𝑓𝑙1 𝑓𝑙2 … 𝑚𝑠-

In equation (1), function 𝑓 is the function of user’s
intent detection. This function computes the state x attime t+1 using the current state 𝑥𝑡 and the sensor values
𝑠𝑡 as input. The new state 𝑥𝑡+1 can be the same ordifferent to the previous state. Using the new state, therobot determines and conducts its next behavior.

III. RESEARCH FOR AN ADAPTIVE
USER’S INTENT METHOD
Two phases are introduced, one which is a training
phase and the other which is an adaptation phase for the
proposed method.
Furthermore, the system is organized as shown in Fig. 1.
In the training phase, an intention estimator, detecting
the user’s intent, is developed by grammatical evolution(GE)[3] with off-line sensor data. After development,

II. DEFINITION OF USER’S INTENT
DETECTION
Before developing a method for adaptive user’s
intent detection, intent detection for both robot andhuman has to be defined. In the case of wearable robots,©ISAROB 2011

(1)
(2)
(3)

1016

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

this block detects the user’s intent from the inputs of therobot’s current state and the sensor data at time t. The
intention estimator is continuously adapted by GE
throughout the robot’s task. This phase is called theadaptation phase. In the adaptation phase, a dynamic
time wrapping algorithm is used as a fitness function of
the intention estimator in an intention estimatorappraiser block.
Robot
State
Managerx’
xs
Sensorsss<code> ::= <line><code> | <line><line> ::= <lhs>=<rhs>; \n<lhs> ::= <var> | <state><state> ::= state<var> ::= r[0]|r[1]|…|r[13]<rhs> ::= <arg1> <op> <arg2><arg1> ::= <var> | <sensor><sensor> ::= s[0]|s[1]|…|s[15]<arg2> ::= <const> | <var> | <sensor><const> ::= <pm>0x<hex><hex><hex><pm> := +|<hex> ::= 0|1|…|f<op> ::= + | - | * | & | BITOR | ^ | << | >>Intention Estimator(evolved by Grammatical
Evolution method)r

Intention Estimator appraiser(made by neural network or
DTW)

The genetic algorithm setting to be used in the
training phase is as follows.
- The generation size is 5000.
- Single-point crossover with probability 0.9- Bitwise mutation with probability 0.01
- The population size is 500.

Fig. 1. The block diagram of proposed method
1. Grammatical Evolution
Grammatical evolution (GE) is a sort of symbolic
regression method evolving the program code using a
genetic algorithm and a Backus-Naur form (BNF). A
BNF is a meta language to describe the syntax ofprogramming languages. BNF grammars are definedand binary chromosomes evolved to make the programcode of the intention estimator.

In the adaptation phase experiment, wedecrease the generation size to 20 and the
population size to 3 in order to get the fastevolution. The intention estimator appraiser is usedas fitness function of the grammatical evolution.

VI. CONCLUSION

2. Dynamic Time Wrapping
Dynamic time wrapping is used to make the fitness
functions for the genetic algorithm described above in
the adaptation phase. This algorithm has the specialty ofmeasuring similarities between two sequences which
may vary in time or speed. In this paper, the similarities
between previous and current foot sensor patterns would
be detected by this algorithm.

An adaptive user’s intent detection method is
proposed in this paper. By combining two machinelearning method and dividing the process of evolution
into two phases, we expect to acquire an adaptive
method to detect user’s intent. In the future, we will
apply this method to our robot and verify its
effectiveness.

IV. EXPERIMENTS PLAN

REFERENCES

To verify our proposed method, experiments areplanned using 16 force sensing register (FSR) sensors,
located in foot part of rehabilitation robots. Two
different experiments will be conducted.
In the training phase experiment, off-line sensor data
is used to make the intention estimator function. Thisdata includes 16 FSR values and the result of user’s
intent detection. A grammar that generates programs for
an intention estimator is given below.

©ISAROB 2011[1] Suzuki K, Mito G, Kawamoto M and et al. (2007),
Intention-Based walking Support for paraplegia patientswith robot suit HAL. Advanced Robotics 21(12):14411469[2] Goffer A, and Tivon K (2002), Gait-locomotor
apparatus. European Patent 1 260 201 A1[3] O’Neill M, and Ryan C (2003), Grammatical
Evolution. Kluwer Academic Publishers.

1017

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Development of touch sensor system of silicone-type artificial skin for aninteractive android robot
Duk-Yeon Lee, Dong-Wook Lee, Dongwoon Che, Hyun-Sub Park, Ho-Gil Lee and Jun-Young Jung
Korea Institute of Industrial Technology (KITECH), Republic of Korea(Tel : 82-31-8040-6315; Fax : 82-31-8040-6330)(proldy@ kitech.re.kr, dwlee@kitech.re.kr,cdw@kitech.re.kr,hsubpark@kitech.re.kr,leehg@kitech.re.kr,
paran1@kitech.re.kr )Abstract: This paper presents a touch sensor system of an android robot developed by authors. In our android, touch
sensors are embedded under artificial skin which is composed of special silicone. Also the thickness of the skin is about
5 to 20 mm in the face and hands. In general, as the skin is thicker, the recognition rate of conventional resistive touch
sensor is worse. In addition, irregular artificial skin surface does not guarantee the recognition of various touching
movements. To improve the recognition rate of touch sensor, we developed capacitive touch sensors with touch
controller module. In order to verify the effectiveness of our touch sensor system, we test it in various conditions ofthickness and touching movements. Also we show some applications of interactive communication using our touch
sensor system between human and android robot.
Keywords: Android robot, Capacitive sensor, Touch sensor, Flexible Tactile sensor.

I.

INTRODUCTION

Using the touch-sensor technology is contacted by thehuman often has been applied to many mobile devices.
This technology is shown in Figure 1 applied to theAndroid robot or Humanoid robot was also used.

Fig. 2 Ever-2 Head : inner structure and facevariety of movements can not be guaranteed. Therefore,various facial expressions are critical problems in the
android robot.
In this paper, a touch sensor system for android robot
interaction is about the development. Change incapacitance value due to contact with the human bodyto detect and obtain data from each of the touch sensor
pad for the touch sensor module; Touch sensor module
to collect the data obtained by the various touchoperation results can be transferred to the host device,
the touch module controller; Analysis of the data
received from the touch module controller for the robot
to determine whether any act came in, the robot is to do
as a result of behavioral patterns were selected togenerate a corresponding control command signal sent
to control the target controller, the host processor(PC:personal computer) unit program Includes.
.

Fig. 1 Robots using tactile sensors
However, the general robot used touch-sensor
technology in the purpose for which power wasmeasured. Touch-sensor technology in the android robot,
rather than for the purpose of measuring the force forthe purpose is to interact with human. In the case of
android robots are composed of thick artificial skin. The
force exerted from the outside is difficult to be passed
inside the touch sensor pad. In addition, the android
robot out of the artificial skin should not be exposed tothe touch sensor pad. Also more than 10mm in thickness
artificial skin was used in the general robotics andforce-based touch technology is difficult to apply. The
outer surface of the android-robot is complex as Figure
2 shows the typical force-based touch sensor using a©ISAROB 2011

II. OPERATING PRINCIPLE OF CAPACITIVETOUCH SENSORS
Capacitive touch sensors that measure the variation incapacitance of the touch sensor pad and the humidity,

1018

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011pressure, location, etc. can be measured. Pressure
sensors, touch sensors, accelerometer sensors, etc. aretypical.
Common measurement for capacitance sensor can
range from the tens of pF is pF. In this paper, we apply
the touch sensor SoC(System On Chip) is used to model
the NTS1006 product by NextChip Ltd. Chip
capacitance measurement precision and resolution has
0.1pF capacitance also has six input channels. In thispaper, in an SoC was used the two capacitive input
channels.
General formula of capacitance relationship between
PAD size and relative permittivity are as follows.
Fig. 3 Touch Sensor System
Communication speed is too high for the sampling
period is short, touch-sensing value is acquired
abnormal value by experimental and applied to thecommunication speed of 100KHz. Calibration of touch
module, initialization, and communications functions
are programmed in the touch module controller.
Contact with touch sensor pad have a relative
premittivity capacitance occurs as should perform
calibration. The reference value is set to occur as thecapacitance and capacitive touch operation changes thevalue of accurate sensor reference value must be
deducted from the acquisition can be.
Communication mode UART / USB / CAN approach
can be used, but in this paper for easy multiple access
CAN2.0A version was used.
Host device (PC: personal computer) 25ms intervals
from the touch module controller touch sensor pad foreach data value is obtained within the sensing. Obtained
touch data based on the behavior patterns which
determine whether the touch and judge the results
shown in the touch module was mounted on the LED.
In this paper, the behavior patterns of touch PAD1,
PAD2 a 25ms cycle Data obtained from the touch-verify
the data obtained were analyzed based on the behavioral
patterns of touch. In the developed touch sensor system
can distinguish between various touch behavior patterns
that looked out through the following experiment.

(1)

Touch sensor pad occurs in the size of the capacitancein the above formula (1) and the same, the touch isinversely proportional to the distance and is
proportional to the pad size. In addition, the touch padand the contacts between the relative permittivity
capacitance can be a significant change in capacity.
Typical relative permittivity of the acrylic 2.5 ~ 3, FR4
PCB 4.3 ~ 4.6, 2.0 to 3.5 rubber, air 1, the water has 80
properties. Silicone artificial skin on the android-robot
picks up the attributes on the relative permittivity
constant of 2 to 3.5 typically rubber like material
properties and the mixing ratio varies depending on thevalue. Low relative permittivity and complex facevarious forms of internal structure of the metal frame ismounted. Thus, the capacitive touch pads are difficult to
operate reliably in the android robot. Between touch pad
and is particularly close to a thick artificial silicone skin,
due to the low relative permittivity constant is difficult
to obtain sufficient capacitance. To overcome these
problems, this paper has developed a touch sensor pad
size of 10X12mm. Also, SoC RF Noise in a number of
input channels to prevent the incoming input channels
were mounted in front of the LPF(Low pass filter). As aresult, a touch sensor operates from a variety of RF
Noise was an error does not happen. The SoC delta
detection algorithm based on the threshold algorithm isapplied to a variety of changes in capacitance values
could be obtained quickly.

IV. EXPERIMENT.
III. DESIGN OF TOUCH SENSOR SYSTEM
The authors have developed a touch sensor system
is shown in Figure 3. Two touch pad with a touch
module, comparative data for each touch pad may besensing the moving action. Touch module was applied
to a high-performance SoC(NTS1006), and I2C
communication with the touch module controller isconnected. 400KHz and 100KHz to 100KHz of the
speed of I2C communication used.

©ISAROB 2011

Fig. 4 Touch operation on the touch sensor pad

1019

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

sufficient change in capacitance could be confirmed.
Figure 7 is stable to have been confirmed bycomparison with Figure 5.

Exposed as shown in Figure 4 on the touch sensor pad
touch action was tried. Touch sensor pad exposed in the
middle do not have a relative permittivity capacitancevalue could be obtained as the results shown in Figure 5.

V. CONCLUSION
In this paper, for Android Robot capacitive touch sensor
system was developed. Android robots applied to theface above the thick artificial silicon skin that stable
operation is performed through the previous
experiments could be confirmed. Android robots
through experiments and a variety of human interaction
can be sure that it could be seen. In this paper, theperfect interaction with the robot did not perform. If
further studies continue to design the entire touch sensor
will be mounted on the robot. Through this pattern,corresponding to various natural touch interaction
research is carried out is expected to continue.

Fig. 5 Touch operation on the results of the touch
sensor pad
Touch sense value, as shown in Figure 5, the sensing
range is 1000-5500, and after a touch more than 100ms
in case of move has a time-shift operations, sensing thata well could be confirmed by experiments.

REFERENCES[1] Dong-Wook Lee, Development of an Android for
Emotional Expression and Human Interaction ,The International Federation of Automatic Control
Seoul, Korea, July 6-11, 2008[2] Kwang-Su, Mu-jin Lee, “A Design of capacitive
Sensing Touch Sensor Using RC Delay with
Calibration”, Journal of Korea Institute of Illuminatingand Electrical Installation Engineers Vol23, No.8,
pp.80~85 August 2009[3] Takashi Minato, et, al., “CB2: A child robot withbiomimetic body for cognitive developmental robotics”,
Humanoid Robots, 2007 7th IEEE-RAS International
Conference on, Nov. 29 2007-Dec. 1 2007[4]”Capacitive Sensors,” Larry K. Baxter, IEEE Press,
1997.
[5] Thorsten P. Spexard, et, al.,“Human-Oriented
Interaction With an Anthropomorphic Robot”, IEEE
TRANSACTIONS ON ROBOTICS, VOL. 23, NO. 5,
OCTOBER 2007
[6] NEXTCHIP Co, Ltd
ULR : http://www.nextchip.com/

Fig. 6 Touch operation on the artificial silicon skin
Figure 6 shows a touch sensor and a relative
permittivity between the chains in the presence ofartificial silicon skin the experimental data shown in
Figure 7. Android robots face a common area ofartificial silicon skin thickness has a thickness of about
0.5cm ~ 1.4cm.

Fig. 7 Touch operation on the results of the artificial
silicon skin
As shown in Figure 7 500-1000 range between thesensor data and experimental results in Figure 5
confirmed that more data can be reduced in scope.
Distance increases and the relative permittivity as
described in Formula 1 is the reason. However, as
shown in Figure 7 from the touch sensor operates©ISAROB 20111020

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Remote Control and Conversation System
between Human and Android Robot via Internet
Dong-Wook Lee, Dongwoon Choi, Jun-Young Joung, Hoseok Ahn, Hyunsub Park, and Ho-Gil Lee
Korea Institute of Industrial Technology, 426-791 Ansan, Republic of Korea
Tel : 81-31-8040-6312; Fax : 81-31-8040-6370
Email {dwlee, cdw, paran1, hoseoka, hsubpark, leehg}@kitech.re.kr

Abstract: In this paper, we propose remote control and conversation system between human and android via internet.
Android is a robot resembles human being not only its appearance but also behaviors. The android EveR-3 which wasdeveloped by authors can express facial expressions and lip synchronizations. Also it can talk with a human, because it
has voice recognition/synthesis engine and conversation database. To control the android in remote place, we develop
an online chatting program with webcam and connect it to the program of the android. At this time, the Android's side
becomes a server and user side is a client. Then, user can order the operation commands to the android using thissystem or talk with it by voice and text messages. In this paper, we will show some examples to verify the effectivenessof our system.
Keywords: Remote control, Remote conversation, android robot
expression, lip synchronization, and vision recognition
are upgraded. EveR-3 is the latest version of ourandroid [3]. She can move the ground because she has
wheel based lower body. So she could be debuted as arobot actress in Korean musical performance, “EveR
comes to Earth” at Feb. 18, 2009. Also the design offacial muscles in head (23 D.O.F) is based on
anatomical structure of human face. So the expression
ability of emotions is more abundant. Fig. 1 shows someexamples of facial expressions of EveR-3.

I. INTRODUCTION
Remote control in robotics, have been developed in
the field of dangerous environments such as nuclear
power plant, deep sea, and space. In recent years, as
intelligent service robots in our home or life have beendeveloped, the application of remote control is wider.
The home service robots, such as pet robot, home guard
robot, education robot, are produced as commercial
goods. In this paper, we proposed remote control andconversation system for applying to a personal service
robot. Proposed system is verified by applying to theandroid robot developed by authors.

II. ANDROID ROBOTAndroid robot resembles human being not only theappearance but also its behaviors. We have developed
android robot for researching the emotional interaction
between human and robot.
Our first android, EveR1-1 is capable of motion fromher torso up because her lower body is a dummy. She
has 35 D.O.F (degree of freedom) with 15 D.O.F inhead [1]. She can do facial expressions of four emotions.
EveR-1 is applied to guidance service of an exhibition,oral narration of fairy tales. EveR-2 is bipedal type [2].
So she can sing a song in standing. EveR-2 has thelower body with 12 D.O.F. She has the function of
EveR-1 and is added the function of whole bodycoordination and dialogue engine. Her gesture, facial

Fig.1. Facial Expressions of EveR-3

III. REMOTE CONTROL SYSTEM
1. Remote control and conversation
To control and have a conversation with robot inremote site via internet, we use one of internet
messaging software which has the function of videoconferencing; Skype (www.skype.com) is used in thisexperiment. Fig. 2 shows the proposed system that iscomposed of three parts; one is user (or client), thesecond is a server with webcam, the other is android
robot, EveR-3. Internet messaging software must be

1

The name EveR derives from the Biblical ‘Eve’, plus
the letter ‘R’ for robot.

©ISAROB 20111021

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

installed in user computer and the server. After a userand server are connected each other in online, the user
can type or tell a message. Server interprets the message
is a command or a conversation.

Another working mode of EveR-3 is scenario mode.
In scenario mode, a robot plays the scenario files. A
scenario file has composed of voice, facial expression,and gesture with the start time of each. As times goes,the voice, facial expression, or gesture is played. User
can create and edit the scenario files using the tool in
Fig. 4.

Fig.2. Components of remote system
The difference between a command and aconversation is whether input message (by typing or
voice) is same with one of reserved scenario files or not.
Scenario files will be explained in sec. III-2. For
example, if ‘right hand up.txt’ is one of scenario files,
also if an input message is ‘right hand up’, and thenEveR-3 plays the scenario files. If the input message is
not matched with reserved scenario files, then the robot
responds the answer as the input message is a question.
At this time, dialogue engine of EveR-3 is used.
Fig. 3 shows the internet messaging software used inthis paper. We can see the robot and can type a message.
Also we can say using microphone which is connectedin PC. The internet messaging software transmits themessage and voice to the server in remote site.

Fig.4. Scenario editing tool

VI. CONCLUSION
In this paper, we developed remote control andconversation system. Using video conference software,user can see the moving images of a robot in remote site,and can control or talk with the robot by text messages
or voice commands. If user types or orders thecommands through the software, then the robot can dosome actions according to the command. If user talks
about something, the robot responds. We think that the
proposed system will be applied to remote medical
examination, remote education, and so on.

ACKNOLEDGEMENT
This research is supported by Ministry of Culture,
Sports and Tourism(MCST) and Korea Creative Content
Agency(KOCCA) in the Culture Technology(CT)Research & Development Program 2010

REFERENCES[1] H.G.Lee, M.H. Baeg, D.W.Lee et al. “Developmentof an Android for Emotional Communication between
Human and Machine,” Proc. of Int. Symp. on Advanced
Robotics and Machine Intelligence, pp.41-47, 2006.
[2] D.W.Lee, T.G..Lee, B.R.So et al. “Development ofan Android for Emotional Expression and Human
Interaction, Proc. of World congress on International
Federation of Automatic Control (IFAC 2008), pp.
4336-4337, 2008.
[3] D.W.Lee, D.W.Che, D.Y Lee et al. “Developmentof an Android for a Performance,” Proc. of Int. Conf on
Ubiquitous Robot and Ambient Intelligence (URAI2010), p. 664, 2010.

Fig.3. Remote control and conversation tool based onvideo conference using Skype
2. Scenario editing tool
EveR-3 has dialogue engine with conversation fairs
that has 5,000 Q&A fairs which is used in ordinary life.
So if one asks a question to the robot, it answers thequestion using dialogue engine.

©ISAROB 20111022

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

THE OPTIMAL METHOD OF SEARCHINGTHE EFFECTIVE COMBINATION OF DITHER MATRIXTsuneyasu Kato and Ken-ichi Tanaka
School of Science and Technology, Meiji University
1-1-1,Higashi-Mita,Tama,Kawasaki,Kanagawa 214-8571 Japan
E-mail:tanaken@isc.meiji.ac.jp |ce91022@isc.meiji.ac.jp
Abstract: In this study, three methods are compared for checking the most effective method of searching the optimal
combination of dither matrix. Three methods are Genetic Algorithm (GA), Simulated Annealing (SA) and Taboo
Searching. The images that are used in this study are Human, Human rotating 180 degrees and Swiss. As a result, themethod of GA is the most excellent in these methods.
Keywords: Dither matrix, Bayer method, GA, SA, taboo searching, optimal combination.

I. INTRODUCTION

III. ALGORITHM OF TABOO SEARCHING

Nowadays, there are some popularly methods of dither

The method of Taboo Searching is as below.

matrix. These methods are good on a viewpoint of image(1) The matrices which have elements set in at

quality. However, the study that is searching effective

random are prepared (it is named individual A)

dither matrix viewpoint of optimization combination(2) Making some individuals based the individual A

without method of GA is not able to be found. Then, the(3) Evaluating and comparing these individuals

best effective method for searching better dither matrix is(4) The best evaluated value individual (it is named

checked. Searching method s are three, GA, SA andindividual B) is selected by these individuals

Taboo Searching. The comparing candidate is Bayer(5) If combination of individual A and B has bemethod. Algorithm of GA, evaluating method and theselected before, return to operation (2)exposition of dither matrix are shown by Tsuneyasu[1].

(6) Comparing individual A and individual B(7) If individual A is better than individual B, return

II. ALGORITHM OF SA

to operation (2)The algorithm of SA is as below.

(8) If individual B is better than individual A, making(1) An individual that have matrices which is set

some individuals based the individual B

elements in at random is prepared.

(9) Running over from (3) to (8) until numbers of(2) New individual is made by changing element of therepeat time surpass numbers of generationsindividual which is made at operation (1).

IV. RESULT(3) Comparing two individuals(4) New individual is made by exchanging element of

The evaluated value of four methods changing eachthe individual getting better evaluated value

images are as below.

(5) Running over from (3) to (4) until the satisfaction(1) The image of humanof the conditions.

Bayer: E=102.846, GA: E=102.785

In this study, the conditions are that temperature is

SA: E=102.823, Taboo searching: E=102.836

under number which is set at hand or evaluated value of(2) The image of human rotating 180 degrees

SA is better than Bayer method.

©ISAROB 2011

Bayer: E=102.848, GA: E=102.817

1023

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

SA: E=102.813, Taboo searching: E=102.929

2. Table(3) The image of Swiss

Table1. The matrix of using GA to human

Bayer: E=101.544, GA: E=101.484
SA: E=101.526, Taboo searching: E=101.532
Then, the best evaluated value of Human rotating 180

7

11

0

8

3

13

6

15

9

2

10

4

12

5

14

1

degrees is SA. However, the best evaluated value of all
patterns is GA. In addition, evaluated values of each
image by using GA are averagely good. Then, as thisTable2. The matrix of using SA to humanresult, the optimal method of searching the effective

5

2

11

12

combination of dither matrix is thought that is GA.

13

14

6

0

10

3

8

4

1

15

7

9

VI. FIGURES/TABLES

Table3. The matrix of using Taboo Searching to
human

Fig1. The image of human by using Bayer method

15

8

7

4

6

14

1

12

2

3

13

10

11

5

9

0

V. CONCLUSION
Unless using same image, the results of GA for humanand human rotating 180 degree are not point symmetry.
Fig2. The image of human by using GA

This reason will be that these results of GA are
convergent local solution. Then, the problem is accuracy
enhancement of GA by making change factors.

REFERENCES
1)Tsuneyasu Kato, “The optimal combination of dither
matrix by using genetic algorithm” Artificial Life andFig3. The image of human by using SA

Robotics. 15, pp137
2) Ken-ichi Tanaka. “Color Images Halftoning Using
Simulated Annealing” The Journal of The Institute of
Image Information and Television Engineers. 61.6,
pp828-837
3) Toshiharu Nagao. “ evolutionary image processing”

Fig4. The image of human by using Taboo Searching©ISAROB 2011

Publication Shoukoudou Inc. Year 2002.

1024

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

AUTHOR INDEX
Notation of session name: PT: Plenary Talk, IT: Invited Talk,
OS: Organized Session, GS: General Session, PS: Poster Session
Note: P-44 / 359 = (page no. in Technical Paper Index) / (page no. in Papers)[A]
Abdul-Rahman
Abe
Abe
Adachi
Ahn
Ahn

O.
N.
T.
R.
B.
H.S.

Aibe

N.

AiharaK.

AikawaAikawaAkama
Akimoto
Akita

C.
M.
K.
T.
S.

Aoki
Aoki

ShigekiShingo

Arai
Arai

E.
T.

Arita

T.

Asada

T.

Aung
Azumagakito

K.T.
T.

OS15-2PS1-2
OS13-3OS14-4
GS21-1
PS1-3
PS1-7
OS15-6OS15-7OS14-1OS14-2OS14-3OS14-4OS14-5
OS3-2
OS15-4OS15-2OS24-4OS15-6OS15-7
GS10-1
GS3-1
GS6-2
GS18-1
GS2-6
IT2
GS21-6
GS1-2
GS3-2
GS3-3
GS7-4
OS9-2
OS9-3
OS9-4
GS1-3
GS3-2

P-44/359
P-29/1008
P-35/322
P-40/346
P-27/979
P-29/1010
P-29/1021
P-45/375
P-45/379
P-39/334
P-39/338
P-39/342
P-40/346
P-40/350
P-24/91
P-45/367
P-44/359
P-28/585
P-45/375
P-45/379
P-25/759
P-28/629
P-42/687
P-37/922
P-34/625
P-22/23
P-27/999
P-47/593
P-28/633
P-28/637
P-35/715
P-36/219
P-36/225
P-36/231
P-47/597
P-28/633

P-31/671
P-39/834
P-38/310
P-46/967
P-28/653
P-38/306
P-43/515
P-45/808

P-42/545
P-35/723
P-31/671

J.I.

Baek
Baek
Bao
Beak
Bhattacharya
Burdelis

G.
K.R.
Z.
J.
B.
M.A.P.

GS5-3
GS13-3OS12-5
GS20-2GS4-2
OS12-4OS21-6
GS12-3[C]
Cai
CastroCha

G.
J.C.H.
Y.C.

OS22-7
GS7-6
GS5-3©ISAROB 2011

C.Y.
J.
D.W.

C.Y.

OS11-1
GS10-4
PS1-6
PS1-7
OS11-1OS10-5OS21-2
GS21-4OS10-1OS11-1OS10-6
GS6-5
GS20-2
GS8-3
GS10-4
GS10-5OS23-1OS23-5
GS20-1
PS1-5
GS21-4
GS21-1
GS8-3
GS10-4
GS10-5
GS21-6OS10-2OS10-4OS21-3

P-32/269
P-25/771
P-29/1018
P-29/1021
P-32/269
P-32/261
P-43/499
P-27/991
P-32/245
P-32/269
P-32/265
P-42/699
P-46/967
P-34/731
P-25/771
P-25/775
P-43/549
P-44/565
P-45/964
P-29/1016
P-27/991
P-27/979
P-34/731
P-25/771
P-25/775
P-27/999
P-32/249
P-32/257
P-43/503

Chen
Chen
Cheng
Cheon
ChiangChiang
Chien
Chiou
Cho
Cho

C.A.
T.H.
Y.
H.S.
F.C.
H.K.
T.L.
Y.R.
B.
J.I.

Cho

Y.I.

Choi

D.W.

Choi
Choi
Choi

G.H.
M.
S.M.

Chumtong
Chung

P.
C.Y.

Cui
Dai
Do
Dong

F.
J.
H.

OS22-1OS12-5OS22-2

P-41/519
P-38/310
P-41/523[E]E
El-fiky
ElshaerM.
M.
R.H.E.

OS22-7
GS4-1
GS16-1

P-42/545
P-28/649
P-25/885

Fan
Fu

S.Z.
B.

Fu
FuchidaFuji
Fujii
Fujimoto
Fujimura
Fujiwara

Y.Y.
T.
K.
N.
N.
T.
T.

GS6-5
GS13-2
GS19-2OS11-5
GS1-3
GS6-1
PS1-1
GS16-2OS9-2
OS17-3OS17-4P-42/699
P-39/830
P-30/944
P-33/285
P-47/597
P-42/683
P-29/1006
P-25/889
P-36/219
P-29/419
P-30/423

[D]

[B]
Bae

Chang
Chang
Che

[F]

I-1

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

FukudaFukudaFukumura
Fukushima
Funahashi
Furukawa

O.
T.
Y.
M.
K.
M.

Furukawa
FurunoFurutani

T.
S.
H.

GS8-1
GS21-5
OS6-2
GS7-6
OS3-4
GS2-3
GS2-4
GS2-5
GS11-2
GS20-3
OS7-4
OS7-5
OS15-1

P-34/727
P-27/995
P-40/150
P-35/723
P-24/99
P-34/613
P-34/617
P-34/621
P-42/789
P-46/971
P-31/187
P-31/191
P-44/354

GS11-5
GS21-7
GS21-2
GS18-1
GS10-1
GS2-1
GS13-6OS10-4OS11-6OS22-7

P-43/797
P-27/1003
P-27/983
P-37/922
P-25/759
P-34/605
P-39/842
P-32/257
P-33/289
P-42/545

OS23-2OS24-4
GS2-6
OS12-1OS21-1
OS19-4
GS14-4
OS7-2
OS17-1
OS6-1
OS6-3
OS6-5
OS20-6
OS8-1
OS8-2
OS8-3
OS8-4
OS8-5
GS16-3
GS16-5
GS19-4
GS19-5OS20-6OS22-5OS12-3
GS21-5
OS1-4
GS20-3
GS17-2
OS6-7
OS3-5
GS17-3
GS1-4

P-43/533
P-28/585
P-34/625
P-38/293
P-43/495
P-46/459
P-44/858
P-31/178
P-29/411
P-40/146
P-40/154
P-40/162
P-41/491
P-33/195
P-33/199
P-33/203
P-33/207
P-33/211
P-26/893
P-26/901
P-30/952
P-30/956
P-41/491
P-41/537
P-38/301
P-27/995
P-23/62
P-46/971
P-38/909
P-40/170
P-24/103
P-38/912
P-47/601

[G]
Gao

W.

Gao
Gejima
Gibo
Gocho
Goto
Guo

X.J.
R.
T.
R.
K.
J.H.

Guo

W.

[H]
Habib
HagitaHagiyaHan
Han
HaradaHarimaHasamaHashimoto
Hashimoto

M.K.
N.
M.
J.
K.Y.
K.
H.
H.
K.
KiyotaHata
HattoriY.
T.

HayashiHayashiHe
HeoHigashimori
Higuchi
HiranoHirayama
Hirokawa
HitakaHo
Hong©ISAROB 2011

E.

T.
P.
J.
M.
K.
G.
K.
S.
Y.
P.M.
Y.S.

I-2

Horiike
Hosokawa
Hsia
Hsu
Hu
Hwang

K.
S.
K.H.
T.Y.
J.
Y.

OS9-5
OS1-5
OS11-3OS10-2OS21-2OS12-2

P-36/237
P-23/66
P-32/277
P-32/249
P-43/499
P-38/297[I]
Ichikawa
Ide

Y.
T.

Igarashi
Iio
Ikeda
Ikeda

H.
T.
A.
K.

Ikeda
Ikeda
Imai
Imai

M.
S.
C.
M.

Imoto
Inenaga
InohiraInoue
Inoue
IshidaIshida

Y.
S.
E.
H.
S.
T.
Y.

Ishigure
Ishiguro

Y.
M.

Ishii

S.

Ishikawa

S.

Ishimaru
Ishitobi
Isokawa
Ito

H.
M.
T.
M.

Ito

T.

Iwadate

K.

GS8-1
OS7-4
OS7-5
GS13-6OS24-4OS22-6
GS12-3
GS16-4OS16-4OS7-4
OS14-5IT1
OS24-3
GS5-4
OS13-5
GS15-3OS17-1
GS3-1
GS2-2
OS7-1
OS19-1
OS19-2OS19-3
OS19-4OS19-6OS20-2OS20-3OS20-4OS20-5OS20-6
GS19-6OS15-6OS15-7
GS14-4
GS14-5
GS10-2
GS10-3
GS10-7
GS11-4
GS19-4OS3-1
GS14-2
OS3-4
GS13-5
GS21-2
GS21-3
OS7-1
OS7-2
OS7-3
OS7-4
OS7-5
GS2-5

P-34/727
P-31/187
P-31/191
P-39/842
P-28/585
P-42/541
P-45/808
P-26/897
P-47/395
P-31/187
P-40/350
P-22/21
P-28/581
P-31/675
P-36/330
P-37/873
P-29/411
P-28/629
P-34/609
P-31/174
P-46/447
P-46/451
P-46/455
P-46/459
P-46/467
P-41/475
P-41/479
P-41/483
P-41/487
P-41/491
P-30/960
P-45/375
P-45/379
P-44/858
P-44/862
P-25/763
P-25/767
P-25/781
P-43/793
P-30/952
P-24/87
P-44/850
P-24/99
P-39/838
P-27/983
P-27/987
P-31/174
P-31/178
P-31/183
P-31/187
P-31/191
P-34/621

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Iwaseya
Iwata
Izumi

M.
M.
K.

Izumi

T.

GS17-1
GS10-1
OS16-4OS16-7
OS8-1
OS8-2
OS8-3
OS8-4
OS8-5

P-38/905
P-25/759
P-47/395
P-47/407
P-33/195
P-33/199
P-33/203
P-33/207
P-33/211

[J]
Jang

I.H.

Jeong

J.C.

Jia
Jia
Jia

C.
X.
Y.M.

Jo
Joung
Jung
Jung
Jung

J.
J.Y.
E.
H.
J.Y.

[K]
Kakudou
Kamisato
Kamitomo
KamiuraKanda
KanekoKanematsu
Kang

T.
S.
H.
N.
M.
M.
H.
J.

Kashima
Katamune

T.
R.

Katayama
Kato
Kato
Kato
Kato
Kato
Katsumata
Kawai
KawanoF.
J.
N.
Tatsuya
Tomonori
Tsuneyasu
Y.
Y.
H.

KawanoKawanobe
Kawasaki
Kawashima

K.
H.
H.
M.

©ISAROB 2011

GS9-1
PS1-5
GS8-3
GS10-5
OS4-1
OS22-4
OS4-2
OS4-3
OS4-5
OS12-2
PS1-7
OS12-5
GS10-7
GS9-1
GS20-1
PS1-5
PS1-6

P-36/741
P-29/1016
P-34/731
P-25/775
P-26/107
P-41/533
P-26/110
P-26/114
P-26/122
P-38/297
P-29/1021
P-38/310
P-25/781
P-36/741
P-45/964
P-29/1016
P-29/1018

OS16-3OS20-4OS13-2
GS14-2OS17-5
GS21-5
OS6-2
GS13-1
GS17-4
GS19-1
GS17-1
OS7-4
OS7-5
OS18-2
OS1-5
GS9-4
OS16-6
OS3-5
PS1-8
OS19-2
GS18-3
OS8-1
OS8-2
OS8-3
OS8-4
OS8-5
OS5-5
OS17-2
GS19-6
GS10-6

P-47/391
P-41/483
P-35/318
P-44/850
P-30/427
P-27/995
P-40/150
P-39/828
P-38/916
P-30/942
P-38/905
P-31/187
P-31/191
P-24/435
P-23/66
P-36/755
P-47/403
P-24/103
P-29/1023
P-46/451
P-37/930
P-33/195
P-33/199
P-33/203
P-33/207
P-33/211
P-24/142
P-29/415
P-30/960
P-25/777

I-3

Kibayashi
Kibe
Kim
Kim
Kim
Kim
Kim

R.
Y.
C.
D.E.
D.J.
HongseukHyoungseop

S.
A.
S.
Y.
S.
K.
K.
S.
N.
Y.
Y.

OS13-4
GS19-4OS12-4
GS3-4
GS13-3
GS21-1
GS10-2
GS10-7
GS11-4
GS8-1
OS23-3OS12-5OS12-5
GS13-6OS23-4
GS13-1
GS17-4
GS19-1
GS17-3
GS6-4
OS20-5
OS9-1
GS20-4OS11-5OS14-1
OS6-6
GS16-3
GS19-6OS14-2OS14-3
GS10-2
GS13-1
GS17-4
GS19-1
PS1-4
GS13-6
GS5-4
OS15-3
GS15-1
GS11-2
OS9-6
OS18-1
OS3-2
OS7-4
OS7-5
GS16-4
PS1-4
GS12-1
OS1-2
OS3-1
GS14-3
GS19-5OS15-4OS22-1
OS24-3OS12-1

P-35/326
P-30/952
P-38/306
P-28/641
P-39/834
P-27/979
P-25/763
P-25/781
P-43/793
P-34/727
P-43/557
P-38/310
P-38/310
P-39/842
P-43/561
P-39/828
P-38/916
P-30/942
P-38/912
P-42/695
P-41/487
P-36/215
P-46/975
P-33/285
P-39/334
P-40/166
P-26/893
P-30/960
P-39/338
P-39/342
P-25/763
P-39/828
P-38/916
P-30/942
P-29/1014
P-39/842
P-31/675
P-44/363
P-37/866
P-42/789
P-37/241
P-24/431
P-24/91
P-31/187
P-31/191
P-26/897
P-29/1014
P-45/800
P-23/54
P-24/87
P-44/854
P-30/956
P-45/367
P-41/491
P-28/581
P-38/293

Kim
Kim
Kim
Kim
Kim
Kim
Kimura

J.
K.
S.
T.
T.H.
Y.J.
Y.

KiritaKitami
Kitamura
Kitaoka
Kitazono
Ko
Kobayashi
Kodama
Koga
Koganemaru
Kohno

S.
K.
K.
T.
Y.
C.N.
W.
S.
K.
K.
T.

Komatsu
Komori

M.
M.

Kon
Kondo
Kondo
Kondo
Kondo
Konishi
Konishi
Kono

K.
E.
K.
T.
Y.
H.
O.
M.

KozawaKubo
KubotaKume
Kunimatsu
KurataKurogiKuroseKushidaKuwayama
Kwon[L]
Leblebicioğlu
Lee

K.
C.M.

GS17-5OS11-5

P-38/918
P-33/285

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Lee

D.S.

Lee

D.W.

Lee

D.Y.

Lee

H.G.

Lee

H.H.

Lee
Lee

J.E.
J.J.

Lee

J.M.

Lee
Lee
Lee
Lee
Lee

Jaehong
Jaeoh
JungjuJunseok
K.S.

Lee

M.

Lee

M.H.

Lee
Lee

S.
Y.J.

Li
Li
Li
Lian

G.
X.
Y.
L.

Liao
Lien
Lim
Lin
Lin
Lin
Lin
Lin
Liu

Y.L.
S.F.
E.G.
C.L.
W.B.
W.C.
Yung-Chien
Yung-Chin
C.

Liu
Liu

G.Y.
Y.

Lu

C.

Lund

H.H.

©ISAROB 2011

GS5-3
GS13-3
GS9-1
GS20-1
PS1-3
PS1-5
PS1-6
PS1-7
GS9-1
GS20-1
PS1-5
PS1-6
GS9-1
GS20-1
PS1-3
PS1-5
PS1-6
PS1-7
OS21-1OS21-3OS21-5OS21-5OS23-4OS23-6OS12-1OS12-2
GS20-2OS12-1OS23-3OS12-2
GS5-3
GS13-3OS12-3OS12-4
GS5-3
GS13-3
GS21-1
GS5-3
GS13-3
GS10-2OS22-3
GS21-3
GS13-2
GS19-2OS10-3OS11-3
GS10-4OS11-2OS11-1OS10-5OS10-5OS10-5
GS13-2
GS19-2OS11-5OS4-3
OS4-4
OS13-1OS13-2
IT4

P-31/671
P-39/834
P-36/741
P-45/964
P-29/1010
P-29/1016
P-29/1018
P-29/1021
P-36/741
P-45/964
P-29/1016
P-29/1018
P-36/741
P-45/964
P-29/1010
P-29/1016
P-29/1018
P-29/1021
P-43/495
P-43/503
P-43/511
P-43/511
P-43/561
P-44/569
P-38/293
P-38/297
P-46/967
P-38/293
P-43/557
P-38/297
P-31/671
P-39/834
P-38/301
P-38/306
P-31/671
P-39/834
P-27/979
P-31/671
P-39/834
P-25/763
P-41/529
P-27/987
P-39/830
P-30/944
P-32/253
P-32/277
P-25/771
P-32/273
P-32/269
P-32/261
P-32/261
P-32/261
P-39/830
P-30/944
P-33/285
P-26/114
P-26/118
P-35/314
P-35/318
P-22/36

I-4

Lund

H.H.

IT5

P-22/43[M]
Ma
Mabrouk
Mackin

Q.L.
E.
K.J.

Mae
Maeda
Maeyama

Y.
T.
S.

Maki
MasudaMatsudaMatsui

M.
S.
N.
H.

MatsuiMatsumotoMatsumoto

N.
H.
K.

MatsunoMatsunoMatsuoMatsuoMatsushita
Mehta

F.
S.
F.
T.
K.
R.

Meng
Mikami
MimuraMinowaMinoyaMiura
Miyamoto
MiyataMiyataMiyauchi

D.
Y.
E.
K.
K.
H.
T.
K.
R.
A.

Mizoguchi
Mizoguchi
MizunoMogi
Mohamad
Moon
Mori
Mori
Mori
Mori
Morie
Munetomo
Murakami
Murao
Murata

F.
N.
T.
K.
E.N.
Y.
A.
K.
Y.
Yukihiro
T.
M.
H.
H.
J.

OS15-1
GS7-6
OS18-3
OS18-4
GS21-6
GS6-1
OS16-5OS16-6
GS5-1
OS8-3
PS1-2
GS9-4
GS12-4
GS12-5
GS14-2OS13-4
GS6-3
GS6-4
GS13-6
OS7-1
PT3
OS17-5
OS18-3
OS7-2
OS7-3
OS4-2
OS6-6
GS3-1
GS16-3
GS1-2
PS1-2
GS10-1
GS20-4
GS14-3
GS7-2
GS7-3
GS15-4
GS9-2
OS14-2
OS1-2
OS2-1
PS1-4
OS12-3
GS12-4OS18-4
GS19-6OS20-4
GS10-3OS15-2
PS1-4
GS1-1
GS12-2

P-44/354
P-35/723
P-24/439
P-24/443
P-27/999
P-42/683
P-47/399
P-47/403
P-30/663
P-33/203
P-29/1008
P-36/755
P-45/812
P-45/816
P-44/850
P-35/326
P-42/691
P-42/695
P-39/842
P-31/174
P-22/15
P-30/427
P-24/439
P-31/178
P-31/183
P-26/110
P-40/166
P-28/629
P-26/893
P-47/593
P-29/1008
P-25/759
P-46/975
P-44/854
P-35/707
P-35/711
P-37/877
P-36/745
P-39/338
P-23/54
P-26/70
P-29/1014
P-38/301
P-45/812
P-24/443
P-30/960
P-41/483
P-25/767
P-44/359
P-29/1014
P-47/589
P-45/804

[N]
Nagai

I.

Nagamatsu

Y.

OS16-3
OS16-7OS14-2

P-47/391
P-47/407
P-39/338

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Nagamine
Nagashima
NagataT.
J.
F.

Nagayoshi
Nagayuki
Naito
Naito
Naitoh

M.
Y.
A.
S.
K.

Nakahira

K.T.

Nakai
Nakai
Nakamura
Nakamura
Nakamura
Nakamura
Nakamura

D.
I.
AkihiroAkira
K.
Masahiro
Masatoshi

Nakamura
Nakamura

Yohei
YosukeNakamura
NakanoYusukeH.

NakanoK.

Nakashima

T.

Nakashima
Nakategawa
NakatohNakayama
Namba
Narimoto
NG
Niimi

Y.
N.
T.
S.
H.
Y.
L. P.
A.

Niki
NishidaNishijima
Nishimuta
Nishio

E.
Y.
K.
K.
K.

Nishiyama
Noda
Noda
NoguchiNoh
NomuraNozawa

H.
J.
M.
K.
C.
Y.
A.

NunohiroE.

©ISAROB 2011

OS22-6
GS7-1
OS16-1
OS16-2
GS1-1
GS12-6
GS6-2
GS14-5OS17-1
OS17-2
OS6-2
OS6-6
OS9-1
GS8-1
OS18-3
GS18-3
GS2-4
GS16-1
OS22-3OS22-6OS14-3
GS8-4
GS18-4
GS19-3OS20-5
GS7-2
GS7-3
GS15-4
OS1-4
OS1-5
GS8-4
GS16-2
GS18-1
GS18-4
GS19-3
GS11-4
OS1-2
OS6-7
GS4-1
OS6-6
GS8-4
OS7-3
OS18-1
OS18-2
OS7-3
PS1-2
OS21-6OS15-5
GS11-1
GS11-2
GS9-2
GS16-5OS24-3OS20-4OS12-4
GS9-4
OS1-1
OS2-3
OS2-4
OS18-3

P-42/541
P-35/703
P-46/383
P-46/387
P-47/589
P-45/820
P-42/687
P-44/862
P-29/411
P-29/415
P-40/150
P-40/166
P-36/215
P-34/727
P-24/439
P-37/930
P-34/617
P-25/885
P-41/529
P-42/541
P-39/342
P-34/735
P-37/934
P-30/948
P-41/487
P-35/707
P-35/711
P-37/877
P-23/62
P-23/66
P-34/735
P-25/889
P-37/922
P-37/934
P-30/948
P-43/793
P-23/54
P-40/170
P-28/649
P-40/166
P-34/735
P-31/183
P-24/431
P-24/435
P-31/183
P-29/1008
P-43/515
P-45/371
P-42/785
P-42/789
P-36/745
P-26/901
P-28/581
P-41/483
P-38/306
P-36/755
P-23/50
P-26/79
P-26/83
P-24/439[O]
Oba
Ochiai
Ogai

S.
H.
H.

OgiharaOgiwaraOgiwara
Ogura
Oh
Ohara
Ohara
Ohnishi

F.
K.
T.
K.
J.H.
K.
M.
K.

Ohta
Ohta
Ohta
Oka

N.
Y.
Yoshihiro
T.

OkamotoOkamotoK.
T.

OkazakiOku
OkumuraOmichi
Omori
Ono
Ooe
Osawa

K.
M.
K.
S.
T.
S.
R.
H.

Osawa
Oshima
Osumi
Otani
Oya

S.
K.
T.
T.
M.

Oyabu

R.

[P]
Pagliarini
Park
Park

L.
H.J.
H.S.

Park
Park

J.B.
J.G.

Park

Minho

Park

GS14-4
GS3-3
OS21-4OS21-6
GS12-2OS16-1
GS18-5
PS1-2
PT1
GS21-6
GS7-5
GS3-5
GS4-4
OS1-1
OS17-5
OS1-4
OS13-3OS13-4OS13-5
GS6-3
OS19-5OS20-2
GS18-5OS14-1
OS5-1
GS13-5
GS1-2
GS4-1
GS2-3
OS24-1
OS24-3
GS5-2
GS13-5OS24-3
GS7-4
OS5-1
OS5-2
OS5-3
OS21-6

P-44/858
P-28/637
P-43/507
P-43/515
P-45/804
P-46/383
P-37/938
P-29/1008
P-22/1
P-27/999
P-35/719
P-28/645
P-28/659
P-23/50
P-30/427
P-23/62
P-35/322
P-35/326
P-36/330
P-42/691
P-46/463
P-41/475
P-37/938
P-39/334
P-23/126
P-39/838
P-47/593
P-28/649
P-34/613
P-27/573
P-28/581
P-31/667
P-39/838
P-28/581
P-35/715
P-23/126
P-23/130
P-23/134
P-43/515

Mison

IT5
OS21-1
GS9-1
PS1-5
PS1-6
PS1-7
GS21-4OS18-3
OS18-4OS23-4OS23-6
GS11-4

P-22/43
P-43/495
P-36/741
P-29/1016
P-29/1018
P-29/1021
P-27/991
P-24/439
P-24/443
P-43/561
P-44/569
P-43/793

Qian

S.H.

GS10-3

P-25/767[R]
RazaliS.

OS16-5

P-47/399[Q]

I-5

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Ryu

S.

OS23-6

P-44/569

Saga

R.

Sagara

S.

Saito
Saito
Saitoh
SaitohSakamoto

K.
R.
Atsushi
Ayumu
M.

GS6-3
GS6-4
OS5-4
OS5-5
GS18-5
GS7-3
OS1-4
GS14-2
OS7-1
OS7-2
OS7-3
OS7-4
OS7-5
OS15-1
GS18-5IT3
OS17-3OS17-4
OS1-5
OS20-3
GS9-3
OS3-3
OS1-3
GS13-6
OS3-2
OS3-3
GS12-7
OS2-2
GS18-5
GS20-2
GS20-4OS6-1
OS6-4
OS22-1OS11-6OS22-6
GS15-4
GS12-5OS20-2
GS6-5
GS10-6OS15-6OS15-7
OS9-3
OS6-3
GS13-5
GS21-3OS24-4OS5-4
OS5-5
GS21-1
GS10-5OS24-3OS24-4OS24-4
GS10-1
GS3-4

P-42/691
P-42/695
P-23/138
P-24/142
P-37/938
P-35/711
P-23/62
P-44/850
P-31/174
P-31/178
P-31/183
P-31/187
P-31/191
P-44/354
P-37/938
P-22/32
P-29/419
P-30/423
P-23/66
P-41/479
P-36/749
P-24/95
P-23/58
P-39/842
P-24/91
P-24/95
P-45/824
P-26/74
P-37/938
P-46/967
P-46/975
P-40/146
P-40/158
P-41/519
P-33/289
P-42/541
P-37/877
P-45/816
P-41/475
P-42/699
P-25/777
P-45/375
P-45/379
P-36/225
P-40/154
P-39/838
P-27/987
P-28/585
P-23/138
P-24/142
P-27/979
P-25/775
P-28/581
P-28/585
P-28/585
P-25/759
P-28/641[S]

SakataSakataSakuraiK.
M.
N.

Sakurama
SamuraSapaty
Sato
Sato
Sato
Sato

K.
T.
P.
A.
K.
N.
O.

Sato
Sato
SekineSeo
Serikawa
Seta

T.
Y.
Y.
W.
S.
K.

Shang
Shiao
Shibasaki
Shibata
Shibata
Shida
Shieh
ShimadaShimada

L.
S.W.
H.
K.
N.
H.
J.S.
A.
H.

ShimadaShimamura
Shimizu

K.
K.
E.

Shimohara
Shimozawa

K.
T.

Shin
Shin
Shinozawa

E.C.
H.C.
K.

Shiomi
Shiozaki
Sim

M.
A.
M.

©ISAROB 2011

I-6

Son
Song
Song
Sorimachi
Su
Su
Su
Su

J.
H.
J.
M.
H.
H.S.
J.P.
K.L.

SuginoSugisakaJ.
M.

SugitaK.

Suh
Sun
SusantoSuzukiSuzukiSuzuki

J.
K.
E.
A.
H.
I.

SuzukiSuzuki

M.
R.

[T]
TabataTabuseY.
M.

TachikawaTadakuma
Tadakuma
TagawaTaguchi
Taguchi
Taira

Y.
K.
R.
S.
R.
Y.
Y.

TajimaTakagiTakahashi
Takahashi

S.
N.
M.
N.

Takahashi
Takahashi
TakakiTakamura
Takeda

S.
T.
R.
M.
K.

OS12-1OS23-3OS22-3OS17-5OS4-5
OS11-6OS11-3OS10-2OS10-3OS10-4OS10-5OS11-2OS11-6
OS5-3
OS22-1
OS22-4OS22-7
GS9-3
GS11-5
GS21-7OS13-3OS13-5OS23-3OS13-1
OS3-1
GS9-4
OS14-5
GS2-3
GS2-4
GS2-5
OS24-2
GS3-2
GS3-3

P-38/293
P-43/557
P-41/529
P-30/427
P-26/122
P-33/289
P-32/277
P-32/249
P-32/253
P-32/257
P-32/261
P-32/273
P-33/289
P-23/134
P-41/519
P-41/533
P-42/545
P-36/749
P-43/797
P-27/1003
P-35/322
P-36/330
P-43/557
P-35/314
P-24/87
P-36/755
P-40/350
P-34/613
P-34/617
P-34/621
P-27/577
P-28/633
P-28/637

OS6-7
OS9-1
OS9-2
OS9-3
OS9-5
OS2-4
GS21-5
GS21-5
OS7-2
OS6-2
GS7-2
OS5-1
OS5-3
OS14-5OS5-3
OS18-2OS3-2
OS3-3
GS20-4OS17-5OS21-4OS21-3
OS8-1
OS8-2
OS8-3

P-40/170
P-36/215
P-36/219
P-36/225
P-36/237
P-26/83
P-27/995
P-27/995
P-31/178
P-40/150
P-35/707
P-23/126
P-23/134
P-40/350
P-23/134
P-24/435
P-24/91
P-24/95
P-46/975
P-30/427
P-43/507
P-43/503
P-33/195
P-33/199
P-33/203

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

TakedaK.

Takei
Takenouchi
TakeokaY.
T.
F.

Takeshita
TakeuchiM.
K.

TakeuchiTaki
TakidaY.
H.
R.

TakuboTamakiT.
H.

TamaruTamura

S.
H.

Tan

J.K.

TanakaTanakaTanakaTanakaF.
HirokiHisayaK.

TanakaTanakaTani
Taniguchi
Tanno

Yoshito
Yutaka
K.
R.
K.

Tarn
Thorsteinsson
Tien
Tokumitsu

T.J.
T.
C.X.
M.

Tokuyama
Tran
Tsuchida
Tsuji

T.
S.D.
Y.
H.

Tsuji
Tsuruhori
Tsuzuki
Tzou

Y.
Y.
Y.
J.H.

[U]
UchidaK.

UchidaM.

UchidaY.

©ISAROB 2011

OS8-4
OS8-5
OS2-3
GS16-4
GS13-1
GS17-4
GS19-1
OS6-6
OS6-1
OS6-3
OS6-5
OS24-2
PS1-2
OS7-1
OS7-2
OS7-3
GS21-6
GS1-1
GS7-5
OS5-2
GS6-1
GS14-1
GS10-2
GS10-3
GS10-7
GS11-4
GS2-6
GS21-5
OS2-2
PS1-1
PS1-8
OS3-5
OS3-5
OS20-3
GS10-6
GS6-1
GS14-1
PT2
IT4
GS1-4
OS19-1
OS20-5
OS9-5
GS17-3
GS15-2
OS6-1
GS3-1
GS6-3
GS5-4
OS1-4
OS19-6OS10-2
OS7-4
OS7-5
OS1-3
OS2-1
OS7-4
OS7-5

P-33/207
P-33/211
P-26/79
P-26/897
P-39/828
P-38/916
P-30/942
P-40/166
P-40/146
P-40/154
P-40/162
P-27/577
P-29/1008
P-31/174
P-31/178
P-31/183
P-27/999
P-47/589
P-35/719
P-23/130
P-42/683
P-44/846
P-25/763
P-25/767
P-25/781
P-43/793
P-34/625
P-27/995
P-26/74
P-29/1006
P-29/1023
P-24/103
P-24/103
P-41/479
P-25/777
P-42/683
P-44/846
P-22/4
P-22/36
P-47/601
P-46/447
P-41/487
P-36/237
P-38/912
P-37/870
P-40/146
P-28/629
P-42/691
P-31/675
P-23/62
P-46/467
P-32/249

Uchikoba
Uchiyama
Ueda
Ueki
Uenishi

F.
N.
T.
S.
T.
J.
K.
S.
A.

GS18-5
GS17-3
OS19-3
GS19-6
GS8-4
GS16-2
GS15-1
GS19-5
GS14-2
GS2-1
GS5-1
GS5-2
GS7-1
GS7-2
GS7-3
GS17-2

P-37/938
P-38/912
P-46/455
P-30/960
P-34/735
P-25/889
P-37/866
P-30/956
P-44/850
P-34/605
P-30/663
P-31/667
P-35/703
P-35/707
P-35/711
P-38/909

Ueno
Ueyama
Umata
Utani

[V]
Voges

K.

OS7-1

P-31/174

[W]
Wada
Wang
Wang

H.
B.
C.C.

Wang
Wang

F.
Jiwu

Wang
Wang
Wang
Wang
Wang

Lan
Lei
Q.
X.
Y.

Wang
WatanabeZ.
K.

Watanabe
Watanabe
Watanabe
Watanabe
Watari
Wu

S.
Takahiro
Tomokazu
Y.
S.
X.

OS5-1
OS22-6OS10-4OS11-2OS11-4
GS4-2
OS22-7
GS11-5
GS21-7OS21-2OS4-4
OS5-2
OS22-6
GS11-5
GS21-7OS21-6OS16-1
OS16-2OS16-3
OS16-4OS16-5OS16-6OS16-7OS17-5
GS4-2
GS16-1
OS20-1
OS20-5
OS6-7

P-23/126
P-42/541
P-32/257
P-32/273
P-33/281
P-28/653
P-42/545
P-43/797
P-27/1003
P-43/499
P-26/118
P-23/130
P-42/541
P-43/797
P-27/1003
P-43/515
P-46/383
P-46/387
P-47/391
P-47/395
P-47/399
P-47/403
P-47/407
P-30/427
P-28/653
P-25/885
P-40/471
P-41/487
P-40/170

Xu

Y.

OS13-2

P-35/318[Y]
YamadaYamadaYamadaYamaguchi
Yamaguchi

S.
Takayoshi
TakayukiD.
TakashiOS24-1
GS16-1
GS15-5
OS18-2
OS18-4

P-27/573
P-25/885
P-37/881
P-24/435
P-24/443[X]

P-31/187
P-31/191
P-23/58
P-26/70
P-31/187
P-31/191

I-7

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

Yamaguchi
Yamamori

TakehiroK.

Yamamoto
Yamamoto
YamamotoA.
Hidehiko
Hisao

YamamotoM.

Yamamoto
Yamanaka
Yamasaki

T.
N.
K.

Yamasaki
Yamashiro
Yamashiro
Yamashita
Yang
Yang
Yang
Yang
Yao
YasudaYasunaga

T.
N.
T.
S.
D.
K.
R.
S.
Y.
G.
M.

Yin
Yokoi
Yokomichi
Yokota

C.
H.
M.
M.

Yoon
Yoshihara

T.S.
I.

Yoshimatsu
Yoshinaga

T.
T.

Yoshioka
Yoshitomi

M.
Y.

Yoshizawa

K.

Yu
Yu
Yukinawa
Yuksel

K.
M.
N.
B.

©ISAROB 2011

OS18-1
OS15-3OS15-4OS15-5
GS10-2
GS16-1
GS2-1
GS5-1
GS5-2
GS7-1
GS7-2
GS7-3
GS17-2
GS2-3
GS2-4
GS2-5
GS11-1
GS17-3OS17-3OS17-4
GS11-2OS20-4OS16-2
GS14-1
GS18-5
GS21-1
PT2
GS18-2OS13-1
GS5-5
OS15-5OS15-6OS15-7
OS6-7
GS15-3OS3-3
OS13-3OS13-5
GS21-4OS15-3OS15-4OS15-5OS15-6OS15-7
GS14-1
OS7-4
OS7-5
GS15-2
OS9-2
OS9-3
OS9-4
OS9-5
OS9-6
OS17-3OS17-4OS12-3OS21-2
GS14-5
GS17-5

P-24/431
P-44/363
P-45/367
P-45/371
P-25/763
P-25/885
P-34/605
P-30/663
P-31/667
P-35/703
P-35/707
P-35/711
P-38/909
P-34/613
P-34/617
P-34/621
P-42/785
P-38/912
P-29/419
P-30/423
P-42/789
P-41/483
P-46/387
P-44/846
P-37/938
P-27/979
P-22/4P-37/926
P-35/314
P-31/679
P-45/371
P-45/375
P-45/379
P-40/170
P-37/873
P-24/95
P-35/322
P-36/330
P-27/991
P-44/363
P-45/367
P-45/371
P-45/375
P-45/379
P-44/846
P-31/187
P-31/191
P-37/870
P-36/219
P-36/225
P-36/231
P-36/237
P-37/241
P-29/419
P-30/423
P-38/301
P-43/499
P-44/862
P-38/918

I-8

[Z]
Zain
Zhang
Zhang

Z-Md.
C.
F.

Zhang

H.

Zhang
Zhang
Zhang
Zhang
Zhang

L.
M.
N.
Q.Y.
T.

Zhang
Zhao
Zhao
Zhao
Zhao
Zheng

Y.A.
H.
M.
Xiaoming
Xin
Z.

Zhou
Zhu
Zhuo
Zou

H.
Y.
J.
J.T.

OS16-7OS22-5
OS3-4
GS21-2
GS13-2
GS19-2
GS20-4PT2
OS22-2
OS4-4
OS22-3OS22-5OS15-1
OS22-4OS22-2
GS4-2
OS21-4OS17-3OS17-4OS22-5OS22-3
OS5-2
OS10-1

P-47/407
P-41/537
P-24/99
P-27/983
P-39/830
P-30/944
P-46/975
P-22/4
P-41/523
P-26/118
P-41/529
P-41/537
P-44/354
P-41/533
P-41/523
P-28/653
P-43/507
P-29/419
P-30/423
P-41/537
P-41/529
P-23/130
P-32/245

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

エム・イーシステム株式会社〒

８７９－１５０６

大分県速見郡日出町９７１－５

ＴＥＬ

０９７７－７２－７７１６

ＵＲＬ http://www.me-system.co.jp

ＦＡＸ０９７７－７２－８７７１

Ｅｰmail info@me-system.co.jp

自動組立機・自動検査装置・画像処理装置の設計製作

5-25 KITA-MATOGAHAMABEPPU-CITY OITAJAPAN 874-0928©ISAROB 2011

PHONE 0977-24-2111

B-1

The Sixteenth International Symposium on Artificial Life and Robotics 2011 (AROB 16th ’11),
B-Con Plaza, Beppu,Oita, Japan, January 27-29, 2011

©ISAROB 2011

B-2