Journal of Robotics, Networking and Artificial Life, Vol. 2, No. 4 (March 2016), 247-251

Dynamic Behavior Selection Model based on Emotional States for Conbe-I robot
Wisanu Jitviriya and Jiraphan InthiamComputer Science and Systems Engineering, Kyushu Institute of Technology.
680-4, Kawazu, Iizuka, Fukuoka, 820-8502, Japan.
wisanu@mmcs.mse.kyutech.ac.jp, jiraphan@mmcs.mse.kyutech.ac.jp
Eiji Hayashi

Mechanical Information Science and Technology, Kyushu Institute of Technology.
680-4, Kawazu, Iizuka, Fukuoka, 820-8502, Japan.
haya@mse.kyutech.ac.jp

Abstract
Currently, the rapid development of non-industrial robots that are designed with artificial intelligence (AI) methods
to improve the robotics system is to have them imitate human thinking and behavior. Therefore, our works havefocused on studying and investigating the application of brain-inspired technology for developing the conscious
behavior robot (Conbe-I). We created the hierarchical structure model, which is called “Consciousness-BasedArchitecture: CBA” module, but it has limitation in managing and selecting the behavior that only depends on theincrease and decrease of the motivation levels. Consequently, in this paper, we would like to introduce the dynamic
behavior selection model based on emotional states, which develops by Self-organizing map learning and Markovmodel in order to define the relationship between the behavioral selection and emotional expression model. We
confirm the effectiveness of the proposed system with the experimental results.
Keywords: Behavior selection model, Self-organizing map (SOM) learning, Markovian model.

1. Introduction
Nowadays, the focus of research of service robots isa development of the robots that are able to express
emotions, imitate human thinking and change theirbehavior based on variety surrounding environment.
However, development of these robots that are still ahuge challenge when they have interaction withhumans. Therefore, Hayashi laboratory’s works [1]-[2]have focused on studying and investigating theapplication of brain-inspired technology by developing
the robots with a hierarchical structure module based onTran Duc Thoa’s research [3]. This model is called“Consciousness-Bead Architecture (CBA)” model
which relates between consciousness field and behavior

group. But the conventional (CBA) model has limitationin managing and selecting the behavior that onlydepends on the increase and decrease of the motivationlevels. For a truly effective of the robotic system, therobots should not only perform the behavior, but also
express the appropriate emotional state. Thus, thegeneral objective of this research, we would like tointroduce the proposed system, which canautonomously select the suitable behavior based ontransition of emotional states.
Our paper is organized as follows: Chapter 2
explains the autonomous behavior selection system
which has three processes. In Chapter 3, our results areshown how our robot can choose behavior depending on

Published by Atlantis Press
Copyright: the authors
247

Wisanu Jitviriya, Jiraphan Inthiam, Eiji Hayashi

its emotional states. In the final section we conclude thepaper and describe our future works.
2. Autonomous Behavior Selection SystemIn this paper, we would like to propose the behavior
selection model which autonomously selects andperforms the suitable action based on variation ofemotional expression. The proposed system has threemajor processes as described below.

2.1.2. Calculation of the robot’s motivation
Since the computation of the naturally occurring
dopamine as described above, the sum of positive andnegative dopamine values that is used as the input
variable for calculating the robot’s motivation. The
motivation waveform can estimate by the 2nd ordersystem of differential equation as expressed by Eq. (3).
Motivation ''(t ) + 2ζωn Motivation '(t )ωn2 Motivation(t ) = ωn2Total _ dopamine(t )2.1. Computation dopamine and robot’smotivation(3)

2.2. Classification emotions and behaviors basedon Self-organizing map (SOM) learning

2.1.1. Determination of occurring dopamine model
The Conbe-I robot uses only the acquired images
from the CCD camera and web camera for operating theactions and emotional expression in recognition process.
The robot is not usable the other sensors (tactile sensor
or laser range finder sensor). Thus, the robot canevaluate the rough position of the target object by onlyusing the image processing. The system may simplify
the acquired images by dividing into four color groups:
green, blue, flesh-color and the other colors. With thesimplification, the sequential recognition process has
the reduced image size, the changed color model ofimage from RGB method to HSV method, the labelingprocess based on the color of visual information and thecalculation of the position of the target object. Then, thevisual information (the color, shape, size, labeling anddistance of the target object) is used to calculate andform the naturally occurring dopamine waveform [4].
The computation of the typical pattern of dopamine
model is explained. The dopamine waveform can beevaluated and divided into rise and fall portions. The
rising part and falling part are created by the 2nd orderand 1st system equations as shown in Eq. (1) and Eq. (2)respectively, where the input variable x(t ) is anaccelerator of dopamine, the output variable y (t ) is anaturally dopamine, ζ is damping factor, ωn is thenatural frequency and Tc is time constant.

y ''(t ) + 2ζωn y '(t ) + ωn2 y (t ) =ωn2 x(t )(1)

Tc y '(t ) + y (t ) =x(t )(2)After recognition process, the robot’s motivation andthe visual information are used to define as the input
parameters of the Self-Organizing Map (SOM) learningin order to generate the behavior and emotion maps. Therobot is set the nine behaviors and six emotions. To
classify and select the most appropriate behavior andemotion correspond to the surrounding environment.
The results of the SOM learning the will be verified.
2.2.1. SOM algorithm
The Self-Organizing Map is a neural networkarchitecture which is proposed by T. Kohonen [5]. It is
well known as an effective pattern classifier that is anunsupervised learning method. The SOM method canprovide topologically preserved mapping from highdimensional input data to a low - dimensional arraybased on competitive learning. The procedure of thebasic SOM learning can explain by the following steps:
Step 1: Each node must to be initialized with weightvalue based on setting the small standardized random
value.
Step 2: Calculate and find the minimum Euclidean
distance that is called “winner node or best matchingunit (BMU)”.
Step 3: After finding the BMU, the weight vectors of theBMU, and its topological neighbors are moved closer tothe input vector. The new weight vectors are updated byEq. (4), where t is the training step index is, η (t ) is thelearning rate, and h j ,c (t ) defines as the neighborhood
kernel function around the winner nodes.

Published by Atlantis Press
Copyright: the authors
248

Dynamic Behavior Selection Modelw j (t + 1)=w j (t ) + h (t )h j ,c (t )  x − w j (t ) 


(4)

Step 4: The training step increases to t+1. Steps 2 to 4
are repeated with decreasing the width of theneighborhood function.
2.2.2. Formation of behavior and emotion mapsIn order to verify the SOM learning, the motivation
value, the slope of robot’s motivation, the pixels ofcolors (green, blue and flesh colors) are defined as theinput data for testing. For example, if the robot
recognizes the green object, which is near the robot’shand. The testing data are the motivation flag = 1(positive level), the motivation value = 0.45, the slopeof robot’s motivation = 0.003, the motivation waveform
is a positive slope = 1, the green pixels = 0.6, the blue
pixels = 0.02 and the flesh-color pixels = 0.2. During
the SOM learning process, we have not only
considering the changing position of the winner nodes
in behavior and emotion maps, but also observe theevolution of the form of both maps as shown in Fig.1
and Fig.2. The blue cells showed on behavior andemotion maps that were the winner nodes of eachbehavior and emotion pattern. The green cells
represented as the response action according the testing
input data. Therefore, in this situation, the robot should
perform the action as “Approach”, and express theemotion as “Happiness” at the same time. Because, thegreen object is a favorite object, thus it tries to possess
the target object.
2.3. Behavior selection based on Emotional States
The last process is the behavioral-emotional
selection process, which is modeled based on a Markovmodel [6]. The Markov model is memoryless property
that means the next state will be changed based on atransition rule and depends only on the current state.
In this study, the Emotional Markovian model thatconsists of the six nodes representing the basic emotion
states. The arrows are the probabilities of getting out ofthe states. The next emotional state can determine byEq. (5), where, A is the emotional state transition
matrix.

Emotionk +1=A ⋅ Emotionk(5)

Fig.1 Behavior map

Fig.2 Emotion map

All elements of the emotional state transition matrix
can be updated by the affective factors, which calculates
by averaging the sum of the weights of individual groupof behavior and emotional patterns in SOM learning
process. In addition, the behavior selection module isbased on its emotional expression, thus the behavior
state will be selected by the emotional state that canestimate with Eq. (6), here, C denotes as the behavioral
state transition matrix.
(6)
Behaviork +1= C ⋅ Emotionk +1
All element values in the behavioral state transition
matrix (cbehavior/emotion) can be modified later based on thebehavioral and affective factors.

Published by Atlantis Press
Copyright: the authors
249

Wisanu Jitviriya, Jiraphan Inthiam, Eiji Hayashi

actions as “Interest” and “Approach” meanwhile itexpressed “Hope” and “Happiness” emotions. Then, theblue object was presented in order to decrease therobot’s motivation. At T5, the motivation modelachieved in the negative level, during periods T5-T6,
the robot felt unhappiness when it confronted thedisliked object, the robot expressed “Sadness” and“Fear” emotions meanwhile it performed the action as“Look around” and “Alertness”. After that (T6-T7), theblue object was moved closer to the robot, it felt dislike
this situation, thus it performed “Avoidance” behavior
and express “Disgust” emotion. Moreover, during
period T6-T8, the robot expressed “Hope” emotion,
when it did not see the disliked object or the robot’s
motivation increased to the positive level.
4. Conclusions

Fig.3 Behavioral and emotional selections at period T1-T8

3.

Experimental results

The intelligent behavioral-emotional selection
system was tested and evaluated, that confirmed its
effectiveness in the surrounding environment.
The robot was verified its behavior and emotional
expression when it recognized the favorite object (greenobject) and disliked object (blue object) in a realisticenvironment. Throughout the task period (T1-T8), therobot was able to act the appropriate behaviors andemotional expressions as illustrated in Fig.3. After therobot was started. The robot selected “Look around”
and “Interest (Flesh)” actions when it recognized theflesh-color. At T1, the robot was able to capture thefavorite object, thus the robot’s motivation increased
and the robot performed the behavior between “Lookaround” and “Interest”, the robot also express emotion
as “Neutral”. From T2-T3, the robot performed theIn this paper, the overview of the proposed systemthat consists of the recognition process, cognitiveprocess and behavioral-emotional selection process. The
system is executed by the Conbe-I robot in the realisticenvironment. All experimental results confirm theeffectiveness of the proposed system such as the system
can classify and generate the behavior and emotion
maps based on the SOM learning method in the
cognitive process, and the robot can suppress andchange all emotional states based on the Markov model.
In addition the robot can perform its behavior based on
updating the emotional transition state. For the future
work, we would like to combine the otherneurotransmitters (noradrenaline and serotonin models)
into the dopamine model in order to generate thedynamic emotional expression model.
References
1. N. Goto and E. Hayashi, “Design of Robotic Behavior
that imitates animal consciousness,” Journal of Artificial
Life Robotics, vol. 12, pp. 97-101, 2008.
2. E. Hayashi, T. Yamasaki, and K. Kuroki, “AutonomousBehavior
System
CombingMotivation
andConsciousness Using Dopamine,” Proc. of 8th Int. IEEE
Symposium on Computational Intelligence in Robotics
and Automation (CIRA 2009), pp. 126-131, 2009.
3. T. D. Thao, D. J. Herman, and D.V. Morano,
Phenomenology and dialectical materialism (Boston
Studies in the Philosophy of Science), D. Reidel Pub.,
1986

Published by Atlantis Press
Copyright: the authors
250

Dynamic Behavior Selection Model

4. H. Kimura, “A trial to analyze the effect of an atypicalantipsychotic medicine, risperidone on the release of
dopamine in the central nervous system,” J. of Aichi
Medical University Association, vol. 33, no. 1, pp. 21-27,
2005.
5. T. Kohonen, “Self-organized formation of topologicallycorrect feature map,” J. Biological Cybernetics, vol. 43,
pp. 56-69, 1982.
6. K. Koiji, and B. Martin, “Towards an emotion core basedon a Hidden Markov Model,” Proc. of 13th IEEE
International Workshop on Robot and Human Interactive
Communication, Sep. 20-22, Kurashiki, Okayama, Japan,
pp. 119-124, 2004.

Published by Atlantis Press
Copyright: the authors
251