Proceedings of 1st International Symposium for Color Science and Art 2019

Education and Communication Research Using Robotics
Based on Color Science
Junji Sone, Daisuke Katagami, Takahito Tomoto, Yuta Ogai, Takenori Obo, andYoshihisa Udagawa
Tokyo Polytechnic Univ., 1583 Iiyama, Atsugi, Kanagawa 243-0297, Japanassessed, we may obtain a novel clue as to how students’emotions vary in a learning situation. Heart rate variability
is widely used as a non-invasive measure of the ANSactivity. The pneumatic pressure sensor inserted into acushion can detect the ballistocardiograph (BCG) signal(Figure 2).

ABSTRACT
Color science psychological approach for educationand communication robotics system based on learningtheory, human agent interaction, biological analysis andcognitive science become more important for education,
driving assistance and so on. Our system was integrated
with advanced topics of information, biological androbotics researches.
1. INTRODUCTION
Many challenges of robotics and human communication
research, verbal and nonverbal communication is main
issue [1]. In our research, we are researching educationand communication with human using robotics based oncolor science, verbal and body representation. Mainapplication is education of students and we are trying to
improve studying motivation using color science androbotics communications.
2. RESEARCH ELEMENTS
2.1 Influence of robot expressing atmosphere using
utterance mind
It has been clarified that a person unconsciously
expresses the feeling "utterance mind" such as "I want totalk" when acting on speech as behavior. In this research,paying attention to the concept of "utterance mind", it isthought that a robot can read human atmosphere byimplementing behavior that expresses "utterance mind" onthe robot [2]. We verify whether it is possible to facilitatecommunication with people by implementing "utterancemind" on emotion recognition humanoid Pepper (Figure 1).

Fig. 2 Principle of the pneumatic pressure sensor.
2.3 Data mining for discovering educational rules from
Log data
Through practices of the education and communication
robots, large amounts of log data are generated. A text
matching algorithm using the Longest Common
Subsequence (LCS), a well-known algorithm of dynamic
programming, is developed to provide partial matching
functionality of texts, and then it is incorporated in an
Apriori-based data mining algorithm to discovereducational rules derived from experiments using theproposed robots for educational communication (Figure 3).

Fig. 1 Emotion recognition experiment system.
2.2 Heartbeat detection based on pulse neuron model
The design of an environment through a variety ofmeans such as lighting and colors can stimulate
perceptual and emotional responses in students and affecttheir behavior. Therefore, if it can be quantitatively

Fig. 3 Example of co-occurrence words extraction andtheir graphic representation.

31Proceedings of 1st International Symposium for Color Science and Art 2019

2.4 Human behavior recognition and control the robot
actions
Recently, many robots to perform the guidance andreceptionists have been utilized in a variety of places, but
those that move around are very few. In this research, we
aim to develop a mobile system for the Pepper robot thatcan be easily introduced at educational sites, such asclassrooms. Teachers are called by a show of hands insome cases of lectures. The Pepper robot needs ability torecognize the show of hands and move to the hand'sposition by following the black line. We developed afunction to recognize the students who are raising theirhands in the classroom using skeleton detection frommonocular camera images. We were able to recognize theshow of hands from the Y axis information of theirshoulders, elbows, and wrists. In addition, we were able todetermine which columns they were sitting from the XY
information of their neck. In order to trace a line for Pepper,
we developed a function to allow Pepper to discriminate
black tape affixed to surfaces in a classroom in directions
using deep learning. The deep learning network was madeby using transfer learning the learned VGG16 network.
Fig. 5 Color changing experiment during education.
3. Results and conclusion
We researched and developed the main element of
human education and communication assist using robotics.
Color phycology effects were observed by robot education.
Our base technologies are human agent interaction,
biological analysis, text mining, deep learning, robot
control and education with mental state method.
We will integrate these technologies and will build highlevels human education assistance and communication
robotics system based on color science phycology in thenear future.
4. Acknowledgement
This work was one research of "Research anddevelopment of educational and communicational robot
system" approved by the Council for Promotion of
Universal Future Society project at MEXT, Japan.

Fig. 4 Moving robot by recognition of student’s hand state.
2.5 Human education based on mental state
In education, not only promoting effective understanding
but also teaching behavior taking into consideration thelearner's mental state becomes important. In order toconduct teaching behavior at the same time as analyzing
the mental state, we need robots and agents. In thisresearch, we analyze the influence of robot's own teaching
behavior on learner's mental state. Furthermore, wepropose an environment including a teacher robot, a TA
robot and student robot (who is a partner robot).
In the first experiment, we investigated the influence ofsome statements ("This is important point." and "Be quiet.",etc.) with colored eyes in robots. As a result, it wasrevealed that the presented color affects the learner'simpression, especially impression changes depending onlearner's way of thinking about learning.
In the second experiment, we investigated the impression
when the robot speaks with colored eyes using script of an
actual teacher. As a result, it was revealed that thepresented color influences learner ‘s mental state also inthe context of the actual lesson (Figure 5).

5. REFERENCES[1] M. Asada, “Cognitive Developmental Robotics: ASurvey,” IEEE Trans. Auto. Ment. Develop., 1(1),pp.12-34, (2009)[2] T. Honya, D. Katagami, M. Yuasa, T. Obo, Y. Ogai, J.
Sone, T. Tomoto, Y. Udagawa, Evaluation of robot's
behavior to human based on understanding thesituation using utterance mind, HUMAN
COMMUNICATION GROUP SYMPOSIUM 2018
HCG2018-B-2-5, (2018)32