Journal of Robotics, Networking and Artificial Life, Vol. 1, No. 2 (September 2014), 140-144

Recognized Face Tracking for CONBE Robot
Sakmongkon Chumkamon

Department of Mechanical Information Science and Technology, Kyushu Institute of Technology, 680-4 Kawazu,Iizuka, Fukuoka, Japan
Eiji Hayashi

Department of Mechanical Information Science and Technology
Kyushu Institute of Technology
680-4 Kawazu, Iizuka-shi, Fukuoka, 820-8502, Japan
E-mail: haya@mse.kyutech.ac.jp
www.kyutech.ac.jp

Abstract
In our research, we develop the robot to combine with the consciousness and behavior which is the conscious andbehavior robot (CONBE). In this paper we present the basis system in the CONBE robot that consist of the twomain sections as the face recognition and robot gaze tracking. This system uses the face recognition using
Fisherfaces that also refer to linear discriminant analysis (LDA). For the robot gaze tracking, we implement withthe head of the CONBE robot that has a camera, two displays used for robot eyes and the actuators of 2 degrees offreedom. The actuators use to control the robot gaze for tracking the human face which the robot can recognize. Inthis paper, we experiment and present the results of the face recognition system and the tracking control systemsuch as the tracking error and time.
Keywords: Consciousness-Based Architecture, Fisherfaces, Face recognition, Cognitive robot, face tracking.

1. Introduction
An autonomous robot requires efficient operation of
their action. Decades, many good approaches have been
proposed for controlling the autonomous robot under
their condition such the development of the self-awarerobot. Due to a robot with self-awareness has a realistic
action to operate under the diversity situation morefunctional than a system without self-awareness. In thispaper, we develop a conscious behavior robot (CONBEI) that imitates a human head. The mechanism of a head
CONBE robot has 2 degrees of freedom that areorthogonal to each other in Cartesian coordinate. In ourlaboratory, we develop the robot to have theconsciousness and behavior as the mention and propose
the consciousness-based Architecture (CBA) system asa hierarchical structure model base on the mechanistic

expressed model of animal consciousness and behavior[1]. The motivational action of animal is derived fromrecognition and comprehension respectively. Therefore,
in this research, we focus on developing the recognitionof the robot to be a basis of CBA system. We develop
the face recognition with the CONBE robot to tracking
face which is recognized. In face recognition, we use
Fisher-faces algorithm that also refers to Linear
Discriminant Analysis (LDA), use a video camera as an
input instant image [2]. For hardware of the head robot,
we use Dynamixel actuators that are controlled via
RS485 bus to control the rotation of yaw and pitch axisof the head. For controlling, in the instant image, we
determine the distances between the center of thetracking object and the center of camera image invertical and horizontal axis. We then calculate thedistances of the angular errors for each rotation in yaw

Published by Atlantis Press
Copyright: the authors
140

axis and pitch axis and command the target angle to theactuators to tracking object. As previously mentioned,
this research studies the face recognition that is one ofrecognition models in our CBA, and studies the part of
the robot behavior that is an action of following gaze.
This system imitates the animal character in the case of
the animal seeing some people. It could recognize thepersons and interested in them, it then acts somebehavior such as a gaze following in our case. Oursystem imitating the animal system, due to our attempt
is to increase the function of the robot.
The robot would operate more than following thecommand from human such as when the robot combines
CBA system; it can introspect to act some suitablebehavior when the robot can recognize the user beingsad, happy and so forth. Moreover, this robot would be
more attractive for human or user with this proposed.

2. Robot system overview
The system based on CBA system combining with robot
mechanism and software that we develop to archive facerecognition and tracking face which is recognized. Thissystem consists of two main such as hardware andsoftware system.
USB video capturedeviceCVBS

TTL

TTL/RS232
convertor

RS232

USB

USB toSerial port

USB

CMOS Camera
USB
RS485

USB/RS485
convertor

Pitch rotation

Yaw rotation

Fig.3.

RS485

Connection diagram of the system.
Obtain image fromCamera
And Initialize robot
Haar classifiercascade profile

Face Detection

Haar classifiercascade profile

Eye detection
Face Preprocessing

Face training
data

Face recognition
using FisherFaces
Determine theposition error of theface and calculate indegree to rotate

Fig.1. Consciousness-based architecture

Send command to
the actuators

Fig.4.

Software flow diagram.

2.1. Hardware

Fig.2. Head robot.

For head robot, we designed and constructed thestructure with the acrylic plastic. The head robot also
assembles with two actuators to rotate in Cartesian
coordinate which the first actuator operates as a robot
neck which base Dynamixel RX-64 and the second
actuator for up down rotating which base Dynamixel
DX-117. These actuators are DC gear motor with plastic

Published by Atlantis Press
Copyright: the authors141

body which are controlled by packet commandcontaining with an identity number of the actuator via
serial RS485 multi drop and yield the feedback such as
position of rotation degree, temperature, load, Input
voltage, speed and so forth. For obtaining the image toprocess, we use embedded CMOS camera based on
SONY FCB-H11 which support full HD (1080i highdefinition) video format with; NTSL and PAL. Thecamera model is equipped with ×10 optical auto focus
zoom lens and the ×12 digital zoom. The camera
module can control the features of the camera with
VISCA protocol via serial port. In this paper, we use
this camera in PAL mode connecting with composite
video channel and convert composite video channel toUSB by USB video capture device. All the device in therobot system is controlled by a laptop which base CPU
i7-2630QM, RAM 4GB.

we provide the environment for recognized facetracking. For the next section, we would experiment thissystem to apprehend the performance and other specificof this system.

2.2. Software

Fig.5. Example image of face and eye detection in different
case; the rectangle is the face region and the circle is theposition of the left and right eye.

Center error (pixel)

The system software has two major such facerecognition and control which are implemented by C++
language in a laptop. For the image processing library,
this software use OpenCV library. At the beginningsoftware from the face recognition, the system would
capture the instant image from the camera thenoperating with human face detection using Haar featuresand Haar classifier cascade profile to determine the faceposition [3]. The software also detects the human eye
due to obtain the definite position to provide the face
preprocessing image due to the face recognition isvulnerable with some difference in lighting condition,
face orientation, face expression, and so forth. For eye
detection, it is used to crop the input image to show only
the approximate eye region. We then perform thepreprocessing
image
combininggeometricaltransformation andcrop;
separate
histogramequalization for left and right side; and smoothing
image using a bilateral filter; and elliptical mask toremain hair and background from the face image.
Eventually we perform the face recognition using
Fisherfaces algorithm with our training data and facelabel. Fisherfaces is the popular algorithm for facerecognition using LDA which is to improve thecapability of classification from PCA which is the coreof the Eigenfaces method that is a powerful method torepresent data. The simple concept of LDA is that thesame classes should group closely together whereas
different classes are more distance [4]. In this section

300

Horizontal

200

Vertical

1000051015202530-100-200-300

Time (s)Fig.6. Distance error between the image center and tracking
object center in horizontal and vertical

3. Experiment
In the system that we already provided, we experiment
this system divided in two main sections; the first is facedetection and recognition; and the second is the tracking
system or the robot gaze following the recognized face.
The face detection is implemented using local basepattern feature cascade classifier with the restrictedcondition of the smallest size image as 25×25 pixels
which can detect the face. For the eye detection, wePublished by Atlantis Press
Copyright: the authors
142pixels, a maximum error as 158, 84 pixels, and theaverage error as 2.7, 13.8 pixels in horizontal andvertical respectively.

Fig.7. Example images in the robot tracking of recognized
face. The left side show the recognized face tracking and theright side show unknown face.
170

Horizontal

120

Center error (pixel)

used Viola-Jones object detection framework with Haarfeature-based cascade classifier that is an effective andpopular object detection method [3]. We experiment theperformance of the face and eye detection in different
condition with some results as shown in Fig. 5. The facedetection also is implemented to incorporate with therobot to experiment for the robot face tracking. The
robot face tracking perform the experiment for the robot
tracking performance as shown in Fig 6. In thisexperiment, we test the robot tracking the face while thehuman moving and facing the camera, the first we movefrom left to right side and return, the second we moveup and down to test. The distance between the humanwhile testing is around 1.5 meters. The experiment finds
out the performance and the characteristic system suchas the average frame rate as 16 frames per second, theaverage error as 5, 15.6 pixels, the minimum error as 272, -83 pixels and the maximum value as 228, 159
pixels in horizontal and vertical axis respectively.
The face recognition algorithm test with the laptop andthe camera in the robot incorporating with the tracking
control. In the experiment, 6 members of our lab havecontributed the sample videos to construct the facerecognition algorithm. The training program acquired
the image by capturing each contributed videos. During
the video recording using the camera in the head robot,
the member would face the camera first then turn left,right and move up down his face in front of the camera
slightly. Every image would perform the face detection
using LBP-based cascade classifier profile andpreprocessing image to preparing for training in
Fisherfaces algorithm to construct the face recognition
model [2], [5]. The training image size after resize andpreprocessing would be 120×120 pixels.
Fig. 7 illustrates the example of the experimental results
face recognition and tracking. There are two member
lab testing the system, left side image which is therecognized face then the robot would track the facewhereas the right side image which is the unknown facethen the robot would not track the face. Fig 8 shows theresults of the recognized face tracking such as the errorof the center image between the center of the faceregion and the center of the camera image in horizontal
axis and vertical axis. In this condition we move theface on the right, left, up and down to find out thecharacteristic and performance of the tracking control.
In this experiment, we find out the average frame rate at
13 frames per second, a minimum error as -121, -59

Vertical70
20-30 0

2.5

5

7.5

10

12.5

15

17.5

20

22.5

25-80
-130

Time(s)Fig.8. Distance error between the image center and tracking
object center in horizontal and vertical.

4. Conclusion
This paper presents the face recognition combining withimage tracking by CONBE robot for an experimental
personal robot in our concept to develop the CBA. In
the proposed visual tracking and the system processes
around 16 frames per second for face detection and 13
frames per second for face recognition incorporatingwith robot face tracking. For the face detection andtracking the robot is able to track the face following
pretty fast but in the recognized face tracking, the robot
following pretty slow as the face recognition algorithm
processing time and depending on the difficulty of the
facial similarity. Because while testing the recognized
face tracking, this system could not recognize the face

Published by Atlantis Press
Copyright: the authors
143every frame as the image is a blur image from themovement reason.
As the results, we first proceed the system to improve
the robust face recognition in real time as well aspossible. We then develop the facial express recognition
to recognize the emotion expressing on the face to fulfill
the part of the CBA operating with the face.
Acknowledgements
We would like to thank the members of our lab who
contributed the image along with for testing the systemand thank Mr. Masato Koike from Department ofMechanical Information Science and Technology,
Kyushu Institute of technology who contributed thesample source code for the robot eye expression.
References
1.

2.

3.

4.
5.

Hayashi E, Ueyama K, Yoshida M (2011), Autonomousmotion selection via consciousness-based architecture. In
Ubiquitous Robots and Ambient Intelligence (URAI),
2011 8th International Conference on, pp. 401-402
Belhumeur P. N., Hespanha J. P., Kriegman D. J. (1997),
Eigenfaces vs. fisherfaces: Recognition using classspecific linear projection. In Pattern Analysis andMachine Intelligence, IEEE Transactions on, pp. 711720.
Viola P., Jones M. (2001), Rapid object detection using aboosted cascade of simple features. In Computer Visionand Pattern Recognition, Proceedings of the 2001 IEEE
Computer Society Conference on Vol. 1, pp. 511-518
Turk M, Pentland A (1991), Eigenfaces for recognition.
In Journal of cognitive neuroscience, pp. 71-86.
He DC, Wang L. (1990), Texture Unit, Texture
Spectrum, and Texture Analysis. In Geoscience andRemote Sensing, IEEE Transactions on, vol. 28, pp. 509 512.

Published by Atlantis Press
Copyright: the authors
144