Journal of Robotics, Networking and Artificial Life
Vol. 5(4); March (2019), pp. 223–227
DOI: https://doi.org/10.2991/jrnal.k.190220.002; ISSN 2405-9021 print; 2352-6386 online
https://www.atlantis-press.com/journals/jrnal

Research Article

Development of Automation Recognition of Hazmat
Marking Chart for Rescue Robot
Wisanu Jitviriya1,*, Poommitol Chaicherdkiat1, Noppadol Pudchuen1, Eiji Hayashi2
Faculty of Engineering, King Mongkut’s University of Technology North Bangkok, Bangsue, Bangkok 10800, Thailand
Mechanical Information Science and Technology, Kyushu Institute of Technology, Iizuka, Fukuoka, 820-8502, Japan

12ARTICLE INFOABSTRACT

Article History

A long history of first place awards in World RoboCup Rescue Robot competitions is Invigorating Robot Activity Project (iRAP)
such as iRAP_PRO, iRAP_FURIOUS, iRAP_JUNIOR, and iRAP_ROBOT. In this paper, we would like to introduce and explain
an autonomous system of our rescue robots for detection and recognition of hazardous material (Hazmat) marking charts. AllHazmat tags are considered and computed by using speeded-up robust feature combined with fast library for approximate nearest
neighbors to match with the templates. Finally, the paper presents experimental results based on real situations to confirm aneffect of the pattern recognition of robotics.

Received 11 October 2018
Accepted 19 December 2018
Keywords
Hazmat marking chart
SURF
FLANN
rescue robot© 2019 The Authors. Published by Atlantis Press SARL.
This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).

1. INTRODUCTION
Invigorating Robot Activity Project (iRAP) is the robotics clubof students from King Mongkut’s University of TechnologyNorth Bangkok, Thailand. Every year, we participate in theWorld RoboCup Rescue Robot competitions, and our teams
have won first place many times in the competitions [1]. Ourrescue robot can be divided into three major parts such as
the mechanical robot part, the electrical system part, and the­software system part. The unique point of the iRAP_ROBOT
is the human–robot interface (the operator console), which canshow and present extensive sensor values and visual data (CO2sensor, visual temperature sensor, laser-­scanner, IMU sensor
and four cameras) in real time as shown in Figure 1. Moreover,
our rescue robot is designed based on proficiency by the agility
and performance tests.

2. STRUCTURE OF iRAP_ROBOTThe structure of the iRAP rescue robot is an original platform
for the rescue robot which has two pairs of flippers (in the front
and in the rear) as shown in Figure 2. In addition, the 6-DOF
robotic arm (one prismatic joint and five revolute joints) is set
up on a body of the robot, to search and observe the victim
and situation in the surrounding area. The multi-sensors are
installed in the end-effector of robotic arm, and also, the cameraon the robotic arm is used for recognizing the Hazmat chartautonomously.
*Corresponding author. Email: wisanu.j@eng.kmutnb.ac.th

3. AUTOMATIC RECOGNITIONOF HAZMAT CHART
The proposed paper that presents the Speeded-up Robust Feature(SURF) method [2] for extraction the information from the image
and then using the Fast Library for Approximate Nearest Neighbors(FLANN) model [3–5], which is used to match the input imagewith Hazmat chart templates. And the system uses blob detection[6] to mark the position of Hazmat chart. The overview system canbe divided into two phases: the initialization phase (Figure 3) andthe real-time processing phase.

3.1. Initialization Phase
3.1.1. Load all template images
The first part of initialization phase is to input all the Hazmat
chart templates. There are 12 templates for training the proposed
methods as displayed in Figure 4.

3.1.2. Pre-processing image
In the pre-processing image, there is a common task to improve oradjust the feature of image (scale, color, edge, area etc.).

3.1.3. Feature extraction
The SURF algorithm is applied to the feature extraction. It consists of detector (Hessian matrix), as expressed and descriptor

224

W. Jitviriya et al. / Journal of Robotics, Networking and Artificial Life 5(4) 223–227

Figure 1 | Operator console of iRAP_ROBOT.

Figure 2 | iRAP rescue robot.

Figure 4 | Hazmat chart templates.

schemes (Haar wavelet). The determinant of a Hessian matrix
in Equation (1) shows the expression of the local change aroundthe area, where H(x, s) is the hessian matrix, as illustrated inEquation (2), and Dxx(x, s), Dxy(x, s) in Equations (3) and (4) are
the convolution of the integral image and the second derivativeof the Gaussian function.
det(H approx ) = Dxx D yy − (wDxy )2 d
 D ( x ,s ) Dxy ( x ,s )
H (x ,s ) =  xx
 Dxy (x ,s ) D yy ( x ,s )
	   
î2Dxx (x ,s ) = I (x ) *g (s )
î x2
Dxy (x ,s ) = I (x ) *Figure 3 | Initialization phase of the proposed method.

î2g (s )
î xy

(1)
(2)
(3)
(4)

After having create the response map, the next step is to consider anon-maximum suppression based on the determents of the Hessian
matrix that is the objective of the SURF detection.

W. Jitviriya et al. / Journal of Robotics, Networking and Artificial Life 5(4) 223–227

225

3.1.4. Compute descriptors

3.1.5. Concatenate descriptors

At each interest point, we have to indicate the unique descriptionof a feature that do not depend on the size and rotation. The SURF
descriptor is based on the Haar wavelet model that can be calculated
efficiently with integral images. The wavelets response in thex- and y- direction is defined as dx and dy respectively as shownin Figure 5. For each sub-area of a vector, descriptors are calculated based on Equation (5).

In this sub-process, it is created by the new platform of descriptors
to the sample feature. Then, we have 12 sample feature descriptors
to match with the input image.

vsub =  ∑ dx , ∑ dy , ∑ dx , ∑ dy (5)3.2. Real-time Processing Phase
The real-time processing phase (Figure 6) is similar to an initialization phase. First, the acquired image is input into the mainprocess. The SURF method is used to extract the interest points
and compute the descriptors. And then, the descriptors of theinput image are compared with all sample images. The similarity
score of the keypoint is calculated in each class, Finally, the input
image is specified in the winner class, which is the maximum
similarity score. Then, the position of the Hazmat image is inthe frame that will be considered to calculate the centroid of aninterest area.

4. EXPERIMENTAL RESULTS
Figure 5 | The wavelets response. Black and white areas show the weight−1 and 1 for the Haar kernels.

The experimental results were verified with the Visual Studio C#program and an open source library such as Emgu CV library.

Figure 6 | Real-time processing phase.

226

W. Jitviriya et al. / Journal of Robotics, Networking and Artificial Life 5(4) 223–227

Figure 7 | One Hazmat chart matching. (a) Oxidizer Hazmat chart. (b) Flammable solid Hazmat chart.

4.1. Verification I
In this case there is only one Hazmat image in the frame, the resultsof the proposed method can be used in recognition and detection
autonomously, it did not depend on the size of input image androtation. In Figure 7a and 7b, the system that will select highest
similarity score as the answer.

4.2. Verification IIBesides, we had more testing of the proposed method by adding
four Hazmat charts as the input image to verify the uncertain imagegroup. In Figure 8, the experimental results show four maximum
similarity scores such as “INFECTIOUS SUBSTANCE (Class6)”,“OXIDIZER (Class5), “EXPLOSIVE (Class1)”, and “CORROSIVE(Class8)”, which were matched with the training images. Theresults that confirm the proposed method can classify and identify
the Hazmat chart accurately and correctly. Moreover, our proposed
method that could draw the frames and mark the position of each
Hazmat chart is based on their density features.

5. CONCLUSION

Figure 8 | Four Hazmat charts matching.

This paper presents the development of the automatic recognitionof a Hazmat marking chart for rescue robots. The results that affirm
the proposed method can be used in detection and recognition in
the real situation. In the future work, we will improve not only thespeed of the recognition system but also the accuracy of method
using deep learning method.

W. Jitviriya et al. / Journal of Robotics, Networking and Artificial Life 5(4) 223–227

REFERENCES[1] A. Phunopas, A. Blattler, N. Pudchuen, iRAP Robot: World Robo­
Cup Rescue Championship 2016 and Best in Class Mobility Award,
in: S. Behnke, R. Sheh, S. Sarıel, D. Lee, (Eds.), RoboCup 2016: Robot
World Cup XX. RoboCup 2016. Vol. 9776, Springer, 2016.
[2] H. Bay, A. Ess, T. Tuytelaars, L. van Gool, Surf: speeded up robustfeatures, Computer Vision and Image Understanding 110 (2008),346–359.
[3] M. Muja, D.G. Lowe, Fast matching of binary features, 2012 Ninth
Conference on Computer and Robot Vision, IEEE, Toronto, ON,Canada, 2012, pp. 404–410.

227[4] M. Muja, D.G. Lowe, Fast approximate nearest neighbors with
automatic algorithm configuration, In Proceedings of the FourthInternational Conference on Computer Vision Theory andApplications (VISAPP’09), 1, 2009, pp. 331–340.
[5] K. Hajebi, Y. Abbasi-Yadkori, H. Shahbazi, H. Zhang, Fast approximate nearest-neighbor search with k-nearest neighbor graph, in:Proceedings of the 22nd International Joint Conference on Artificial
Intelligence, 2011, pp. 1312–1317.
[6] Wikipedia, “Blob detection—Wikipedia, the free encyclopedia,” 2011. Available from: https://secure.wikimedia.org/wikipedia/en/wiki/Blob detection [Online; accessed 14 July2010].

Authors Introduction
Dr. Wisanu Jitviriya

Mr. Noppadol Pudchuen
He received his PhD from Kyushu
Institute of Technology, Japan in
2016. He is a lecturer in the Depart­
ment of Production Engineering,
Faculty of Engineering, King Mong­
kut’s University of Technology North
Bangkok, Thailand.
His main research interests areRobotics and Automation systems.

Mr. Poommitol Chaicherdkiat
He received his Bachelor degreefrom the Department of Electrical
Computer Engineering, Faculty ofEngineering, King Mongkut’s Univer­sity of Technology North Bangkok,Thailand. His main research interests
are Computer Vision and Robotics
Systems.

He received his Master degree fromDepartment of Instrumentation andElectronics Engineering, Facultyof Engineering, King Mongkut’s
University of Technology North
Bangkok, Thailand in 2017. Heworks as a lecturer in Faculty ofEngineering at King Mongkut’s
University of Technology North
Bangkok. His main research interests
are the Computer Vision and SLAM for Robotic systems.

Prof. Dr. Eiji Hayashi

He received his PhD from Graduate
School, Division of Science andEngineering, Waseda University,
Japan in 1994. He is a Professor
in Faculty of Computer Scienceand Systems Engineering, Kyushu
Institute of Technology, Japan. His
main research interests are Perce­
ption information processing andIntelligent robotics system.