Journal of Robotics, Networking and Artificial Life, Vol. 1, No. 1 (June 2014), 45-48

Study on the Improvement of Flexibility for an Industrial Robot Based on Machine Vision
Jiwu Wang1

School of Mechanical, Electronic and Control Engineering, Beijing Jiaotong University
Beijing 100044, China1
Xianwen Zhang1, Weining Zhang1

School of Mechanical, Electronic and Control Engineering, Beijing Jiaotong University
Beijing 100044, China1
Sugisaka Masanori 2

ALife Robotics Corporation Ltd.,Oaza Shimohanda, Oita 870-1112, Japan2
E-mail: jwwang@bjtu.edu.cn; ms@alife-robotics.co.jp

Abstract
In order to improve the applications for an industrial robot, it is necessary to increase its flexibility and accuracy.
The machine vision technology is an effective solution. In order to reduce the influence of the size, deformation etc.,
the target extraction method with fusion of color space and moment invariants is developed. It makes the multiple
targets extracted accurately and robustly. The experiments results showed that the developed algorithms are robust,and the robot’s flexibility can be improved.
Keywords: Arm robot, Target recognition, Flexible control, Machine vision

1. Introduction
With the development of new technology and the higher
requirements of product quality and production
efficiency, more and more production lines are used inmodern companies. 1 But skillful workers withexperience can not be trained in a short time and its
number is not enough. Many automation machines are
required to operate like skillful workers. Compared with
an operator, the robot rarely makes mistakes if its
working conditions are not changed. And its efficiency
and reliability is higher than an operator. Especially inthe danger or high working intensity environment, it ismore appropriate than human beings. Then various

robots are developed and applied in the most advancedcompany.
Generally, most robots firstly can only be used for somesimple, repetitive and single jobs. But now the further
applications are required to be developed. On the other
hand, the repetitive positioning accuracy can not besatisfied if an arm robot is used for a longer time.
Although some feedback or sensor fusion technology isadopted, the effective solutions are not enough.
With the development of the CCD camera technology,
machine vision gets more attention. The CCD camera
not only works like human being eyes, but also canprovide more information than other sensors. With thepattern recognition algorithms, the CCD camera canPublished by Atlantis Press
Copyright: the authors
45

J. Wang, X. Zhang, W. Zhang and M. Sugisaka

provide multiple target positions in a captured image. 2
Moreover, it can calculate the end position of the arm
robot in a captured image if necessary. In a word, with aCCD camera, the operating flexibility and repetitive
positioning accuracy can all be improved.
In the application of machine vision, in order to getrobust and reliable image processing results, the lighting
conditions and image processing efficiency should beconsidered carefully. In this paper, considering thepractical application requirements, the fusion techniqueof true color space and moment invariant is applied in
the pattern recognition.
The structure of the arm robot is shown as figure 1. A is
the CCD camera, B is the control box, C is a step motor
to control the upper arm, D is a step motor to control theforearm, E is a step motor to control the hand, F is a
support just like the shoulder, G is the upper arm, H isthe forearm. The length of the upper arm and forearm is400mm respectively. The height between the camera
and ground is 700mm.

Fig. 2 The control relationship about all hardware

The developed software system is shown in figure 3. In
the centre part, it is the target display area. The rightpart is the motor control area.

Fig. 3 Illustration of the developed software interface

2. The Target Extraction Method

Fig. 1 The structure illustration of the arm robot

The control relationship among all hardware isillustrated in figure 2. The CCD camera is used todetermine the target position. Three motors (C, D, E) is
controlled by the PC through motion control card. The
working process is as following: 1) After some targets
are detected by the CCD camera, they will be extracted
by the image processing algorithms in PC(An industrialcomputer), and their positions will be calculated. 2)
Based on the determined target positions, the computer
will calculate rotation angles for each joint, and three
motors will run together to make the hand reach eachtarget.

In order to extract targets from the captured image at ahigher speed, the color based target detection method isdeveloped. At the same time, to improve recognition
ratio, the moment invariant algorithms are then applied
to determine whether the separated objects are targets ornot.
In normal case, the triple components (R, G, B) of eachpixel can be used to extract the target area in a common
RGB image. But it is more difficult to determine anappropriate color threshold suitable for different lighting
conditions, which will bring unstable for the following
target recognition.
In order to reduce the influence of different lighting
conditions, a color model in the chromatic color space isbuilt. Chromatic colors, also known as pure colors in theabsence of luminance, are defined by a normalization of
the triple components (R, G, B). The process to build
target color model is as follows:

Published by Atlantis Press
Copyright: the authors
46

Study on the Improvement1)

Capture some common RGB color images of thetarget under different lighting conditions, and takethem as sample images. Then extract the typical
RGB target color areas manually from thesesample images as sample colors.
2) Transform these sample colors from the RGB
color space to the chromatic color space by anormalization process. Here color green isredundant because of r  g  b  1 .
r  R /( R  G  B )(1)b  B /( R  G  B )3) Calculate the mean value and covariance of sample
colors in the chromatic color space.
xx  n(2)nT
 ( x  x )( x  x )C  nn 1
Where, x is a pair of normalization components (r, b)
for each pixel in the chromatic color space; x is themean value and C is the corresponding covariance.
4) The possibility of the target color for each pixel inthe detected image can be calculated as follows:(3)
Px  exp[  0 . 5 ( x  x ) C  1 ( x  x )]When the color model is used to extract target, weshould firstly transform each pixel from the RGB color
representation to the chromatic representation. Then apair of chromatic color values for each pixel is input thecolor model to decide whether that pixel has the samecolor as the target. By this way, the whole image will
be separated by the target areas and non-target areas.
The next step is to extract the targets with moment
invariants in the above separated image based on thecolor model. The geometric moment invariants arecalculated by the following formula3,4:
Mpq =x p y q f ( x , y ) dx dy , p , q  0,1, 2...
(4)

3. Extraction on Multiple Targets andPositioning
In order to verify the above developed algorithms, thefollowing experiments are carried.
The first experiment is done as shown in figure 4. We
take a picture with black circle marks. After the target
areas are separated from non-target areas, the target
areas are located with the moment invariant algorithms.
Just like the results shown in figure 4-2, the center of
each black circle is marked with a red dot. The resultshows the multiple targets can be extracted accurately in
one captured image with our developed image
processing algorithms.

1) Sample image with circles
2) Results after processing
Fig.4 Extraction on multiple targets and center positioning

The second experiment is done in our lab with theimage just like the picture shown in figure 3. In thatimage, there are three marks: two purple circle marks
and one green circle mark. It can be easily found theinfluence of the lighting in figure 3.
Based on the steps explained in section 2, the purple
color mark modal and green mark modal are built. With
the color modals, the target areas are separated from
non-target areas. In figure 5, the targets can be extracted
effectively even under the influence of lighting. Thenusing the moment invariants, the center of each mark islocated.

DWhere, D is the area of each separated independent area,x p y q as the transform kernel, M pq is called ordergeometric moments.
Here only the center of each target is required, therefore,
M00, M01, M10 are necessary. The center (x, y) of eachobject will be calculated as following:
M 10M 01(5)
x ,y M 00

M 00

1)Two purple marks2) One green markFig.5 Positioning multiple marks in lab lighting environments

Figure 6 shows the arm robot move to one target basedon the position obtained with image processing.

Published by Atlantis PressCopyright: the authors
47

J. Wang, X. Zhang, W. Zhang and M. Sugisaka

Fig. 6 Illustration on the arm robot picking one target

4.

Conclusions

In this paper, an arm robot and the position control
system based on CCD camera is developed. By fusionof color based target detection and moment invariants,
the multiple targets can be extracted efficiently and theirpositions can be calculated. Because the multiple targets
can be obtained in one captured image, unlike previous
arm robot that can only move based on each input
signal, the robot can be operated automatically to moveto each target based on their different locations.
References
1. Y. B. Wang, Y. Z. Hu, and M. Y. Lu, Siemens machine
vision system and its application in automobile engine
assembly line, Automation systems engineering , 1 (2006)
20-22.
2. K. L. Zhuang, J. Z. Wang, and J. Zhou, Application ofmachine vision in angle inspection, Equipment
Manufacturing Technology, 4(2011) 4-10.
3. G. L. David, Distinctive image features from scaleinvariant key points. Int. Journal of Computer Vision,2(60)(2004) 91-110.
4. A. Khotanzad, Y. Huahong, Rotation invariant image
recognition using features selected via a systematic
method, Pattern recognition, 23(10)(1990) 1089-1101.
5. M. Sugisaka, X. Wang, Intelligent control of a mobilevehicle using on-line learning, System science, 25(1999)
41-50.
6. M. Sugisaka, X. Wang, J. Lee, Intelligent control strategy
for a mobile vehicle, Applied mathematics andcomputation, 20(1998) 91-98.

Published by Atlantis Press
Copyright: the authors
48