Journal of Robotics, Networking and Artificial Life, Vol. 4, No. 3 (December 2017) 217–220___________________________________________________________________________________________________________Study On The Target Positioning For An Omni-Directional 3 DOF Mobile Manipulator Based
On Machine Vision
Jiwu Wang
School of Mechanical, Electronic and Control Engineering, Beijing Jiaotong University
Beijing 100044, China1
Yao Du1, Wensheng Xu
School of Mechanical, Electronic and Control Engineering, Beijing Jiaotong University
Beijing 100044, China1
Masanori Sugisaka
Alife Robotics Corporation Ltd, Japan and Open University, United Kingdom2
2E-mail: 15121244@bjtu.edu.cn1; ms@alife-robotics.co.jp
www.bjtu.edu.cn

AbstractThe omni-directional mobile robot with multi DOF, because the operation posture and operation accuracy of themanipulator can be better controlled in a narrow or crowded workplace compared with the general manipulator, isgetting more interested in practical applications. The present problem is to improve its flexibility for operating
multiple different targets. Target recognition with image processing is an effective solution. Based on the image
processing, the position and posture of the target can be determined. Then the signal will be sent to the arm controlsystem. In this paper, the illumination conditions, distortion, etc. are studied in the target recognition. The target
position with image processing, is verified with real coordinates. The experiments show target recognition withimage processing can effectively improve the flexibility of our robot.
Keywords: mobile robot; machine vision; 3 DOF manipulator; Omni-direction.

1.

D-H method, and on the basis of this model, derived
solving method to the manipulator kinematics equationof the manipulator and the inverse kinematics problem,the analytical solution of problem of manipulator
inverse kinematics.
Then the analytical solutions of the forward kinematics
equations and inverse solutions are verified andanalyzed.
The experimental results prove the effectiveness of themethod and the method of the target location.

Introduction

Today, the degree of automation of factories andwarehouses becomes increasingly higher and higher. 1
However it is difficult to expand manipulator’sfunctions because its fixed base. This article describes
an Omni-directional 3 DOF mobile manipulator, which
can increase flexibility of the manipulator and greatly
increase its function.
Firstly, through the analysis of the 3 DOF manipulator,established the manipulator kinematics model by theCopyright © 2017, the Authors. Published by Atlantis Press.
This is an open access article under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).
217

Journal of Robotics, Networking and Artificial Life, Vol. 4, No. 3 (December 2017) 217–220___________________________________________________________________________________________________________

2.

Figure 4 shows the Mecanum wheel four-wheel robot
chassis model coordinate system. The coordinate
system OiXiYi(i=1,2,3,4) is established with the centerof each wheel position as the origin, and the reference
coordinate system OXY is established with the center ofthe robot as the origin.

System Overview

In order to realize the automatic control of the robot,
the robot control system plays an indispensable role. In
this paper, the robot uses four Mecanum wheels torealize the omnidirectional movement of the robot
platform. Robot control system is divided into fourparts, they are IPC, the main, chassis drives andmechanical arm drive plate, as shown in Figure 1.

Fig.4. Coordinate modeling of omnidirectional mobile robot

As shown in Figure4, the robot body length is 2L, the�⃗=body width is 2l, the speed of the robot chassis is 𝑉(𝑉𝑥 , 𝑉𝑦 ), the angular velocity of the robot around the�⃗𝑖𝑤,Z-axis is 𝜔𝑧, the linear velocity of the wheel is 𝑉�⃗
there𝑉 𝑖𝑤 = 𝜔𝑖𝑅. (Where 𝜔𝑖 is the angular velocity ofeach wheel and R is the radius of the wheel). The roller�⃗ 𝑖𝑟.
and the ground contact when the line speed 𝑉�
⃗Suppose that the speed of each wheel is 𝑉 𝑖= (𝑉𝑖𝑥, 𝑉𝑖𝑦).
Under OiXiYi(i=1,2,3,4), can be decomposed along 𝑋𝑖
�⃗𝑖𝑤 and 𝑉�⃗𝑖𝑟, and 𝑉𝑖𝑥 can be expressed asand 𝑌𝑖, as 𝑉�⃗𝑖𝑤 and 𝑉�⃗𝑖𝑟 cos𝛼𝑖. Which−𝑉𝑖𝑟 sin𝛼𝑖, 𝑉𝑖𝑦 is the sum of 𝑉
satisfy the relation:

Fig.1. System Overview

IPC runs the Windows operating system, through theserial port and the main control board to achievecommunication between the main control board and thetwo driver boards through the CAN buscommunication.
2.1

Design of Omnidirectional Mobile Chassis
Based on Mecanum Wheel

In this paper, four Mecanum wheels are installed torealize the omni-directional movement of the platform,as shown in Figure 2. 2

𝑉1𝑥 = - 𝑉1𝑟 sin𝛼1，𝑉1y = - 𝑉1𝑤+V1𝑟 cos𝛼1𝑉2𝑥 = - 𝑉2𝑟 sin𝛼2，𝑉2y = - 𝑉2𝑤+V2𝑟 cos𝛼2𝑉3𝑥 = - 𝑉3𝑟 sin𝛼3，𝑉3y = - 𝑉3𝑤+V3𝑟 cos𝛼3𝑉4𝑥 = - 𝑉4𝑟 sin𝛼4，𝑉4y = - 𝑉4𝑤+V4𝑟 cos𝛼4(1)𝑉1𝑥 = 𝑉x –L𝜔z，𝑉1y = 𝑉y +l𝜔z
𝑉2𝑥 = 𝑉x –L𝜔z，𝑉2y = 𝑉y –l𝜔z
𝑉3𝑥 = – 𝑉x +L𝜔z，𝑉3y = 𝑉y –l𝜔z
𝑉4𝑥 = 𝑉x +L𝜔z ，𝑉4y = 𝑉y +l𝜔z

(2)

In the formula 𝛼𝑖-i round roller axis and the hub axisangle (𝑖 = 1, 2, 3, 4).
�⃗𝑖 is the speed of theOverall, the speed of each wheel 𝑉
robot and the relative speed of the wheel center O speed�⃗ = 𝑣⃗ 𝑖 + ωvector, that is, 𝑉��⃗ z∗ �����⃗OO l, consider vector
directionality, along the coordinates and 𝑋𝑖𝑌𝑖
Coordinate decomposition formula can be obtained:

Fig.2. Mecanum wheel

From Figure 3 you can see the four Mecanum wheels
combined installation situation. Each wheel is driven bya DC brush motor alone, with both left-handed andright-handed rotations.

Taking into account the pipe speed for thenon-controlled volume, to eliminate, that is derived
omni-directional robot inverse kinematics equation is:W1 =Fig.3.Analysis of chassis movement

218

1𝑅

[– 𝑉x cot𝛼 + Vy + (l + L cot𝛼) 𝜔z ]

Journal of Robotics, Networking and Artificial Life, Vol. 4, No. 3 (December 2017) 217–220___________________________________________________________________________________________________________W2 =
W3 =
W4=

1R
1𝑅

1

𝑅

2.2.2 Robot manipulator arm speed control[𝑉x cot𝛼 + Vy – (l + L cot𝛼) 𝜔z ][– 𝑉x cot𝛼 + Vy – (l + L cot𝛼) 𝜔z ]It can be seen that the relationship between the spatialposition of the robot hand and the joint variable is(3)[– 𝑉x cot𝛼 + Vy + (l + L cot𝛼) 𝜔z ]�Design of Multi - DOF Gripping

𝑑𝑑𝑦 =

𝜕𝜕

𝑑𝜃1 +𝜕𝜃1
𝜕𝜕𝑑𝑑𝑧 =

𝜕𝜃1𝑝𝑦2 +𝑝𝑧2 −𝑙12 −𝑙22

1 2
�𝑝𝑦 (𝑙1 +𝑙2 𝑐𝑐𝑐𝑐2 )−𝑝𝑧 𝑙2 𝑠𝑠𝑠𝑠2
𝜃1 = 𝑎𝑎𝑎𝑎 (𝑙 +𝑙 𝑐𝑐𝑐𝑐 )2+𝑙2𝑠𝑠𝑠2 𝜃
22(4)

2

For the robot hand space position, the line isinterpolated as it moves from (𝑦1, 𝑧1) to (𝑦2, 𝑧2) in astraight line.
Because of simple interpolation from the x-axis ory-axis, there is a great deal of non-uniformity.
Assuming the slope of the line segment is k, when𝑦1≠𝑦2, 𝑘 = (𝑧2- 𝑧1)/( 𝑦2- 𝑦1), the straight line inclination
angle is:α = arctank(5)
Assuming that d is the distance from any point (𝑦, 𝑧) to(𝑦1, 𝑧1) on this line segment, the point corresponding tod (𝑦, 𝑧) is𝑦 = 𝑦1 + 𝑑𝑑𝑑𝑑𝑑
�𝑧 = 𝑧1 + 𝑑𝑑𝑑𝑑𝑑

𝑑𝜃2 = (𝑙2 𝑠12 −𝑙1 𝑐1 )𝑑𝑑1 +(𝑙2 𝑐12 )𝑑𝑑2

(8)𝑙2 𝑐1 𝑐2 𝑑𝑑1
���𝑙2 𝑐12 𝑑𝜃2(9)𝑙2 𝑐1 𝑐2�𝑙2 𝑐12(11)Jacques array is mainly used to describe the joint
displacement and space pose differential relationship,
the relationship between the spatial velocity of the robot
hand and the angular velocity of the joint is given
v=J𝜃̇
(12)
Conversely, the speed of movement of each joint can be
determined from the desired hand speed 𝜃̇ =J-1 v.
According to calculations:

Calculation shows that the position of the robot joint
position and the relationship between the hand space:2𝜕𝜃2−𝑙 𝑠 −𝑙 𝑠𝐽 = � 2 12 1 1𝑙2 𝑐12 +𝑙1𝑐1

Linear Interpolation

1𝑑𝜃1 +Is known from the definition of the Jacobian matrix:
𝑝̇ = 𝐽𝑞̇(10)The Jacobian matrix of the manipulator can be obtained

Motion Control of Manipulator 's Hand

2𝑙 𝑙𝑑𝜃2 = (−𝑙2 𝑠12 −𝑙1 𝑠1 )𝑑𝑑1 +(−𝑙2 𝑠12 )𝑑𝑑2

𝑑𝑑𝑦−𝑙 𝑠 −𝑙 𝑠�� = � 2 12 1 1𝑙2 𝑐12 +𝑙1 𝑐1𝑑𝑃𝑧

The design of the robot arm a total of three degrees offreedom, the direction of the X-axis rail, twoconnecting rods were Y, Z axis.

𝜃2 = 𝑎𝑎𝑎𝑎

𝜕𝜕𝜕𝜃2𝜕𝜕Written in matrix form

Manipulator

2.2.1(7)

Derivative of the above formula can be obtained

From the above equation, the control of the whole robot
can be realized.
2.2

𝑝𝑦 = 𝑙2 𝑐1 𝑐2 −𝑙2 𝑠1 𝑠2 +𝑙1 𝑐1
�𝑝𝑧 = 𝑙2 𝑐2 𝑠1 −𝑙2 𝑐1 𝑠2 +𝑙1 𝑠1

-1

−𝑐12

𝑙1 𝑐12 𝑠1
�𝑙2𝑐12+𝑙1𝑐1

J =𝑙1 𝑙2 𝑐12 𝑠1−𝑙1 𝑠12 𝑐1−𝑙1 𝑙2 𝑠12 𝑐1−𝑠12

𝑙1 𝑐12 𝑠12
𝑙2 𝑠12 +𝑙1 𝑠1
𝑙1 𝑙2 𝑐12 𝑠1− 𝑙1 𝑠12 𝑐1−𝑙1 𝑙2 𝑠12 𝑐1(13)�2.3 Design of Image Processing.
Template matching is to find the target image in theimage. 3 The principle is to measure the similarity
between two images by some similarity criteria.
2.3.1 Template Matching Based on Gray Value
When the template slides in the image, do similarity
matching. The simplest measure of similarity is tocalculate the sum of the absolute values of thedifferences between the template and the image or thesum of squares of all the differences (SAD and SSD).

(6)

Wheredisintherangeof[0, �(𝑥2 − 𝑥1 )2 + (𝑦2 − 𝑦1 )2 ]. When d is evenly
changed in this interval, the movement of the hand (𝑦, 𝑧)obtained by the above-described equation is substituted
into the joint position expression so as to be able torealize a straight line movement of the hand𝑠𝑠𝑠(𝑟, 𝑐) =1
�|𝑡(𝑢, 𝑣) − 𝑓(𝑟 + 𝑢, 𝑐 + 𝑣)|
𝑛(𝑢,𝑣)∈𝑇

1

𝑠𝑠𝑠(𝑟, 𝑐) = ∑(𝑢,𝑣)∈𝑇 𝑥|𝑡(𝑢, 𝑣) − 𝑓(𝑟 + 𝑢, 𝑐 + 𝑣)|2219

𝑛(14)Journal of Robotics, Networking and Artificial Life, Vol. 4, No. 3 (December 2017) 217–220___________________________________________________________________________________________________________(𝑟+𝑢,𝑐+𝑣) denotes the gray value of the row r + u
column c + v of the image to be measured, and 𝑡(𝑟,𝑐)denotes the gray value of the template image T row r
column c. 4 The difference between the values after
the change and their mean values will be smaller
regardless of whether all the gray values become larger
or smaller and then become zero mean and unit
variance by dividing the variance so that the image isless sensitive to changes in the brightness level , andhas strong robustness.
1𝑛𝑛𝑛(𝑟, 𝑐) = ∑(𝑢,𝑣)∈𝑇
𝑛𝑡(𝑢,𝑣)−𝑚𝑐
�𝑠𝑡2∙𝑓(𝑟+𝑢,𝑐+𝑣)−𝑚𝑓 (𝑟,𝑐)�𝑠𝑓2 (𝑟,𝑐)As shown in figure 6. First, the original image
distortion correction and smoothing noise; then extract
the image of the saturation S component from theimage HSV space, we can clearly see that saturation
component of the object we want to crawl is relatively
high; The third step is to get the image region with highsaturation through the dynamic threshold technique.
The final step is to display the obtained area and get theorientation information of the area and the orientation
information of the bottle.

(15)

3.

𝑚𝑐 is the average gray value of the template, 𝑠𝑡2 is theConclusion

This paper introduce an Omni-directional 3 DOF
mobile manipulator based on machine vision. However,in real-world experiments, the robot was controlled tograsp the bottle. Our experiments have shown that thedesired relative position and orientation between thecamera and the object have been kept within errors offew centimeters. The next step is to further improve theimage processing method to make the image processing
faster, so that the robot more accurate understanding of
the surrounding environment. Adding some sensors,
such as lidar, gives the robot a path planning andobstacle avoidance function. Increase the chassis
suspension system to adapt to a richer terrain.

variance of all the gray values of the image to bemeasured, (𝑟,) is the average gray value of the image tobe measured.

Fig.5.A schematic diagram of the image pyramid

As shown in figure 5, in the pyramid-based searchstrategy: from the top to the bottom of the search, in thehigh-level image search to track the template instanceto the bottom of the image pyramid, then in a small areato match, in the next search region to locate the resultsof the matching of a small area the similarity degree iscalculated, segmented by the threshold and the local
extrema is extracted.

References
1. Wang J, Zhang X and Zhang W, Study on theImprovement of Flexibility for an Industrial Robot basedon Machine Vision, J. Journal of Robotics, Networking &Artificial Life, 2014, 1(1):45-48.
2. Du B, Zhao J and Song C, Dexterity Analysis for
Omni-directional Wheeled Mobile Manipulator Based on
Double Quaternion, J. Chinese Journal of MechanicalEngineering,2013,03:585-593.
3. Wang J, Zhang X and Dou H, Study on the Target
Recognition and Location Technology of Industrial
Sorting Robot based on Machine Vision, J. Journal ofRobotics, Networking & Artificial Life, 2015,2(2):108-110.
4. Wang J, Dou H and Zheng S, Study on the Target
Recognition based on Machine Vision for Industrial
Sorting Robot, J. Journal of Robotics, Networking &Artificial Life, 2015, 2(2):100-102.

2.3.2 Realization of crawl
In order to achieve the robot automatic items crawl, wemust get the actual location of items in the world
coordinate system.

Fig.6. Steps to find items

220