Journal of Robotics, Networking and Artificial Life
Vol. 6(3); December (2019), pp. 195–198
DOI: https://doi.org/10.2991/jrnal.k.191203.004; ISSN 2405-9021; eISSN 2352-6386
https://www.atlantis-press.com/journals/jrnal

Research Article

Research on the Realization Method of Augmented
Reality based on Unity3D
Jiwu Wang*, Weixin Zeng
School of Mechanical, Electronic and Control Engineering, Beijing Jiaotong University, Beijing, China

ARTICLE INFOABSTRACTArticle History

Aiming at the problems of low stability and limited application conditions in traditional augmented reality, Unity 3D software
is applied to realize augmented reality by combining with Vuforia SDK and EasyAR SDK (Software Development Kit). The
developed results can be used on mobile terminal and augmented reality glasses-Hololens. At the same time, the 3D registration
technique is studied for the augmented reality. Moreover, the principle and process to realize augmented reality in Unity3D andSDK are given. Finally, with the common image markers, the stability of the two SDKs was tested and analyzed with the angularvariation experiment. Compared with the traditional system, the model stability is improved.

Received 20 October 2018Accepted 20 November 2018
Keywords
Augmented reality
Vuforia SDK
EasyAR SDK
Unity3D© 2019 The Authors. Published by Atlantis Press SARL.
This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).

1. INTRODUCTION

2. IMPLEMENTATION SCHEME

With the help of computer graphics, computer vision and othertechnologies, virtual objects are superimposed onto real scenes by
Augmented Reality (AR). It enables users to perceive a new environment where the real scene and the virtual scene are seamlessly
integrated together, and it presents users with a real-time interactive sensory effect [1].

In this paper, with OpenCV, some markers are be developed forthe automatic recognition and positioning in a captured image,and then superimposes the existed virtual objects under the camera
image by OpenGL to realize augmented reality. The implementation scheme is shown in Figure 1.

Many researchers have made some achievements in the fieldof augmented reality. Feiner et al. [2] published a paper on theprototype of the AR system at the International Conference on
Graphic Images, which was subsequently widely cited by theAmerican Computer Society. In 1999, the open source augmented reality development kit-ARToolKit, was released by
the University of Washington [3]. With the release of the development kit, a large number of augmented reality applications
gradually emerge. In 2015, Microsoft released the AR head-up
Microsoft HoloLens.
However, traditional augmented reality has the problems of low
stability and limited application range. Here we use Unity3D(San Francisco, USA) ­software combined with Vuforia SDK andEasyAR SDK to realize augmented reality. And it can also beapplied on mobile terminal and AR glasses-Hololens, in which thetraditional black and white identification images in ARTOOLKITare extended to any ordinary images for recognition. In this paper,
considering the applications of the common marker images, the3D registration technology is studied, and two SDKs in Unity3D
platform are developed to realize augmented reality. At last, theireffect is analyzed and compared.
Corresponding author. Email: jwwang@bjtu.edu.cn; 17121253@bjtu.edu.cn*

In this scheme, the feature points of the image marker are extracted
in the captured image of real-time scene. The relevant feature point
information is obtained after detection and description, and the points
are matched with the Hamming distance, filtered the matching points
with the RANSAC (Random Sample Consensus) algorithm to get
the final matching points. Then, by the conversion of the coordinatesystem, the position of the virtual object in the real scene is determined.
At last, with the OpenGL the augmented reality can be realized [4].

3. KEY TECHNOLOGIESIn order to realize augmented reality, three core technologies arenecessary: three-dimensional registration technique, display technology and real-time interaction technology [5].
• Three-dimensional registration: In order to realize the melting
between the virtual and real world, the position of the tracking
camera is calculated, at the same time, the position of the virtual object in the real scene is calculated too.
• Display technology: In the real world, virtual images or models
present a virtual but fused in the real world to the user.
• Real-time interaction: It means an interaction between theuser and the virtual object. The information flow and feedback
can be timely and effective performed between them.

J. Wang and W. Zeng / Journal of Robotics, Networking and Artificial Life 6(3) 195–198

196

Figure 2 | Illustration of the coordinate systems.
Figure 1 | Implementation scheme.

In 3D registration, the marker in the world coordinate systemneeds to be converted into pixel coordinate system in the camera.
Figure 2 shows the transformation between coordinate systems.
Point P is a point in the world coordinate system, and Ow − Xw,Yw, Zw is the world coordinate system, describing the position ofthe camera. Oc − Xc, Yc, Zc is the camera coordinate system and thecamera optical center is used as its origin. The xoy is the image
coordinate system, and (x, y) corresponds to the physical coordinates; the uo0v is the pixel coordinate system, (u, v) corresponds tothe pixel coordinate.
Assuming that the coordinates of the point O0 in the pixel coordinate system are (u0, v0), the corresponding relationship between thepixel coordinate system and the image coordinate system is shownin Equation (1).
1	     u   dx
v  =  0   1  
0Figure 3 | Camera and image coordinate system.

u0   x  (1)v0   y   1
1  			 y = f Yc (4)Zc

The transformation matrix from world coordinate system (Xw, Yw,Zw) to camera coordinate system (Xc, Yc, Zc) is shown in Equation (2).
The transformation matrix is composed of rotation matrix R andtranslation matrix T, R is 3 × 3 orthogonal identity matrix, T = (t1,t2, t3)T is a three-dimensional column vector.

(5)01dy0é Xw ùé Xc ùêY úêY ú éRTùw3 *1	     ê c ú = ê 3*3ú ê Z ú (2)01Z
ûê w ú
ê cú ë
êë 1 úû
êë 1 úûFigure 3 shows the transformation between camera coordinate
system and image coordinate system. The coordinate value of point
P in the image coordinate system is (x, y), the coordinate value inthe camera coordinate system is (Xc, Yc, Zc), and the focal lengthof the camera is f, then according to the proportion relationship[Equations (3) and (4)]:		     x = f

Xc
(3)
Zc

We can get the following Equation (5):éx ù é f		 Zc ê y ú = ê 0ê ú ê
êë 1 úû êë 00f0

éX ù0 0ù ê c ú 
Y0 0ú ê c úú ê Zc ú1 0 úû
êë 1 úû

By substituting Equations (1) and (2) into Equation (5), Equation (6)for the transformation relation between world coordinate systemand pixel coordinate system can be obtained: fdu   x
Zc v  =  0   1  
0
 f
d
 x
Set A =  0

00f
dy0

u0
v0
10f
dy0

u0v01

0
 Xw  Y  (6)RT

3 *10  3*3
 w1   Zw 
 0
 1 0

0
T3*1 ù
éR
0 , M = ê 3*3. Therefore, it can be
01 úûë
0

simplified as Equation (7):

J. Wang and W. Zeng / Journal of Robotics, Networking and Artificial Life 6(3) 195–198

197

é Xw ùéu ùêY ú		 Z c ê v ú = AM ê w ú (7)ê úê Zw úëê 1 ûú
êë 1 úû
where A is the camera internal parameter matrix and M is thecamera external parameter matrix.

4. THE DEVELOPING PROCESSUnity3D is a very compatible platform. It provides a variety ofplugins that are necessary for further higher level development andvarious development environments. Vuforia SDK is a ­development
kit developed by Qualcomm. And EasyAR SDK is an AR engine
developed by China Company. They can both work in the Unity3Dplatform. So they were used to develop our applications in theUnity 3D platform with the augmented reality process.

(a)

(b)

(c)

(d)

(e)

(f)

Figure 5 | A comparison test with augmented reality.

Figure 4 shows the AR implementation steps.
In a word, before realizing augmented reality with SDK and Unity3D,a database should be built on the official website of that SDK tomanage the identifying images. Then a series of identification images
are selected and uploaded to the cloud database for image preprocessing, training and matching, etc. After that, the prepared model
imported into Unity3D to develop the AR applications.

Figure 6 | Image angle variation experiment with Vuforia.

5. EXPERIMENT AND RESULTS
The experiment is done in Win 10 system with the model ofthe graphics card of GTX 960. And the Unity3D with versionof 2018.2.10-f1 is applied. The development kit is Vuforiaunity-6-2-10 and Easyar-2.0.0-basic. The experiment environment
was in good lighting conditions.
Figure 5 shows the comparison results with augmented reality.
Here the 3D model is superimposed on the recognized image andcan move according to the operator’s commands, in which, (a) isthe 3DS MAX model; (b) and (c) are the AR effect with Vuforia
SDK; (d–f) are the AR effect with EasyAR SDK.
Figures 6 and 7 are the AR angle variation experiments.
In the experiments, using the same image marker and the experimental equipment, etc., the angle between the image marker andFigure 7 | Image angle variation experiment with EasyAR.
Table 1 | Identification success rate
Marker image angle (°)
01530456090Vuforia (%)

EasyAR (%)

070.5
81.4
95.6
98.0
98.2

00
30.5
90.4
94.8
96.7

camera are adjusted by more than 50 times, then their identification success rate is recorded as shown in Table 1.

Figure 4 | AR implementation process.

Based on Table 1, we could find when the image marker is at angleof more than 45, the identification success rate is high for bothmethods. It can be found that the identification success rate withVuforia method is higher when the angle is less than 45.

J. Wang and W. Zeng / Journal of Robotics, Networking and Artificial Life 6(3) 195–198

198

6. CONCLUSION
In this paper, the key technologies of augmented reality are studied,and some augmented reality applications are developed and verified. At the same time, the identification success rate and stability
are compared respectively with Vuforia and EasyAR method. Here
only some main functions are tested, then more functions will be
developed in the future work.

[3]

head-mounted display, in: Proceedings of the Conference onGraphics Interface. British Columbia, Canada, Vancouver,
1992, pp. 78–85.
H. Kato, M. Billinghurst, Marker tracking and HMD calibration for a video-based augmented reality conferencing system,

REFERENCES

Proceedings 2nd IEEE and ACM International Workshop on
Augmented Reality (IWAR’99), IEEE, San Francisco, CA,USA, 1999, pp. 85–94.
[4] J. Park, S. You, U. Neumann, Natural feature tracking forextendible robust augmented realities, Proceedings ofthe international workshop on Augmented reality, 1998,

[1]

[5]

R.T. Azuma, A survey of augmented reality, Presence Teleop.
Virtual Environ. 6 (1997), 355–385.
[2] S. Feiner, B. MaCintyre, D. Seligmann, Annotating the realworld with knowledge-based graphics on a see-throughpp. 1–2.
Q. Li, Y. Li, J. Tian, D. Ren, Augmented reality registration
method based on natural feature points, 2017 IEEE 17th

International Conference on Communication Technology(ICCT), IEEE, Chengdu, China, 2017, pp. 1937–1941.

Authors Introduction
Mr. Weixin Zeng

Dr. Jiwu Wang
He is an Associate Professor at Schoolof Mechanical, Electronic and ControlEngineering, Beijing Jiaotong University,China. He obtained his PhD degree atTsinghua University, China in 1999. Hismain interests are computer vision, robotic
control and fault diagnosis.

He is a master student at School ofMechanical, Electronic and Control
Engineering, Beijing Jiaotong University,Beijing China. His main interests are VRand AR.