Journal of Robotics, Networking and Artificial Life
Vol. 6(3); December (2019), pp. 179–182
DOI: https://doi.org/10.2991/jrnal.k.191202.007; ISSN 2405-9021; eISSN 2352-6386
https://www.atlantis-press.com/journals/jrnal

Research Article

Modeling Virtual Reality Environment Based on SFM Method#Jiwu Wang*, Chenyang Li, Shijie Yi, Zixin Li
School of Mechanical, Electronic and Control Engineering, Beijing Jiaotong University, Beijing, China

ARTICLE INFOABSTRACT

Article History

Virtual Reality (VR) technology is widely used in digital cities, industrial simulation, training, etc. where the environment
modeling is a necessary component. Comparing with the difficulty to build the three dimension (3D) environment with theconventional methods, the Structure from Motion (SFM) method is proposed in this paper. The modeling accuracy is studied
by comparing with the real dimensions. The results show the SFM method can give a high precision reconstructed 3D model ina short time.

Received 25 October 2018
Accepted 09 December 2018
Keywords
Virtual reality
SFM methodmodelingunity© 2019 The Authors. Published by Atlantis Press SARL.
This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).

1. INTRODUCTION
Virtual Reality (VR) is an artificial media space built by computer.
It makes people enter a virtual environment by interaction devices
of multimedia sensors. Using external devices, such as helmets anddata gloves, users see an interactive and dynamic 3D views formed bymulti-source information. VR has been applied in many fields suchas simulated military training, digital museum, gaming, etc. [1].
Modeling the virtual world, that is, the construction of VR environment, is a necessary step. Because of the complexity of the realworld environments, it is tedious and difficult to model the working environment in any real world. So many people have to simplify the model process by construct the environments with only
necessary components for conventional methods. In this paper, theStructure from Motion (SFM) [2] method is studied to build the 3D
environments. The results show it can provide more accurate andvivid 3D model and reduce the modeling time.

(3) Using modeling software and Virtual Reality Modeling
Language (VRML) to build, such as 3Dmax and SolidWorks,etc. Yuan et al. [4] used SolidWorks and VRML to modelingthe VR environment. In this situation, the modeling software
could modeling the object accurately, but it is also not suitable
for modeling the real environment as not realistic enough.

2. APPLICATION OF SFM METHODIN VR MODELING
2.1. The SFM Method
In the SFM method, the camera parameters and three-dimensional
environment information are recovered with geometry methods.
Taking a bear model as an example, its main process is shown inFigure 1.

Till now, many modeling methods are developed to build the 3D
environments for VR, such as:

Structure from motion method is mainly divided into four steps
in Figure 1:(1) Using VB, C++, OpenGL graphics library and other tools tobuild. These methods generally are inconvenient and require
much time to study.

(1) Input and align photos: Input the photos of the captured
model or scene into the SFM program to complete the alignment of the images. In the process of alignment, the featurepoints whose shape or texture information is more prominent
in the photos are extracted first. And then the feature points
between the photos are matched. After matching is completed,the positional relationship between the cameras is calculatedby different feature points in the photo.

(2) Using VR software, such as WorldToolKit (WTK) andMultiGen Creator, to model complex 3D graphics. Dang [3]completed the creation of virtual campus environment using
MultiGen Creator. By this method, the actual campus scene isbasically restored. But the model was relatively simple and not
realistic enough so the user cannot feel sufficient immersion.
And it is very difficult and time-consuming too.
Corresponding author. Email: jwwang@bjtu.edu.cn; 17121244@bjtu.edu.cn
This work was supported by Project KMGY318002531.

*#(2) Generate sparse point cloud: The three-dimensional coordinates of each feature point can be solved by the basic principleof stereo vision. Then, according to the texture informationof each feature point, a point cloud with color information isgenerated.

180

J. Wang et al. / Journal of Robotics, Networking and Artificial Life 6(3) 179–182

Figure 2 shows a stone table model reconstructed in this ­experiment.
In order to facilitate the dimension measurement in later step, several markers with different number were pasted in the scene.
By comparing the distance in the model and the actual scene, themodeling accuracy of SFM method is analyzed. Table 1 gives theerror analysis data.
According to the data in Table 1, error analysis is carried out furtherly in horizontal and vertical directions of the model. Error
is mainly classified into Mean Error (ME), Mean Absolute Error(MEA), Standard Deviation of Error (SDE) and Root Mean SquareError (RMSE) in Table 2.

Figure 1 | The process of SFM. (1) Input and align photos. (2) Generatesparse point clouds. (3) Generate dense point clouds. (4) Complete the 3Dreconstruction.

(3) Generate dense point cloud: Using the PMVS tool to get thethree-dimensional coordinates of points which are aroundthe feature points and in high photo-consistency, thereby thedense point cloud is generated.
(4) Complete the 3D reconstruction: After generating dense point
cloud, the outliers in the generated dense point cloud need to beremoved firstly, the culling operation ensures that the final model
has no defects. Then the three adjacent points are used to form afacet, and those facets are connected to form the final 3D model.
One advantage of the SFM method is that the modeling steps areautomatic. Another is that it can modeling large-scale scenes, which
is very suitable for modeling of natural terrain and urban landscape.
But as SFM method restore the shape and texture information of scenefrom the feature points in the photos, the method may lead to a bad
result in the following cases. One is the environment is low texture,such as a blank wall. There are not enough feature points to extract, so
the scene cannot be restored. Another is the problem of shade, somestructure cannot be extracted feature points when they were shelteredby other object or other part of their own. Users need to take photos
from as many angles as possible in order to avoid that problem.

2.2. Accuracy Analysis of Model­Reconstructed by SFM Method
Due to the high requirements on the dimensional accuracy of themodel in VR, the modeling accuracy of SFM method is tested withan experiment in this section.

Figure 2 | A stone table model.
Table 1 | Error of every distance
Distance
betweentwo target

Distance DistancePlace thatin actual in thedistancescene
modelexist(mm)(mm)

Error(mm)Relativeerror(%)

Target 1_target 2
Target 5_target 6
Target 13_target 14
Target 15_target 16
Target 19_target 20
Target 1_target 3
Target 1_target 4
Target 1_target 7
Target 1_target 8
Target 2_target 3
Target 2_target 4
Target 2_target 7
Target 2_target 8
Target 3_target 4
Target 3_target 7
Target 3_target 8
Target 4_target 7
Target 4_target 8
Target 5_target 11
Target 5_target 12
Target 6_target 11
Target 6_target 12
Target 7_target 8
Target 9_target 10
Target 9_target 11
Target 9_target 12
Target 10_target 11
Target 10_target 12
Target 11_target 12

Desktop
GroundVerticalVerticalVerticalDesktop
Desktop
Desktop
Desktop
Desktop
Desktop
Desktop
Desktop
Desktop
Desktop
Desktop
Desktop
Desktop
GroundGroundGroundGroundDesktop
GroundGroundGroundGroundGroundGround−0.1260.378−0.255−0.226−0.3473.888
3.723−1.357−1.377−1.346−1.725−1.882−1.439−0.063−1.66−0.316−1.2913.209−1.358−1.885−1.497−2.124−0.15−0.513−0.566−0.611−1.136−0.323−0.162

−0.15
0.44
−0.30
−0.27
−0.41
0.96
0.93
−0.20
−0.20
−0.29
−0.36
−0.29
−0.22
−0.07
−0.27
−0.06
−0.18
0.49
−0.20
−0.26
−0.20
−0.27
−0.18
−0.60
−0.09
−0.11
−0.20
−0.07
−0.19

85
85
85.2
85
85.2
403
402
677
677
470.3
479.5
651
661
85
624
572
705
652
668.2
720
731
777
85
85.1
618.5
554
563
492
85.1

84.87485.37884.94584.77484.853406.888405.723675.643675.623468.954477.775
649.118
659.561
84.937622.34571.684703.709655.209666.842718.115729.503774.876
84.85
84.587617.934
553.389561.864491.677
84.938

J. Wang et al. / Journal of Robotics, Networking and Artificial Life 6(3) 179–182

Table 2 | Model error analysis

ME (mm)MEA (mm)
SDE (mm)
RMSE (mm)Horizontal

Vertical

Totality−0.498
1.400
1.665
1.738

−0.2865
0.287
0.081
0.293

−0.432
1.204
1.525
1.585

1812.3. Application of SFM Methodin VR ­ModelingAfter the modeling is completed, the material information can beexported through a 3D format file such as OBJ, and the texture fileof the model is also exported as a picture. The exported model andtexture information are respectively shown in Figures 5 and 6.
The files obtained in the above steps can be imported into theUnity3D software successively to complete the subsequent edit ofthe VR. The imported model in Unity3D is shown in Figure 7.

Figure 5 | The material information of model.
Figure 3 | Error in all directions.

Figure 6 | The texture information of model.
Figure 4 | Average relative error in all directions.

Figure 3 shows the trend of the error in each direction, and Figure 4
shows the trend of the average relative error in each direction.
It can be seen from Figure 2, Tables 1 and 2 that the model reconstructed is basically the same as the actual scene. From Figures 4and 5, it can be concluded that the error value of the horizontal
and vertical directions is low. But the error value of the horizontal
direction is larger as the distance between makers in the horizontal direction is larger. After comprehensive analysis, the final average error of the model is <1 mm, the maximum error is <4 mm, andthe average relative error is <0.1%. Moreover, the modeling time is
all within 2 h.

Figure 7 | The model in Unity3D.

J. Wang et al. / Journal of Robotics, Networking and Artificial Life 6(3) 179–182

182

3. CONCLUSION
Using SFM method to modeling the VR environment based onreal environment is discussed in this paper. Compared with conventional modeling methods, the advantages of SFM method areshown as follows:(1) Reduce the difficulty of modeling by less operations. Using theSFM method, the modeling only needs to take a multi-angle
photos of the real environment in the early stage, and theninput photos into the program, in which the modeling can beoperated step-by-step. Compared with the conventional methods, the operation is very simple and requires little on modeling skill in SFM method.
(2) Reduce the modeling cycle with less modeling steps. The
SFM method needs less operation steps than the conventional method, and the time period is also shorter, even within
2 h, which can be even faster on a computer with a high-­
performance GPU.
(3) It is more suitable for the VR modeling. Compared with conventional method modeling, the texture information modeledby the SFM method extracts from a photo rather than a manual

selection of tones, so a higher degree of realism can be obtained
when the texture is finally mapped onto the model. And as discussed in Section 2.2, the error of model reconstructed by SFM
method is low enough to meet requirements of VR.
On the other hand, there are also two cases to be attention: one isthat the SFM method should not be used to modeling the scenes inlow texture; another is that users should take photos of the scenes
in as many angles as possible.

REFERENCES
[1]
[2]
[3]
[4]

X. Shuo, M. Kun, L. Shuqin, D. Meng, W. Lijun, Research on key
technologies of online experiential application based on VR/AR,Intell. Comput. Appl. 7 (2017), 28–31.
L. Shapiro, G. Stockman, Computer vision, Prentice Hall, Upper
Saddle River, New Jersey, 2001.
B. Dang, Research on virtual scene generation in virtual reality,
Beijing Jiaotong University, Beijing, 2010.
F.W. Yuan, B.W. Li, B. He, Accurate modeling of VR based on
SolidWorks-VRML, Mech. Electr. Eng. Magazine 27 (2007),
103–105.

Authors Introduction
Dr. Jiwu Wang

Mr. Shijie Yi
He is an Associate Professor at Schoolof Mechanical, Electronic and Control
Engineering, Beijing Jiaotong University,China.

He is a master student at School of Mechanical,Electronic and Control Engineering, Beijing
Jiaotong University, Beijing China. His maininterest is computer vision for robotic.

He obtained his PhD degree at Tsinghua
University, China in 1999. His main interests are computer vision, robotic control
and fault diagnosis.
Mr. Chenyang Li

Ms. Zixin Li
He is a master student at School ofMechanical, Electronic and Control
Engineering, Beijing Jiaotong University,Beijing China. His main interests are 3D
reconstruction and object detection.

She obtained her master degree at Schoolof Mechanical, Electronic and Control
Engineering, Beijing Jiaotong University,
China in 2019. Currently, she works at AECC
Beijing Hangke Control System Science andTechnology Co., Ltd. Her main interest is 3D
reconstruction.