# Visual SLAM Based Spatial Recognition and Visualization Method for Mobile AR Systems
## 3 Visual SLAM Based Mobile AR System
### SLAM Design on Android  
The overall work procedure in the mobile SLAM system is shown in Figure 5.  
Continuous image data is collected using the camera of the mobile device. The camera pose is approximately estimated by extracting the ORB feature for each image data and by calculating the position of the feature point compared to the nearest key frame. To create a map as similar as possible to the real environment by determining the correlation with the real environment using the camera pose information, BA (Bundle Adjustment) co-visibility graph, which indicates the relationship between key frames, was used, and a regional map, which is the map data at the time the camera is looking at, is drawn. Loop Closure Detection is performed on the local map to remove the accumulated drift; there are too many map points in the global map obtained by adding all the local maps, so localization is performed only with the camera pose, excluding the map points through optimization. BA (Bundle Adjustment) uses a non-linear optimization library called g2o to calculate the pose of the current frame as Motion-only BA, the pose of the recent key frame as Local BA, and the overall optimization as Loop Closing's pose graphing optimization and Full BA. All four optimizations are performed independently on different threads. Motion-only BA optimizes only the current pose with the key points of the map fixed, and Local BA optimizes the pose of the public visible key frame and the key points visible in the corresponding pose when a new key frame occurs. Loop Closing performs pose graph  
Figure 5: System Workflow.  
optimization in a way of optimizing only with the pose perspective, excluding key points from the co-visibility graph, and then, the key frames and key points added through Full BA are updated [12; 13].  
To make SLAM available in the Unity Engine where only dynamic libraries are available, the SLAM system is made into a library through Clang build of LLVM and processed as a Unity plug-in. The complier-based LLVM helps to easily implement optimizations regardless of the programming language at compile time, link time, and runtime [15]. Clang is a compiler frontend for C, C++, Objective-C, and Objective-C++ programming languages that use LLVM as a backend [15]. It increases the convenience of development by provides faster compilation speed and faster execution speed than GCC compiler, and by providing more accurate error location and information. In order to use the function that delivers the image along with the initialization function and setting values of the SLAM system in the Unity Engine through the SLAM plugin created by Clang build, in this paper, the Unity Engine's [D11 Import (\"PluginName.d11\")] function is used, and the required SLAM function is controlled by calling it. In this way, it is possible to implement SLAM-based AR technology in the Unity Engine through the built-in SLAM plug-in.  
Continuous image data collected from Android mobile devices is delivered to the C++ SLAM library to provide pose data and point cloud data to the Unity Engine. Camera pose data can represent camera movement in a 4 \(\times\) 4 matrix, and point cloud data is 1 \(\times\) 3 vector data representing the position in space. This makes sure that implementing visualization of the current camera pose and surrounding environment are stored inside the SLAM.