# Visual-Based Localization Using Pictorial Planar Objects in Indoor Environment
## 1 Introduction
### Pictorial Planner Object Based Localization  
In this paper, a pictorial planar surface based VBL framework is presented for producing the 3D pose of indoor devices with a camera. An example robot with our VBL support is shown in Figure 2; the robot can localize itself whenever some pre-trained pictorial planar surface is visible by its camera. We have designed and implemented two object detection methods for centimeter-precision-level localization, ArPico and PicPose (Figure 1b,c). Our methods are motivated by an open-source project, ArUco [10], which uses fiducial markers (Figure 1a) for AR processing. ArUco produces 2D barcodes with a black frame and an inner grid of binary blocks. The black frame is used to detect the marker, and the inner blocks are used to recognize its ID. The pictorial planner object based VBL framework presented in this paper has three components: (a) Offline object learning and library creation; (b) libraries and map management; and (c) real-time device localization. This framework is applicable to both ArPico and PicPose methods. The ArPico method detects and recognizes framed pictures (Figure 1b) before converting pictures into vectors of binary blocks (similar to ArUco markers) for matching with known marker codes in a library [11]. It then uses the picture's corner points to identify the camera's pose relative to the picture's actual position stored in the 3D map. In contrast, the PicPose method detects the pictorial planar surface (Figure 1c) of an object without requiring it to have any specific shape or frame. PicPose is designed to extract feature points of a pictorial planar object from a camera view, and produce the camera's pose  
Figure 1: Planar object examples. Figure 2: Robot navigating using PicPose localization. by matching those feature points on the planar surface of a 3D object with its real-world 3D models stored in the 3D map. Compared with some earlier localization works, our VBL methods have the following benefits:  
* Our methods only use a simple monocular camera, no need for high-resolution or multiple cameras. It is a low cost solution for localization. * Our methods can learn to recognize any user-selected pictorial object, not restricted to system-generated fiducial markers or patterns. It is a flexible solution for any user space. * The PicPose method can produce a successful localization with a partially visible picture, and with any known shape. It provides a robust solution in the practical setting. * Our methods are not subject to radio interference or blocking concern, or other sensing signal disturbances. It provides an ideal indoor and underground solution for localization. The contribution of this paper is that we present an attractive localization support that uses existing natural pictorial objects in a target environment as pictorial planar markers. Our approach is novel since we do not place system-generated fiducial markers in an environment [10] but instead pre-learn distinctive objects already present in the environment and use them for producing robot localization. It is similar to the human localization perception strategy of referencing to known visual objects from our past visual knowledge. Our approach is marker-based but uses pre-runtime object learning to collect good strong markers from the operational environment for runtime recognition and to produce localization. To improve the localization speed and precision, we have designed two new techniques to improve the efficiency and precision of pictorial planar object recognition. We have developed an algorithm to reduce the number of extracted feature points from a robot view before matching them to a pictorial object in the library. By inspecting the relative positions of feature points and selecting just a sufficient number of useful points, we can reduce the number as well as improve the quality of feature points for picture matching. Our study shows that this technique can reduce the average matching time by 50%.