# Cross-View Outdoor Localization in Augmented Reality by Fusing Map and Satellite Data
## 1 Introduction  
Finding an image's exact location and orientation is a challenging task, but it is essential to obtain these data accurately for the purpose of augmented reality applications where an image's precise location is important, such as in AR navigation. Using GPS data, an image's location can be localized to a small area; however, the precision of this approach is limited. This problem is an active topic of research, and previous attempts have included using ground-level images for 3D model mappings [1; 2; 3; 4] and ground-level images for 2D street view images mapping [5; 6]. These techniques, however, all share a common problem: the data to map against are hard to acquire. Creating a 3D model of a city is tough, and although many techniques to build 3D models of cities exist [7; 8], making a robust model that does not rely on these models would allow for scaling beyond preprocessed areas. Otherwise, street view data can be outdated in just a few years or when the seasons change [9].  
To circumvent these issues, we propose a technique based on top-down imagery, such as satellite and OpenStreetMap (OSM) data, instead. This technique has been an active topicof research for the last few years [10; 11; 12; 13; 14; 15; 16; 17; 18; 19]. The basis of these techniques involves matching the ground-level image to specific positions in the top-level image to estimate where the image was taken. In this paper, we build upon previous work to build a high-performing model for this task.  
The task is a particularly difficult challenge because of the large information gap between the top-level and ground-level images. Previous work used metric learning to learn deep representations of both modalities to make matches without taking the geometry into account [19]. To go beyond metric learning, we base our model on the work by [PERSON] et al., who used geometric projection to bridge this gap [13]. In particular, we use a U-Net architecture for both top-level and ground-level input, use a geometric projection module to translate the top-level input to the ground-level modality, and use the Levenberg-Marquadt optimization algorithm [20] to optimize the position over steps. The LM algorithm aims to align the projected top-level image with the ground-level image. As such, we call our model AlignNet, of which we give an overview in Figure 1.  
To go beyond the previous work, we make significant contributions to their model and show the benefits of those contributions. In particular, we introduce a fusion of satellite and OpenStreetMap data to improve the input; we also improve the architecture using a different U-Net architecture and show improvements regarding how the model is trained. Furthermore, we contribute towards the research of this iterative sample-wise optimization model by providing an analysis of the behavior of this model. Finally, we conduct an ablation study on the model to show the importance of the various design choices made in our model.  
The next section discusses recent works on visual localization using satellite data. In Section 3, we describe our augmented reality-related method for cross-view outdoor localization by fusing map and satellite data. Then, in Section 4 we analyze the performance of the model proposed and also provide information regarding an ablation study. In the last section, we discuss and conclude on the findings of the current research.