# Cross-View Outdoor Localization in Augmented Reality by Fusing Map and Satellite Data
## 3 Methods
### Top-Level to Ground-Level Projection  
We provide details of our geometric projection, which was carried out following the steps for implementation provided by [PERSON] et al. [13]. The geometric projection module projects the deep-encoded top-level images to the ground-level. This was carried out using a pinhole geometric projection from a point on the top-level image, where we used a flat-world assumption. The camera was set a fixed amount above this flat plane and with a given camera intrinsic \(K\), as given by the KITTI dataset.  
The world coordinate system was set to the center of the top-level image; the \(x\) axis was parallel to the \(v^{s}\) direction of the top-level image, the \(y\) axis was pointing downward, and the \(z\) axis was parallel to the \(u^{s}\) direction. A 3D point \((x,y,z)\) in the world coordinate system was mapped to a top-level image pixel using the orthogonal projection,  
\[\big{[}\ u_{s},v_{s}\ \big{]}=\big{[}\ \frac{z}{\alpha}+u_{s}^{0},\frac{x}{ \alpha}+v_{s}^{0}\ \big{]}^{T}, \tag{1}\]  
where \(\alpha\) is the meter-per-pixel distance of the top-level image and \((u_{s}^{0},v_{s}^{0})\) correspond to the top-level image's center.  
Supposing a rotation given by matrix \(\mathbf{R}\) and translation given by vector \(\mathbf{t}\), we calculate the world coordinates as follows:  
\[\big{[}\ x,y,z\ \big{]}^{T}=\mathbf{R}\big{(}\ \ x_{c},y_{c},z_{c}\ \big{]}^{T}+ \mathbf{t}). \tag{2}\]  
The projection from the 3D point to the pin-hole camera image plane is formulated as  
\[w\big{[}\ u^{s},v^{s},1\ \big{]}^{T}=\mathbf{K}\big{[}\ \ x_{c},y_{c},z_{c}\ \big{]}^{T}, \tag{3}\]  
where \(\mathbf{K}\) is the camera intrinsic and \(w\) is a scale factor.  
Figure 2: **OpenStreetMap input examples.** Left columns shows pre-cropped satellite image. The right-most column shows the data, which were extracted with a legend for human readability, while columns 2–4 shows the input the model receives. The extracted images are 512 \(\times\) 512 pixels, with roads and nodes having a set size of 4 pixels and where each different color is a different class. The outlines of each areas as shown in the full OSM column are not part of the input to the model.  
Following Equations (1)-(3), [PERSON] et al. derived the following mapping from a ground-view pixel to a satellite pixel as  
\[\left[\begin{array}{c}u_{s}\\ v_{s}\\ z\end{array}\right]=\left[\begin{array}{ccc}\frac{1}{a}&0&0\\ 0&\frac{1}{a}&0\\ 0&0&1\end{array}\right]\left(w\mathbf{R}\mathbf{K}^{-1}\left[\begin{array}{c }u_{g}\\ v_{g}\\ 1\end{array}\right]+\mathbf{R}\mathbf{t}\right)+\left[\begin{array}{c}u_{s}^{0 }\\ v_{s}^{0}\\ 0\end{array}\right]. \tag{4}\]  
As said before, we make a flat-world assumption in these calculations. Given these equations, this is easy to do by setting \(y_{c}\) in Equation (3) to the distance from the camera to the ground plane. As the camera height is given, this is easy to calculate. Then, the scale factor \(w\) can be calculated using that equation. Alternatively, if this distance is not known such as in a diverse dataset, a strategy of using multiple different scale factors could be used [12].