# Development of Height Indicators using Omnidirectional Images and Global Appearance Descriptors
## 3 Development of Height Indicators Using Global Appearance Descriptors
### Method 4: Change of the Camera Reference System (CRS)  
The fourth method consists in simulating an artificial movement of the camera and calculating the new coordinates of the pixels of the image after the movement. Some researchers (such as [PERSON] et al. [44]) have used this technique to simulate a displacement of the Camera Reference System (CRS) using the epipolar geometry.  
To obtain the new image after the artificial displacement, the next steps are followed. First, each pixel of the original omnidirectional image is retroprojected to obtain its coordinates with respect to the WRS. Let \(m\in\mathbb{R}^{2}\) be the coordinates of one pixel of the omnidirectional image (with respect to the IRS) and \(a\) the distance from this pixel to the center of the omnidirectional image. The function \(f(a)\) (an example is shown in Equation (15)), obtained from the calibration of the catadioptric system, permits calculating the point of the mirror \(Q\) that projects onto this pixel \(m\). This point can be retroprojected onto the unit sphere and, as a result, the coordinates of this projection \(M\in\mathbb{R}^{3}\) respect to the CRS can be obtained. After that, a movement of the camera is simulated through a change of the CRS and the new coordinates \(M^{\prime}\) respect the new CRS system will be:  
\[M^{\prime}=M+\rho\cdot\overline{T}, \tag{13}\]  
being \(\overline{T}\) the unitary displacement vector, and \(\rho\) the scale factor, which is proportional to the amount of displacement. In this work, to simulate a vertical displacement \(\overline{T}=[0,0,1]\).  
Using the new coordinates of the projection of the point onto the unit sphere \(M^{\prime}\), the corresponding point on the mirror \(Q^{\prime}\) can be obtained and projected onto the new image plane, where the new coordinates respect the IRS will be \(m^{\prime}\). Repeating this operation for all the pixels of the original image, the result will be the new omnidirectional image after the simulated movement. After this process, some pixels of the original image may lay out the new image plane and some pixels of the new image may be empty. In this case, the value of these pixels is estimated as the average value of its 8 nearest neighbours.  
Once the new omnidirectional image after the artificial movement has been calculated, different projections can be obtained. Specifically, in this work, the orthographic view, the panoramic image and the unit sphere projection are considered. Figure 9 shows the simulated vertical movement of the catadioptric system. On the one hand, Figure 9a shows the projection of a world point \(P\) onto the original image plane \(m\) (plane of projection 1) and onto the new image plane after the simulated vertical movement \(m^{\prime}\) (plane of projection 2) obtained using epipolar geometry. \(F_{1}^{\prime}\) and \(F_{2}^{\prime}\) are the focal points of the hyperbolic mirror before and after the movement and \(F_{1}\) and \(F_{2}\) are the focal points of the camera. On the other hand, Figure 9b shows two sample omnidirectional images and panoramic projections considering the original image plane (\(\rho=0\)) and the new image plane after the simulated movement (\(\rho=0.4\)).  
In order to estimate the relative height between two scenes, the algorithm simulates several displacements of the reference image \(im_{R}\) by giving different values to \(\rho\), and compares each of them with the test image \(im_{T}\), considering \(\rho=0\), i.e., without CRS movement, and using global appearance. Finally, the algorithm selects the best match. The coefficient \(\rho\) associated with this match is a topological measurement of the vertical distance between images. In the experiments, the coefficient \(\rho\) of the reference image will be given values between \(-0.3\) and \(0.3\).