## 3 Proposed Approach
### Mapping Fisheye Image to Cubemap Image  
After calibrating the fisheye camera, we use the calibration parameters to generate the cubemap image from the fisheye image. As shown in Figure 5a, the cubemap model consists of five image planes generated by virtual pinhole camera models. Each virtual pinhole model has the same intrinsic but different extrinsic parameters because of different viewing directions. Thus, the fisheye image is divided into five virtual pinhole cameras. Meanwhile, the intrinsic parameters of the virtual cubemap depend on the size of each cubemap face. We calibrate the intrinsic parameters \([f_{Facx},x_{P},f_{Facx},y,\)\(F_{Facx},x_{Facx},y]\) of the virtual pinhole cameras so that the pinhole image resolution is half of the cubemap image resolution. Here,\(f\) is the focal length, \(c\) is the principal point, and the subscript _Face_ is the index of the five cubemap faces such that \(Face=\{Front,\ Left,\ Right,\ Bottom,\ and\ Top\}\).  
Figure 5b shows the relationship between the cubemap image plane and the cubemap model. To generate a cubemap image, we first define the transformation from the cubemap image coordinates to the cubemap model coordinates, as shown in Equation (14). Here, \(R_{Face}\) is a 3D rotation matrix of the index _Face_ cubemap plane.  
Figure 4: A graphical model of the enhanced unified camera model.  
\[R_{Front}=\left[\begin{array}{ccc}1&0&0\\ 0&1&0\\ 0&0&1\end{array}\right], \tag{8}\]  
\[R_{Left}=\left[\begin{array}{ccc}0&0&-1\\ 0&1&0\\ 1&0&0\end{array}\right], \tag{9}\]  
\[R_{Right}=\left[\begin{array}{ccc}0&0&1\\ 0&1&0\\ -1&0&0\end{array}\right], \tag{10}\]  
\[R_{Bottom}=\left[\begin{array}{ccc}1&0&0\\ 0&0&1\\ 0&-1&0\end{array}\right], \tag{11}\]  
\[R_{Top}=\left[\begin{array}{ccc}1&0&0\\ 0&0&-1\\ 0&1&0\end{array}\right], \tag{12}\]  
\[\textbf{X}_{CI}=\left[\begin{array}{ccc}\mathbf{u},&\mathbf{v},&1\end{array}\right], \tag{13}\]  
\[\textbf{X}_{C}=R_{Facc}\,\textbf{X}_{CI}. \tag{14}\]  
We transform the cubemap image coordinates \(\textbf{X}_{CI}\) into the model coordinates \(\textbf{X}_{C}\), which is on one of the cubemap faces, as shown in Figure 6. The concept of the EUCM-Cubemap projection model involves the use of the EUCM parameters to transform the point of the cubemap face to the pixel of the fisheye image or vice versa. First, we transform point \(\textbf{X}_{C}\) to another point \(\textbf{X}_{p}\) on the curve plane \(\mathbf{P}\) of the EUCM, as shown from Equations (15) to (18). We replace \(\textbf{X}\) in Equation (1) with \(\textbf{X}_{C}\). Here, the back-projection \(z_{P}\) is not required in this step.  
\[\textbf{X}_{C}=[x_{C},y_{C},z_{C}], \tag{15}\]  
\[\rho\,=\sqrt{\beta({x_{C}}^{2}+{y_{C}}^{2})+{z_{C}}^{2}}\, \tag{16}\]  
Figure 5: (**a**) Illustration of cubemap model. (**b**) Illustration of relation between cubemap image and cubemap model.  
\[x_{P} =\frac{x_{C}}{\alpha\rho+(1-\alpha)z_{C}}, \tag{17}\] \[y_{P} =\frac{y_{C}}{\alpha\rho+(1-\alpha)z_{C}}. \tag{18}\]  
Next, we project \(\mathbf{X}_{P}\) to \(\mathbf{X}_{M}\) on the EUCM normal image plane M using orthogonal projection, as shown in Equation (6). Finally, \(\mathbf{X}_{M}\) is projected to a pixel point \(\mathbf{X}_{I}\) on the fisheye image by multiplying it with the intrinsic matrix \(K\), as shown in Equation (7).  
By following this process, pixel coordinates in the fisheye image are transformed into the cubemap image coordinates. Using the backward transformation from the cubemap image to the fisheye image, we can identify the correspondences between the cubemap and the fisheye images. Because this process is computationally expensive, the coordinate transformation was performed once when the proposed method starts. Then, the index table of the pixel coordinates mapping between images is stored and used. Figure 6 shows the EUCM-cubemap projection model, and Figure 7 shows the difference between the fisheye image and the cubemap image projections.